Accountability in Higher Education: Bridge over Troubled Water?

Author(s): Jeroen Huisman and Jan Currie Source: Higher Education, Vol. 48, No. 4 (Dec., 2004), pp. 529-551 Published by: Springer Stable URL: . Accessed: 07/04/2011 11:01
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at . JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at . . Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact

Springer is collaborating with JSTOR to digitize, preserve and extend access to Higher Education.

Higher Education 48: 529-551, 2004. ? 2004 KluwerAcademicPublishers. Printedin the Netherlands.


Accountability in higher education: Bridge over troubled water?

1Centerfor Higher EducationPolicy Studies, Universityof Twente,P.O.Box 217, 7500 AE 2Schoolof Education, Enschede, TheNetherlands(; MurdochUniversity, SouthStreet,Murdoch,Western Australia6150, Australia(E-mail: Abstract. This article discusses the impact of accountabilityon higher education policies in Europe and the United States. We describe how the accountabilitymovement relates to other policy trendsin higher education,providingempirical data on how accountabilitywas implementedand how academics and managersin four universitiesperceivedthese policies. We close the article with a reflection on the observed shift from professional to political accountabilitythat uses 'soft' mechanisms that seem to offer little change in the quality of educationin these countries. Keywords: academics,accountability, higher educationpolicy, universitymanagement

Accountabilityis on the higher education policy agenda in many systems. In a numberof countries accountabilityis institutionalizedand commonly accepted, in others it is a recentphenomenon,and in others it is a contested issue on the higher education agenda. Some analysts think that governments and other stakeholdersdo not have the right to make academics formally accountablefor their performance.To supporttheir view most of these analysts refer to the concepts of academic freedom and professional autonomy.Othersbelieve that the increasingattentionto public, measurable accountabilityis the logical consequence of governmentsretreatingfrom closely monitoringhighereducationand allowing an increasein institutional autonomy. Moreover, others are preoccupied with the intended and unintended consequencesof the growing attentionto accountability. Given these concernsmany interestingquestionsarise regardingaccountability. This article has five aims. First, we present a conceptual explorationof the concept of accountability.Second, we investigate the rise of accountability: why and how has accountabilityenteredhigher education?Third,we review what has happenedat the national policy level regardingaccountability in France, the Netherlands,the United States and Norway. Fourth, we reporton how accountabilityis actually implementedand perceived by



staff members and managers in four universities in France (University of Avignon), the Netherlands (University of Twente), Norway (University of Oslo) and the United States (Boston College). Fifth, we discuss the paradox emergingfromthe empiricaldata:despite growingattentionto accountability at the nationallevel, at the shop-floorlevel staff membersare to some extent cynical about the ability of currentaccountabilitymechanisms to improve quality. Although some of our respondentsstated that accountabilitycould lead to greaterimprovements,they felt thatthe currentmechanismswere not very beneficial. At the extreme end there were those who disliked external forms of accountabilitypreferringinstead to rely on internalmotivationto improvetheirteachingand who countedon the professionalintegrityof their colleagues for quality improvement.There was a noticeable gap between policy rhetoricdemandingharsher,managerialforms of accountabilityand the lack of its implementation our four case studies. This is exploredin the in conclusion.

Conceptual exploration
Analysts of accountabilitygenerally agree that it is the "answerabilityfor (Romzek 2000, p. 22) or "the obligationto reportto others,to performance" explain, to justify, to answer questionsabout how resourceshave been used, and to what effect" (Trow 1996, p. 310). Both Romzek and Trow supplement these definitionswith the question:who is to be held accountable,for what, to whom, and through what means? Trow (1996) also questions the consequences (see Wagner(1989) for a similarapproach,and Kogan (1986) for a slightly differentmethod). Romzek (2000) offers the most comprehensiveframeworkfor analyzing types of accountabilityrelationships.She identifies four basic types: hierarchical,legal, professionaland political. The last two are the types that are more often found in higher educationcurrently.In some countriesthere has been a movement from professional to political accountabilityas national governmentsbegin to 'steer from a distance' (Kickert 1991; Marceau 1993), allowing institutionsgreaterautonomyat the same time as makingthem more accountable. Professionaland political accountabilitysystems reflect situations"where the individualor agency has substantiallymore discretionto pursuerelevant tasks than underlegal or hierarchicaltypes. And the review standards,when they are invoked, are much broader"(Romzek 2000, p. 25). Romzek notes
that the difference between professional and political accountability is the source of the standard for performance. "Professional accountability systems are reflected in work arrangements that afford high degrees of autonomy to



individualswho base their decision-makingon internalizednorms of appropriate practice" (2000, p. 26). Political accountabilityrelationshipsafford managersthe discretion or choice to be responsive to the concerns of key interest groups, such as elected officials, clientele groups, and the general public. Trow (1996) adds to Romzek's frameworkby more explicitly pointing to the functions of accountabilityand more specifically focusing on the higher education context. Regardingthe functions, he first maintains that power, therebydiscouragingfraud accountabilityis a constrainton arbitrary and manipulation,and strengtheningthe legitimacy of institutionsthat are groups. Second, accountabilityis claimed obligated to reportto appropriate to sustain or raise the quality of performanceby forcing those involved to examinetheiroperationscriticallyand to subjectthem to criticalreview from can outside. Third,accountability be used as a regulatorydevice throughthe kind of reportsandthe explicit and implicitcriteriato be met by the reporting institutions.

Accountability in higher education: Where does it come from? Why has there been an increasedemphasis on accountabilityin the 1980s and 1990s in both Europe and the United States? Here we explore some globalizing practices which have led to this increase and the shift towards more public, political accountability.A number of writers (Henry et al. 2001; Rhoades and Sporn 2002; Vidovich 2002) have argued that a global model of qualitypolicy in highereducationhas emergedthroughprofessional mechanisms, such as annual conferences and the internationalcirculation of professionals, as well as throughthe influence of internationalorganizafor tions, such as the Organisation Economic Cooperationand Development Network of Quality Assurance Agencies in and the International (OECD) Education (INQAAHE). Rhoades and Sporn convincingly demonHigher stratethe diffusion of quality assurancemodels between two 'core' regions, the USA and Europe. Quality assurancein Europe as in most parts of the world emerged in the 1990s whereas it existed in a variety of forms in the USA since the late 1800s. However,the global model that is emerging and influencing most countries differs to some extent from the accreditationof institutionscommon in the USA. It resembles more the quality issues introduced at the state level by state boardsand legislaturesemphasizing"quality reviewprocesses ... in the contextof strategicmanagementeffortsto refocus institutions"(Rhoades and Sporn 2002, p. 361). These forms of quality managementhave been taken from business and the federal governmentin the USA and are relatedto the efficient and effective use of public resources.



Similartrendswhich had alreadyexisted in the USA for at least two decades priorto the 1990s began impactingon highereducationin Europe.However, as much as it is clear that "the cross-Atlanticand intra-European patterns of influence are evident in the professional discourse of higher education" (Rhoadesand Sporn2002, p. 369), there are also local differencesand resistance on the partof some Europeanpolicy makersto following the American model. Furthermore, there are countries that are taking the lead in Europe, such as England and the Netherlands,and developing their unique brands of quality assurancesystems. In coining the term 'glonacal' Marginsonand Rhoades state that "at every level - global, national and local - elements and influences of other levels are present"(2002, p. 289). There are policy networks at every level that are influencing the creation of quality agendas that are similar to each other, yet have unique attributesthat significantly relateto their geographicand historicalcontexts. At the same time, there are overridinginfluencesthatdeterminemajorshifts in highereducationpolicies. The ideological shift towardsthe New Right led to greaterprivatizationof higher education and was a major influence bringing marketforces to bear on universities. In addition, the following global trends influencing higher education systems from the 1980s to the present have affected the type of mechanisms)established qualityassuranceprograms(andthus accountability in differentnationalsystems (see also e.g., De Boer et al. 2002; WorldBank 2002). * Changing relationshipsbetweengovernmentsand universities:In most systems there was a relatively strong bond between government and higher education institutions through funding, legislation, and planning mechanisms. However, governments have retreatedand opened the arena for greaterautonomy and marketmechanisms (Gornitzkaet al. 1999). In this context, Neave's (1988, 1998) analysis of developments in WesternEuropeis revealing.He points to the strikingchange from ex ante governmentalcontrol by legislation and proceduresto ex measures.This post justificationby qualityassuranceand accountability visible in WesternEuropein the 1980s and was particularly development in Centraland EasternEuropein the 1990s. The change occurredearlier in the United States wherepublic policies combined with marketmechanisms particularly the 1970s and 1980s when political accountability in overtook professional judgment in universities as the quality mechanism. According to Trow (1996), accountabilitybegan to replace trust in professionalintegrityin the USA duringthis period. * Efficiencyand valuefor money:A related,yet autonomousdevelopment is the growing trendof governmentsto documentvalue for money. This is partly due to the massificationof higher educationaroundthe world



pressuringgovernment budgets.Withincreasingstudentnumbersthe cry for efficiency and effectivenessbecame louder,for instance parentsand tax-payersbegan to challengethe presumedqualityof highereducation. Duringthe pastdecade in manycountries,a specificelement of the value for money issue shiftedfromconsideringhighereducationas a public or quasi-publicgood towards consideringhigher education as more of a privategood. Within this context debates occurredregardingthe introduction of tuition fees and student grant systems or interest-bearing loans. Understandably such debatesinfluencedthe accountabilityissue. Students confrontedwith increasedprivate costs for higher education became more criticalof the services deliveredin exchange. * Internationalizationof higher education and globalization: National borderswere once evident;howevertoday,globalizationof the economy, the free flow of goods, services, ideas, and people, blurs these boundaries. Globalization facilitates the entrance of foreign higher education institutions and business organizationsinto national arenas and alters the previouslyhomogeneousculturaland normativeexpectations concerningthe natureof higher education.This culturalchange, which may only be a gradual long-term change, raises questions related to Should foreigninstitutionsbe treatedin a similarmanner accountability. to national institutionsor should they be treateddifferentlyaccording to their position, possibilities, and duties within the higher education landscape?Additionally,should foreign institutionsbe accountableto the government in their home country or to the government in the debatesregarding countrywherethey preside?In this contextthe current the inclusion of education in the General Agreement on Trades and Services (GATS)are also relevant(see Altbach 2001). If higher education is included in the WTO agreement, does this imply that such for global arrangements supersedenational or supra-national, instance on accountability? European,agreements I* nformation and communication technology developments: The increasing technological possibilities particularly in the context of informationand communication technologyhave hastenedglobalization This adds to the previouspoint in two ways. First,the actual processes. location of a higher education institution becomes less relevant as technologies allow institutions to work globally and easily across national boundaries. Second, questions regarding legal and political controlover less tangibleor virtualinstitutionsbecome more urgentand complex. In sum, variousinterrelated trendsare affecting higher educationand the role and instruments accountabilityin higher education.However,its role of



will differ depending on the historicalcontext and the way national governments decide to implement accountabilitymechanisms and how they are approaching globalizationas a neo-liberaleconomic ideology. Accountability in national contexts On the basis of an analysisof developmentsin the fourcountries(see Currieet al. 2003 for more details), we observedthat accountabilityand globalization were particularlyvisible in policies that stressed the importanceof higher educationin its competitiverole, that is, supportingthe nation in the global economy. This challenges nationalgovernmentsto keep a close watch on the effectiveness and efficiency of higher educationinstitutionsand make them more accountable. Our conclusions for universitiesare rathersimilar to those of Leithwood et al. regardingaccountabilityat the school level: "Thecurrent preoccupation with educational accountabilityappears to have begun in most developed countriesin the 1960s, acquiringsignificantnew energy during the mid-tolate 1980s. The reasonsfor these calls for greateraccountability, furthermore, are to be found in the wider economic, political, and social context of which schools are a part.These contexts are not uniformacross all countries"(1999,

p. 11). we There somenoteworthy are the differences among countries examined. In Norway, publicmanagement andthe country's new ideas in position the European that but political landscape, is partof Europe, not of the for were European greater accountability. Community, triggers introducing
mentdecentralized number activities, initiating institutional thus a of greater of werestripped theirthorns As a result, accountability the autonomy. policies the becameless effectual than during implementation processandin practice
the governmentplanned The governmentintended to introduceaccountabilitymechanisms, such as activity planning and quality assurance;however, concurrentlythe govern-

in Dutchuniversities gainedgreater of also autonomy a number areas.

This general trend requirestwo qualifications.First, the governmentmain-

tainedits powerovera number aspectsof highereducation. of Second,in some areasit was not so mucha questionof more or less autonomy but

of shifting responsibilities and accountabilitymechanisms, in other words, 'steeringfrom a distance'. The most importantdevelopmentin practice was the introductionof a national quality assurancesystem. The original intentions of the governmentseemed far-reaching. But, similar to the Norwegian case and due to the Dutch corporatistmodel of policy making, the actual

of turned to be moremodest. out implementation accountability



The face of accountability the United Stateschangedovertime. Broadly in speaking, there was a shift from an internally-oriented system of accountaaimed at improvement towardsan explicit, externally-oriented The one. bility reasons for the change lie significantlyin the fact that the costs of higher educationgrew enormouslywith consequencesfor nationaland statebudgets and the generalpublic's view that highereducationwas not deliveringvalue for money.These changesclearlysaw a movementfromprofessiornal politto ical accountabilityas universitiessought to gain greaterpublic approvalfor theirquality. The French higher education system is still largely controlled by the Ministry,despite some developments towards deregulationand decentralization. The efforts of the governmentto implement formal accountability mechanisms were not accepted wholeheartedly and turned out to focus mainly on monitoringdevelopmentsin higher education.The quality assurance throughthe Comitw National d'Evaluation (CNE) was introducedas a voluntaryscheme and there have been few consequences for the institutions involved, otherthan increasingtheirprestigeand reputation.

Accountability in institutional contexts We now take a furtherstep in ourinvestigationof accountability looking at by how it was implementedwithinfour universities.We describethe impactthat nationaldebatesandpolicies actuallyhad on daily practicesand on the views of academics and administrators, summarizingthe main findings according to three central themes/questionswe discussed with the interviewees: the measuresat the government universitylevels, the measures and accountability at the individuallevel, and the effectiveness of these measures. Sample We gatheredqualitativedatathroughin-depthinterviewswith academicsand managersfrom four diverse universitiesin France,the Netherlands,Norway, and the United States. It is importantto emphasize that this research was not a strictly comparativestudy because we did not control the sample of institutionsor the interviewrespondentsto enable a statisticalor explanatory in comparisonof our findings.However,we chose a similarset of participants each of the universitiesand they were asked a similarset of twenty questions aboutgovernance,accountability, competitionand generatingfunds, and new technologies, ending with a few questions regardingthe role of tenure and the futureof the university.Thus, this allowed us to observe the similarities and differences in the trends that existed during 1998 and 1999 across the



four universities.In this article, we focus on the three questions asked about accountability. There were 131 interviews with a small numberof senior managersand an approximately equal numberof academic staff from professional schools (education, applied languages, and/or law), sciences, and social sciences (arts). The interviews were conducted face-to-face, almost entirely by one of the authors,ensuringconsistency in questioningand depthof probing.The sampleincluded37 individualsfromBoston, 32 fromAvignon, 31 from Oslo, and 31 from Twente. The academics interviewed ranged from professors to assistantprofessors,consisting of more men than women, particularlyat at Avignon andTwente,with a more equal representation Boston College and Oslo. The senior managersinterviewedincludedpresidents,vice rectors/vice presidents,provosts,and universitysecretaries/registrars. We chose particularuniversities that were not representativeof universities in their countries but were chosen to represent different types of universities (large/small; capital-based/provincial; public/private;researchoriented/teaching-oriented;managerial/collegial). Boston College is a medium-sized (12,500 students), private, Jesuit university located in the United States,which became highly manageddue to a brushwith bankruptcy in the early 1970s. The University of Avignon is a small (7,100 students), provincial,liberalartsuniversitylocated in southernFranceand is mainly an institution.The Universityof Oslo is a large (34,400 students) undergraduate universitylocated in the capital of Norway and is mainly researchfocused. The Universityof Twenteis a small (5,500 students),entrepreneurial univerlocated in the easternpart of the Netherlandsand combines technology sity and social sciences. Oslo and Twentedeterminetheir futures,having greater autonomy than Avignon, yet not as much as Boston College with its need and capacity to garnerprivatefunds. However,Twenteis less dependenton governmentfunding, becoming an 'entrepreneurial' universityby building up money from researchand consulting contracts.Avignon and Oslo staff members are beginning to embrace and serve local community economic interests, yet their traditionalnature and reliance on government funding remainsintactmore thanBoston College or Twente.Studentsat Oslo did not pay tuitionfees; at Avignon,they paid a few hundreddollarsin administrative costs; at Twentethey paid about$1,500; and at Boston tuitionfees were about $21,500 (US dollarequivalentsin 1999). Accountabilitymeasuresinforce at universityand departmentlevel Here respondents were asked to mention the accountabilitymeasures, for example research indices, quality reviews, and teaching evaluations that were introducedby the governmentto monitoruniversitiesand departments.



The Universityof Avignon seems the least preoccupiedwith accountability requirements the government.A small number(16 percent)of responses by indicatedthat there is no control or monitoringin place and a little less than a third (29 percent) of the responses stated that there is no change or only In debatestakingplace on the issue of accountability. sum, almost half of the statedthattherewas a lack of accountabilitymechanismsor a lack responses of change otherthanthe monitoringof studentnumberswhich was seen as a form of accountability. traditional I would say that the structures of evaluation haven't really changed much.Asfor your examples,I am not really convincedthat the Minister takes muchnotice of the pass rates; howeverthe percentage of students enrolled,yes, evidently.But the only thing that this is usedfor is so that the Ministercan establish a budgetarynotationto allocate creditsto the but university, after this has been done, we are the ones who decide what we are going to do with thefunds and whetheror not we are going to cut certain courses or keep them. So the Minister in a way rids himself of this responsibility.OtherwiseI would say that in this area, nothing has really changed. Therehas been a change in the Minister's discourse, a change of methods, but not a change in the procedures of evaluation. (Avignon,Junior,Male, Academic, ProfessionalSchool) About half (47 percent) of the responses indicatedthat accountabilityexists 'out there' but no immediateeffect was noticeable. Most of the elements of for accountability, instancemonitoringof studentchoice, pass rates,required relatedfundingfor small parts of academics, and performance qualifications of the university budget seemed relatively harmless. Whereas the general tendency in the responses was that accountabilitymechanisms were not necessary,a few responses (8 percent)mentionedthat there should be some externalscrutinyfrom the government.The following quote identifies that the evaluationshouldbe on the contentof the course, not on the performance of the professor,which seems to be an important distinctionthat the French would want to maintain. We are in the middle of an importanttransformationat the moment, which is based on the English and Americansystems. Up until now, we had no studentcourse evaluationsat all, and I feel that this was missing from our system.Thestudentsshouldbe allowed to evaluatethe contents of their courses, (as opposed to evaluatingor makingcriticisms of the actualprofessors)and to makean evaluationof the overall way in which theirchoice of subjecthas beenpresentedto them- theyshould be given an opportunityto say which areas of the course could be improved, or what needs to be added, etc. I think that this is a very good idea,



howeversome of my more traditionalistcolleagues have a very negative attitudetowardsthis kind of thing, saying that it would underminetheir authority and prestige, which may I add, I think is quitefar from the truth.(Avignon, Senior,Male, Academic, Social Sciences) Boston College could be positioned at the other end of the spectrum.It must be stressedthat althoughthe university,being private,is not monitored by the state legislature,74 percentof the responses indicatedthat accountability is all around the place, mainly by external reviews. Nevertheless, a numberof responsesindicatedthatexternalreviewswere a fairlyrecentmonitoring device, introducedto improve the performanceof some departments and used to rewardothers. Theuniversity as puts togetheran evaluation the department a whole. for Thereare sticks and carrots with respect to monitoring.The university hasjust come out withan awardfor teachingfor faculty witha little cash prize of $4000, not much.It's more recognitionthan a monetaryreward. Ourperformanceis measuredindividuallybut also departmentalwise. In the university'sopinion, are we allocating too many resourcesto the Do graduateprogramversus undergraduate? we have enough electives on the book?(Boston College, Senior,Male, Academic, Social Sciences) In additionto the externalreview mechanisms, 15 percent of the responses named explicitly the internalscrutiny of class sizes and other elements of the educationalprocess. The attentionpaid to accountabilitydoes not always mean thatrespondentsare seriously 'bothered'by accountability. The academic vice presidentand deans do not seem to monitordepartments all that directly. There is an annual report that's put in, but I don't know exactly what happens to these. It goes up to the dean, and I have neverhad anyfeedback. (Boston College, Senior,Male, Academic, ProfessionalSchool) Most of the responses in this category,however, accepted external reviews and stated that these were helpful for improvingtheir programsand procedures.Also a numberof responsesindicatedthatthere were departments that were hardlyevaluatedor held accountableto managementat a higher level. In additionto the accountabilitymeasuresdescribedabove, some responses (6 percent)relatedto nationalrankingsof theiruniversity.This should not be literally taken as a direct accountabilitymeasure,for externalorganizations use universitydatato rankinstitutions,indirectlyimplying thatthe university could be asked to explain or justify its performancevis-&-visits regulatory bodies (in this case, its Board of Trustees). Others (5 percent) mentioned



to directactionby the department chair(exhortations do better)or dismissing a department chairif reportswere very negative. The Universityof Twente and the University of Oslo took middle positions between Boston College and the University of Avignon. At Twente almost all respondents(78 percent)referredto externalreviews of different aspects, teaching and research separately,by national visiting committees. The compulsoryparticipationin these quality assuranceprocesses implies for that each study program for education and each department/faculty researchwould write a self-evaluationand a peer review committee would on visit the programto assess the reportsand makes recommendations areas thatmay need improvement. The universityis fairly well set up in terms of quality assuranceprocedures and accountabilityprocedures. Thefaculties are reviewedtwice yearly, and all that is done in termsof performanceindicators,reviews, student evaluations - everything you can imagine. (Twente, Senior, Male, Academic, Social Sciences) A number of responses (15 percent) mentioned the funding mechanisms that partlytook into accountthe performanceof the universitybased on the number of graduatesat the Masters level, time to complete a degree, and numberof PhDs granted.A small number(7 percent)of the responsesrelated to the internalmonitoringpractices of the university,partly as a preparation for the national quality assurance system, and partly as a preparation for obtaining accreditation.At the time of the survey, this was a voluntary mechanisms. activity,soon to become partof the obligatoryaccreditation At the University of Oslo, most responses (57 percent) reported the role of completionrates in accountabilityproceduresand about a third (31 percent) mentionedthe annualproductivityforms. A minority of responses (7 percent) mentioneddiscipline reviews. It was clear from responses that completion rates and annualproductivityreportingpractices did not lead to severe consequences in terms of the budget, as these indicatorsdetermine only a small partof the budget. Completionrates of studentswere introducedseveral years ago by the government,so part of the funding is related to creditpoints. It is still not a major element, but it is there. Research indices are still rather primitive. It is essentially a question of publications and the number of doctoral candidates we produce. It's not sophisticated,and so far it is not specifically linked to the budgets or any rewardsor punishment. Quality reviews are done not on an extensive, regular basis, but the governmentasks the Research Council to conductperiodic reviewson the state of a discipline. It is done on a national basis. And teaching



evaluations, again there has been a consistentpush I would say from the Ministry,even supportedby prizes, for those who do this the way the former Minister would like it to be done. So yes, again there are really no heavy sanctions for those who don't use these evaluations. The pressure so far has been essentially to use course evaluations on a regular and a fairly systematic basis. The results of the evaluations havefew consequences those involved.As yet, thereis no mechanism for for translatingthe resultsof these reviewsinto decisions about budgets. (Oslo, Senior,Male, Academic, Social Sciences) There is a push to pay attentionto quality monitoringand quality reviews, but there are no mechanisms for translatingthe results of the reviews into decisions about budgets. Some parts of the university appear to take the accountabilitymechanismsseriously. Thereis a lot of evaluation, and that's new. Wedidn't have that at all manyyears ago, and of course,I thinkthat it is quite useful. Wehave had internationalcommitteesevaluating our research.And as to teaching, this institute has been doing that for a longer time than other institutes. The studentorganizationpicks out a couple of differentteaching units every semester to evaluate. Togetherwith the teacher in charge they choose the questions,and one of the studentsis given some money to do the statistics. I think it works quite well. (Oslo, Senior, Female, Academic, Sciences) Given the fact that many interviewed at the University of Oslo were not awareof all the mechanismsin place to monitortheir activities,the accountability practicesseem ratherceremonialwithoutany real sanctionsthat might threatenthe survivalof departments. Accountabilitymeasuresinforce at the individuallevel Although there is some overlapin the responses on accountabilitymeasures at the organizationaland the individuallevels, we decided to discuss them separately. At the University of Oslo, teaching evaluations (43 percent of the responses) and annualreports(40 percentof the responses) were mentioned the most as accountabilitymechanisms at the individual level. For many respondents,thereappearsto be some frustration expressedaboutthe amount of time the variousreportstake and the amountof overlapin demandsfor the same information. We have, of course, now an increased amount of reporting: that you recordyour plans for teachingfor the term, then after each term, then



after each year, you report what you have been doing. How much teaching you have done. Whatkind of researchyou have carried out. Also all sorts of publications and things like that. This is new to the university.It is not somethingthat happened in thatformat earlier on. And the head will have a conversationwith each memberof the staff once a year (Oslo, Senior,Male, Academic, ProfessionalSchool) Eleven percent of the responses relatedto the international peer reviews of theirresearch,such as grantapplicationsandjournalarticles.A small number of responses (7 percent) regardeduniversityprizes and dialogue meetings between centraladministration the faculties as other monitoringmechand anisms. The latter can easily be interpretedas 'soft' monitoring, that is, discussing problemsand raisingpossible solutions;the formeris more difficult to directlyconnect to accountability. One shouldinterpret mentionof the as a mechanismthroughwhich individualsare not so much prizes, however, held accountable,as rewardedon the basis of a comparisonof their merits or achievements.In sum, the following quote suggests that quality is not so much the game as quantityat the Universityof Oslo. Maybe we should call it countability,because it is always a questionof quantitynot quality.Theyare countingteaching hours, articles written, conferences attendedand projects planned. Weuse quite a lot of time to report about what we are doing. (Oslo, Senior, Male, Academic, ProfessionalSchool) At Boston College, 62 percent of the responses referredto the annual reviews, built on the evaluationof each individual'steaching, research,and communityservice. Everybodyhas to submitan annual report.In collaboration,the department chair;dean and associate dean scrutinizethat annualreport.So we look at the publications,the teaching evaluations,and the service. And the assumptionis that everybodywill be given a small raise, and then on top of that somewherebetween one and maybe up to two percent in meritpay can be added to that depending on ourjudgment. So faculty membersare aware that they are expected to be productive.I thinkthe criteriafor productivityarefairly well known. Wetend to rely on some quantitativemeasuresacross the board and the chair can add the voice that looks at qualityas well. (Boston College, Senior,Male, Academic, ProfessionalSchool)
A minority of responses (13 percent) mentioned mentoring programs and peer teaching evaluations and another 13 percent mentioned tenure reviews. With



respect to the latter:those without tenure go through the tenuringprocess, as has always happenedtraditionally, those who are going for promotion and to full professorare scrutinizedin great detail. There is now also discussion of post-tenurereviews at Boston College, which could involve sanctions for those whose productivityis below the normor whose teachingis evaluatedas below average. A small proportion(10 percent) identified no formal evaluation mechanisms, suggesting some discrepancy between departmentsat Boston College. Othersare not particularlysatisfied with the feedback they get as individuals. I don't think they are very effective at all. The sanction or rewardof a salary increase is I thinkeffective.It will tell you prettydirectlywhat the institutionthinksof you, but the lack of any kind offormativeevaluation I thinkshows and the institutionwould be betteroff if it did a betterjob of that. (Boston, Senior,Male, Academic, ProfessionalSchool) At the University of Twente, four direct methods of monitoring the performanceof academics were mentioned.Over a third (35 percent) of the responses reportedthat annual individualreviews were used without sanctions, 28 percent of the responses mentioned teaching surveys, 20 percent mentioned annual reportsgiven to the departmentchair, and 13 percent of the responses identified annual reviews with bonuses, task reassignments, or performance assessment plans. Only four percent mentioned indirect measures,such as countingthe numberof PhD studentsandformulafunding. We have some system of personal interviews every year linked to the annual report. Theremay be some sort of task reassignment,and some don't seem to have any system. This is a majorpolicy issue departments that the board of the universitywants to deal with, and it is in favor of a systematic approach of academic managementby the dean and the departmentchairs. Thiswould enable departmentalchairs or group leaders to implementa system of interviewsevery half a year and link these to an assignmentof tasksand resultsof the tasks, lookingbackand lookingforward. (Twente,Senior,Male, Manager) The personal interviews were criticized for their lack of usefulness, that is, they did not resultin salaryincrementsor even promotionunless an academic asked for specific advice in this regard. Thus the annual review meetings
were not usually used to discuss career development. Regarding the teaching

surveys, respondentsindicated that these were mostly applied to first year courses and that the focus was to a larger extent on the courses and the ratherthanon the individualacademicparticipating a program.A in programs



good example of a rathereffective methodof evaluatinga course is described by a female academicin the sciences. Our favourite type of evaluation is the oral method, evaluating the course witha small groupof students.Wehave representatives each of of the studentproject groups,say aboutfive or six students,and one of my colleagues and I meet with themabout once everymonthor so. Because this way when somethingis going wrong you can immediatelychange things. Weprefer that to having the writtenevaluation after the course because then the changes can only be made the next year. (Twente, Senior,Female, Academic, Sciences) At the Universityof Avignon, the patternwas similarto that of accountability at the organizationallevel. Altogether 68 percent said there was no evaluation;although out of this percentage, 20 percent reportedthat there were discussions on this issue and evaluationsmay occur in the future.This latterresponseis expressedin the following quote: For the moment,there are none at all, but I know that there is talk of establishing accountabilitymeasures during the new reformand that these measures will include student evaluation of courses. This has been planned for sometime this year; but up until now, nothing has materialized.(Avignon, Senior,Male, Academic, Sciences) Some felt there was a need for indirect and/or informal evaluations (15 percent), and clearly some actually felt that some type of individual accountabilitymeasures(17 percent)would be of benefitto the university. I would very much like to show the inspectors a piece of paper with a list of all my accomplishments, the things thatI am proudof, because all takes into account what actually happened during the year. nobody Qualitygoes unnoticedin this system/qualityis sacrificedfor quantity. Figures are used as an indication of quality. (Avignon, Senior, Male, Manager,ProfessionalSchool)

How effectiveare these measures? Here the respondents were asked to react to the question of whether the accountability mechanisms in place were effective in either monitoring quality or improvingthe quality of teaching and researchwithin the university.Do accountabilitymeasures improvethe quality of teaching and research?



Almost half (44 percent) of the responses at the University of Oslo indicatedthat accountabilitymeasuresdo not improvethe quality. In lots of instances, you can see that this has been a major source of stress and resulted in somewhatunproductiveadjustments.Whenyou rewarda specific kind of activityand not somethingelse, which may be equally important, can see unintendedconsequences.(Oslo, Senior, you Male, Academic, Social Sciences) About a similar percentage (47 percent) thought that they improved as a result of accountability,although a fair share of this percentage qualified theircomments.Examplesof these commentsinclude thatthe improvements in quality were mainly a result of a person's internal motivation or that it was necessary to encouragebut not to punish staff to improvequality.There were a few responses (8 percent)suggestingthatthey were not sureaboutthe impactor found it difficultto assess. At Boston College the responses were in general more negative than positive. About 56 percent of the responses stated that the measures were not effective. The annual review in which the dean sends you a letter about your strengthsand weaknesses,I don't get a sense that this is used to improve the quality of teaching or research.I have never heard of people acting on these things. I don't have a sense that this method has been used to create new ways or better ways to teach. (Boston College, Junior, Female, Academic, ProfessionalSchool) They described the kind of evaluation that they thought would be more effective. It appearsthat most want more formative ratherthan summative evaluations, more collaborative and peer-review types of evaluations and more one-on-one feedback, in a developmental,supportiveatmosphere.A few respondentsfelt thatthe current mechanismswere not harshenough.Also some commented on the fact that formalmechanismswere not as important as having a culturein which teaching was taken seriously.About 44 percent of the responses saw the currentmechanismsas effective. They felt that the teaching surveys gave good feedback and the rewards and sanctions may make people more productive,also that the tenure and promotionprocesses helped to focus attentionon teachingand research. I felt the effortsto do internalteaching observationsand meetings with faculty were really good. I really enjoyed those. At the suggestion of the chair I have gone to the Academic Development Center and had thatperson observe my class and make suggestions.Actually she came back twice and that was very helpful. I find fewer avenues with regard



to researchfor mentorshipbut there are some good teaching avenues. (Boston College, Senior,Female, Academic, Sciences) No one felt thatthe salaryincrementswere enough to really changebehavior, but receiving a negative incrementor even just an averageincrementwould signal disapprovalof performanceand may effect a change. Teaching and researchawardswere also seen as beneficial.Only one responseindicatedthat tryingto judge the effectivenessof these measureswas really too difficult. At the Universityof Twentethere is a more positive pictureof the impact of accountabilitymeasures.Slightly over half (54 percent) of the responses indicatedthatthe measureswere effective. Yes,I think they do have positive influences in the long run. I've been working at the university twentyyears, and in the beginning there for were few monitoringactivities in research or in teaching. And now I would say that these monitoring activities have improvedthe quality of teaching and research. (Twente, Junior, Male, Academic, Social Sciences) A little over a third (38 percent) of the responses showed that there were doubts about the effectiveness of accountabilitymeasures. The comments made were aboutthe lack of effectivenessof the currentmechanisms,because they did not think that these mechanismsreally changed the motivationto research,and they believed thatthere might be other mechanismsthat could have greaterimpact. It is a difficultquestionto because the quality of educationhas answer, sure improvedin the last decade and research the same. But I'm for not sure if the major impactof this improvement due to the measures is we have been discussing or pressurefrom outside, such as international competitionfor funding, national competitionfor students and so on. (Twente,Senior,Male, Manager) A few responsesin this categoryalso doubtedwhetherthe qualityof research or the effectivenessof teachingcould be assessed. A small number(8 percent) of the responses explicitly indicatedthat the mechanismsin force needed to be changed. They suggested introducingmore collaborative,non-individual measures,peer reviews, and more formativeevaluations. Giventhe lack of accountability mechanismsat the Universityof Avignon, to a general question of whether they thought these people responded
measures could potentially be effective in improving the quality of teaching and research in their university. About a fifth (22 percent) of the responses implied a positive response. They argued that the university needed to be



more efficient and open-mindedto the scrutinyof externaldemands.Over a half (56 percent)of the responseswere doubtful,but in a positive sense. They qualifiedtheirresponsesby saying thatinternalmotivationwas also necessary and thattherewas a risk if assessmentwould only take into accountquantitative indicators.. this category of responses, there were also academicsand In administrators indicatingthat it would be difficult to develop accountability mechanisms. I believe that academics should have the responsibility evaluating for their own work; it is part of theirjob. But it is difficultto know exactly which mechanisms should be put into place to make sure that this procedure is put to use effectively,and that it actually does become part and parcel of their role in the university.(Avignon,Junior,Female, Academic, ProfessionalSchool) Some arguments relatedto the choice of criteriaor what specificallyshouldbe evaluated.A minorityof responses (14 percent)indicatedthat accountability would not improveacademicperformance.A further8 percentfelt that they did not know enough to comment.In sum, some might suggest thatthereis an anarchistictendencyat the Universityof Avignon. If this is more widespread thanjust this university,then it will make it difficult for the governmentto mechanismson academics.The solutionmay be impose 'hard'accountability to implement'softer' mechanismsthataremore formative,usingpeer reviews and voluntaryteachingevaluations. Discussion When we look at the type of mechanismsin use, both at the organizational andthe individuallevel, it is strikingthatmost mechanismswould be categorized as 'soft' measures: monitoring and explanation, and a few of them would be described as strong measures:justification.Regardingthe effects of accountabilitymechanisms,many academics in this study were skeptical aboutthe effectivenessof currentmeasures.Therewas some oppositionto the bureaucratic procedures,the amountof work involvedandthe focus on quantifiable indicators.Many respondentsdoubtedwhetherthe procedureswould indeed have the presumedimpact.Across the four countries,many respondents argued for less formal, more individualisticprocedures, and pleaded for a culture in which informal procedures were accepted as part of the working environment.On the other hand, there were also argumentsfrom a minorityof respondentswho wantedto introduceaccountabilitymechanisms thatrewardedgood practicesand punishedlow qualityperformance. The responses clearly were influenced by the contexts in which the accountabilitydebate took place and the experiences gained with accounta-



bility. The four universities apparentlywere in different stages of development. Boston College and the University of Twente seemed to be the two institutionsin which accountabilitymeasureswere to an extent institutionalized, althoughthe Boston case showed considerablevarietyby department. Anotherdifferencewas the fact thatin the Dutch case, the universityfollowed the national quality assurance requirements,whereas in the United States case, there was much more variety in proceduresused by the departments. Aside from regional accreditationagencies in the United States, there is no national body to monitor accountabilitymechanisms and most universities set up theirown accountability The Universityof Oslo seemedto procedures. be representative universitiesin which accountabilitywas to some extent of implemented,but at the same time practiceswere mostly ceremonial.At the Universityof Avignon accountabilitywas least visible, debates were taking place, but in practicenot many mechanismswere implemented.In particular, the practicesat the Europeaninstitutionswere reflectionsof developmentsat the nationallevels. Connecting the findings at the case study level to those at the national levels and the theoretical framework,there are some interesting relationships. Romzek's (2000) types of accountabilitymost found in professional organizationswere true of these universitieswith a slight movementtowards greaterpolitical accountability.The use of performanceindicatorsby state level authoritiesin the United States - to show 'value for money' illustratesthis shift towardspolitical accountability. Anothertype of dynamismis shown in most nationalcases thatillustratelegitimateauthorities'shift from invoking one type of accountability,proactiveby legislative requirements, towards other output-orientedforms, such as accreditation,performancebased funding and reporting performanceindicators. Most accountability mechanismsbelonged to the category of 'soft' mechanisms.They were not set up to sanctionindividualsor theiractivities. The national and institutional case studies seem to indicate that the changing relationship between government and universities is the most importantfactor affecting the rise of accountability.The shift in types of steering relationships towards more institutional autonomy and to some extent increasing market mechanisms invoked accountabilitymechanisms or new types of accountability mechanisms. Neither administratorsnor academicswholeheartedlywelcomed many of these. Conclusion: Are accountability policies failing universities? One of the striking findings was that 'soft' accountabilitymeasures were favouredover 'hard'measuresthatwould involve rewardsand sanctions.This implies a deviationfrom the contentsof most of the policy proposals(either



at the national or institutionallevel). How is it possible that strong cries for moreaccountabilityfromhighereducation'sstakeholders variouscountries in have - seemingly - no direct, severe impact on the day-to-daypractices of academics?There are differentexplanationsfor this paradox;some of these were evident in our empiricaldata.We addressthese briefly. Shiftfrom professional to political accountability One could maintainthat despite the governments' attempts to bring about change by implementingaccountabilitymechanisms, the academic professionals within the organizationswere able to resist - and possibly subvert - these policies because they saw them as a sort of 'window dressing'. The shift from professionalto political accountabilityhas allowed universitiesto demandedby theirpublic stakeholders satisfy the accountability requirements and legislaturesby introducingaccountabilitymechanismsthat for the most partcount existing activities. Sometimesthey introducenew measureswhich take the time of academics but they do not necessarily exact new activities upon them. They generally use.those measuresthat are alreadyin existence, such as teaching evaluations and annual reviews which count grants and publicationsof academics. These do not change the day-to-daybehaviourof academicsand do not necessarilylead to any increasedqualityfor the clients of the universities,namely the students. Within universities, as highly professionalized sites, there is internal controloverthe processes of educationandresearchas in most public bureaucracies. Governmentsat a greatdistancefrom these processes are not able to get a grip on the internalworkingsof universities.For one reason, they lack the specialized knowledge or the ability to judge the quality of education given to students.For anotherreason the cost of gaining such knowledge is usually consideredto be too great. This argumentexplains why even in cases where the government has been able to implement some elements of accountabilitymechanisms (e.g., in the Netherlandsand Norway), the ultimate impact is less than expected. Researchon the impact of quality assurancemechanismsand activity planning in Norway seems to be in line with this contention; for example, activityplanningwas merely experiencedas a ritualwith hardlyany harmful effects (Bleiklie et al. 2000) and the universitiesmerely turnedthe quality assessmentproceduresinto processesthatsuitedtheirstrongbottom-uptraditions (Stensaker1997). The Dutch experience seems to be a case where the academicsand theirrepresentative bodies have been able to stripthe intended accountabilitymechanismsof their thorns.Already in the corporatistpolicy process, specific universityorganisationswere able to influence the debate. For example, the Association of Dutch Universities(VSNU) was able to gain



control over the quality assurancesystem and to 'convince' the government not to collect performanceindicatorsto gain insight into the quality of the universities(Huisman2003). Policy rhetoric One interpretation the subversionof accountabilitymechanismscould be of thatthe government policies simply failed to implementmore severepolicies. This would not be surprising,for there are many examples of policy failures or partlysuccessful policies in highereducation(Cerychand Sabatier1986). On the other hand, it would be surprising,given the variety of contexts in which the policies were formulatedand the varietyof instruments suggested, that all attemptsacross the countries and institutionsto implement harsher forms of accountability. Another explanation could be that most attention to accountability in (government) policy papersis merelyrhetoric.Thatis, governments plead for accountabilitymeasuresbut actually refrainfrom enforcing specific policy instruments. is actuallydifficultfor them to monitorwhetherhighereducaIt tion institutionsreally accountfor theirperformance. Thereis also usually an on processes ratherthan outcomesand rarelyare sanctionsapplied. emphasis Valueof 'soft' mechanismsand professional accountability A thirdline of reasoningpoints to the managementof highereducationinstitutionsas the weakestlink in the accountability chain.Governments may have been successful in puttingforwardaccountability policies, but if institutional leaders do not 'translate' the policies into institutionalmechanisms, then nothing changes. There are some indications in the empirical materialthat indeed institutionalmanagementwas hesitantto implement such measures. An important explanation- at least in the Europeancontinentalcontext - is that universitymanagementis still in its infancy.That is, only recently have institutionalleaders been grantedthe power to really manage their institutions. Before the mid-1980s (in some countriesearlieror later) institutional leadershipto a considerableextent implied ceremonialbehaviorand routine leadership.Since the 1990s, the roles and functions of these leaders became much strongerand from the late 1990s on, we began to see a new generation of leadersin power (again:thereare important differencesbetween the countries).These new leadersstill have to get used to theirnew roles, which could imply that strongerleadershipis emerging,but only gradually.Some leaders
have grasped the opportunity to lead and implemented far-reaching changes in their institutions. Others have tended to stay in a more traditional role and decided against managing in an overt and aggressive way. This could explain



the ratherweak implementationof accountabilitymeasuresin the European countries. Anotherexplanationof the hesitancyof universityleaders is not so much based on a lack of managerialskills, but based on a positive readingof their currentleadership:purposively the managers side-step stronger accountability measures for softer ones which they believe are more effective in manner. Thusthey gain greaterprofesleadingin a collegial and collaborative sional integrityfrom their staff. In the case of Boston College, which has a strong managerialculture and where a range of accountabilitymechanisms exist, theremay be good reasonswhy the case for more punitivemeasureshas not been made. The managerstheremay have recognizedthat 'soft' mechanisms are more effective in a professional environment.Based on Romzek's types, universitiesare places of high autonomywhere the kind of monitoring that would suit them best is one based on professional expertise. This is why qualitative judgmentsmade by one's peers are preferableto hierarchical assessments based on efficiency and hard quantitativeindicatorsthat often eschew qualitative judgments. There is already considerablejudgment exercised in universities that is dependentuponpeer expertise,for examplein peerreviews of grants,articles, promotionapplications, and teaching awards. There are limits to the type of managerialaccountabilityimposed from above. It may be effective for some type of workers, but for university scholars 'soft' monitoring may be better because they are more often motivated by intrinsic rather than extrinsicrewards.Managersin these four case studies may be intuitively in touch with this idea and have avoided the worst excesses of hierarchicalor managerialaccountabilityor 'hard' monitoring,which would use quantifiable performanceindicators and sanctions that may not produce the most effective results in a universityenvironment.If this explanationholds, the universitymanagershave been able to build a bridgeover troubled(accountability) water,easing the minds of their academics. If the managers'strategy seems effective, policy makersat nationallevels may want to heed the advice of university managers and especially academic researcherswho may be best placed to assess the type of accountabilitymechanisms most suited to universities.

P.G. and education theWTO: run Globalization amok', International Altbach, (2001).'Higher Education 2-4. 23, Higher R. Bleiklie,I., Hgstaker, and Vab0,A. (2000). Policyand Practicein HigherEducation. London: Universities. JessicaKingsley. Reforming Norwegian



Boer, H. de, Huisman, J., Klemperer,A., Meulen, B. van der, Neave, G., Theisens, H. and Wende, M. van der (2002). Academia in the 21st Century.An Analysis of Trends and Perspectivesin Higher Education and Research. The Hague: Advisory Council for Science and TechnologyPolicy. Cerych, L. and Sabatier,P. (1986). Great Expectationsand Mixed Performance.Stoke-onTrent:Trentham Books. Currie, J., Boer, H. de, DeAngelis, R., Huisman, J. and Lacotte, C. (2003). Globalizing Practices and UniversityResponses.Westport:GreenwoodPress. A., Gornitzka,A., Huisman,J., Klemperer, Maassen, P., Heffen, O. van, Maat, L. van de and EuropeanHigher Vossensteyn,H. (1999). State SteeringModels with Respect to Western Education.Enschede:CHEPS. Henry, M., Lingard, B., Rizvi, F. and Taylor, S. (2001). The OECD, Globalisation and EducationPolicy. London:Pergamon. reformin higher education:Foreverchanges?', in Denters, Huisman,J. (2003). 'Institutional B., Heffen, 0. van, Huisman,J. and Klok, P-J. (eds.), InteractiveGovernanceand Market Mechanisms.Dordrecht:Kluwer,pp. 113-128. Kickert, W.J.M. (1991). 'Steering at a distance: A new paradigm of public governance in Dutch higher education'. Presented at the European Consortium Political Research, for Universityof Essex. An Kogan,M. (1986). EducationAccountability. Analytic Overview.London:Hutchinson. Leithwood, K., Edge, K. and Jantzi,D. (1999). EducationalAccountability:The State of the Art. Gtitersloh: Bertelsmann. Marceau,J. (1993). Steeringfrom a Distance: International Trendsin the Financing and AustralianGovernment Governanceof Higher Education.Canberra: PublishingService. S. andRhoades,G. (2002). 'Beyond nationalstates,marketsand systems of higher Marginson, education:A glonacal agency heuristic',Higher Education43(3), 281-309. Neave, G. (1988). 'On the cultivationof quality, efficiency and enterprise:An overview of recent trends in higher education in WesternEurope', European Journal of Education 23(1/2), 7-23. Neave, G. (1998). 'The evaluativestate reconsidered',EuropeanJournalof Education33(3), 265-284. Rhoades,G. andSporn,B. (2002). 'Qualityassurancein Europeandthe U.S.: Professionaland political economic framingof highereducationpolicy', HigherEducation43(3), 355-390. Romzek, B.S. (2000). 'Dynamics of public accountabilityin an era of reform', International ReviewofAdministrativeSciences 66(1), 21-44. The role of qualityassessmentsin Stensaker,B. (1997). 'From accountabilityto opportunity: in Higher Education3(3), 277-284. Norway', Quality Trow, M. (1996). 'Trust, markets and accountability in higher education: A comparative perspective',Higher EducationPolicy 9(4), 309-324. Vidovich, L. (2002). 'Quality assurance in Australianhigher education: Globalisation and 'steeringat a distance",Higher Education43(3), 391-408. Wagner,R. (1989). Accountabilityin education:A philosophical inquiry.New York/London: Routledge. EducaWorldBank (2002). ConstructingKnowledgeSocieties: New Challengesfor Tertiary tion. Washington: WorldBank.

Sign up to vote on this title
UsefulNot useful