See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/317239505

Internal Quality Assurance: A New Culture or Added Bureaucracy?

Conference Paper · September 2016

CITATIONS READS

2 1,354

4 authors:

Maria J. Rosa Sónia Cardoso

University of Aveiro Interdisciplinary Research Centre for Education and Development - ULusófona an…
135 PUBLICATIONS   1,803 CITATIONS    61 PUBLICATIONS   552 CITATIONS   

SEE PROFILE SEE PROFILE

Pedro Videira Alberto Amaral

ISCTE-Instituto Universitário de Lisboa Centre For Research In Higher Education Policies
51 PUBLICATIONS   226 CITATIONS    159 PUBLICATIONS   4,093 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Performance Measurement and Management in Research View project

SMART-QUAL: Structured indicators to manage HEI Quality System View project

All content following this page was uploaded by Pedro Videira on 30 May 2017.

The user has requested enhancement of the downloaded file.

CHER 29th Annual Conference
University of Cambridge, UK
Internal Quality Assurance: A New Culture or Added Bureaucracy?

Track 4 Other research topics related to the conference theme

Maria J. Rosa, DEGEIT and CIPES, University of Aveiro, Aveiro,Portugal

Sónia Cardoso, A3ES and CIPES, Matosinhos, Portugal
Pedro Videira, CIPES, Matosinhos, Portugal
Alberto Amaral, A3ES, Lisboa, Portugal

Corresponding author: Maria J. Rosa

Corresponding author’s email: m.joao@ua.pt

Structured Abstract:
Purpose:
This paper analyses teaching and non-teaching staff’s perceptions on the implementation of
internal quality assurance (IQA) practices at higher education institutions in order to find out
which perspective(s) on quality – as culture, as compliance and as consistency (Cardoso, Rosa
and Stensaker, 2015) – is(are) prevalent among these professionals.
Approach and methodology:
Theoretically the study is based on the perspectives on quality proposed by Cardoso et
al.(2015): quality as culture refers to both the academic community’s commitment to quality
and to the structures, processes and procedures contributing to its enhancement; quality as
compliance stands for the external policies and guidelines influencing institutions’ definition of
their IQA framework; quality as consistency links to the academic standards defined for the
assessment and validation of higher education processes and end results, as the new
standards for quality assurance.
Empirically the study uses data from a survey sent to the professionals of all Portuguese
institutions.
Findings:
Professionals emphasised the legal requirements and demands from A3ES, as well as
institutional efforts to promote continuous quality improvement as the most important factors
for developing IQA practices. Furthermore, these practices were seen as displaying the idea of
quality assurance (QA) as both improvement, self-awareness and self-reflection of the
institution and a set of processes centrally managed and guided by external requirements. IQA
practices were also seen as producing a number of effects, which seem to reveal an increase in
bureaucracy and over formalisation. Hence the results suggest that both the perspectives of
quality as culture and as compliance seem to permeate Portuguese institutions.
Research limitations/implications:
This study only addresses the Portuguese case. The possibility of a comparative perspective
would strengthen the analysis. Furthermore it is based only in quantitative data mainly from
teaching staff. This paves the way for further research based on a more qualitative approach.
Practical implications:
This study allows for a better understanding on how IQA practices and their effects are
perceived within the academia. It can contribute to instigate institutions’ critical reflection
about how quality assurance can be more effective and aligned with academia’s expectations.
Original/value:
The study focuses on a subject that is still relatively under-researched. Furthermore it includes
a professional group usually absent from research on higher education.

Introduction
Quality is a multidimensional concept that can be seen as having multiple and interrelated
meanings which can be distinguished but not completely separated (Green, 1994; Harvey and
Green, 1993; Kleijnen, Dolmans, Willems, and Van Hout, 2013).
This multidimensionality may be explained by the fact that quality is a relative concept, which
depends on the way it is perceived by different stakeholders (Green, 1994; Harvey and Green,
1993; Westerheijden, 2007). Furthermore, quality encompasses different higher education
dimensions, namely “the quality of inputs, outputs and processes, which have to be combined
with the demands put forward by students, universities and society each time one intends to
assess quality” (Sarrico, Rosa, Teixeira, Cardoso, 2010: 40).
The fact that quality does not appear as something unique and absolute in higher education
led to the emergence of different quality definitions sustaining QA schemes over time (Harvey
and Stensaker, 2008). While in the late 1980s and in the 1990s, most attention was given to a
fitness-for-purpose and a value-for-money approach to quality, in recent years, and most
probably due to the Bologna Process and to an increasing interest in the outcomes of higher
education, quality as excellence seems to be gaining prominence.
Looking back specifically at the development of quality assurance schemes, a distinct
characteristic is that they are almost constantly changing (Westerheijden, Stensaker, Rosa, and
Corbett, 2014). The reasons for these changes have varied over time, emphasising the debate
concerning whether they should be improvement or accountability oriented, at the end of the
1980s and throughout the 1990s (Thune, 1996), to discussions on whether attention should be
given to the assessment of programmes or to institutional routines and structures, in the
2000s (Stensaker, 2008), to the current emphasis on how quality assurance can accommodate
the current political interest for learning outcomes in higher education (Kohoutek, 2013). Over
time, HE has also changed and institutions have been given greater autonomy, and often been
mandated to establish their own internal systems for quality assurance (Westerheijden,
Hulpiau, and Waeytens, 2007).
In line with Cardoso, Rosa and Stensaker (2015), this multiplicity of conceptions and
approaches to quality and its assurance may be read using three different perspectives of
quality in higher education: quality as culture, as compliance and as consistency. These
perspectives ‘can be said to exist side by side as proxy measures for quality’ (p. 4).
In this paper it is assumed that these perspectives of quality collect different degrees of
support by institutional actors, therefore reinforcing or, otherwise, compromising the
implementation of internal practices aiming to promote quality.
Teaching staff has a pivotal role in higher education. They are the ones responsible for
developing educational programmes, for delivering them and are also involved in the
assessment of those programmes’ quality. Therefore what they think about quality and its
assurance, namely regarding the factors for its development, its implementation features and
its effects, emerges as relevant (Cruickshank, 2003; Kleijnen et al., 2013; Newton, 2000).
Although at a different level, non-teaching staff also has a position regarding the
implementation of quality assurance practices. They are usually the ones working in the
organisational structures that put in place such practices. Therefore it seems also important to
understand their views on this issue. This is even more pertinent as, at least to our knowledge,
there are no studies focusing on how non-teaching staff embraces quality in higher education.
Taking the Portuguese higher education context as its starting point, this paper aims to analyse
teaching and non-teaching staff perceptions of the implementation of IQA practices in higher
education institutions. The goal is to understand what is(are) the perspective(s) on quality
prevalent among these groups of actors. Additionally it discusses how this(ese) perspective(s)
relate to the overall satisfaction induced by quality assurance practices.
The paper is organised as follows: firstly, a discussion of the three perspectives on quality is
presented stressing for each one of them the factors that may lead to the development of IQA
practices, the implementation features characterising these practices and their possible
effects; secondly, the empirical study conducted to collect and analyse relevant data is
presented; thirdly, teaching and non-teaching staff perceptions are examined in order to
identify the prevalent quality perspective(s); finally, the main conclusions are put forward as
well as the study implications both for research and practice.

Three perspectives on quality

As suggested by Cardoso et al. (2015), three different perspectives or ideal-types can emerge
when discussing quality in higher education (HE): quality as culture, as compliance and as
consistency. In what follows of this section each of these perspectives will be further
elaborated bearing in mind that frequently IQA practices comprised by each one of them,
although developed under different rationales, may overlap in terms of implementation
features and effects produced.

Quality as culture
This first perspective has to do with quality as including both a cultural/psychological and a
structural/managerial element. The former element corresponds to the higher education
institutions’ values, expectations and commitment regarding quality as well as to the academic
community’s engagement with it (EUA, 2006). The commitment of teaching and non-teaching
staff on the processes designed to improve quality is decisive for the quality of educational
provision (Cardoso et al., 2015). The structural/managerial element relates to a more formal
dimension of quality including the institutional structures, processes and procedures aimed at
enhancing quality. This structural element includes the internal structure supporting the
institution’s staff in their daily work, including that related to the promotion of quality
(Cardoso et al. 2015; Yorke, 2000).
Under this perspective quality is a responsibility of all the staff meaning that all must
contribute to ensure that the right things are done right. Furthermore, quality is based on the
ability of an institution to fulfil, first of all, the requirements of the academic community
(Kleijnen et al., 2013). This is consistent with the idea of quality as perfection and maybe also
as transformation and continuous improvement (Cardoso et al. 2015; Harvey and Green,
1993). In fact, the focus is much more on whether ‘transformation can be achieved and a
qualitative change can be effected’ than on how ‘formulated objectives can be attained as
effectively as possible’ (Kleijnen et al., 2013: 153).
Some of the factors that may influence the development of internal quality assurance practices
within this perspective can be thought to be the promotion of the institutions’ quality
continuous improvement or of a favourable environment for innovation. According to Rosa,
Sarrico and Amaral (2012), improvement and innovation are two possible purposes for
assuring quality in higher education. The improvement purpose reflects the idea that quality
assurance can be a powerful driver for improvement and that often it is enough to assess that
improvement will follow. The focus of quality assurance practices include the improvement of
teaching and learning processes, the development of teaching staff skills and competences or
a better link between teaching and learning (Rosa et al., 2012). The innovation purpose
assumes that quality assurance looks at what exists and tries to create something new, namely
at the level of academic practices, teaching and learning methodologies and the link between
teaching, research and institutional management (Rosa et al., 2012).
Based on Veiga, Rosa, Cardoso and Amaral (2014), the development of internal processes to
improve institutions’ self-awareness and reflection as well as the definition of formal and
systematic QA processes, revealing a strong commitment to quality, can be seen as the
implementation features which can be thought of as being mostly aligned with this
perspective.
Moreover, one can think that the IQA practices associated to this perspective may lead to
effects such as the development of a good environment among different professional groups,
a greater focus on innovation and experimentation on teaching and learning as well as a
greater pedagogical training of the teaching staff and an increased awareness of teaching
quality issues. Associated specifically to the structural element encompassed in this
perspective, it is expected that bureaucracy will also emerge as an effect, namely resulting in
greater demand and time availability for non-academic tasks, greater formalisation of
procedures and an increased monitoring of academic performance.

Quality as compliance
Quality as compliance has to do with quality as a result of institutions’ compliance with the
quality policies and guidelines externally defined, which sometimes lead to unintended
consequences (Cardoso et al., 2015). Under this perspective the external aspects conditioning
institutions and the way these cope with them determines what is understood by quality and
how it is assured. Indeed, when designing and implementing their IQA practices, institutions
‘are driven by external requirements as those defined by the state and/or by its agencies’
(Westerheijden, 2007: 14). The aim is to promote institutions’ transparency, accountability,
efficiency and responsiveness to changing external demands (Green, 1994; Westerheijden,
2007). Accountability emerges then as a central aspect of this perspective linked to the
existence of internal procedures based on externally defined goals and criteria (Kleijnen et al.,
2013; Thune, 1996; Rosa et al., 2012).
In this context, quality and its assurance are closely linked to the ideas of fitness for purpose,
and, to some extent, value for money. Institutions are stressed to be accountable both to their
funders (tax payers) and the users of higher education services and ‘products’ (Green, 1994;
Harvey and Green, 1993; Westerheijden, 2007; Yorke, 2000).
Internal quality assurance practices framed by this perspective can be seen as being designed
and developed in response to legal requirements from the national QA systems, the need to
be accountable towards society or in order to promote the institutions’ social recognition.
These factors go in line with the control and communication purposes for assuring quality in
higher education proposed by Rosa et al. (2012). The control purpose reflects the idea of
quality assurance providing internal and/or external feedback based on which appropriate
actions can be taken. At supra-institutional level, the communication purpose relates to the
promotion of transparency and trust. Institutions should be able to communicate to all of their
stakeholders that they offer quality and value for money (Rosa et al., 2012).
Quality assurance processes centrally managed and guided by external requirements is one of
the features that can be seen as characterising IQA practices’ implementation within this
perspective (Veiga et al., 2014). As for the effects eventually produced by IQA practices they
include improvements in accountability to society at large and in the quality of public
information about the institutions’ activities. Effects may also include, as previously
mentioned, unintended consequences, such as an increase in bureaucracy, reflected in greater
demand and time availability for non-academic tasks and greater formalisation of procedures.

Quality as consistency
This third perspective encompasses a relatively recent notion of quality assurance, much more
focused on the processes and their results than in the structures and procedures supporting
them. It therefore links to the academic standards defined for the assessment and validation
of higher education processes and end results (concerning teaching and learning, research or
relations with society) and, specifically, to learning outcomes (Jessop, McNab, and Gubby,
2012; Kohoutek, 2014; Stensaker, 2014). It embeds a more evidence-based approach to quality
sustained on the idea that the academic standards should be the new standards for IQA
practices. This is in line with an idea of quality as consistency (Green 1994; Harvey and Green
1993).
This perspective emerges because according to some authors (Aamodt, Frølich and Stensaker,
2016; Stensaker, 2014) quality assurance may have not produced the expected effects on
institutions’ quality. Assessing processes and end results against academic standards defined
for them, such as the learning outcomes, may then be an alternative to the traditional QA
mechanisms (Hussey & Smith 2008). Even if learning outcomes definition is frequently an
internal response to external policy requirements, this definition potentially ‘addresses
academic issues more than the organizational dimensions that tend to be in focus in traditional
quality assurance’ (Aamodt et al., 2016: 2).
Since this perspective on quality is focused on processes, it is expected that it will encompass
IQA practices leading to improvements in the quality of teaching and learning, research,
outreach activities and internationalisation and a greater focus on innovation and
experimentation in teaching and learning. Furthermore, and as with other perspectives,
another possible - unintended - effect may be an increase in bureaucracy, reflected in an
increased monitoring of academic performance. Even if the focus on processes and end results
may collect a more positive attitude from teaching staff towards IQA practices it also ‘may
have effects that can cause bureaucracy and have unintended consequences for higher
education’ (Aamodt et al., 2016).

Method
The Portuguese higher education system is binary, comprising a university and a polytechnic
subsystem. Institutions may belong either to the public or to the private higher education
sector. 14 universities and 15 polytechnics constitute the public sector while in the private
sector there are 8 universities, 20 other institutions offering university degrees and 49
polytechnic institutions (DGES, 2015). In 2014/15, the teaching staff population in the
Portuguese higher education system was 32,346 academics (DGEEC, 2014). 75.7% were in the
public sector and 24.3% in the private. The university subsystem integrated 62.5% of the
academic population while the polytechnic assembled 37.5%. Furthermore, 56% of academics
were male. Academics average age was of 46 years (DGEEC, 2014).
The non-teaching staff is a group of professionals within the academia that is frequently left
out of official statistical analysis. There is no information about the numbers and profile of
non-teaching staff in private institutions and even in public institutions the information is not
always up to date. However, non-teaching staff plays an increasingly important role and as
such they can provide relevant insights to the changes occurring in institutions and in their
professional activities.
A study was conducted aiming to understand how the Portuguese higher education
institutions are responding to the challenges facing higher education globally and how recent
initiatives have influenced this response. Four core areas and their interplay were the focus of
the analysis: institutional governance; performance management systems; internal quality
assurance; and professionals’ roles.
The entire population of teaching staff and of non-teaching staff working at the level of
‘qualified technician’ (or similar) from both the public and private sectors was targeted.
Regarding the teaching staff this was thus a census on the entire population. As referred, the
non-teaching staff included only the ‘qualified technicians’ which is a professional category
within Portuguese academia representing professionals working in different services but which
are usually more qualified and in higher responsibility positions. The assumption for selecting
these professionals was that this group would be more affected by the changes occurring in
institutions.
In order to collect teaching and non-teaching staff perceptions on the transformations
occurring in Portuguese institutions, a questionnaire was developed covering the previous four
core areas. The questionnaire was made available online and answers were collected between
October 2014 and October 2015. 1,661 members of the teaching staff (corresponding to about
5% of the Portuguese academic population) and 339 members of the non-teaching staff
answered the survey.
For the purpose of this paper, only the quality core area was addressed namely the questions
on: (i) the importance of different factors for the creation and development of internal quality
assurance practices at institutions; (ii) the features underlying the implementation of these
practices; (iii) the effects of this implementation; (iv) and the overall level of satisfaction with
the existent internal quality assurance practices.
Descriptive statistics and non-parametric tests were run to analyse the data resulting from the
answers to these questions. Descriptive statistics allowed to synthetize and represent
information available in the data set. Statistical measures (mean, median, mode and standard
deviation) were computed for each variable under analysis. As one of the goals was to see if
different groups of staff had different perceptions Mann-Withney tests (for a 0.05 significance
level) were used to identify statistical significant differences between the answers’
distribution. An exploratory factor analysis using Promax rotation (oblique) was run to
understand if the different effects of IQA practices’ implementation could be grouped, helping
to uncover the quality perspective at stake.
Table 1 characterises the study sample. Teaching staff, mostly male (53.4%), came evenly from
universities and polytechnics (50.2% and 49.8% respectively), mainly belonged to the public HE
sector (77.5%), and were predominantly aged between 40 and 59 years old. Non-teaching
staff, mostly female (68.9%), were predominantly aged under 49 years old, working in the
public sector (85.5%) and in the university subsystem (59.3%).

Table 1 – Sample’s characterisation

Teaching staff Non-teaching staff
N % of valid N % of valid
answers answers
Gender Male 850 53.4 118 31.1
Female 742 46.6 261 68.9
Total 1592 100 379 100
Age Less than 40 289 17.7 129 33.4
years
40 to 49 years 623 38.1 156 40.4
50 to 59 years 548 33.5 86 22,3
60 years or 175 10.7 15 3.9
older
Total 1635 100 386 100
HE sector Public 1246 77.5 330 85.5
Private 361 22.5 56 14.5
Total 1607 100 386 100
HE subsystem University 807 50.2 229 59.3
Polytechnic 800 49.8 157 40.7
Total 1607 100 386 100
Perspectives on quality: HEIs’ professionals perceptions
The majority of the academic staff (63.8%) recognised the existence of an internal quality
assurance system at their institutions, while 18.4% identified the existence of practices for
internal quality assurance, but not of a formal system. Although to a less extent, also the
majority of the non-teaching staff acknowledged the existence of a system (53.4%), while
21.1% recognised at least the existence of some practices.
In the following subsections the perceptions of these professionals are presented in relation to
the factors influencing the development of internal quality assurance practices at institutions,
the main features underlying their implementation, the effects resulting from it, and the
overall satisfaction with internal quality assurance practices. Differences between teaching and
non-teaching staff are pinpointed when significant (from a statistical point of view).

Factors influencing the development of internal quality assurance practices

Table 2 presents professionals’ perceptions on the importance of different factors for the
creation and development of internal quality assurance practices at their institutions. Overall,
professionals tend to consider all the factors as important. However, this importance presents
some nuances. For professionals the least important factor is the accountability towards
society (median of 3), which aligns with a perspective of quality as compliance. As previously
discussed, this perspective comprises a communication purpose for the development of
quality assurance. Nevertheless this purpose is also reflected by the idea of developing IQA
practices as a way to promote the institution’s social recognition, with which professionals
tend to agree (median of 4). Therefore only a partial support of this perspective exists
regarding this purpose. Interestingly though, the same perspective is valued when the purpose
at stake is related to control: professionals agree that one of the most important factors for
the development of internal quality assurance practices were indeed the legal requirements
and demands from the national system of evaluation and accreditation of higher education
(A3ES) (median of 4). Moreover, as the results also emphasise the importance of the factors
related to the perspective of quality as culture, it is possible to argue that the two perspectives
(quality as culture and as compliance) coexist as leading to the development of IQA practices.

Table 2. Professionals’ perceptions on the importance of different factors for the creation and
development of internal quality assurance practices
N Mean Median Mode SD
Legal requirements and demands from the national system 1491 4.1 4 4 0.82
of evaluation and accreditation of higher education (A3ES).
Promoting continuous improvement of the institution’s 1502 4.1 4 4 0.83
quality.
Accountability towards society. 1413 3.3 3 4 1.10
Promoting the institution’s social recognition. 1479 3.9 4 4 0.93
Promoting a favorable environment for innovation. 1465 3.8 4 4 1.04

Differences were found between the answers’ distribution of teaching and non-teaching staff
only for one of the factors recognised as being important for the implementation of quality
assurance practices. Non-teaching staff tend to agree more with the factor accountability
towards society (p=0.001), which aligns with the perspective of quality as compliance. One of
the possible explanations for this result may be the fact that non-teaching staff usually have
support roles within services and structures that receive and provide information on quality
assurance, namely in what regards externally defined requests. Therefore, they may have
internalised more the idea of quality practices’ development as an answer to accountability
requirements.
Main features underlying the implementation of internal quality assurance
practices
Overall, professionals’ perceptions on the main features underlying the implementation of
internal quality assurance practices seem to suggest that these practices display again, to some
extent, both the perspectives of quality as culture and quality as compliance (Table 3).
The quality assurance processes being centrally managed and guided by external requirements
(mean of 3.4; median of 4) was the feature collecting a higher level of agreement by
professionals in regard to the implementation of the internal quality assurance practices. This
aligns with the perspective of quality as compliance.
On the other hand, professionals show less agreement with internal quality assurance
practices implementation being about vague and occasional processes, manifesting a
superficial commitment to quality (mean of 2.7; median of 3). Therefore, the perspective of
quality as culture, based on the academic community’s commitment to quality as well as to the
processes, structures and procedures, seems to be favoured.

Table 3. Professionals’ perceptions on the implementation features of internal quality

assurance practices
N Mean Median Mode SD
Internal processes to improve self-awareness and reflection 1429 3.2 3 3 1.04
on the institution.
Quality assurance processes which are centrally managed 1369 3.4 4 4 0.94
and guided by external requirements.
Vague and occasional processes, manifesting a superficial 1322 2. 7 3 2 1.13
commitment to quality.

At this level, differences between teaching and non-teaching staff were only found in regard to
the implementation of IQA practices as promoting internal processes to improve self-
awareness and reflection on the institution (p=0.046). Non-teaching staff tends to agree more
with this feature, maybe because they share a more positive view about the how and why IQA
practices should be implemented.

Effects of internal quality assurance practices’ implementation

As evidenced by the results in Table 4, IQA practices seem to produce a number of effects,
although the level of agreement with them tended to be moderate. The most significant
effects were (medians of 4): increased monitoring of academic performance; greater demand
and time availability for non-academic tasks; greater formalization of procedures; increased
awareness of teaching quality issues; and improvements in the quality of public information on
the activities of the institution. The nature of these effects seems to reveal bureaucratic and
over formalised IQA practices, in line with a perspective of quality as compliance while a
culture component also emerges in relation to the structural/managerial elements of the IQA
practices. However, especially the identification of the increased monitoring of academic
performance as a significant effect suggests also some alignment with the perspective of
quality as consistency.
Table 4. Professionals’ perceptions on the effects of internal quality assurance practices
implementation
N Mean Median Mode SD
Increased tension among the teaching staff. 1381 3.1 3 3 1.06
Increased tension between managers and the teaching staff. 1372 3.3 3 4 1.06

Increased tension between the teaching staff and the non- 1401 2.7 3 2 1.02
teaching staff.
Increased monitoring of academic performance. 1452 3.7 4 4 0.92
Greater demand and time availability for non-academic 1447 3.9 4 4 0.96
tasks.
Greater formalization of procedures. 1469 4.2 4 4 0,76
Increased awareness of teaching quality issues. 1459 3.6 4 4 1.00
Greater focus on innovation and experimentation in 1435 3.2 3 4 1.07
teaching and learning.
Greater pedagogical training of the teaching staff. 1422 2.9 3 3 1.09
Improvements in the quality of teaching/learning. 1436 3.3 3 4 1.06
Improvements in the quality of research. 1400 3.1 3 4 1.11
Improvements in the quality of outreach activities. 1315 3.0 3 3 1.04
Improvements in the quality of the internationalization. 1370 3.3 3 4 1.06
Improvements in the quality of public information on the 1422 3.4 4 4 1.02
activities of the institution.
Improvements in accountability to society at large. 1314 3.1 3 3 0.99

In order to better understand the quality perspectives emerging in relation to the effects of
IQA practices an exploratory factor analysis was run. This revealed three factors with
eigenvalues>1 (corresponding to 69.6% of total variance explained). Factor analysis resulted
then in an effects scale consisting of 15 items and 3 subscales, each one corresponding to one
of the three extracted factors. All loadings > 0.3 were considered to be significant (Hair,
Anderson, Tatham, and Black, 1998). Whenever an item presented loadings > 0.3 in more than
one factor, the decision was to allocate it to the factor were the loading was higher. Table 5
presents the statistics computed for each of the three effects subscales. These subscales were
formed by combining all items loading higher on the factor and using their mean as a
replacement variable – the subscale (Hair et al., 1998). Internal consistency (Cronbach α) of the
scale was 0.86, while for the three subscales it ranged from 0.68 to 0.94 (Table 5).
Correlations (Spearman’s coefficient) between the subscales were all significant at the 0.01
level (2-tailed) which allows to conclude that the three subscales were highly correlated (Table
1 in Annex). Furthermore, t-tests for paired samples revealed the existence of statistically
significant differences (for a two-tailed significance level of 0.05) among the mean scores of
the subscales. This means that there were in fact differences to be considered among
professionals’ perceptions about these three different types of effects (Table 2 in Annex).

Table 5. Professionals’ perceptions on the effects of IQA practices: statistics for the three
effects subscales
Effects subscale N Items Mean Median SD Cronbach’s α
Improvement 1160 9 3.2 3.2 0.86 0.94
Increased tension within academia 1331 3 3.0 3.0 0.90 0.83
Bureaucracy 1411 3 3.9 4.0 0.69 0.68

The Improvement subscale comprises the effects related to the improvement of the different
components of higher education mission (teaching and learning, research, outreach activities,
internationalisation); development of teaching staff; and enhancement of institutions’
awareness on quality issues, accountability and information to society.
The second subscale – Increased tension within academia - encompasses the effects linked to
the increased tension promoted by the implementation of IQA practices among the teaching
staff, between this staff and academic managers and between teaching and non-teaching staff.
Finally, the Bureaucracy subscale includes the effects related to IQA practices as inducing the
increased monitoring of academic performance, formalization of procedures and time
availability for non-academic tasks.
The results presented in Table 5 corroborate the previous idea that professionals tend to see
IQA practices as producing effects linked to increasing bureaucracy which aligns with the three
perspectives of quality. Interestingly the Bureaucracy subscale correlates positively with the
two other subscales, evidencing that independently of other types of effects, IQA practices
seem to always induce bureaucratic processes (Table 1 in Annex).
Also worth noting is the fact that the correlation coefficient between the improvement and the
increased tension within academia subscales is negative. This means that the less IQA practices
promote an increase in tension between different professionals, the more they lead to quality
improvement.
Finally, when comparing the answers’ distribution of teaching and non-teaching staff,
statistically significant differences emerge for all but three of the effects under analysis:
increased tension among the teaching staff, increased tension between managers and the
teaching staff and increased monitoring of academic performance. For all the remaining
effects the non-teaching staff tends to show higher agreement except for IQA practices
inducing a greater demand and time availability for non-academic tasks and a greater
formalization of procedures. This last result is not completely surprising since on the one hand
non-teaching staff is already used to perform more bureaucratic and procedural tasks, and on
the other hand most of the work involved with IQA practices implementation lies with
teaching-staff. As for the other effects assembling a higher agreement from the non-teaching
staff one possible explanation may again be the previously mentioned more positive view of
these professionals regarding IQA practices.

Satisfaction with internal quality assurance practices

Overall, professionals tended to be only relatively satisfied with IQA practices (mean of 3.2;
median of 3). Furthermore, no differences could be detected at this level between teaching
and non-teaching staff. This only relative satisfaction may be related to the factors influencing
the development of IQA practices, the main features underlying their implementation and to
the effects produced by these practices.
The results of an analysis of correlation (Spearman correlation coefficient) between the
previous variables and professionals’ satisfaction reveals that these are significantly correlated
(Tables 3, 4 and 5 in Annex).
All the correlations between the importance of different factors for the development of IQA
practices and professionals’ satisfaction were statically significant and positive. This means
that the higher the importance given by professionals to each of the factors, the higher they
are satisfied with IQA practices. As professionals tended to recognise as important most of the
factors, a higher degree of satisfaction was eventually expected. It is then possible to assume
that other variables may condition this satisfaction.
When analysing the correlations between IQA practices implementation features and
professionals’ satisfaction with these practices, it is possible to note that all of the correlations
were statistically significant. Moreover, all but one were positive, the exception being that
between the feature vague and occasional processes, manifesting a superficial commitment to
quality and professionals’ overall satisfaction. This suggests that satisfaction depends, to a
certain extent, on the institutional will to formally implement IQA practices independently of
the features assumed by this implementation.
Finally, the analysis of the correlations between each of the IQA practices effects and
professionals’ satisfaction shows that for most of the effects the higher the level of agreement
with them, the higher the satisfaction with IQA practices. There were nevertheless three
effects for which the correlation coefficients were negative meaning that the level of
satisfaction decreases as the professionals recognise these effects as occurring. These effects
include an increased tension between the teaching staff, the teaching staff and managers and
between the teaching and non-teaching staff. Interestingly, no statistically significant
correlations existed (for a 0.05 significance level) for the greater demand and time availability
for non-academic tasks and formalisation of procedures effects. This suggests that satisfaction
with IQA practices seems not to be related with bureaucracy (at least linearly). In sum, since
overall professionals only moderately agreed with most of the effects, it is not surprising that
the level of satisfaction with IQA practices was also moderate.

Conclusions
This paper analysed teaching and non-teaching staff perceptions on the development,
implementation and effects of IQA practices at Portuguese higher education institutions.
Resorting to previous work on different perspectives on quality - as culture, as compliance and
as consistency (Cardoso et al. 2015), the goal was to understand what is(are) the perspective(s)
on quality prevalent among these professionals.
In relation to all aspects under analysis in regard to IQA practices professionals emphasise both
the perspectives of quality as culture and quality as compliance. This suggests that the two
perspectives coexist as leading to the development, implementation and effects of such
practices. The perspective of quality as consistency was more difficult to uncover. This was
mainly due to the fact that the empirical component of the study (questionnaire) did not
specifically addressed factors and implementation features fitting this perspective.
Nevertheless this perspective was comprised by the IQA practices’ effects related to the
improvement of higher education components, even though, they did not collect a relevant
agreement from professionals. To some extent, this corroborates the idea that, at least in the
Portuguese context, this perspective is still relatively recent and, thus, not sufficiently evident.
Furthermore the study suggests that professionals were only moderately satisfied with IQA
practices. This may possibly be justified by the fact that they do not see relevant effects
resulting from these practices.
Even with the need to be careful in understanding the perspectives on quality at stake in this
study, because of its empirical limitations, the results achieved provide nevertheless an
interesting insight about the way IQA practices are being perceived by teaching and non-
teaching staff.
Moreover, this is a first attempt to include non-teaching staff in research on quality and its
assurance in the Portuguese context, even if the number of answers from this staff was
relatively small and therefore compromised to a certain extent the conclusions about the
differences between the two groups of professionals. This justifies the need for further and
maybe more qualitative research addressing specifically non-teaching staff which, as referred,
are quite absent especially in regard to their relation to quality assurance. Indeed this absence
raised problems when trying to explain why non-teaching staff perceptions vary from those of
the teaching staff.
Despite its limitations, the study contributes to advance in knowledge in this field. Thus it may
be helpful to institutions, namely by instigating them to critically reflect about how their
quality assurance practices can be more effective and aligned with academia’s expectations.
Keywords: quality; internal quality assurance practices; higher education institutions’
professionals

References
Aamodt, P., Frølich, N. and Stensaker, B. (2016). Learning outcomes – A useful tool in quality
assurance? Views from academic staff, Studies in Higher Education,
http://dx.doi.org/10.1080/03075079.2016.1185776.
Cardoso, S., Rosa, M.J. and Stensaker, B. (2015). Why quality in higher education institutions is
not achieved? The view of academics, Assessment and Evaluation in Higher Education, DOI:
10.1080/02602938.2015.1052775.
Cruickshank, M. (2003). Total quality management in the higher education sector: A literature
review from an international and Australian perspective. TQM & Business Excellence, 14(10):
1159–67.
DGEEC – Direção Geral de Estatísticas da Educação e Ciência. (2014). Principais resultados do
REBIDES 14 – Docentes 2014/15 [REBIDES main results 14 – Academics 2014/15]. Accessed
June 2016 at:
http://www.dgeec.mec.pt/np4/EstatDocentes/%7B$clientServletPath%7D/?newsId=138&fileN
ame=DGEEC_DSEE_DEES_2015_Destaque_Docentes_1.pdf.
DGES - Direção Geral do Ensino Superior. (2015). Rede De Ensino Superior: Estabelecimentos
[Higher Education Network: Institutions]. Accessed 2015 at:
http://www.dges.mctes.pt/DGES/pt/Estudantes/Rede/Ensino+Superior/Estabelecimentos/.
EUA - European University Association. (2006). Quality culture in European universities: A
bottom-up approach. Report of three rounds of the quality culture project. Brussels: EUA.
Green, D. (1994). What is quality in higher education? Buckingham: Open University Press.
Hair, J.F. Jr., Anderson, R.E., Tatham, R.L. and Black, W.C. (1998), Multivariate data analysis, 5th
Ed. New Jersey: Prentice-Hall.
Harvey, L., and D. Green. (1993). Defining quality, Assessment & Evaluation in Higher
Education, 18 (1): 9–34.
Harvey, L., and B. Stensaker. (2008). Quality culture: Understandings, boundaries and linkages,
European Journal of Education, 43 (4): 427–442.
Jessop, T., McNab, N., and Gubby, L. (2012). Mind the gap: An analysis of how quality
assurance processes influence programme assessment patterns, Active Learning in
Higher Education, 13(2): 143–154.
Kleijnen, J., Dolmans, D., Willems, J. and Van Hout, H. (2013). Teachers’ conceptions of quality
and organisational values in higher education: compliance or enhancement? Assessment &
Evaluation in Higher Education, 38(2): 152–166.
http://dx.doi.org/10.1080/02602938.2011.611590.
Kohoutek, J. (2013). European standards for quality assurance and institutional practices of
student assessment in the UK, the Netherlands and the Czech Republic, Assessment &
Evaluation in Higher Education, 39(3): 310-325. DOI: 10.1080/02602938.2013.830694.
Kohoutek, J. (2014). European standards for quality assurance and institutional practices of
student assessment in the UK, the Netherlands and the Czech Republic, Assessment &
Evaluation in Higher Education, 39(3): 310–325. DOI:10.1080/02602938.2013.830694.
Newton, J. (2000). Feeding the beast or improving quality? Academics’ perceptions of quality
assurance and quality monitoring, Quality in Higher Education, 6(2): 153–63.
Rosa, M.J. Sarrico, C.S., and Amaral, A. (2012). Academics’ perceptions on the purposes of
quality assessment, Quality in Higher Education, 18(3): 349-366. DOI:
10.1080/13538322.2012.733550.
Sarrico, C.S., Rosa, M.J., Teixeira, P.N., and Cardoso, M.F. (2010). Assessing quality and
evaluating performance in higher education: Worlds apart or complementary views? Minerva,
48: 35–54. DOI 10.1007/s11024-010-9142-2.
Stensaker, B. (2014). European trends in quality assurance: New agendas beyond the search
for convergence? In M.J. Rosa and A. Amaral (Eds.) Quality Assurance in Higher Education
(135–148). Basingstoke: Palgrave MacMillan.
Stensaker, B. (2008). Outcomes of quality assurance: A discussion on knowledge, methodology
and validity, Quality in Higher Education, 14: 3–13.
Thune, C., (1996). The alliance of accountability and improvement: the Danish experience,
Quality in Higher Education, 2(1): 21–32.
Veiga, A. Rosa, M.J., Cardoso, S. and Amaral, A. (2014). Ascribing meaning to quality cultures in
the Portuguese context, Quality Assurance in Education, 22(3): 255 – 272.
http://dx.doi.org/10.1108/QAE-05-2013-0020.
Westerheijden, D., V. Hulpiau, and K. Waeytens. (2007). From design and implementation to
impact of quality assurance: An overview of some studies into what impacts improvement,
Tertiary Education and Management, 13(4): 295–312.
Westerheijden, D., B. Stensaker, M. J. Rosa, and A. Corbett. (2014). Next generations, catwalks,
random walks and arms races: Conceptualising the development of quality assurance schemes,
European Journal of Education, 49(3): 421–434.
Yorke, M. (2000). Developing a quality culture in higher education, Tertiary Education and
Management, 6(1): 19–36.

Acknowledgments
This research has been supported by the FCT through the project PEst-OE/CED/UI0757/2013
(funded by the Programme COMPETE) and by grant EXCL/IVC-PEC/0789/2012 - GLONATINS -
Global Challenges, National Initiatives, and Institutional Responses - Mapping the
Transformation of Portuguese Higher Education Institutions at the Dawn of the Twentieth-First
Century

Biographical Details:
Maria J. Rosa has a PhD in Industrial Management and is an Assistant Professor at the
Department of Economics, Management and Industrial Engineering at the University of Aveiro.
She is also a researcher at CIPES (Centre for Research in Higher Education Policies). Her main
research topics are quality management and quality assessment in higher education
institutions. She is a member of CHER and of the executive committee of EAIR.

Sónia Cardoso holds a PhD in Social Sciences and is a researcher both at the Office of Research
and Analysis of A3ES and at CIPES. Her main research interests are related to quality assurance
in higher education and higher education institutions’ actors.

Pedro M. Videira is a researcher at CIPES and a PhD candidate at ISCTE –University Institute of
Lisbon. His main research interests are on the Internationalization of Higher Education and
Research and the Sociology of Science.

Alberto Amaral is a professor at the University of Porto and a researcher at CIPES. He was the
rector of Porto University from 1985 to 1998 and former chair of CHER. He is life-member of
IAUP and the chair of the administration council of the Portuguese Assessment and
Accreditation Agency for Higher Education.
Annex

Table 1 - Spearman’s correlation coefficients between the three subscales emerging from the
factorial analysis (correlations are significant for a 0.01 significance level)
Improvement Increased tension Bureaucracy
within academia
Improvement 1 -0.133 0.271
Increased tension within academia 1 0.193
Bureaucracy 1

Table 2 – Paired-sample t-tests for statistically significant differences between the three
subscales emerging from the factorial analysis (t-value and p-value for each one of the three
pairs)
Improvement Increased tension within Bureaucracy
academia
Improvement - 4.293 (0.000) -27.458 (0.000)
Increased tension within academia - -33.615 (0.000)
Bureaucracy -

Table 3 – Spearman non-parametric correlations between professionals’ perceptions on the

importance of different factors for the creation and development of institutional quality
practices and the overall level of satisfaction with these practices at their institutions
Overall level of
satisfaction with IQAS.
Legal requirements and demands from A3ES. ρ=0.178
(0.000)
Promoting continuous improvement of the institution’s quality. ρ=0.210
(0.000)
Accountability towards society. ρ=0.093
(0.000)
Promoting the institution’s social recognition. ρ=0.189
(0.000)
Promoting a favorable environment for innovation. ρ=0.133
(0.000)

Table 4 - Spearman non-parametric correlations between professionals’ perceptions on the

implementation features of internal quality assurance practices and the overall level of
satisfaction with these practices at their institutions
Overall level of
satisfaction with IQAS
Internal processes to improve self-awareness and reflection on the ρ=0.549
institution. (0.000)
Quality assurance processes which are centrally managed and guided by ρ=0.206
external requirements. (0.000)
Vague and occasional processes, manifesting a superficial commitment to ρ=-0.331
quality. (0.000)
 Table 5 - Spearman non-parametric correlations between professionals’ perceptions on the
effects of the implementation of internal quality assurance practices and the overall level of
satisfaction with these practices at their institutions
Overall level of satisfaction
ρ=-0.223
Increased tension among the teaching staff.
(0.000)
ρ=-0.256
Increased tension between managers and the teaching staff.
(0.000)

ρ=-0.131
Increased tension between the teaching staff and the non-teaching staff.
(0.000)

ρ=0.313
Increased monitoring of academic performance.
(0.000)
ρ=0.0010
Greater demand and time availability for non-academic tasks.
(0.692)
ρ=-0.000
Greater formalization of procedures.
(0.998)
ρ=0.459
Increased awareness of teaching quality issues.
(0.000)

Greater focus on innovation and experimentation in teaching and ρ=0.509

learning. (0.000)

ρ=0.447
Greater pedagogical training of the teaching staff.
(0.000)
ρ=0.515
Improvements in the quality of teaching / learning.
(0.000)
ρ=0.476
Improvements in the quality of research.
(0.000)
ρ=0.503
Improvements in the quality of outreach activities.
(0.000)

ρ=0.408
Improvements in the quality of the internationalization.
(0.000)

Improvements in the quality of public Information on the activities of the ρ=0.466

institution. (0.000)

ρ=0.463
Improvements in accountability to society at large.
(0.000)

View publication stats