The current issue and full text archive of this journal is available on Emerald Insight at:

https://www.emerald.com/insight/1741-0401.htm

Performance appraisal, justice and Appraisal,

justice and
organizational effectiveness: effectiveness

a comparison between
two universities 87
Kurmet Kivip~old and Kulno Tu
€rk Received 5 June 2019
Revised 21 January 2020
School of Economics and Business Administration, University of Tartu, Accepted 25 January 2020
Tartu, Estonia, and
Lea Kivip~old
Department of Quality, Social Insurance Board of the Republic of Estonia,
Rakvere, Estonia

Abstract
Purpose – The purpose of this paper is to identify how the design of a performance appraisal system (PAS)
affects the perceived justice of academic employees (AE) about their performance appraisal (PA) and how this
is associated with organizational effectiveness in terms of organizational leadership (OL).
Design/methodology/approach – The study subjects are two economic faculties of two Estonian public
universities. The data for the study were collected using the PA Survey with a total of 82 AEs, OL
Capability Questionnaire with a total of 72 AEs and the organizations’ documents to analyze PAS.
Assessment and analysis of the data included: the measurement of PAS design; the measurement of
perceived justice from PA; the measurement of organizational leadership capability; analysis of the results
gained from studying perceived justice from different PAS designs and organizational effectiveness in
terms of OL.
Findings – Ultimately, the study reveals that PAS design affects academic employees’ perception of
distributive justice and organizational external effectiveness in terms of OL but does not affect academic
employees’ perception of procedural justice and organizational internal effectiveness in terms of OL.
Research limitations/implications – This study suggests that organizational effectiveness depends on
perceived justice of employees from the design of PAS. However, the results of this study are valid in the
arrangements of academic jobs in universities and in similar or close context of Estonian culture.
Originality/value – This paper demonstrates the role of PAS design in conditions of intellectual job
arrangement in universities with its influence on organizational effectiveness in the context of OL.
Keywords Academic employees, Organizational effectiveness, Performance appraisal, Procedural justice,
Distributive justice, Organizational leadership
Paper type Case study

1. Introduction
There has been a sharp increase in the proportion of knowledge work in the world and
knowledge workers have become the most valuable asset for an organization in the 21st
century (Sunalai and Beyerlein, 2015). Knowledge work is characterized by high complexity
and team-based cooperation. This also brings changes in the work arrangement where
flexible and virtual work emerges. Academic occupations in higher education represent such
kind of jobs. These type of works demands great responsibility and self-organization from
employees. In these conditions employees need management/performance management
systems which align them to organizational goals and provide feedback in their everyday International Journal of
activities. Performance appraisal systems (PAS) are one of the important management Productivity and Performance
Management
Vol. 70 No. 1, 2021
pp. 87-108
Funding Information: The Estonian Ministry of Education, Research and Innovation Policy Monitoring © Emerald Publishing Limited
1741-0401
Programm DOI 10.1108/IJPPM-05-2019-0229
IJPPM systems in universities (Chatelain-Ponroy et al., 2018; Pekkola et al., 2018), allowing the
70,1 analysis and assessment of the performance of academic employees (AE) – their activities,
achievements and effectiveness of group processes (Agasisti, 2017; Pekkola et al., 2018).
Therefore, universities constantly develop their PAS for their faculties and AEs by taking
into account the motivational messages and consequences for them (Hardre and
Kollmann, 2012).
Different types of PAS, such as performance-based (Kivist€o et al., 2017), competence-
88 based (Mikkonen et al., 2018), and position-based (Yin, 2016), exist in HRM practice.
Individual, quantitative and short-term measures characterise PAS used for the AEs in
higher education (Melo et al., 2010; Alach, 2017). This has been criticised by many authors
(e.g. Bogt and Scapens, 2012; Chatelain-Ponroy et al., 2018; Pekkola et al., 2018) who
suggested to utilise more long-term, qualitative, process and team-based measures in the
PAS. In addition, organizational effectiveness which links with PAS and its design (T€ urk,
2016; Su and Baird, 2017) has been measured mainly by individuals’ behaviour, such as job
satisfaction, turnover intent (Hassan and Hashim, 2011), organizational commitment
(Meyer and Smith, 2000) and organizational citizenship behaviour (Teh et al., 2012).
Effectiveness measures, which are based on collective phenomenon such as organizational
learning (Marsick and Watkins, 2003) or organizational leadership (OL) (Kivip~old and
Vadi, 2010), are more suitable to use together with PAS concerning long-term, process and
team-based measures. Additionally, it is important how employees perceive justice from
their PA (Wu and Wu, 2015), which in turn influences the organizational effectiveness as
well (Cheng, 2014).
The design of PAS influences effectiveness in terms of OL (Longenecker and Fink, 2017)
through the perceived justice of individuals (Lambert and Hogan, 2010). Therefore, it is
important to investigate how organizational performance and effectiveness in terms of OL
are influenced by the design of PAS in the context of perceived justice. In most studies, either
the relationship between PAS and perceived justice or between perceived justice and
effectiveness are studied (Phin, 2015; Kim, 2016). However, we are aware of only a few studies,
even these have weak links that have examined PAS design influence on organizational
effectiveness influenced by perceived justice (Meyer and Smith, 2000). Our exploratory study
is unique because it explores all three factors together.
The purpose of this paper is to identify how the design of PAS affects the perceived
justice of AE about their PA and how this is associated with organizational effectiveness
in terms of OL. Therefore, our main research questions are: 1) How is the design of PAS
associated with AE’s perception about procedural and distributive justice? and 2) How
does perceived justice from the design of PAS affect organizational effectiveness in terms
of OL?
In order to explore linkages between the designs of PAS with organizational effectiveness
the logic of the comparative-case study was used. The rationale of this study is based on the
design of PAS associated with AE’s perception about the procedural and distributive justice
in universities which in turn are associated with organizational effectiveness. This study was
conducted in the economic faculties of two Estonian universities where PAS were assessed by
document analysis, procedural and distributive justice by survey, and OL by the
measurement tool of Organizational Leadership Capability.

2. Theoretical background and research questions

2.1 Performance appraisal system design
PA is defined as a process of measuring and developing the individual and/or the team
performance; a PA consists of goal setting, monitoring and feedback of employees (De
Andres et al., 2010). It is an integral part of the managerial control and work-related
information system aimed at employee work activities and work results. Organizations Appraisal,
change their PAS and even remove numerical rating scales because the PAS failed to develop justice and
adequate criteria, the inconsistent effect of performance feedback, or the weak relationship
between research and practice. The arguments for PA are that the performance is always
effectiveness
evaluated in some manner and the alternatives to numerical ratings may be worse and
evaluations have positive merits for improving organizations (Adler et al., 2016). Therefore, it
is important to ensure that a PA measurement tool is reliable, free of bias, practical, and
acceptable to the user (Phin, 2015). 89
PAS design includes a clearly defined purpose, extensive input from employees and
managers on how the appraisal system should function, appraisal procedures and
performance-based rating forms that are easy to follow, and systematic information of
managers and employees on how the appraisal system operates. Kivist€o et al. (2017) find that
despite the extensive use of performance-based appraisal and management in European
higher education and the many research works, little is known about its impacts on teaching
and research; further empirical studies will be needed in order to determine the actual impact
of performance-based management on behaviour, and on the performance of individual
academics and institutions as a whole. Two factors frequently drive the review and re-design
of an organization’s existing appraisal process: a new OL regime, or tensions and frustration
with the PA process. (Longenecker and Fink, 2017).
The performance-based PAS design is the most employed design at the individual level in
the practice of performance management – mainly in production but also in higher education
organizations. The PAS’s design in the higher education is characterised by a clearly defined
goal, criteria and procedures and mainly short-term (one year) outcomes, which is based on
detailed quantitative indicators of teaching and research. The purpose of the PAS is to
increase employees’ effectiveness through the process of PA (Stanton and Nankervis, 2011)
by combining and integrating individual and group efforts with organizational goals
(Moreland, 2009). However, performance-based systems are shifting control from the personal
(individual) to managerial level (Pekkola et al., 2018), which is not suitable (sufficient) in an
academic work environment. The same aspect based on research of Finnish universities is
supported by Kivist€o et al. (2017, p. 14) “. . . that the motivation for high performance among
academics still relates primarily to acknowledgement from the academic community and
academic achievement – in other words, from something closer to intrinsic motivation – than
from measurement and financial incentives”. From this point of view, PAS for academic
(intellectual) work should be designed in way which generates self-evaluation, self-efficacy,
self-regulation, etc. (Melo et al., 2010; Bogt and Scapens, 2012; Browne and Rayner, 2015).
Position and competency-based design of PAS differ considerably from performance-
based design. Both have many similarities in universities where getting and keeping a
research and/or teaching position depends on competences (knowledge and skills) of AE. The
core competences of AE are positive attitude to research, knowledge about the research and
teaching subjects, pedagogical and research skills, international networking and leadership
etc. (McAllistair and Flynn, 2016; Mikkonen et al., 2018) from which arise the abilities to
execute the duties of the academic position. In addition to the individual competences PA of
AE, the assessment of their real job tasks for securing the organizational performance is also
important (Mikkonen et al., 2018). Therefore, PA in teaching and research mostly use
qualitative indicators, such as quality of teaching, learning, curriculum and research
(Varouchas et al., 2018). Hence, contrary to the performance-based design approach, position
and competence-based PAS use long-term indicators which correspond to the specific nature
of academic posts and with AE long-term contracts (usually 4–5 years). Additionally, based
on the position-based design approach Yin (2016) brings out the importance of taking into
account in the PA process efficiency the managerial and leadership activities between the
persons holding different positions.
IJPPM The main problems are attributable to the usage of individual-based PA, which have
70,1 helped to increase the quantitative aspect of work, but at the expense of quality. Some studies
have shown that PA inhibits the creativeness of AE (Bogt and Scapens, 2012), and decreases
the voice and freedom of AE (Melo et al., 2010). All this has led, for example, to the increased
number of publications and has led to lower quality of those publications. Mingers and
Willmott (2013) have emphasised that while AE publish the needed number of articles, in
reality it is just “shaping performance”. To reduce this negative tendency, it is important to
90 calculate the impact of the papers and indexing of journals (T€ urk, 2016) and to apply more
robust and trustworthy indicators for appraisal and selection of AE. For example, as an
indicator of academic effectiveness, Yang et al. (2019) use the H-index; Wang et al. (2019),
however, recommend using not only the H-index but also publication and citation numbers.
Many universities also use student assessments; however, the quality of these evaluations
is questionable, and they should be handled with care (Clayson, 2013). Despite significant
positive developments in regard to student surveys, we also point out that the first
impressions and likeability of the lecturer’s personality play an important part in the student
ratings. PAS can sometimes be judgemental and pose a threat to performance development
systems, inhibit creativity and innovation in the teaching and research process. Still,
judgemental indicators are one way of establishing a practical instrument for providing early
signals about changes in performance (Bogt and Scapens, 2012).
Mone et al. (2011) have emphasised the need to look beyond measuring results (outputs),
but also focus on inputs and process. According to Maillard and Savage (2012), the traditional
approach to PA has been criticised as not keeping pace with the move towards more
professional leadership. Modern approaches of PA enable open and collective leadership and
allow administrative control to be replaced with a system of feedback orientated to
development. From this point of view, the design of PAS gives feedback and feedforward to
organizational members (Budworth et al., 2015; Bracken et al., 2016; Grant, 2017) by allowing
control and regulation in the context of collective behaviour. In accordance with Marion and
Uhl-Bien (2001), this type of bottom-up coordination gives greater fitness and creativity to an
organization as compared to a top-down leader-controlled system. All the above is achieved
by empowered organizational members where individuals’ self-control and leadership are
converged into the collective phenomenon of leadership – the so-called OL.
The purpose of PAS is not only to measure performance, but also to justify how valid the
indicators are. Liao et al. (2017) propose that employee perceptions of procedural fairness and
leader-member exchange to a certain extent mediate the positive relationship between
individual deals and effectiveness in-role performance and helping behaviour. Perceived
justice is an important factor with a strong influence on personal and organizational
effectiveness/outcomes (Kim, 2016); it introduces new activities for designing PAS, which
ensures a greater sense of justice of AE during the PAS.

2.2 Organizational justice and performance appraisal

Different research has affirmed a positive relationship between performance and employee’s
perception about justice in the organization (Cropanzano et al., 2007; Devonish and Greenidge,
2010; Hassan and Hashim, 2011). Organizational justice describes employees’ feeling about
their fair treatment in the workplace which depends on it how their interests are taken into
account and how much they have control over decisions in the organization (Ybema and Bos,
2010; Lambert et al., 2010). The perception of fairness may increase employees’ feeling of
psychological inclusion with an organization – organizational identity that motivates them to
behave more for the benefit of the organization (Mahajan and Benson, 2013; Choi et al., 2014).
The two main dimensions of organizational justice such as procedural and distributive
justice make up the structural part of justice (Greenberg, 1993) which deals with PA as well as
effectiveness in organizations (Meyer and Smith, 2000; Levy and Williams, 2004; Chang and Appraisal,
Hahn, 2006; Heslin and VandeWalle, 2011). Procedural justice is a system of policies and justice and
procedures used in the decision-making process of appraising employees in the organization
(Greenberg, 2005; Lambert et al., 2010; Tuytens and Devos, 2012). If the system seems to be
effectiveness
fair employees presume the organization to be trustworthy. Increased interpersonal trust
leads to sharing more information within the team. Therefore, the need for fair procedures in
PA is higher in teams which handle non-routine tasks (Mahajan and Benson, 2013; Choi et al.,
2014; Ganesh and Gupta, 2015). When employees feel they are treated fairly, they may even 91
accept decisions that are not advantageous for them (Lambert et al., 2010).
Distributive justice deals with identifying which distribution of rewards are perceived as
fair (Fisek and Hysom, 2008). When an employee compares the rations of their outcomes and
inputs to the corresponding rations of others in similar situations, they feel the distribution to be
fair if the ratio matches the others (Lambert et al., 2010). Inequity is usually perceived more
distinctly to under-benefit than over-benefit (Greenberg, 2005; Ybena and Bos, 2010). The
feeling of fairness in rewarding will increase network ties among employees and improve
cooperation among them. It helps to collect and share more unique information and reduces
opportunistic or negative behaviour of employees (Mahajan and Benson, 2013; Choi et al., 2014).
Distributive justice based on equity theory is influenced by different norms, of which the
most important are equity and equality (Cropanzano et al., 2007). While the equity norm
proposes the distribution of rewards to be fair when the distribution of a payoff is
proportional to the contributions of every employee, the equality norm stresses the equal
sharing of rewards among group members is fair (Messick and Sentis, 1979; Hysom and
Fisek, 2011).
Equity could be used in the distribution of a payoff only if it is possible to measure each
employee’s contribution to the performance which is difficult in non-routine and complex
tasks. In this case, assessments of contributions may also include the employee’s task-
relevant characteristics as work experience, education and some social criteria or status
characteristics as organizational title. To include task-relevant characteristics into PAS it is
necessary to identify which of them have a significant influence on work performance and
how to combine different inputs to assess input value (Greenberg, 2005; Fisek and Hysom,
2008). A PAS based on the principles of equity motivates individuals by rewards to reach
higher performance and productivity, but it also leads to greater reward differentiation by
increasing social tensions among employees. On the other hand, a more equal rewarding
system integrates employees and generates cooperation between them. Therefore, it is
reasonable to use weighted combinations of equity and equality suggested by Cropanzano
et al. (2007) and Hysom and Fisek (2011) in the design of a PAS.
Employees’ perceptions about procedural and distributive justice is an outcome of PA
(Meyer and Shmith, 2000; Sumelius et al., 2014). Grace (2017) highlights the same in the
context of higher education institutions. Additionally, the perception of justice depends on the
characteristic of work (Hysom and Fisek, 2011). Academic (knowledge) work demands certain
skills, capabilities and team-orientation (Lee et al., 1999); and it is also non-routine with high
complexity characteristics. Additionally, AEs from different cultural backgrounds perceive
organizational justice from PAS differently (Hassan and Hashim, 2011). Therefore, it is
important how PAS is designed and how AEs perceive procedural and distributive justice.
Corpanzano et al. (2007) propose to pay attention to both – procedural and distributive justice
in PAS design, because they interact and also compensate each other to a certain level. Taking
into consideration all the above, it is important to find out how a different design of PAS
implicates employees’ perception of organizational justice. Here arises the research question:
RQ1. How is the design of PAS associated with academic employees’ perception about:
a) procedural justice; and b) distributive justice?
IJPPM 2.3 Organizational effectiveness – outcome functioning in terms of
70,1 organizational leadership
Organizational effectiveness is a multidimensional construct related to many domains of
activities in organizations. It is defined as the extent to which an organization is able to
survive, perform its mission, and maintain favourable earnings, financial resources (Yukl,
2008) as well as organizational fitness – an organization’s ability to adapt, thrive and survive
(Osborn and Hunt, 2007). Additionally, organizational effectiveness has been distinguished
92 into criteria such as “internal” – employee satisfaction, supervisory practices and work
progress, and “external” – market share, competitiveness in terms of profitability and return
on assets (Boal and Hooijeberg, 2000). More precisely, it could be distinguished as “means” –
criteria reflecting organizational processes, and “ends” – criteria reflecting organizational
outcomes (Quinn and Rohrbaugh, 1983).
First, effectiveness criteria that reflect organizational processes describe a wide range of
varieties of organizational functions which could be operationally defined, such as job
satisfaction, employees’ turnover, coordination processes, managerial task and interpersonal
skills, product quality, group process satisfaction and group cohesion (Zummuto, 1984:
Edmondson, 2003; Zornoza et al., 2009). Second, effectiveness criteria that reflect
organizational outcomes describe organizational performance in three main categories,
such as: a) financial performance divided into liquidity and profitability dimensions, b)
market and organizational performance concerns the organization size dynamics – sales,
market share, etc., and c) stock market performance concerns capital market indicators – total
shareholder return, economic value added, etc. (Richard et al., 2009; Hamann et al., 2013). In
addition, most academic institutions assessed academic performance outcomes such as
quantity and quality of journal and conference papers, research and consulting projects,
patents, etc. instead of financial performance (Sahney and Thakkar, 2016). According to the
above-mentioned, organizational effectiveness is revealed via organizational functioning on
the one hand, and organizational performance is the basis for evaluating organizational
effectiveness on the other (Figure 1).
Effectiveness is an important measure at the collective/organizational level (Gerhart et al.,
2009) where cooperation and cohesiveness among the group members marks the
effectiveness of collective behaviour (Weinberg, 1998) represented by OL – a collective
view of leadership at the organizational level. The OL approach is distinct from traditional
vertical top-down approaches of leadership in the same way as distributed (Gronn, 2002),
organic (Avery, 2006), and complexity leadership (Uhl-Bien et al., 2007). All these approaches
are based on the multiple leadership mechanism characterised by main common principles: a)
it is created by collective collaborative processes (Hiller et al., 2006) which cascades

Criteria reflecting Organizational

organizational processes Functioning

Criteria of Organizational
Effectiveness Effectiveness

Figure 1.
Organizational Criteria reflecting Organizational
effectiveness and organizational outcomes Performance
criteria of effectiveness
throughout the entire organization (Avolio and Bass, 1995); b) have emergent nature that Appraisal,
coordinates organizational processes largely by bottom-up dynamics (Marion and Uhl-Bien, justice and
2001); c) multiple leadership activities are embedded into organizational structure as a meta-
capability (Hazy, 2008; Kivip~old, 2015). OL framework dimensions – organizational
effectiveness
orientation and adaptation – developed by Kivip~old and Vadi (2010) satisfy both
effectiveness criteria (see Figure 1) where organizational orientation reflects organizational
collective movement towards the desired future end or superior outcome, and organizational
adaptation reflects organizational everyday processes to coping with the dynamics of the 93
external environment.
Organizational effectiveness is strongly linked with PAS in higher education institutions
(Harkness and Schier, 2011) where teaching, research, and overall motivation as well as the
quantity and quality of faculty teaching and research are the main effectiveness measures
(Terpstra and Honoree, 2009; T€ urk, 2016). However, the link between PA and organizational
effectiveness is not simple; it also depends on person perception about the justice how they
and their activities are treated in accordance with the design of PAS. Perceived justice is an
important actor for the motivational function in PAS (Levy and Williams, 2004) and its non-
alignment with expectations of justice give rise to the turnover intention (Greenbaum et al.,
2015). Additionally, Roberson and Colquitt (2005) emphasise the collective context of
organizational justice with its relationship to the effectiveness, measured by team attitudes,
processes, withdrawal, and performance. The same positive effect of team-orientation on the
relationship between organizational justice and organizational citizenship behaviour is
revealed from Erkutlu’s (2011) study on ten Turkish public universities. This is important
because the perception of justice at the collective level has a stronger relationship with the
effectiveness compared to the individual level perception of justice (Whitman et al., 2012).
Supported from the above-mentioned research, we assume that organizational effectiveness
in terms of OL depends on how the organizational members collectively perceive justice from
the design of PAS. Therefore, is important to explore how the design of PAS in academic
educational institutions affects their organizational effectiveness in terms of OL. This forms
our second research question which is stated as follows:
RQ2. How does perceived justice from the design of PAS affect organizational
effectiveness in terms of OL in the academic environment of universities?

3. Methodology
3.1 Case study design
In order to explore the linkages between the design of PAS and organizational functioning in
terms of OL in a context of organizational performance, the logic of a comparative-case study
was used. A case study is suitable for exploratory purposes (Yin, 2018) and it gives an
important initial understanding about the investigated phenomenon (e.g. Eisenhardt, 1989;
Routio, 2007). Two cases – economics faculties from two universities (U1 and U2, names
disguised) – were chosen as the only appropriate cases satisfying Lijphart (1975) and
Gummesson (1991, pp. 84–85) comparative sampling conditions. In order to avoid disturbing
influences from environment factors, the similarities of three main characteristics were kept:
1) the general economic environment – both are in the Estonian economic environment; 2)
specific institutional regulation and financing – both are state universities and treated as a
public organizations by the Ministry of Education and Research; and 3) the teaching portfolio
and research field activities are qualitatively similar (full offering in the fields of economics
and business administration), but differ quantitatively – U1 has 49% senior and 51% junior
academic employees, while U2 has 40% senior and 60% junior academic employees. U2 is
also approximately 1.6 times larger in terms of staff levels. It is important that the
IJPPM dissimilarities concern only the PAS design of the compared institutions’ units – the unit from
70,1 U1 had adopted a performance-based design and the unit from U2 had adopted a position-
based design. All these together allow us to investigate the PAS design relationship with
organizational effectiveness and AE performance without significant influence of any other
external and internal factors.
The full study follows the logic of three main steps according to Figure 2. The first of them
estimates the PAS design by using document analysis. The second step estimates AE’s
94 subjective perception of PAS from the point of view of procedural and distributive justice by
using a survey; and finally, the third step estimates organizational effectiveness in terms of
OL by using questionnaire as well as subjective estimation of organizational internal and
external performance by using a questionnaire and academic performance by using a dataset
from the Estonian Research Portal (ERP).

3.2 Data collection and analysis for design of PAS

In order to find out the specifics of position-based and performance-based design of PAS,
objective data from official documents of universities were analysed: Guidelines for the
assessment of the work performance, Job descriptions of academic staff, Regulations for
recruitment of teaching and research staff, Regulations for professional reviews of teaching and
research staff, Regulations for evaluation of teaching and research staff, Reports AE workload,
Web-Based student feedback and Principles for revenue planning (usage only in U1), and on the
Salary Rules (Salary Grades, base Salaries and Remuneration Rules).
In accordance with Kim (2016), document analysis focuses on the relationship between PA
criteria, politics and rewards. Therefore, we determined the extent of use of the following
categories (and indicators) which express procedural justice: a) formalisation of processes
and information, b) teaching and teaching development, and c) research and development
(Pignata et al., 2016) and distributive justice: d) outcomes for the post and promotion, and e)
rewards according to outcomes (Scarpello and Jones, 1996; Lee et al., 1999).
The extent of use of the different components in PAS were estimated into three distinct
levels: 1) small extent – description of the procedures is insufficient (0–4 indicators), position-
based compensation (rewards) are determined subjectively by the leaders, AE are not
integrated into development of PAS and remuneration process; 2) medium extent –
description of the procedures is good (5–9 indicators), mainly used in position-based
compensation, AE are partially integrated into the development of PAS and remuneration
process (leaders of faculty and subunits take part in PA and remuneration system
development); and 3) great extent – description of the procedures is of high level (10 and more
indicators), mainly uses a remuneration system, AE are involved in the development of the
PA and remuneration system and take an active part in the decision-making process of the
faculty to achieve their objectives.

Organizational
effectiveness
PAS design Procedural
Organizational
Figure 2. Performance-based RQ1a RQ2
Perceived Leadership
Methodological and Justice and
framework for Position-based RQ1b Performance
investigate the
relationship between Distributive
PAS design and
organizational
effectiveness Questionnaires and
Document analysis Survey Dataset from ERP
3.3 Data collection, survey and analysis for perceived justice Appraisal,
In order to assess perceived procedural and distributive justice the survey was further justice and
developed by the authors. The survey consisting of 15 different questions divided into five
main groups: understanding about the PAS – 2 questions based on Moorman (1991) and
effectiveness
Sweeney and McFarlin (1997); reflection about the procedural justice from PAS – 3 questions
based on Sweeney and McFarlin (1993) and Scarpello and Jones (1996); simplicity/complexity
of PAS (Engle et al., 2008; Tuch et al., 2009) – 3 questions; subjectivity in PAS – 3 questions
based on Sweeney and McFarlin (1997); and fairness perception from PAS – 4 questions 95
based on Joy and Witt (1992) and Lee et al. (1999). Each question was closed-ended with a
symmetrical four-point scale (1–not agreed, 2–partly not agreed, 3–partly agreed, 4–agreed).
Survey questions were answered by the following: n1 5 44 respondents from U1 (58.1%
senior and 41.9% junior AE); and n2 5 38 from U2 (52.6% senior and 47.4% junior AE). Due
to the small sample size, differences between these two groups were estimated using the non-
parametrical Mann–Whitney U-test. In order to find out differences between the perceptions
of AE in the two universities about their PAS the results of objective document analysis and
subjective survey analysis were compared and analysed.

3.4 Data collection, measures and analysis for organizational effectiveness

The OL was estimated by assessing the Organizational Leadership Capability measurement
tool, developed by Kivip~old and Vadi (2010). The data were collected with a total sample
n 5 72, from U1 n1 5 35 (68.6% senior and 31.4% junior academic staff) and from U2 n2 5 37
(56.8% senior and 43.2% junior AE). The measurement tool uses closed statements with a
seven-point scale (strongly disagree to strongly agree) and consists of three main factors
referred to as alignment and cohesion (X1), the architecture of the internal network (X2) and the
control-feedback system (X3) with four statements for each factor. The reliability of each
construct was assessed using Cronbach’s α test resulting in the following: alignment and
cohesion (0.82), architecture of internal network (0.71) and control-feedback system (0.82). A
value for Cronbach’s α of 0.7 or more indicates good internal reliability (Nunnally, 1978, p. 245).
Three separate aspects of OL were estimated: perception of AE towards job arrangement
and management coordination (Kivip~old and Ahonen, 2013); organizational cohesiveness;
and collective ability of organizational members to explain their everyday activities within
strategic objectives (Kivip~old and Vadi, 2013). The Wilcoxon test was performed to identify
differences in the mean estimations of the OL factors themselves in each university sample
and the Mann–Whitney U-test was performed to identify differences between OL of the two
universities.
Finally, to verify and extend our explanations for our results about OL as an outcome, the
other subjective and objective performance indicators were examined and compared with OL
and perceived justice from PAS. Subjective performance divided into internal and external
organizational performance was measured by six (three internal and three external)
statements, as suggested by Kivip~old and Vadi (2013), with internal consistence Cronbach’s α
respectively 0.8 and 0.91. The Mann–Whitney U-test was used to identify differences
between the means of the two universities. For the evaluation of academic performance two
indicators were examined: 1) total amount of published papers from the period 1999 to 2013;
and 2) dynamics of publications from the period 2005 to 2013 that corresponds with data
collected by the survey and questionnaire. Three most important categories of publications
were used, such as 1.1 – scholarly articles indexed by Thomson Reuters Web of Science and/
or by ERIH, 1.2 – peer-reviewed articles in research journals with an ISSN code or of ERIH
category, and 3.1 – articles/chapters in books published by the publishers listed in the Annex,
including collections indexed by the Thomson Reuters Conference Proceedings Citation
Index from the ERP dataset.
IJPPM 4. Results
70,1 4.1 PAS design
The analysis consists of two parts: the objective data document analysis, and subjective data
(collected survey and questionnaires) analysis. During the first stage, we objectively
estimated the design of the PAS (more towards performance-based or position-based), also
the extent of use indicators that reflects procedural and distributive justice in the PAS by
analysing the documentation at universities U1 and U2.
96 The authors determined the extent of use of procedural justice indicators, including
formalisation of processes and information, regulation of teaching and research activities, etc.
In the Job descriptions and other documents of the universities we can see that formalisation of
PAS and processes are much more detailed at U1. Additionally, U1 uses a Balanced scorecard
for the entire university and its faculties. (see Table 1)
The Formalisation of processes and information (level of formalisation of processes,
transparency and clarity of information), Teaching and teaching development (teaching and
supervision of students, assessing the work of students and providing feedback to students,
creation textbooks), and Research and development (number and quality of publications,
research grants and projects) are used to a markedly great extent at U1 compared to U2 (see
Table 1).
Finally, our analysis reveals that the PAS design of U1 has used more principles of
procedural justice compared to the PAS design of U2. Also, this indicates that the PAS of U1
lean towards performance-based design while the PAS of U2 lean towards position-based
design.
The authors objectively determined the extent of use of distributive justice indicators –
post, promotion, rewards. In the Job descriptions and other documents regarding PAS of AE,
the category of the Post and promotion equity and equality balance is of a similar level in both
universities (see Table 2). The election and promotion of AE is based on a work contract and
AE are guaranteed the university’s minimum salary by contract. An individual’s
remuneration at university level consists of bonuses divided into the following categories:
remuneration for additional duties (including management duties), additional pay for
outstanding performance in a research project and additional pay for effective supervision of
a PhD student.
The analysis of the documents reveals that the procedures for the rewards
(estimation of teaching and research, feedback, career and remuneration) are markedly
more precisely regulated in U1. The remuneration system of AE at U1 is markedly more
differentiated in regard to salaries and more detailed and transparent. In the U1, AE are

Procedural justice U1 U2

Formalisation of processes and information

Level of formalisation of processes and procedures Great extent Small extent
Sufficiency and transparency of information Great extent Medium extent
Clarity of information Medium extent Medium extent
Teaching and teaching development
Number of lectures and seminars Great extent Medium extent
Number of supervised thesis Great extent Medium extent
Creation textbooks, aids and courses Great extent Small extent
Table 1.
The extent of use of the Research and development
indicators of the Number of publications Great extent Medium extent
procedural justice in Quality of publications (quoted and indexes) Medium extent Small extent
PAS at U1 and U2 Research grants, agreements and projects Medium extent Medium extent
integrated into the development of the remuneration system – they take an active part Appraisal,
in the decision-making process for each year and they are very actively involved in the justice and
development of PAS.
The remuneration of AE in U1 is based on a work contract, but the final pay is still
effectiveness
determined based on the teaching load, research and publications, management and
development activity. The performance ratings are directly linked to remuneration,
culminating in an annual appraisal interview and salary negotiation. During the stage of
further development at U2, it would be necessary to pay more attention to qualitative criteria, 97
indicators and methods of PAS (T€ urk, 2016). Therefore, the reward system of AE at U2 is
shifted towards equality and at U1 is shifted towards equity.
All this leads us to the conclusion that the position-based system in U2 is more orientated
towards equality, and the performance-based system in U1 is more orientated towards equity.

4.2 Perceived justice and PAS design – RQ1

In the second stage, we estimate how AE perceive justice (procedural and distributive) caused
by different PAS designs. Therefore, survey answers from AE of different universities were
accessed and compared with objective data analysis results of PAS. The perception of AE
about the main aspects of procedural justice with understanding and sufficiency of
information from PAS are not different according to the Mann–Whitney U-test. Table 3
summarises these U-test results which stand between 0.06 and 0.63.
RQ1a. How is the design of PAS associated with AE’s perception about procedural
justice?
In order to find the answer to research question RQ1a, we compared results of the document
analysis about the main similarities and dissimilarities in the design of PAS and procedural
justice in PAS with the results of the survey analysis about the perceived procedural justice.
The document analysis reveal differences between the economic faculties of the two
universities where the PAS design of U1 concerns more detail and procedural justice
compared to the PAS design of U2 (see Table 1). In contrast, the survey analysis reveals no
differences between U1 and U2 about the perception of procedural justice. It follows that PAS
design towards granularity and procedural justice has no influence on the perception of
procedural justice of AE.
The perception of AE about the distributed justice in estimation aspects such as
normativity, subjectivity and differentiating accordance with position are different according
to Mann–Whitney U-tests (p < 0.05). The other aspects from the main category of
remuneration system such as differentiating accordance with form and level of study, and

Distributive justice U1 U2

Post and promotion

Qualification and promotion Medium extent Medium extent
Election and academic post Medium extent Medium extent
Administrative activities and post Medium extent Medium extent
Rewards
Estimation of teaching and research Great extent Medium extent Table 2.
(feedback, development interview, career) The extent of use of
Remuneration indicators of the
Representation and involvement AE Great extent Medium extent distributive justice of
(for development PA and rewards system) Great extent Small extent PAS at U1 and U2
IJPPM remuneration accordance with personal input are not different – Mann–Whitney U-test
70,1 results are 0.16 and 0.56 respectively (see Table 4).
RQ1b. How is the design of PAS associated with AE’s perception about the distributive
justice?
In order to find the answer to research question RQ1b we compared the results of the
document analysis about distributive justice in PAS with the results of the survey analysis
98 about the perceived distributive justice. Document analysis reveal differences between the
economic faculties of the two universities where the PAS design of U1 concerns more aspects
and therefore is orientated more to “equity” compared to the PAS design of U2 (see Table 2).
The same results appear from the survey analysis, where AEs from U1 perceive their PAS to
be more normative, complex and less orientated to the academic position, compared to U2 AE

U1 (n 5 44) U2 (n 5 38) U-test

Survey question Mean SD Mean SD (p-value)

Understanding about the PAS

1. I have enough information about PAS 2.82 0.99 2.39 0.97 0.06
2. I understand how my salary is formed 2.75 0.94 2.55 0.80 0.29
Reflection about the procedural justice from PAS
Table 3. 1. Remuneration calculation should include more indicators 2.90 0.96 2.81 0.88 0.55
Academic staff 2. Remuneration calculation should consider all activities 3.21 0.84 3.16 0.76 0.63
perception of done at the university
procedural justice 3. Remuneration calculation should consider more qualitative 3.05 0.91 2.84 1.03 0.43
from PAS indicators

U1 (n 5 44) U2 (n 5 38) U-test

Survey question Mean SD Mean SD (p-value)

Simplicity/complexity of PAS
1. It is necessary to simplify the PAS 3.30 0.83 2.78 0.75 0.00*
2. Assessment of publications should be simplified 3.10 0.80 2.55 0.95 0.01*
3. The PAS should be developed more towards normative 2.17 0.85 3.26 0.95 0.00*
approach
Subjectivity in PAS
1. Remuneration should not be based mainly on standard 2.53 0.92 2.08 0.75 0.03*
hours and official salaries
2. PA should be finalised with results of the appraisal 3.29 0.97 2.42 0.95 0.00*
interview
3. Remuneration calculation might consider students’ 2.57 0.99 2.16 0.75 0.03*
satisfaction as well
Fairness perception from PAS
1. Differentiating remuneration according to form of study is 2.45 0.90 2.74 0.89 0.16
fair
Table 4. 2. Differentiating remuneration according to level of study is 3.09 0.72 2.87 0.81 0.16
Academic employees’ fair
perception of 3. Differentiating remuneration according to qualification 2.86 0.81 3.39 0.72 0.00*
distributed justice (position) is fair
from PAS 4. The remuneration system reflects fairly my work input 2.43 0.91 2.58 0.60 0.56
perception. From the point of view of distributive justice, AE from U1 perceive their PA and Appraisal,
remuneration system to be orientated to “equity” and AE from U2 perceive accordingly their justice and
system to be orientated to “equality” compared to each other. Therefore, AE perceive
performance-based PAS as a “equity-orientated” and position-based PAS as an “equality-
effectiveness
orientated”.

4.3 Organizational effectiveness and perceived justice – RQ2 99

In the third stage we are trying to answer research question RQ2, which concerns the
understanding of relations between perceived justice from the PAS and organizational
functional and outcome effectiveness.
RQ2. How does perceived justice from the design of PAS affect organizational
effectiveness in terms of OL in the academic environment of universities?
According to the fact that AEs do not perceive differences in procedural justice, therefore only
the perception of distributive justice can affect organizational effectiveness and performance.
At first, we estimated organizational effectiveness in terms of OL by two separate functioning
qualities: level of OL factors – reflecting the satisfaction of extrinsic aspects as a work
conditions and arrangements (Kivip~old and Ahonen, 2013); and interaction between OL
factors – reflecting OL capability to create superior market orientation (Kivip~old and Vadi,
2013). Table 5 summarises the results of OL factor levels and compares them according to the
two universities. Mann–Whitney U-tests (p > 0.05) indicate no significant differences
between the two universities (see Table 5); therefore, we can conclude that there is no
significant relational effect between perceived differences about the distributive justice and
OL in terms of job extrinsic aspects such as work conditions and arrangements.
OL capability was estimated according to interactions calculated between the OL
factors (X1 5 X2 Λ X1 < X3) suggested by Kivip~old and Vadi (2013). Organizational
cohesiveness was estimated as a ratio of the factor’s alignment and cohesion and the
architecture of the internal network (X1/X2), where the ratio X1/X2 5 1 indicates the
highest interaction value and X1/X2 < 1 indicates the lower value (see ratios in
Appendix). Therefore, the value 0.85 for U1 corresponds with a “low” value and the value
1.0 for U2 corresponds with a “high” value of organizational cohesiveness. Collective ability
of organizational members to explain their everyday activities within strategic objectives
was estimated as a ratio of the factor’s alignment and cohesion and the control-feedback
system (X1/X3), where the ratio X1/X3 < 1 indicates the highest interaction value and X1/
X3 ≥ 1 indicates the lower value (see ratios in Appendix). Therefore, the value 1.0 for U1
corresponds with a “low” value and the value 0.89 for U2 corresponds with a “high” value
of collective ability of organizational members to explain their everyday activities within
strategic objectives. Table 6 summarises the results of OL capability of both universities,
indicating the highest level of both interactions are found in university U2, leading to the
conclusion that university U2 has a higher total OL capability compared to university U1.
From all of these we can conclude that differences in perceived distributive justice have a
relational influence on OL capability, more precisely perceived “equality-orientation” from
PAS associated with higher OL capability.

U1 (n 5 35) U2 (n 5 37) U-test

OL factor Mean SD Mean SD (p-value) Table 5.
Level of OL factors and
Alignment and cohesion (X1) 3.60 1.50 3.81 1.70 0.51 their differences
Architecture of internal network (X2) 4.26 1.40 4.13 1.63 0.51 between two
Control-feedback system (X3) 3.94 1.48 4.27 1.52 0.24 universities
IJPPM The subjective assessment of organizational performance divided into two categories, where
70,1 internal performance expresses on internal functioning effectiveness and external
performance expresses on external functioning effectiveness. The results show no
difference (U-test p > 0.05) between the two universities in estimated internal performance
and a difference (U-test p < 0.05) in estimated external performance (see Table 7). These
results correspond exactly with the results of functioning effectiveness in terms of OL. The
two universities have no difference on effectiveness in terms of level of OL factors which
100 represents internal functioning in terms of satisfaction with working conditions and
arrangement (see Table 5) which corresponds with the estimation of internal performance
(see Table 7). At the same time, these universities are different on effectiveness in terms of OL
capability which represents external functioning in terms of market orientation (see Table 6)
and it corresponds with the result of external performance estimation (see Table 7)
where U2 > U1.
Academic performance was estimated using two different measures: 1) total amount of
publications from the period 1999 to 2013; and publication dynamics from 2005 to 2013. There
were no differences in total amount of publications between universities U1 and U2, at 680
and 696 publications respectively. A difference appears in the dynamics of publications from
2005 onwards. University U1 academic performance dynamics have a fluctuating
characteristic, while U2 dynamics have a stable increase, which might be the reason why
it falls behind U2 (see Table 8).
Finally, in accordance with research question RQ2 we can conclude that perceived
differences in distributive justice are not related to organizational functioning in terms of
internal effectiveness but are related to organizational functioning in terms of external
effectiveness as well as a dynamics of academic performance.

5. Discussion
This paper makes a contribution to the academic higher education organizations by
exploring how the design of PAS affects organizational effectiveness in terms of OL. We
found that the connection of design of PAS with organizational effectiveness depends on how

Table 6. Collective ability of organizational members to

The level of Organization Organizational cohesiveness explain their everyday activities within strategic objectives
organizational
leadership capability University U1 Low Low
with its interactions University U2 High High

Table 7.
Subjective U1 (n 5 35) U2 (n 5 37) U-test
performance Performance factor Mean SD Mean SD (p-value)
estimation differences
between two Internal performance estimation 4.10 1.47 4.48 1.60 0.32
universities External performance estimation 3.97 1.30 4.70 1.55 0.01*

Table 8.
The number and
dynamics of Publications in the most 1999–2004 2005–2010 2011–2013 Total Dynamics from
publications in the important categories (pcs) (pcs) (pcs) (pcs) 2005
three main categories
from the period U1 (categories 1.1; 1.2; 3.1) 92 360 228 680 þ212
1999 to 2013 U2 (categories 1.1; 1.2; 3.1) 59 268 369 696 þ310
AE perceive justice about their job evaluation within the frame of PAS. Our main result is that Appraisal,
the position-based design of PAS is associated with higher organizational effectiveness in justice and
terms of OL as well as organizational performance.
The discussion of results focuses on two main aspects: the AE justice perception about the
effectiveness
design of PAS and the PAS design connection with organizational effectiveness.
First, the paper demonstrates how the design of PAS influences the perceived justice
(procedural and distributive) of AE. Our findings bring out that PAS design has no influential
effect on the perceived procedural justice of AE. Perceived procedural justice explains 101
employees’ ability to have control over the organizational decisions and procedures by
improving their outcomes (Thibaut and Waler, 1975; Folger and Cropazano, 1998). Here,
control over the processes and the extent to which employees feel they can have their voices
heard in this process (Konovsky, 2000; Sumelius et al., 2014) is important. Two aspects could
explain why different designs of PAS do not influence AE perception of procedural justice:
AE have sufficiently engaged into the design process of PAS; AE as highly educated
professionals have enough knowledge and skills capacities to cope with a different design in
terms of granularity and complexity of PAS.
Contrary to the AE perception of procedural justice, our study reveals that PAS design has
an influential effect on the AE perception of distributive justice. PAS design towards
simplicity and less detail is perceived as more equality-orientated compared to design
towards granularity and complexity which is perceived as more equity-orientated. Equity-
equality balance in the context of distributive justice is not a simple phenomenon; it depends
on different sets of situational factors such as a task, group, and organization. According to
Hysom and Fisek (2011), the non-routine and independent task tends to be more equality-
orientated than routine and interdependent task, in addition small groups with three
members tends to be more equality-orientated compare to groups with five members. In
addition, the complexity level of task is important, in-group member situation individuals
tend to follow equality rules when personal input is high and follow equity rules when it is low
(Morris and Leung, 2000). The nature of non-routine with complexity characterises academic
work at the full scale. Furthermore, universities’ teaching activities are mostly characterised
by independent tasks while research work (considered publications) is carried out in small
groups up to three members in the sample of our case. Additionally, personal development-
orientated organizations preferred need-based distribution (Mannix et al., 1995) which could
be taken as a specific form of equality-orientation and universities as expert-dependent
organizations (Blackler, 1995) corresponds to this type of organizations.
Second, the study brings out some important aspects of the connection between the design
of PAS and organizational effectiveness in terms of OL. Our study reveals that different
designs of PAS have an influence on external strategic effectiveness but have no influence on
internal organizational effectiveness in academic organizations. In terms of internal
effectiveness of our two studied universities, no differences were found among the levels
of OL main factors (Table 5) and internal performance (Table 7). The level of OL factors
reflects the extrinsic part of job satisfaction (Kivip~old and Ahonen, 2013) which is one of the
mostly used indicators to measure internal functioning effectiveness of organizations. Here, it
is important that job satisfaction has the strongest relationship with perceived procedural
justice compared to other internal functioning measures in academic work settings in
universities (Hassan and Hashim, 2011; Pignata et al., 2016).
Differences of external effectiveness come to light from the estimation of OL capability
expressed by organizational cohesiveness and collective ability of organizational members to
explain their everyday activities within strategic objectives (Table 6) as well subjective estimation
of external performance (Table 7). These components of OL capability behaviours were seen as
a market orientation of an organization (Kivip~old and Vadi, 2013). We found that equity-
equality balance is important in the design of PAS, and in academic work settings equality-
IJPPM orientated design have an advantage compared to equity-orientated design in terms of
70,1 external effectiveness. Literature about the organizational justice focuses mainly on the
relationship between distributive justice and internal organizational effectiveness. Only a
couple of studies have some closeness to the external effectiveness. For example,
organizational identity (Choi et al., 2014), guidance for priority setting and decision-making
(Arnold et al., 2006), and social capital mediating ability to firm performance (Mahajan and
Benson, 2013) and customer orientation (Trivellas and Kakkos, 2015).
102
5.1 Limitations and future research
Finally, there are some limitations that also need to be addressed. The results of this study are
valid close or similar to the context of Estonian culture. It is a quite complicated situation
because many cross-cultural studies based at the level of country analysis have not
established the impact of culture on perception of justice (Fischer and Smith, 2003; Patterson
et al., 2006). Even Fisek and Hysom (2008) highlight significant differences in justice
perception in two different cultures – individualistic (American) and collectivistic self-
construal (Turkish) – they reported a weak effect of culture only. As Hui et al. (1991) argue, it
might be caused by individualistic-collectivistic construct which is too global to explain
specific resource allocation behaviours. Therefore, other specific cultural aspects such as
economic collectivism (sharing resources with the group) and economic individualism (belief
in achievement through competition) in combination with hierarchy might influence the
perception of justice more uniquely (Morris and Leung, 2000).
Another important aspect that influences justice judgement and its change is rapid
economic and social changes in society (Morris and Leung, 2000), expressed by changes in
technology and work arrangement in organizations. From this point of view, flexible work
forms, such as part-time employment, distance and virtual work on the one hand, and on the
other, complexity of work characterised by a high level of intellectual competences, has
increased in societies of developed countries. Additionally, our results are valid in the
academic job context in universities. The academic job in universities is characterised by high
complexity where AE might carry out teaching, research, and administrative duties at the
same time. Other types of intellectual capital-based jobs are very different by nature which
leads to the different level of complexity with different work arrangements there. Therefore, it
is important for the future studies to use a wide range of samples with variability of cross-cut
of intellectual job arrangements in society.

References
Adler, S., Campion, M., Colquitt, A., Grubb, A., Murphy, K., Ollander-Krane, R. and Pulakos, E. (2016),
“Getting rid of performance ratings: genius or folly? A debate”, Industrial and Organizational
Psychology, Vol. 9 No. 2, pp. 219-252.
Agasisti, T. (2017), “Management of higher education institutions and the evaluation of their
efficiency and performance”, Tertiary Education and Management, Vol. 23 No. 3, pp. 187-190.
Alach, Z. (2017), “The use of performance measurement in universities”, International Journal of Public
Sector Management, Vol. 30 No. 2, pp. 102-117.
Arnold, L., Drenkard, K., Ela, S., Goedken, J., Hamilton, C., Harris, C., Holecek, N. and White, M. (2006),
“Strategic positioning for nursing excellence in health systems: insights from chief nursing
executives”, Nursing Administration Quarterly, Vol. 30 No. 1, pp. 11-20.
Avery, G.C. (2006), Understanding Leadership, SAGE Publications, London.
Avolio, B.J. and Bass, B.M. (1995), “Individual consideration viewed at multiple levels of analysis: a
multi-level framework for examining the diffusion of transformational leadership”, The
Leadership Quarterly, Vol. 6 No. 2, pp. 1218-1999.
Blackler, F. (1995), “Knowledge work and organizations: an overview and interpretation”, Appraisal,
Organization Studies, Vol. 16 No. 6, pp. 1021-1046.
justice and
Boal, K.B. and Hooijberg, R. (2000), “Strategic leadership research: moving on”, Leadership Quarterly,
Vol. 11 No. 4, pp. 515-549.
effectiveness
Bogt, H.T. and Scapens, R.W. (2012), “Performance management in universities: effects of the
transition to more quantitative measurement systems”, European Accounting Review, Vol. 21
No. 3, pp. 451-497.
103
Browne, L. and Rayner, S. (2015), “Managing leadership in university reform: data-led decision-
making, the cost of learning and deja vu”, Educational Management Administration and
Leadership, Vol. 43 No. 2, pp. 290-307.
Bracken, D., Rose, D. and Church, A. (2016), “The evolution and devolution 3608 feedback”, Industrial
and Organizational Psychology, Vol. 9 No. 4, pp. 761-794.
Budworth, M.H., Latham, G.P. and Manroop, L. (2015), “Looking forward to performance
improvement: a field test of the feedforward interview for performance management”,
Human Resource Management, Vol. 54 No. 1, pp. 45-54.
Chang, E. and Hahn, J. (2006), “Does pay-for-performance enhance perceived distributive justice for
collectivistic employees?”, Personnel Review, Vol. 35 No. 4, pp. 397-412.
Chatelain-Ponroy, S., Mignot-Gerard, S., Musselin, C. and Sponem, S. (2018), “Is commitment to
performance-based management compatible with commitment to university ‘publicness’?
Academics’ values in French universities”, Organization Studies, Vol. 39 No. 10, pp. 1377-1401.
Cheng, S.Y. (2014), “The mediating role of organizational justice on the relationship between
administrative performance appraisal practices and organizational commitment”, The
International Journal of Human Resource Management, Vol. 25 No. 8, pp. 1131-1148.
Choi, B.K., Moon, H.K., Ko, W. and Kim, K.M. (2014), “A cross-sectional study of the relationships between
organizational justices and OCB Roles of organizational identification and psychological contracts”,
Leadership and Organization Development Journal, Vol. 35 No. 6, pp. 530-554.
Clayson, D. (2013), “Initial impressions and the student evaluation of teaching”, Journal of Education
for Business, Vol. 88 No. 1, pp. 26-35.
Cropanzano, R., Bowen, D.E. and Gilliland, S.W. (2007), “The management of organizational justice”,
Academy of Management Perspectives, Vol. 21 No. 4, pp. 34-48.
De Andres, R., Garcia-Lapresta, J.L. and Martinez, L. (2010), “A multi-granular linguistic model for
management decision-making in performance appraisal”, Soft Computing, Vol. 14 No. 1,
pp. 21-34.
Devonish, D. and Greenidge, D. (2010), “The effect of organizational justice on contextual performance,
counterproductive work behaviors, and task performance: investigating the moderating role of
ability-based emotional intelligence”, International Journal of Selection and Assessment, Vol. 18
No. 1, pp. 75-86.
Edmondson, A.C. (2003), “Speaking up in the operating room: how team leaders promote learning in
interdisciplinary action teams”, Journal of Management Studies, Vol. 40 No. 6, pp. 1419-1452.
Eisenhardt, K.M. (1989), “Building theories from case study research”, The Academy of Management
Review, Vol. 14 No. 4, pp. 532-550.
Engle, A.D., Dowling, P.J. and Festing, M. (2008), “State of origin: research in global performance
management, a proposed research domain and emerging implications”, European Journal of
International Management, Vol. 2 No. 2, pp. 153-169.
Erkutlu, H. (2011), “The moderating role of organizational culture in the relationship between
organizational justice and organizational citizenship behaviors”, Leadership and Organization
Development Journal, Vol. 32 No. 6, pp. 532-554.
Fischer, R. and Smith, P.B. (2003), “Reward allocation and culture”, Journal of Cross-Cultural
Management, Vol. 12 No. 4, pp. 17-35.
IJPPM Fisek, M.H. and Hysom, S.J. (2008), “Status Characteristic and reward expectations: a test of a theory
of justice in two cultures”, Social Science Research, Vol. 37 No. 3, pp. 769-786.
70,1
Folger, R. and Cropanzano, R. (1998), Organizational Justice and Human Resource Management, Sage,
London.
Ganesh, M.P. and Gupta, M. (2015), “Impact of procedural justice perception on team commitment Role
of participatory safety and task routineness”, Journal of Advances in Management Research,
Vol. 12 No. 2, pp. 176-191.
104
Gerhart, B., Rynes, S.L. and Fulmer, I.S. (2009), “Pay and performance: individuals, groups, and
executives”, The Academy of Management Annals, Vol. 3 No. 1, pp. 251-315.
Grace, C.C. (2017), “Exploring the potential for and promise of incorporating distributive and
procedural justices into post-secondary assessment of student learning”, Teaching in Higher
Education, Vol. 22 No. 3, pp. 304-317.
Grant, A.M. (2017), “Solution-focused cognitive-behavioral coaching for sustainable high performance
and circumventing stress, fatigue, and burnout”, Consulting Psychology Journal: Practice and
Research, Vol. 69 No. 2, pp. 98-111.
Greenbaum, R.L., Mawritz, M.B. and Piccolo, R.F. (2015), “When leaders fail to ‘walk the talk’:
supervisor undermining and perceptions of leader hypocrite”, Journal of Management, Vol. 41
No. 3, pp. 929-956.
Greenberg, J. (2005), “Equity theory”, in Nichilson, N., Audia, P.G., Madan, M. and Pillutla, M.M. (Eds),
Organizational Behavior, The Blackwell Encyclopedia of Management, 2nd ed., Blackwell
Publishing, Malden, MA, pp. 109-110.
Greenberg, J. (1993), “The social side of fairness: interpersonal and informational classes of
organizational justice”, in Cropanzano, R. (Ed.), Justice in the Workplace Approaching Fairness in
Human Resource Management, Erlbaum, Hillsdale, NJ, pp. 79-103.
Gronn, P. (2002), “Distributed leadership as a unit of analysis”, The Leadership Quarterly, Vol. 13
No. 4, pp. 423-451.
Gummesson, E. (1991), Qualitative Methods in Management Research, SAGE publications, Newbury
Park, California.
Hamann, P.M., Schiemann, F., Bellora, L. and Guenther, T.W. (2013), “Exploring the dimensions of
organizational performance: a construct validity study”, Organizational Research Methods,
Vol. 16 No. 1, pp. 67-87.
Hardre, P.L. and Kollmann, S.L. (2012), “Motivational implications of faculty performance standards”,
Educational Management Administration and Leadership, Vol. 40 No. 6, pp. 724-751.
Harkness, P. and Schier, M. (2011), “Performance related pay in Australian universities”, Australian
Universities Review, Vol. 53 No. 2, pp. 50-58.
Hassan, A. and Hashim, J. (2011), “Role of organizational justice in determining work outcomes of
national and expatriate academic staff in Malaysia”, International Journal of Commerce and
Management, Vol. 21 No. 1, pp. 82-93.
Hazy, J.K. (2008), “Leadership or luck? The system dynamics of intel’s shift to microprocessors in the
1970 and 1980s”, in Uhl-Bien, M. and Marion, R. (Eds), Complexity Leadership, Part I: Conceptual
Foundations, Information Age Publishing, Charlotte, NC, pp. 347-378.
Heslin, P.A. and VandeWalle, D. (2011), “Performance appraisal procedural justice: the role
of a manager’s implicit person theory”, Journal of Management, Vol. 37 No. 6, pp. 1694-1718.
Hiller, N.J., Day, D.V. and Vance, R.J. (2006), “Collective enactment of leadership roles and team
effectiveness: a field study”, The Leadership Quarterly, Vol. 17 No. 4, pp. 387-397.
Hui, C.H., Triandis, H.C. and Yee, C. (1991), “Cultural differences in reward allocation: is collectivism
the explanation?”, British Journal of Social Psychology, Vol. 30 No. 2, pp. 145-157.
Hysom, S.J. and Fisek, M.H. (2011), “Situational determinants of reward allocation: the equity-equality
equilibrium model”, Social Science Research, Vol. 40 No. 4, pp. 1263-1285.
Joy, V.L. and Witt, L.A. (1992), “Delay of gratification as a moderator of the procedural justice – Appraisal,
distributive justice relationship”, Group and Organization Management, Vol. 17 No. 3,
pp. 297-308. justice and
Kim, J. (2016), “Impact of performance appraisal justice on the effectiveness of pay-for-performance
effectiveness
systems after civil service reform”, Public Personnel Management, Vol. 45 No. 2, pp. 148-170.
Kivip~old, K. (2015), “Organizational leadership capability – a mechanism of knowledge
coordination for inducing innovative behaviour”, Baltic Journal of Management, Vol. 10
No. 4, pp. 478-496. 105
Kivip~old, K. and Ahonen, M. (2013), “Relationship between organizational leadership capability and
job satisfaction: exploratory study in the small-sized IT service organization in Estonia”, Review
of International Comparative Management, Vol. 14 No. 1, pp. 95-115.
Kivip~old, K. and Vadi, M. (2013), “Market orientation in the context of the impact of leadership
capability on performance”, International Journal of Bank Marketing, Vol. 31 No. 5, pp. 368-387.
Kivip~old, K. and Vadi, M. (2010), “A measurement tool for the evaluation of organizational leadership
capability”, Baltic Journal of Management, Vol. 5 No. 1, pp. 118-136.
Kivist€o, J., Pekkola, E. and Lyytinen, A. (2017), “The influence of performance-based management of
teaching and research performance of Finnish senior academics”, Tertiary Education and
Management, Vol. 23 No. 3, pp. 260-275.
Konovsky, M.A. (2000), “Understanding procedural justice and its impact on business organization”,
Journal of Management, Vol. 26 No. 3, pp. 489-511.
Lambert, E.G., Hogan, N.L., Jiang, S., Elechi, O.O., Benjamin, B., Morris, A., Laux, J.M. and Dupuy, P.
(2010), “The relationship among distributive and procedural justice and correctional life
satisfaction, burnout, and turnover intent: an exploratory study”, Journal of Criminal Justice,
Vol. 38 No. 1, pp. 7-16.
Lee, C., Law, K.S. and Bobko, P. (1999), “The importance of justice perception on pay
effectiveness: a two-year study of a skill-based pay plan”, Journal of Management, Vol. 25
No. 6, pp. 851-873.
Levy, P.E. and Williams, J.R. (2004), “The social context of performance appraisal: a review and
framework for the future”, Journal of Management, Vol. 30 No. 6, pp. 881-905.
Liao, C., Wayne, S.J., Liden, R.C. and Meuser, J.D. (2017), “Idiosyncratic deals and individual
effectiveness: the moderating role of leader-member exchange differentiation”, The Leadership
Quarterly, Vol. 28, pp. 438-450.
Lijphart, A. (1975), “The comparable-cases strategy in comparative research”, Comparative Political
Studies, Vol. 8 No. 2, pp. 158-177.
Longenecker, C. and Fink, L. (2017), “Lessons for improving your formal performance appraisal
process”, Strategic HR Review, Vol. 16 No. 1, pp. 32-38.
Mahajan, A. and Benson, P. (2013), “Organisational justice climate, social capital and firm
performance”, Journal of Management Development, Vol. 32 No. 7, pp. 721-736.
Maillard, J. and Savage, S.P. (2012), “Comparing performance: the development of police performance
management in France and Britain”, Policing and Society: An International Journal of Research
and Policy, Vol. 22 No. 4, pp. 363-383.
Mannix, E.A., Neal, M.A. and Northcraft, G.B. (1995), “Equity, equality, or need? The effects of
organizational culture on the allocation of benefits and burdens”, Organizational Behavior and
Human Decision Processes, Vol. 63 No. 3, pp. 276-286.
Marion, R. and Uhl-Bien, M. (2001), “Leadership in complex organizations”, The Leadership Quarterly,
Vol. 12 No. 4, pp. 389-418.
Marsick, V.J. and Watkins, K.E. (2003), “Summing up: demonstrating the value of an organization’s
learning culture: the dimensions of the learning organization questionnaire”, Advances in
Developing Human Resources, Vol. 5 No. 2, pp. 132-151.
IJPPM McAllister, M. and Flynn, T. (2016), “The capabilities of nurse educators (CONE) questionnaire:
development and evaluation”, Nurse Education Today, Vol. 39 No. 4, pp. 122-127.
70,1
Melo, A., Sarrico, C. and Radnor, Z. (2010), “The influence of performance management systems on
key actors in universities”, Public Management Review, Vol. 12 No. 2, pp. 233-254.
Messick, D.M. and Sentis, K.P. (1979), “Fairness and preference”, Journal of Experimental Social
Psychology, Vol. 15 No. 4, pp. 418-434.
106 Meyer, J.P. and Smith, C.A. (2000), “HRM practices and organizational commitment: test of a mediation
model”, Canadian Journal of Administrative Sciences, Vol. 17 No. 4, pp. 319-331.
Mikkonen, K., Ojala, T., Sj€ogren, T., Piirainen, A., Koskinen, C., Koskinen, M., Koivula, M., Sormunen,
M., Saaranen, T., Salminen, L., Koskim€aki, M., Ruotsalainen, H., L€ahteenm€aki, M.L., Wallin, O.,
M€aki-Hakola, H. and K€a€ari€ainen, M. (2018), “Competence areas of health science teachers – a
systematic review of quantitative studies”, Nurse Education Today, Vol. 70 No. 11, pp. 77-86.
Mingers, J. and Willmott, H. (2013), “Taylorizing business school research: on the ‘one best way’
performative effects of journal ranking lists”, Human Relations, Vol. 66 No. 8, pp. 1051-1073.
Mone, E., Eisinger, C., Guggenheim, K., Price, B. and Stine, C. (2011), “Performance management at the
wheel: driving employee engagement in organizations”, Journal of Business and Psychology,
Vol. 26 No. 2, pp. 205-212.
Moorman, R.H. (1991), “Relationship between organizational justice and organizational citizenship
behaviors: do fairness perceptions influence employee citizenship?”, Journal of Applied
Psychology, Vol. 76 No. 6, pp. 845-855.
Moreland, J. (2009), “Investigating secondary school leaders’ perceptions of performance management”,
Educational Management Administration & Leadership, Vol. 37 No. 6, pp. 735-765.
Morris, M.W. and Leung, K. (2000), “Justice for all? Progress in research on cultural variation in the
psychology of distributive and procedural justice”, Applied Psychology: An International Review,
Vol. 49 No. 1, pp. 100-132.
Osborn, R.N. and Hunt, J.G. (2007), “Leadership and the choice of order: complexity and hierarchical
perspectives near the edge of chaos”, The Leadership Quarterly, Vol. 18 No. 4, pp. 319-340.
Patterson, P.G., Cowley, E. and Prasongsukarn, K. (2006), “Service failure recovery: the moderating
impact of individual-level cultural value orientation on perceptions of justice”, International
Journal of Research in Marketing, Vol. 23 No. 3, pp. 263-277.
Pekkola, E., Siekkinen, T., Kivist€o, J. and Lyytinen, A. (2018), “Management and academic profession;
Comparing the Finnish professors with and without management positions”, Studies in Higher
Education, Vol. 43 No. 11, pp. 1949-1963.
Phin, L. (2015), “The effectiveness of performance appraisal in the private education industry in
Malaysia”, International Journal of Business and Information, Vol. 10 No. 1, pp. 95-124.
Pignata, S., Winefield, A.H., Provis, C. and Boyd, C.M. (2016), “A longitudinal study of the predictors
of perceived procedural justice in Australian university staff”, Frontiers in Psychology, Vol. 7
No. 1271, pp. 1-14.
Quinn, R.E. and Rohrbaugh, J. (1983), “A spatial model of effectiveness criteria: towards a competing
values approach to organizational analysis”, Management Science, Vol. 29 No. 3, pp. 363-377.
Richard, P.J., Devinney, T.M., Yip, G.S. and Johnson, G. (2009), “Measuring organizational
performance: towards methodological best practice”, Journal of Management, Vol. 35 No. 3,
pp. 718-804.
Roberson, Q.M. and Colquitt, J.A. (2005), “Shared and configural justice: a social network model of
justice in teams”, The Academy of Management Review, Vol. 30 No. 3, pp. 595-607.
Routio, P. (2007), “Models in the research process”, available at: http://www2.uiah.fi/projekti/metodi/
177.htm (accessed 03 August 2007).
Sahney, S. and Thakkar, J. (2016), “A comparative assessment of the performance of select higher
education institutes in India”, Quality Assurance in Education, Vol. 24 No. 2, pp. 278-302.
Scarpello, V. and Jones, F.F. (1996), “Why justice matters in compensation decision making”, Journal Appraisal,
of Organizational Behavior, Vol. 17 No. 3, pp. 285-299.
justice and
Stanton, P. and Nankervis, A. (2011), “Linking strategic HRM, performance management and
organizational effectiveness: perceptions of managers in Singapore”, Asia Pacific Business
effectiveness
Review, Vol. 17 No. 1, pp. 67-84.
Su, S. and Baird, K. (2017), “The association between performance appraisal systems, work-related attitudes
and academic performance”, Financial Accountability and Management, Vol. 33 No. 4, pp. 356-372.
107
Sumelius, L., Bj€orkman, I., Ehrnrooth, M., M€akel€a, K. and Smale, A. (2014), “What determines
employee perceptions of HRM process features? The case of performance appraisal in MNC
subsidiaries”, Human Resource Management, Vol. 53 No. 4, pp. 569-592.
Sunalai, S. and Beyerlein, M. (2015), “Exploring knowledge management in higher education
institutions: processes, influences, and outcomes”, Academy of Educational Leadership Journal,
Vol. 19 No. 3, pp. 289-309.
Sweeney, P.D. and McFarlin, D.B. (1993), “Workers’ evaluation of the ‘ends’ and the ‘means’: an
examination of four models of distributive and procedural justice”, Organizational Behavior and
Human Decision Processes, Vol. 55 No. 1, pp. 23-40.
Sweeney, P.D. and McFarlin, D.B. (1997), “Process and outcome: gender differences in the assessment
of justice”, Journal of Organizational Behavior, Vol. 18 No. 1, pp. 83-98.
Teh, C.J., Boerhannoeddin, A. and Ismail, A. (2012), “Organizational culture and performance appraisal
process: effect on organizational citizenship behavior”, Asian Business and Management,
Vol. 11 No. 4, pp. 471-484.
Terpstra, D.E. and Honoree, A.L. (2009), “Merit pay plans in higher education institutions:
characteristics and effects”, Public Personnel Management, Vol. 38 No. 4, pp. 55-77.
Thibaut, J. and Walker, L. (1975), Procedural Justice: A Psychological Analysis, Lawrence Erlbaum, Hillsdale, NJ.
Trivellas, P. and Kakkos, N. (2015), “Understanding the impact of procedural justice, job characteristics,
and resources on emotional exhaustion and its marketing-related implications”, Anatolia – An
International Journal of Tourism and Hospitality Research, Vol. 26 No. 1, pp. 45-60.
Tuch, A.N., Bargas-Avila, J.A., Opwis, K. and Wilhelm, F.H. (2009), “Visual complexity of websites:
effects on users’ experience, physiology, performance, and memory”, International Journal of
Human-Computer Studies, Vol. 67 No. 9, pp. 703-715.
Tuytens, M. and Devos, G. (2012), “Importance of system and leadership in performance appraisal”,
Personnel Review, Vol. 41 No. 6, pp. 756-776.
urk, K. (2016), “Performance management of academic staff and its effectiveness to teaching and
T€
research – based on the example of Estonian universities”, Trames, A Journal of the Humanities
and Social Sciences, Vol. 20 No. 1, pp. 17-36.
Uhl-Bien, M., Marion, R. and McKelvey, B. (2007), “Complexity leadership theory: shifting leadership from
the industrial age to the knowledge era”, The Leadership Quarterly, Vol. 18 No. 4, pp. 298-318.
Varouchas, E., Sicilia, M.A. and Sanchez-Alonso, S. (2018), “Academics’ perceptions on quality
in higher education shaping key performance indicators”, Sustainability, Vol. 10 No. 12, pp. 1-16.
Wang, J., Khurshid, K., Jalal, S., Nicolaou, S., White, S., Englander, M., Salazar, G. and Khosa, F. (2019),
“Influence of academic productivity on gender disparity in academic interventional radiology”,
American Journal of Roentgenology, Vol. 212 No. 6, pp. 1370-1376.
Weinberger, T.E. (1998), “A method for determining the equitable allocation of team based pay:
rewarding members of a cross-functional account team”, Compensation and Benefits
Management, Vol. 14 No. 4, pp. 18-26.
Whitman, D.S., Caleo, S., Carpenter, N.C., Horner, M.T. and Bernerth, J.B. (2012), “Fairness at
the collective level: a meta-analytic examination of the consequences and boundary
conditions of organizational justice climate”, Journal of Applied Psychology, Vol. 97 No. 4,
pp. 776-791.
IJPPM Wu, X.N. and Wu, X. (2015), “Do the compensatory effects of outcome and procedure on policy acceptance
depend on trust in authority?”, Social Behavior and Personality, Vol. 43 No. 9, pp. 1429-1440.
70,1
Yang, H., Rhee, G., Xuan, L., Silver, J., Jalal, S. and Khosa, F. (2019), “Analysis of H-index in assessing
gender differences in academic rank and leadership in physical medicine and rehabilitation in
the United States and Canada”, American Journal of Physical Medicine and Rehabilitation,
Vol. 98 No. 6, pp. 479-483.
Ybema, J.F. and van den Bos, K. (2010), “Effects of organizational justice on depressive symptoms and
108 sickness absence: a longitudinal perspective”, Social Science and Medicine, Vol. 70 No. 10,
pp. 1609-1617.
Yin, R. (2018), Case Study Research and Applications: Design and Methods, 6th ed., Sage Publication,
Thousand Oaks, California.
Yin, G. (2016), “The study of position performance appraisal based on total relationship flow management
theorems”, Journal of Human Resource and Sustainability Studies, Vol. 4 No. 2, pp. 110-116.
Yukl, G. (2008), “How leaders influence organizational effectiveness”, The Leadership Quarterly,
Vol. 19 No. 6, pp. 708-722.
Zammuto, R.F. (1984), “A comparison of multiple constituency models of organizational effectiveness”,
The Academy of Management Review, Vol. 9 No. 4, pp. 606-616.
Zornoza, A., Orengo, V. and Penarroja, V. (2009), “Relational capital in virtual teams: the role played
by trust”, Social Science Information, Vol. 48 No. 2, pp. 257-281.

Further reading
Ball, S. (2007), “Leadership of academics in research”, Educational Management Administration and
Leadership, Vol. 35 No. 4, pp. 449-477.
Decramer, A., Smolders, C. and Vanderstraeten, A. (2013), “Employee performance management
culture and system features in higher education: relationship with employee performance
management satisfaction”, The International Journal of Human Resource Management, Vol. 24
No. 2, pp. 352-371.

Appendix

The ratio of collective ability of

The ratio of organizational members to explain
organizational Wilcoxon their everyday activities within Wilcoxon
Table A1. Organization cohesiveness (X1/X2) (p-value) strategic objectives (X1/X3) (p-value)
Organizational
leadership capability U1 (n 5 35) 0.85 0.00* 1 0.20
interactions ratios with U2 (n 5 37) 1 0.06 0.89 0.01*
Wilcoxon test results Note(s): U1 – university 1; U2 – university 2

Corresponding author
Kurmet Kivip~old can be contacted at: Kurmet.Kivipold@ut.ee

For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: permissions@emeraldinsight.com