Performance management of academic staff and its effectiveness to teaching and research--based on the example of Estonian universities.
Turk, Kulno
1. Introduction and method
The aim of the research is to identify options for developing
performance management (hereinafter PM) of the academic staff
(hereinafter AS) based on the example of Estonian universities. For the
purposes of this research, the scope of PM is limited to the performance
appraisal (hereinafter PA) and remuneration systems of AS, enabling more
analyses of interplay between those two components. The transition from
the traditional university to the modern university after the economic
crisis has brought several changes in the management of universities in
the last decade. There have been significant decreases in student
numbers and the end of private funding for studying in Estonia too.
The objects of the research are the PM systems for AS in the three
leading universities and their subunits in Estonia: the Faculty of
Economics and Business Administration at the University of Tartu
(hereinafter U1); the School of Economics and Business Administration at
the Tallinn University of Technology (hereinafter U2); and the private
Estonian Business School (hereinafter U3). The main focus is the PM of
the AS in the Economics and Business Administration faculties, where
there were the biggest changes in teaching and research. The research
question is to identify the effectiveness of PA and remuneration systems
and to investigate the options for developing PM in the universities.
The management of university faculties has been changing and has
increasingly used management instruments of the private sector, which
need professional managers.
The research methodology was developed incorporating exploratory
methods, including questionnaire-based surveys and interviews.
Exploratory analyses and qualitative methods were conducted involving
the analyses of documentation universities had on their PA and
remuneration systems (Remuneration and benefits, Salary Rules, Staff
recruitment etc.), focus group and semi-structured interviews with
academic leaders, and participatory observations within a case study in
U1. The case study holistically analyses PM policies in U1 by using
several methods and includes quantitative and qualitative evidence.
Quantitative methods included a questionnaire-based survey of AS in
three university subunits--13 questions and open questions (see Table
4). The author used the Kruskal-Wallis test for finding the differences
in opinions of AS at different universities. The data from U1 was
gathered over a period of 10 years, similar surveys were carried out in
2004, 2007 and 2013. The same survey was gathered at U2 and U3 in 2013.
To determine the particularities of the aforementioned systems, a total
of 108 AS were surveyed in 2013 and nine academic leaders were
interviewed at their universities (subunits). Seven focus group
interviews (the type of Delphi method discussions) were also conducted
at U1. Descriptive statistics and correlation analysis were used to
analyse the results obtained.
2. Literature review
2.1. PM in general
There are many definitions of PM. However, in principle it is
defined as a process of measuring and developing the individual and the
team performance; a set of PM practices with goal setting and planning,
monitoring and feedback, appraising and remunerating of employees
(Aguinis and Pierce 2007, De Andres et al. 2010). Some scientists have
taken an even broader approach, incorporating topics like managing by
values, empowerment and participative management (Mone et al. 2011,
Biron et al. 2011), or organisational performance and leadership
(Kivipold and Vadi 2013), or have associated it with creating a shared
vision of the aims of the organisation (Decramer 2013), or emphasise the
need to give equal prominence to leadership and management to achieve
their objectives in schools and colleges (Bush and Middlewood 2013). PM
is also an integral part of the managerial control system aimed at
employee work activities and work results. It has developed into an
exchange system of work-related information, where both employer and
employee express their wishes and ideas towards creating mutually
beneficial relationships. Special emphasis is on informal communication,
which assumes a development of corresponding organisational culture
(Miah and Hossan 2012).
The latest academic literature on PM has moved from the employee PA
to a variety of HRM activities, where the importance of employee
development and self-assessment are emphasised, which requires the
creation of a suitable work environment (Decramer 2013, Gravina and
Siers 2011, Edler et al. 2012, Haines and St-Onge 2012, Soss et al.
2011). Modern approaches of PM enable open and collective leadership and
allow administrative control to be replaced with a system of feedback
oriented to development. We can see a variety of approaches to the PA
(see Table 1), but we want to concentrate on PA and remuneration and its
interaction.
The goal for measuring performance is to increase performance and
align the means of PA with strategic goals of an organisation and to
include goal setting and PA under the domain of PM (Walker et al. 2010,
Stanton and Nankervis 2011). It also lays the foundations to strategic
management and provides support to quality management and the creation
of a learning organisation environment (Brudan 2010). PA enables the
consolidation of work effort of individuals and teams for achieving
strategic goals and must operate as a balanced system integrating the
individual PM and strategic management on using a balanced scorecard
approach also in education (Moreland 2009).
PA is a significantly narrower concept than PM, focusing on
evaluating employee work activities and results in order to improve
employee performance. Annual PA is an integral part of the managerial
control system and primarily based on past performance. The latest
literature emphasises the need to look beyond the appraisal's
evaluative component and look at it as a two-component bundle, where the
focus is also on developmental side of the appraisal process (Gravina
and Siers 2011, Edler et al. 2012). Mone, Price and Eisinger (2011) have
emphasised the need to look beyond measuring end results (outputs), but
focus also on inputs and process. This course has replaced
administrative control with substantive and multilateral monitoring and
thus ensures the information is based on the so-called organisational
control, which is defined as "... engineering human behaviour in
organisations to be fit for purpose" (Kindsiko 2014:58). Williams,
Rayner and Allinson (2012) believe that PM success depends on how
competent and committed the leaders of an organisation are and to what
extent it can be linked to changes in the organisational support for
employees. Studies show that the involvement of senior management and
their support to PM is crucial (Biron et al. 2011, Dewettinck and Van
Dijk 2013). The author of this paper shares this view, his opinion is
based on his research at the U1.
The author poses the following hypothesis H1: The detailed and
annual PA of AS is necessary.
The prevailing trend in PM has been in linking PA with remuneration
systems, which are mainly pay for performance (hereinafter PFP) or
performance-related pay (PRP) (Heinrich and Marschke 2009). Many studies
have shown that employee PM could improve overall organisation
performance (Wang and Wang 2008, Crowell, Hantula and McArthur 2011,
Haines and St-Onge 2012, Dewettinck and Dijk 2013). Some researchers
(Laursen 2002, Atkinson et al. 2009, Gielen et al. 2010, Malik et al.
2014, Feng et al. 2014, Lucifora and Origo 2015) have found a positive
effect of PFP systems on individual performance. Besides the
motivational aspect, PFP systems are considered to have a signalling
function to potential employees, emphasising organisations'
willingness to pay higher salaries to those who have the capability to
perform (Milanowski 2007, Gerhart et al. 2009). (see Table 2)
PFP systems are considered to have a positive impact on
quantitative aspects of work, but a negative impact on the quality of
work (Rosenhtal and Frank 2006). It is generally accepted that external
fees undermine intrinsic motivation and reduce creativity of employees
(Fang and Gerhart 2012). The author's survey results indicated the
same.
There is strong evidence that PFP has a positive impact on
motivation, however, there are also negative implications present which
are difficult to avoid (Gerhart et al. 2009). Some researchers have
found that those problems can be compensated with other managerial tools
if employees have a naturally high level of motivation and creativity
(Malik et al. 2014, Atkinson, et al. 2009). (see Table 2) In the
author's opinion, these results are controversial; objects and the
PM systems are different and varied.
The author poses the hypothesis H2: The PFP system helps to get
positive results from individual performance.
2.2. PM in universities
The education sector has been subject to a growing level of
regulation by central government (Egginton 2010) and fundamental changes
to academic work by mixing an increased market-driven transparency with
accountability in institutional and organisation management, utilizing
evidence-informed practice (Browne and Rayner 2015:292). In conditions
of declining budgets and scarce resources in the public sector (and in
the educational sector), there has been a need to implement more
effective PM methods to control outputs and results.
The focus in managing universities is to improve their performance,
but at the same time the autonomy of institutions is decreasing and the
power of central institutions is increasing (James 2014). According to
OECD reports, the majority of member countries have implemented PA and
remuneration (PFP) systems as management tools in public organisation
(Cardona 2007). This led to several positive trends--decentralisation of
management, a larger scope of management and autonomy in
decision-making, self-management and increased transparency of
activities (Melo et al. 2010, Bogt and Scapens 2012, Browne and Rayner
2015). Estonian universities have great levels of autonomy in selecting
and remunerating its AS (similar to high levels found in Sweden,
Switzerland and Czech Republic); Estonian universities' financial
autonomy and economic independence is one of the highest in Europe, on a
par with England and Luxemburg (Estermann et al. 2011).
All aforementioned trends have sharply increased the need for new
professional managers and resulted both in increased managerialism and
decreased dominance of classical managerial roles (Bogt and Scapens
2012). Several dilemmas have been established in the implementation of
PM and PA--verification of performance measures, transparency in public
provision of information and actions taken to meet those goals (Zia and
Koliba 2011). These problems are magnified if PA is conducted
hierarchically and becomes distorted, so that rather than having a
motivating effect, it results in demotivation and loss of job
satisfaction (Pascal and Marschke 2008).
Some studies have shown that PA inhibits the creativeness of both
teaching and research (Bogt and Scapens 2012), and increases the
authority and power of nonacademic staff (managers) in decision-making
and decreases the voice and freedom of AS (Melo et al. 2010). Some
studies have shown that PA increases the effectiveness of teaching and
research. (see Table 3) In author's opinion, these results are
controversial; objects and the PM systems are different and varied.
The traditional approach to PA has been criticised as not keeping
pace with the move towards more professional management (Sousa et al.
2010, Maillard and Savage 2012) or the move towards participative
organisations, which has led many organisations to adopt newer PA
systems that use multiple feedback (e.g. 360-degree) systems. PA
requires the creation of a feedback system, including appraisal and
development interviews in the organisations. Budworth, Latham and
Manroop (2015) recommended the use of a feedforward interview as opposed
to a traditional PA interview; feedforward interview should enable to
increase the PM of organisations. The author of this paper shares this
view, his opinion is based on his research at the U1.
The author poses the hypothesis H3: PA requires the creation of a
feedback system, including development interviews.
The main problems are attributable to the usage of individual-based
PA, which have helped to increase the quantitative aspect of work, but
at the expense of quality, innovation and commitment of AS. Besides
that, the academic atmosphere was weakened as people started to focus on
quantitative aspects of their work and were less willing to contribute
to other aspects crucial for the development of the university. All this
has led to the increased number of publications, especially in journals
ranked higher by university performing appraisals and have led,
according to Gil-Anton (2011), to a lower quality of those publications.
Academic atmosphere may be diluted as people have started to focus on
quantitative aspects of their work and are less willing to contribute to
other aspects for the development of academia.
Besides research quantity or quality, many universities also use
student assessments as bases for PA and as Browne and Rayner (2015)
write "putting students at the heart of the system has led to an
increasing use of managing by performance smart-data".
However, the validity of those evaluations according to Jones,
Gaffney-Rhys and Jones (2012) and Clayson (2013) is relatively low and
prone to cognitive biases, so the general suggestion is not to use this
data as a base for remuneration systems. Research has found that a first
impression had a great effect on the score of student evaluation,
additionally the use of humour, clarity, appearance and personality of
lecturers is of great importance in regards to the results of
students' evaluations, thus the quality of these evaluations is
highly questionable and they should be handled with care (Symbaluk and
Howell 2010). These results are controversial; objects or research are
different and varied.
The author poses the hypothesis H4: The student ratings used for
evaluating teaching quality are not sufficiently reliable.
While a business organisation's main objective is profit and
it is easy to measure, the goals of universities are not easily
measurable. Evaluations of academic activities and outputs (teaching and
research) are based on measurable quantitative measures and the usage of
not easily subjectively evaluated qualitative measures. For example Okas
et al. (2014) show that experienced teachers placed emphasis on the role
of educator--professional teachers must be good communicators and
willing to cooperate with colleagues and students. Sutrop (2015) has
argued that teacher's professionalism includes a value educator; in
order to advise students to reflect those values, the teacher must
acquire the same values. The author's survey results indicated the
same, but this is difficult to assess. One of the possibilities in
appraising AS is in the multitude of goals and targets that academia
needs to meet as a result of multitasking, that is, the simultaneous
effect of different activities aimed at improving performance (Nannerup
and Olsen 2014). In the author's opinion, this is difficult to
achieve.
For example Mingers and Willmott (2013) have emphasised that AS do
not engage in developing performance, but rather in shaping performance,
for example, to publish a needed number of articles. The manifestation
of this trend has increased the number of articles with controversial
results that in turn reduces the value of the papers. These trends are
particularly visible for junior researchers who do not have tenure and
who are therefore more exposed to the pressures of distorted PM systems
(Lukka 2010). To reduce this negative tendency, indexing of journals has
commenced and is adopting different systems in order to calculate the
impact of the papers. The author's survey results indicated the
same, but this is difficult to assess.
In order to decrease the negative aspects of quantitative
measurements, British universities have started to combine PA with a
greater degree of academic freedom and development and have introduced
qualitative-measures systems, for example, EFQM (Bogt and Scapens 2012).
Improving the management of universities, concentrating on competences
and quality management systems are a prerequisite for raising the
quality of higher education as a whole, including in Estonia (Heidmets
and Vilgats 2012). Subunits of universities have great levels of freedom
and rights to shape their own quality management systems, taking into
account the high standards and publication specifications of the high
rated international journals and considering the external opinions given
to the specific published articles.
This step, however, calls for an increase in subjectivity in
appraising the AS and for the introduction of the judgemental types of
PA systems. This in turn enables a reduction in the weight of the
objective and quantitative indicators of teaching and research and their
negative impact on the quality and innovation. At the same time, some of
the objective indicators (for example, grouping of scientific journals
based on their level) are seemingly objective, but in practice and
essence subjective indicators (Bogt and Scapens 2012), whilst
judgemental PA systems are not protected against quantitative evaluation
criteria bias towards past performance. New PA systems seem more
judgemental, but pose a threat to performance development systems and
inhibit creativity and innovation in the teaching and research process.
Still, judgemental indicators are seen as one way of establishing a
practical instrument for providing early signals about changes in
performance (Hallin et al. 2012). The author's survey results are
similar.
The author poses the hypothesis H5: The PA system of AS research
based on detailed indicators does not guarantee the quality of the
publications and research.
3. Results and discussion
3.1. AS opinions of PA and PFP at U1
The PM of AS in U1 is based on a work contract (every three to five
years) and on the detailed-indicator-based annual PA system, where since
1995 a unique PA and PFP system of the AS has been utilised. While AS
are guaranteed the university's minimum salary by contract, the
final pay is still determined based on the teaching load, research and
publications, and management activity. The performance ratings are
directly linked to remuneration, culminating in annual development
interviews and salary negotiations. Funding of the U1 subunits and AS
appraisal and remuneration policy is transparent; this system has been
developed each year in cooperation with AS.
The teaching load is based on the number of papers defended under
the supervision of an academic person and the amount of contact hours.
The latter is adjusted depending on the level and language of teaching.
The number of publications is taken into account during the last three
years, adjusted depending on the place of publication, with the aim of
appraising the quality of the research (research is grouped into 30
groups; the difference in weights is sixty fold).
The quantitative PA and PFP system has motivated the AS to do more
work. In the course of a study in 2011, AS members of U1 estimated their
work time as an average of 58 hours a week (taking into account all the
activities related to the position), which enabled to pre-emptively
increase AS salaries. In this decade, the workload of the AS has
decreased due to a reduction in student numbers (due to demographic
changes). This development was further influenced by new state policies,
which brought about a steep decline in fee-based higher educational
places on offer. This change has caused severe budgetary restrictions to
U1 and a stabilisation in the salaries of AS. Such a workload causes
frustration and a negative effect on the motivation and satisfaction of
the work. Some members of the AS adapt quickly to the situation and, for
example, seek opportunities to earn extra money with easier publishing.
Although annual adjustments of the PA allow the reduction of abuse of
the appraisal system, it remains a serious challenge and source of
ineffectiveness. For example, some members of the AS shape performance,
"play safe" and avoid controversial topics; this finding is
similar to that of Mingers and Willmott (2012).
The shortcomings become amplified; AS are increasingly concerned
with the complexity, administration and resource-intensity of the
system. However, despite the complicated system for PA, it had paid
dividends over many years, but it started to slow U1 development, and
dissatisfaction with the current system has increased in this decade.
Despite the broad-based discussions and democratic decision-making
processes, an increasing number of AS consider it a hindrance to
cooperation and prone to incite conflict.
AS criticism of the PM system has increased since the beginning of
the economic crisis. AS claim that for many years they have done more
work for the same money. Some teachers also decry the high teaching
workload and insufficient motivation, which is also negatively
influenced by the uneven distribution of academic workloads. The
existing PM system motivates spending financial resources, and does not
encourage procuring resources. Many members of AS have focused on
raising their own performance rating and increasing their remuneration.
They have not paid sufficient attention to developing their abilities
and that of their subunits; this needs to change. The measures applied
for raising the quality of the work have enabled to reduce the one-sided
effect of the quantitative evaluation system. These situation are
primary due to economic and demographic crisis and these opinions
expressed in the AS surveys and interviews.
AS members believe that the annual PA system is necessary and
important but that it is "... very detailed and cumbersome and
needs to be simplified." Managers should talk more frequently with
AS and "... a development interview held once a year cannot replace
it". Many AS feel that insufficient attention to quality is a
problem, above all with regard to teaching work. Opinions vary
diametrically in some cases regarding feedback from students. Twice
yearly student feedback tends to be representative; the majority of the
AS accept the student ratings as personal feedback and consider these a
good source of background information, furthermore, these feedbacks
enable to determine the "problem" subjects and teachers.
However, some of the AS see the use of student reviews as a component of
the PA as a threat to quality of teaching--"... popularity of the
teacher and quality of teaching can be different." It is emphasised
that drawing specific conclusions and implementing measures requires a
deeper analysis in each situation (including direct conversations with
students and visits to the lectures by colleagues).
It also became evident that an "... annual review of teaching
is considered to be too short a period" and "a reasonable
period would be 2-3 years." Constant changes in PA indicators are
also disapproved of--we only recently "were ETIS (1) proponents,
now we proceed from the impact factor of publications." PA in
regard to publications "... has become devalued and the system
promotes sloppy work or working to one's own detriment",
likewise, "there could be more value placed on high-calibre
research articles, not so much chapters in science books."
The measurement of the levels of fulfilment of the qualitative
objectives for the purpose of PA is complicated. This includes, e.g.
"serving society and top-ranked achievements," which would
allow AS to deviate from the main goals. U1 has started to manage
indicators and "... the system for evaluating performance has freed
managers from actual managing". Directly linking performance to
remuneration "... the salary gap among employees is too great"
and there is also a need for more "recognition and value accorded
to employees who work in a dedicated fashion for modest pay." There
is a greater desire to follow "requirements established for
different positions" and establish a "ceiling" for
teaching for professors and associate professors "so that they
would have enough time for R&D". This would allow deviations
from quantitative assessments to be reduced and place more value on
quality work in the form of non-monetary recognition. Some members of
the AS pointed out the lack of non-monetary motivational techniques,
especially in conditions of economic crisis where budgetary limitations
have ruled out pay raises.
In addition to the abovementioned shortcomings, several paradoxes
(polarised opinion) were expressed. The most conflicting opinions are:
1) the PA system is too detailed, versus all activities and results are
not sufficiently considered, and 2) employees' low sense of job
security versus the need for rapid changes. The conflicting opinions
point, on the one hand, to different interests and goals, and on the
other, they signal frustration among some AS. Some of the AS accuse the
PM (PA and PFP) system of causing their workload to increase, but fail
to consider the global reasons behind it. At the same time, in both 2007
and 2013 U1 AS continue to support the use of a remuneration system
based on quantitative PA, and opinions from AS have become more uniform
(see Table 4).
Seven focus group interviews were formed to develop the U1 PA
system and proposed to prepare a PA system based mainly on qualitative
indicators. The latter was used as the basis for developing a new model
(indicators) for the PA, for example of professors': (1) the
quantity of teaching on the doctoral level, (2) defence of doctoral
dissertations and master's theses supervised by the professor, (3)
student ratings in all subjects, (4) receiving research grants and
taking part in projects, (5) publishing in top-ranked journals, (6)
Google Scholar H-index, (7) social visibility and service to society.
The same principle would be followed for developing performance review
indicators for employees in other academic positions.
When a PA system based on qualitative criteria was proposed in the
course of PM and PA reform in 2013, many members of AS changed their
opinion and once again supported the quantitative PA system. One reason
for this is the increased indefiniteness spawned by the new system in
conditions that were already unstable due to the economic crisis and
contraction of the education market. The old quantitative PA system was
seen as a key guarantee for ensuring one's interests, while
qualitative PA was associated with subjective assessment. The increasing
of the share of qualitative indicators inevitably led to a significant
increase in the subjectivity of PA which most AS did not favour or were
not willing to accept as a basis for developing qualitative systems for
evaluating the performance of AS.
3.2. The particularities and effectiveness of the PM systems of U1,
U2 and U3
The AS professional suitability and performance are evaluated at
Estonian universities mainly at the time of the candidate's
application. Additional thorough PA systems are rarely utilised -
systematic and thorough annual PA of AS is used only in some subunits at
Estonian universities and has been organised most comprehensively at U1.
The first hypothesis, the detailed and annual PA of AS is
necessary, was supported. Descriptive statistics and interviews show
that AS requires annual PA; this is significant part of evaluation of
the AS. The AS attitudes towards the necessity of the PA are positive
(averages in U1, U2 and U3 3,2-3.5). (see Table 4)
The attitudes of AS towards PA systems vary, with U2 AS (as opposed
to U1) desiring further development of a quantitative indicator-based PA
system (averages 3.3 and 2.2 respectively; p-value 0.0). The desire of
U1 AS to develop a PA system has diminished sharply--there was a much
greater preference for this in 2007 as compared to 2013. U1 AS would
like to simplify the quantitative PA and PFP system and have greater
reliance upon position-based salary (see Table 4). In the position,
where assistants and lectures (versus researchers and senior
researchers) are in favour of developing the quantitative evaluation
system (averages 2.5 and 1.6 respectively).
PM of the AS requires the PA to be linked with remuneration, and
this varies widely from one university/subunit to the next. While the
remuneration of U1 AS proceeds directly from their PA, at U2 and U3, the
links between PA and remuneration are markedly more modest, where
results and quality of work is generally taken into account only in the
process of deciding the appointment to the position. The AS at all
university subunits believe that PA should be directly linked to
remuneration (average 3.0-3.2). At both U1 and U2 there are significant
correlative relationships between the need for development of PFP system
(performance rating must be directly related to the remuneration) and
quantitative PA system (R 0.52 and 0.45 respectively; see Table 5).
The second hypothesis, the PFP system helps to get positive results
from individual performance, was partly supported via the AS surveys,
interviews and participatory observations within the framework of a case
study in U1. The quantitative component of PA of AS at U1 enables the
measurement of "visible" activities--teaching load (in contact
hours) on different stages of study and defence of doctoral,
master's and bachelor's theses supervised by the AS.
It was evident from the (focus group) interview results that while
U1 AS understand and accept past-based PA, the AS at other universities
have a markedly lower acceptance of this. In the latter institutions,
the lag in remuneration compared to work done is criticised, including
also in the case of development activity.
The universities' AS are most supportive of the implementation
of indicators characterising the number of contact hours and the number
of papers defended under academics supervision, depending on the level
of study and language (see Table 4). For instance, U1, U2 and U3 AS
provide PFP for supervisors of successful defences of doctoral
dissertations, at an average rate of more than the monthly salary earned
by the supervisor.
To assess the quality of research publications, the publication
source is used. Publications receive the most detailed evaluation based
on the publication source at U1, while at U2 and U3 the evaluation is
limited to the most general ETIS classification. Compared to U3 and U2,
U1 AS have a much greater preference for simplifying the publication
system used, which shows that excessive detail does not necessarily
ensure employee satisfaction. The AS at all investigated universities
would like an increase in the weight given to qualitative indicators. AS
at U1 favour the implementation of a remuneration system based on salary
grades and the development of a much simpler PA system as a basis for
remuneration. They want to increase qualitative indicators and have a
greater acceptance of objective indicators for assessing teaching
quality. (see Table 4)
It is crucial how thoroughly AS have been informed about PA and how
salaries (PFP) are determined. There is a very strong correlation
between these aspects at U1 in contrast to U2 (see Table 5). The level
of information given to AS regarding PA (including ratings assigned to
staff) and determination of salary is one requirement for ensuring a
sense of justice. Based on the above, the AS desire to use a
quantitative PA system for evaluating performance is
understandable--above all at U2 and U3, where there is no experience
with the negative manifestations of such systems.
The effective implementation of PA requires the AS to participate
in developing the systems or that at least they be kept informed of the
special nature of the PA system. The extent to which U1 AS are informed
regarding the particularities of PA and remuneration is, thanks to their
participation in managing the faculty, relatively high compared to staff
at other universities and subunits. At the same time, dissatisfaction
has increased in the last decade regarding information in the field of
PA and remuneration. While data from a survey carried out in 2004
according to Turk (2010) showed that 20% U1 AS were not satisfied with
sufficiency of information on PA, in 2007 the figure had risen to 28%
and in 2013, 36%.
AS require information about their appraisal system and they
require information about the outputs of their work that influence their
salary. Informing AS about the PA system is positively related to AS
opinions about the need of PA system (Annual PA of AS is necessary) in
U1 (R= 0, 46; see Table 5). The correlation analysis did not indicate
statistically significant relationships between these aspects in U2,
which is due to the lack of annual evaluation of AS in U2. AS
involvement in the process of creating PA system are positively related
to AS opinions about reliability and openness of PA system in the U1.
The prerequisite for effective implementation of PM is the
involvement of the AS in PA and in the development and implementation of
the remuneration system and transparent budget policy. The budget
process of U1 is more decentralised and more transparent and has been
delegated to the level of subunits. The budget process takes place along
the same principle as the evaluation of AS performance and it depends on
whether the indicators are for teaching or research.
The third hypothesis, PA requires the creation of a feedback
system, including development interviews, was partly supported via
surveys, interviews and participatory observations in U1, U2 and U3.
Much depends on the practices used to conduct the surveys and
development interviews on how representative they are. The practices
used to implement these methods (qualitative ratings) and the
reliability of the results obtained are greater at U1, where the
suitability of these methods for PA of remuneration is considered more
acceptable than it is by AS at other universities. The results of
development interviews should focus above all on staff development and
reducing control rather than the role of one-sided interviews. In
implementing subjective evaluations, the focus should lie on employee
development; however, objective indicators are the best basis for
remuneration. Results indicate that at U2, where development interviews
are rarely and incidentally used and AS lack positive experiences in
regards to these interviews, AS show less desire to use them. (see Table
4)
The fourth hypothesis, the student ratings used for evaluating
teaching quality are not sufficiently reliable, was partly supported via
surveys, interviews and participatory observations in U1, U2 and U3. The
results of student surveys are not considered trustworthy enough and
some of the AS believe that students are not qualified to rate teachers
and base their assessment on whether they like the teacher's
personality (see Table 4). Rating teachers can also be manipulated and
used to "pay back" teachers, such as for lower grades.
The studies conducted at U1 are somewhat more optimistic and allow,
provided that a rigorous method is implemented, sufficiently reliable
results to be obtained regarding the teaching quality. Such opinions are
expressed somewhat seldom at U2, where there are markedly more problems
in terms of representativeness of student surveys. The analysis of
student ratings results should focus above all on staff development and
reducing control, official feedback in providing a guide for AS
activities. In implementing subjective evaluations, the focus should lie
on employee development.
University student-assigned ratings are the means predominantly
utilised for evaluating teaching quality. These should be seen as one
possibility, the results are merely an indication and provide necessary
input for development; however, no sweeping conclusions should be drawn
on their basis. In spite of significant positive developments with
regards to student surveys, various studies also point to the
unreliability of student ratings (Jones et al. 2014, Clayson 2013),
where first impressions and likeability of the teacher's
personality play an important part (Symbaluk and Howell 2010).
The fifth hypothesis, PA system of AS research based on detailed
indicators does not guarantee quality of the publications and research,
was partly supported via an analysis of documentation, interviews and
participatory observations in U1 and U2.
It is difficult to evaluate the quality of the publications of AS,
but a clear link between PA of AS and publishing papers or textbooks can
be detected (see Table 6). The leaders of university subunits found that
recognition and rewarding AS on the basis of their higher publications
positively influenced research activity. In order to keep the PM system
from dominating and exerting a one-sided (above all quantitative)
influence on teaching and research, it will be necessary to use also
other management instruments, such as quality management.
The effectiveness of PM systems is difficult to ascertain, as AS
performance depends concurrently on many other factors. Still, some
relations are evident, for example, research publications and university
textbooks and on the other hand, the annual PA and PFP systems of AS. At
U1, where since 1995 an annual PA is used, an average of 115 original
university textbooks have been published by approximately 50 members of
AS. This is markedly greater than by other members of AS at others
economics and business administration subunits in Estonia combined.
The higher ratings ascribed to top-ranked publications (through the
sharp increase in weights) have ensured a rapid increase in the number
of publications at U1 compared to U2 and other universities. According
to ETIS classification, the 1.1, 1.2 and 3.1 type research publications
(see Table 6) receive the highest rating, which are published in
international journals with a high impact factor or as monographs
(chapters of monographs) by internationally regarded publishing houses.
It is difficult to evaluate the quality of teaching and research of
AS, and as results of the surveys and case study show, quantitative
indicators did not guarantee the quality of work. Surveys, interviews
and participation observations reflected the positive impact of PA and
PFP for teaching and research only in U1.
4. Conclusions
In the conditions of decreased availability of resources for higher
education, the budgets of universities are under strain and they thus
need to improve the usage of scarce resources by making steps to improve
the effectiveness of management and adopting modern PM practices. By
using simple PM systems, AS activities can be managed in a more robust
way, which can be negative to internal motivation and the quality of the
teaching and research. An extensive use of PA instrument combined with
PFP can ensure the external motivation of AS. However, it has created
excessive competition between AS that in turn results in reduced
cooperation.
PM systems used in Estonian state and private universities have
some similarities. These universities use profession-based PM systems
that rely on a work contract which takes place in the majority of cases
every three to five years. U1 additionally used a detailed quantitative
PA and remuneration system for AS, which enabled to measure objective
indicators--contact hours and number of master theses or dissertations
mentored on different stages of study, the number of articles published
in different outlets, etc. Detailed quantitative appraisal offers a
solid base for PM and provides control over budgets and resource
allocation. However, it is mainly a short-run tool and the introduction
of qualitative aspects of appraisal needs to be implemented. It would be
necessary to develop long-term goals--setting such goals would allow
staff to focus their activities on raising the quality of teaching and
research, based on the main goals of the university.
AS emphasised the need for improvement in the quality of research
and teaching even in conditions of scarce funding. As the motivation to
engage in knowledge work is primarily intrinsic, the usage of detailed
quantitative measures in PM should be handled with care. The impact of
PA should not dominate teaching and research; it is also crucial to use
other management instruments, for example strategic management and
quality management.
Kulno Turk
University of Tartu
Address:
Kulno Turk
School of Economics and Business Administration
University of Tartu
Tartu, Narva 4-A219, 50090, Estonia
E-mail: kulno.turk@ut.ee
Tel.: +372 5036 542
Acknowledgments
This publication has been partially supported by European Social
Foundation through the Research and Innovation Policy Monitoring
Programme.
References
Aguinis, H. and C. Pierce (2007) "Enhancing the relevance of
organizational behavior by embracing performance management
research". Journal of Organizational Behavior 29, 139-145.
Atkinson, A., S. Burgess, B. Croxson, P. Gregg, C. Propper, H.
Slater, and D. Wilson (2009) "Evaluating the impact of
performance-related pay for teachers in England". Labour Economics
16, 3, 251-261.
Biron, M., E. Farndale, and J. Paauwe (2011) "Performance
management effectiveness: lessons from world-leading firms". The
International Journal of Human Resource Management 22, 6, 1294-1311.
Bogt, H. and R. W. Scapens (2012) "Performance management in
universities: effects of the transition to more quantitative measurement
systems". European Accounting Review 21, 3, 451-497.
Browne, L. and S. Rayner (2015) "Managing leadership in
university reform: data-led decision-making, the cost of learning and
deja vu?". Educational Management Administration & Leadership
43, 2, 290-307.
Brudan, A. N. (2010) "Rediscovering performance management:
systems, learning and integration". Measuring Business Excellence
14, 1, 109-123.
Budworth, M.-H., G. P. Latham, and L. Manroop (2015) "Looking
forward to performance improvement: a field test of the feedforward
interview for performance management". Human Resource Management,
54, 1, 45-54.
Bush, T. and D. Middlewood (2013) Leading and Managing People in
Education. 3rd Ed. Sage Publications Ltd.
Cardona, F. (2007) Performance related pay in public service. In
OECD and EU Member States. Sigma.
Clayson, D. (2013) "Initial impressions and the student
evaluation of teaching". Journal of Education for Business 88, 1,
26-35.
Cranet Survey on Comparative HRM (2011) International executive
report. Cranet. Available online at
<http://www.ef.uns.ac.rs/cranet/download/cranet_report_2012_
280212.pdf >. Accessed on 28.07.2014.
Crowell, C. R., D. A., Hantula, and K. L. McArthur (2011)
"From job analysis to performance management: a synergistic
rapprochement to organizational effectiveness". Journal of
Organizational Behavior Management 31, 4, 316-332.
De Andres, R., J. L. Garcia-Lapresta, and L. Martinez (2010)
"A multi-granular linguistic model for management decision-making
in performance appraisal". Soft Computing 14, 1, 21-34.
Decramer, A., C. Smolders, and A. Vanderstraeten (2013)
"employee performance management culture and system features in
higher education: relationship with employee performance management
satisfaction". The International Journal of Human Resource
Management 24, 2, 352-371.
Dewettinck, K. and H. van Dijk (2013) "Linking Belgian
employee performance management system characteristics with performance
management system effectiveness: exploring the mediating role of
fairness". The International Journal of Human Resource Management
24, 4, 806-825.
Ederer, F. and G. Manso (2012) Is pay-for-performance detrimental
to innovation? Available online at
<http://faculty.haas.berkeley.edu/manso/em.pdf>. Accessed on
20.07.2014.
Edler, J., L. Georghiou, K. Blind, and E. Uyarra (2012)
"Evaluating the demand side: new challenges for evaluation".
Research Evaluation 21, 1, 33-47.
Egginton, B. (2010) "Introduction of Formal Performance
Appraisal of Academic Staff the Management Challenges associated with
effective implementation". Educational Management Administration
& Leadership 38, 1, 119-133.
Estermann, T, T. Nokkala, and M. Steinel (2011) University autonomy
in Europe II. The scorecard. European University Association.
Fang, M. and B. Gerhart (2012) "Does pay for performance
diminish intrinsic interest?". The International Journal of Human
Resource Management 23, 6, 1176-1196.
Feng, T., D. Wang, and D. Prajogo (2014) "Incorporating human
resource management initiatives into customer services: empirical
evidence from Chinese manufacturing firms". Industrial Marketing
Management, 43, 1, 126-135
Figlio, D. N. and L. W. Kenny (2007) "Individual teacher
incentives and student performance". Journal of Public Economics
91, 901-914.
Gerhart, B., S. L. Rynes, and I. S. Fulmer (2009) "Pay and
performance: individuals, groups, and executives". The Academy of
Management Annals 3, 1, 251-315.
Gil-Anton, M. (2011) "The merit pay system in a Mexican
university: the case of Metropolitan Autonomous University". The
Journal of the Professoriate 4, 2, 122-140.
Gielen, A. C., M. J. M. Kerkhofs, and J. C. Ours (2010) "How
performance related pay affects productivity and employment".
Journal of Population Economics 23, 1, 291-301.
Gravina, N. E. and B. P. Siers (2011) "Square pegs and round
holes: ruminations on the relationship between performance appraisal and
performance management". Journal of Organizational Behavior
Management 31, 4, 277-287.
Haines, V. Y. and S. St-Onge (2012) "Performance management
effectiveness: practices or context?". The International Journal of
Human Resource Management 23, 6, 1158-1175.
Hallin, C. A., S. Tveteras, and T. J. Andersen (2012)
"Judgmental forecasting of operational capabilities: exploring a
new indicator to predict financial performance". Available at
<http://openarchive.cbs.dk/bitstream/handle/10398/8507/Hallin%20.pdf?sequence=1>. Accessed on 12.05.2014.
Heidmets, M. and B. Vilgats (2012) "Control, freedom and
trust: seeking for a higher education model in a post-communist
country". Problems of Education in the 21st Century 40, 88-96.
Heinrich, C. J. and G. Marschke (2009) "Incentives and their
dynamics in public sector performance management systems". Journal
of Policy Analysis and Management 29, 1, 183-208.
Harkness, P. and M. Schier (2011) "Performance related pay in
Australian universities: the case of Swinburne university".
Australian Universities' Review, 53, 2, 50-58.
James, C. (2014) "Trends in the governance and governing of
school in England". Local Government Studies 40, 6, 893-909.
Jones, J., R. Gaffney-Rhys, and E. Jones (2014) "Handle with
care! An exploration of the potential risks associated with the
publication and summative usage of student evaluation of teaching (SET)
results". Journal of Further and Higher Education 38, 1, 37-56.
Kallio, K. M. and T. J. Kallio (2014) "Management-by-results
and performance measurement in universities--implications for work
motivation". Studies in Higher Education 39, 4, 574-589.
Kindsiko, E. (2014) Organisational control in university
management: a multiparadigm approach on the example of the University of
Tartu. PhD Thesis, University of Tartu, Estonia.
Kivipold, K. and M. Vadi (2013) "Market orientation in the
context of the impact of leadership capability on performance".
International Journal of Bank Marketing, 31, 5, 368-387.
Lucifora, C. and F. Origo (2015) "Performance-related pay and
firm productivity: evidence from a reform in the structure of collective
bargaining". ILR Review, 68, 3, 606-632.
Lukka, K. (2010) "The roles and effects of paradigms in
accounting research". Management Accounting Research 21, 2,
110-115.
Maillard, J. and S. P. Savage (2012) "Comparing performance:
the development of police performance management in France and
Britain". Policing and Society: An International Journal of
Research and Policy 22, 4, 363-383.
Malik, M., A. Butt, and J. Choi (2014) "Rewards and employee
creative performance: moderating effects of creative self-efficacy,
reward importance and locus of control". Journal of Organizational
Behavior 36, 1, 59-74.
Melo, A. I., C. S. Sarrico, and Z. Radnor (2010) "The
influence of performance management systems on key actors in
universities". Public Management Review 12, 2, 233-254.
Miah, M. K. and C. G. Hossan (2012) "Performance management
system in UK retail industry: a case study". Far East Journal of
Psychology and Business 7, 3, 13-25.
Milanowski, A. (2007) "Performance pay system preferences of
students preparing to be teachers". Education Finance and Policy 2,
2, 111-132.
Mingers, J. and H. Willmott (2013) "Taylorizing business
school research: on the 'one best way' performative effects of
journal ranking lists". Human Relations 66, 1051-1073.
Mone, E., C. Eisinger, K. Guggenheim, B. Price, and C. Stine (2011)
"Performance management at the wheel: driving employee engagement
in organizations". Journal of Business Psychology 26, 2, 205-2012.
Mone, E., B. Price, and C. Eisinger (2011) "Performance
management: process perfection or process utility". Industrial and
Organizational Psychology 4, 2, 184-187.
Moreland, J. (2009) "Investigating secondary school
leaders' perceptions of performance management". Educational
Management Administration & Leadership 37, 6, 735-765.
Nannerup, N. and K. K. Olsen (2014) "Multitasking and
performance measurement". Discussion Papers on Business and
Economics, Working Paper 20. University of Southern Denmark. Available
online at <http://papers.ssrn.com/sol3/papers.cfm?
abstract_id=2540063>. Accessed on 02.10.2014.
Neal, D. (2011) "The design of performance pay in
education". NBER Working Paper, nr. 16710, January 2011.
Okas, A., M. van der Schaaf, and E. Krull (2014) "Novice and
experienced teachers' views on professionalism". Trames, 18,
4, 327-344.
Pascal, C. and G. Marschke (2008) "A general test for
distortions in performance measures". Review of Economics and
Statistics 90, 3, 428-441.
Philippe, A., M. Dewatripont, C. Hoxby, A. Mas-Colell, and A. Sapir
(2008) Higher aspirations: an agenda for reforming European
universities. Brueghel Blueprint.
Pepper, A., J. Gore, and A. Crossman (2013) "Are long-term
incentive plans an effective and efficient way of motivating senior
executives?" Human Resource Management Journal 23, 1, 36-51.
Schulz, E. R. and D. M. Tanguay (2006) "Merit pay in a public
higher education institution: questions of impact and attitudes".
Public Personnel Management 35, 1, 77-88.
Soss, J., R. Fording, and S. F. Schram (2011) "The
organization of discipline: from performance management to perversity
and punishment". Journal of Public Administration Research and
Theory 21, 2, 203-232.
Sousa, C. A. A., W. F. Nijs, and P. H. J. Hendriks (2010)
"Secrets of the beehive: performance management in university
research organizations". Human Relations 63, 9, 143-1460.
Stanton, P. and A. Nankervis (2011) "Linking strategic HRM,
performance management and organizational effectiveness: perceptions of
managers in Singapore". Asia Pacific Business Review 17, 1, 67-84.
Symbaluk, D. G. and A. J. Howell (2010) "Web-based student
feedback: comparing teaching-award and research-award recipients".
Assessment and Evaluation in Higher Education 35, 1, 7-82.
Sutrop, M. (2015) "Can values be taught? The myth of
value-free education". Trames 19, 2, 189-202.
Terpstra, D. E. and A. L. Honoree (2009) "Merit pay plans in
higher education institutions: characteristics and effects". Public
Personnel Management 38, 4, 55-77.
Turk, K. (2008) "Performance appraisal and the compensation of
academic staff in the University of Tartu". Baltic Journal of
Management 3, 1,40-54.
Turk, K. (2010) "Performance management of academic staff in
the faculty of economics and business administration at Tartu University
(pluses and minuses)". In 5th IEEE International Conference on
Management of Innovation and Technology. Proceedings of the 2010 IEEE
ICMIT, 624-629. Singapore.
Walker, R. M., F. Damanpour, and C. A. Devece (2010)
"Management innovation and organizational performance: the
mediating effect of performance management". Journal of Public
Administration Research and Theory 21, 367-386.
Wang, Z. M. and S. Wang (2008) "Modelling regional HRM
strategies in China: an entrepreneurship perspective". The
International Journal of Human Resource Management 19, 5, 945-963.
Williams, H. M., J. Rayner, and C. W. Allinson (2012) "New
public management and organisational commitment in the public sector:
testing a mediation model". International Journal of Human Resource
Management 23, 13, 2615-2629.
Zia, A. and C. Koliba (2011) "Accountable climate governance:
dilemmas of performance management across complex governance
networks". Journal of Comparative Policy Analysis: Research and
Practice 13, 5, 479-497.
(1) Estonian Research Portal, the publications are grouped in 22
groups.
Table 1. PM components and relation to organisation performance
(compiled by the author, sources presented in the table)
PM components and organisation performance Author, year, survey
PM has moved from the PA to a variety of Aguinis and Pierce 2007,
HRM activities and started the role of De Andres et al. 2010,
employee development; PM practices include Walker et al. 2010
goal setting and planning, feedback,
appraising and remunerating of employees.
PM is positively related to organisational Wang and Wang 2008
performance; such relationship is stronger (103 Chinese firms)
with adaptive capability.
PM associated with creating a shared vision, Decramer 2013, Edler et
organisational leadership, managing by al. 2012, Gravina and
values, empowerment and participative Siers 2011
management; the goal of PA is to increase
performance and align the means of PA with
strategic goals.
Results indicate positive associations Haines and St-Onge
between employee recognition and 2012; 312 (Canadian
PM effectiveness; organisation culture, private and public
climate and strategic integration of HRM organisations)
are also related to positive PM outcomes.
PM system facilitators include strategic Biron et al. 2011
and tactical elements, involving senior (16 world-leading firms)
managers in the process, clearly
communicating performance expectations and
formally training performance raters.
Major activities include setting Mone et al. 2011 (large
performance and development goals, corporation, other
providing feedback and recognition, research)
building a climate of trust and
empowerment, PM can be used to increase
employee engagement.
Informal performance reviews have stronger Dewettinck and Van Dijk
relation to PM system effectiveness than 2013 (cross-industry,
formal performance reviews; the manner in 3192 employees,
which PM systems are shaped is very Belgium)
important for their effectiveness.
Table 2. PA and remuneration relations with organisation
performance and outcomes (compiled by the author,
sources presented in table)
PA and PFP relations with organisation Author, year, survey
performance
PRP stimulates labour productivity Gielen et al. 2010
and maximum effort, and encourages (Dutch firms)
the most able workers to apply for
vacancies.
PFP results in higher levels of effort but Ederer and Manso 2012
inhibits creativity and innovation; (experiments)
the combination of tolerance for failure
and remuneration for long-term success
allows innovation.
Intrinsic interest does not appear to be Fang and Gerhart, 2012
harmed by PFP (PFIP); organisations that (white-collar employees,
place greater emphasis on PFIP plans tend 8 companies in Taiwan)
to have employees with motivation
orientations matching their PFIP
plans--this reduces the detrimental
effect of PFIP.
Long-term incentives are often ineffective Pepper et al. 2013
in meeting their objectives; there is weak (FTSE executives),
correlation between managers' remuneration Backes-Gellner and Pull
and organisation's performance; 2013 (Germany)
wage differences must not be based on
productivity but instead differences
between individuals.
Table 3. PA and remuneration (AS and teachers) relations with
organisation performance and outcomes (compiled by the author,
sources presented in the table)
PA and PFP relations with organisation Author, year, survey
performance
The use of merit pay is increasing in Schulz and Tanguay
universities; merit pay had a positive 2006, Terpstra and
effect on faculty performance in teaching, Honoree 2009, Figlio
research, and service; relations between and Kenny 2007
teacher incentives (merit pay) with (USA, Canada)
student performance (test scores) could
be due to teacher incentives eliciting
more effort from teachers.
Teachers' experiences may be the Milanowski 2007
dominant influence on attitudes towards (USA university)
PFP; young teachers favour PFP higher
than experienced teachers.
PFP can generate powerful motivation Gerhart et al. 2009
effects; PFP depend on the circumstances (USA, education,
and the organisation; individual and group metaresearch)
PFP may be beneficial for some objectives,
but detrimental to others.
PA and PFP enable motivation of AS Turk 2010, Turk 2008
(research and teaching) during the growth (Estonian universities)
period of university, and create conflicts
and dissatisfaction amongst employees,
and decrease innovation.
The nature of academic work has undergone Harkness and Schier
substantial change over recent decades; 2011 (Australian
PRP practices are likely to be effective universities)
in a higher educational institution.
Total teacher effort rose following the Neal 2011
introduction of PFP; it is difficult to (metaresearch)
use one assessment system.
Management-by-results is in conflict with Kallio and Kallio 2014
intrinsic motivation and has a negative (3 Finnish universities)
effect on motivation; creative work
in universities is essentially intrinsic
motivation.
Fundamental changes to academic work Browne and Rayner
by mixing an increased market- driven 2015 (England, higher
transparency; English policy discourse education)
in higher education is putting students
at the heart of the PM system.
Table 4. Perceptions of PA and PFP systems of AS in U1,
U2 and U3 in 2007 and in 2013 (scale: 1 - not; 2 - rather not;
3 - rather yes; 4 - yes)
Questions U1 U2 U3
2007 2013 2013 2013
Annual PA of academic staff 3.5 3.3 3.2 3.5
is necessary
I am sufficiently informed about 3.3 2.8 2.4 2.2
the PA system
The PA system should be further 3.2 2.2 3.3 3.0
developed based on a quantitative
PA system
PA results should be directly 3.0 3.0 3.2 3.0
reflected in remuneration decisions
PA should conclude with development 3.3 3.3 2.4 3.1
interview
The significance of qualitative - 3.0 2.8 2.7
measures of teaching should
be increased in remuneration systems
The student feedback should be used 2.2 2.6 2.2 2.3
as a quality-component when making
remuneration-related decisions
The salary-related appraisals should 2.8 2.9 2.8 3.0
be supplemented with additional
criterions measures
(e.g. grants received)
The system for evaluating 2.3 3.1 2.6 2.6
publications should be simplified
The system of remuneration should -- 3.2 3.2 3.1
incorporate all aspects of work
I am sufficiently informed about the -- 2.8 2.6 2.8
outputs of my performance that are
going to influence my salary
The remuneration system should be 2.4 2.5 2.1 2.3
based primarily on nominal work
hours and position-based pay
The remuneration system should -- 3.3 2.8 2.8
be based on significantly simplified
PA system
Table 5. Correlations between different aspects
of PM of AS in U1 and U2 in 2013
Questions I am I am sufficiently
sufficiently informed about
informed about the outputs of my
the system of performance that
PA are going to
influence
my salary
University U1 U2 U1 U2
Annual PA of academic .46 ** - .42 ** -
staff is necessary
System of PA should - - - -
be further developed based
on quantitative PA system
I am sufficiently informed x x
about the system of PA
I am sufficiently informed .86 ** .53 ** x x
about the outputs of my
performance that are going
to influence my salary
PA results to be directly .32 * - .31 * -.44 **
reflected in salary
decisions
Questions PA results to
be directly
reflected
in salary
decisions
University U1 U2
Annual PA of academic - -
staff is necessary
System of PA should .52 ** .45 **
be further developed based
on quantitative PA system
I am sufficiently informed ,32 * -
about the system of PA
I am sufficiently informed .31 * -.44 **
about the outputs of my
performance that are going
to influence my salary
PA results to be directly x x
reflected in salary
decisions
** Spearman correlation is significant at the 0.01 level;
* Spearman correlation is significant at the 0.05 level.
Table 6. The number of publications published
by AS in U1 and U2 in 2013 (2)
Type of 1.1 * 1.2 * 3.1 *
publication
U1 1999-2004 6 40 46
U1 2005-2010 59 110 191
U1 2011-2013 45 103 80
U1 total 110 253 317
U2 1999-2004 7 22 30
U2 2005-2010 45 130 93
U2 2011-2013 36 225 108
U2 total 88 377 231
According to ETIS; author's calculations. Number of fulltime
posts AS in U1 (44 positions) and in U2 (113 position)
in 2013.
(2) 1.1* --Scholarly articles indexed by Thomson Reuters Web of
Science and/or by ERIH
1.2* --Peer-reviewed articles in research
journals with an ISSN code or of ERIH category
3.1* --Articles/chapters in books published by the publishers listed in
Annex, including collections indexed by the Thomson Reuters
Conference Proceedings Citation Index
