Appraisal and compensation of the academic staff in Estonian public and private universities: a comparative analysis.

Turk, Kulno ; Roolaht, Tonu

1. Introduction

The appraisal of performance and especially the compensation paid to educators have been the major subjects in the public discussions about the future and quality of the Estonian education system. The situation is in many respects similar in universities that who need to establish performance appraisal systems in order to have clearly defined causality between the performance and pay of their personnel. The universities have somewhat longer experience in performance appraisal as they periodically carry out elections of the academic staff to their posts. However, the appraisal systems in the universities have to be in continuous development and their dynamic nature should reflect the changes in the demand for the university education.

In this paper, though, the primary focus will be on the inherent interconnection of performance appraisal and compensation systems used by universities. Our intention is to determine to what extent these systems are market-driven. One would expect that in private sector the market forces play a bigger role in determining the appraisal and compensation policies, while in the public sector intra-organizational traditions retain a strong position. Yet, it is possible that also public universities have to take into account the growing role of private funding, which could be seen as organizational innovation towards the new set of performance values. The authors aim, on the basis of several sources of information, to compare the performance appraisal and compensation systems used by Estonian public and private universities and reveal the differences. This is done in order to understand the dynamics of these systems towards incorporating aspects that account for shifts in the education financing. And even if this predominantly qualitative analysis fails to disclose major differences, it will nevertheless allow us to discuss the implications for the theory and practice of human resource management in the education sector.

The paper starts with a literature overview on performance appraisal and compensation in general terms and specifically in educational institutions. In the next section, general features of performance appraisal and compensation systems in Estonian universities are characterized. Thereafter the authors offer a qualitative comparison, though incorporating some quantitative measures, of systems in public and private institutions. In terms of methodology, the paper relies upon data triangulation by using several sources of secondary data (earlier appraisal system studies) as well as primary data (ongoing survey of appraisal systems). The empirical analysis provided will be based on survey responses, but due to the limited number of observations (questionnaires were filled in mostly by personnel managers or other experts), the authors are going to use predominantly qualitative generalisations.

2. Theoretical conceptualization of the performance appraisal and compensation

Performance appraisal can be defined as a process aimed at determining the results of an employee's work, one of its main functions being to offer a justified compensation for his/her efforts. It can be based directly on a particular employee's work results, or on his/her activities or competencies and is regarded as the main component of performance management, through which it is also possible to evaluate the effectiveness of an organization. The performance appraisal activities enable determination of whether the employees' performance accords with the established objectives and are primarily based on the appraisal of employees' work results. A performance appraisal criterion has to be relevant, reliable and justly measurable, while also closely linked with the objectives of the organization and its subdivisions.

Boyd and Kyle (2004) also stress that one of the antecedents to distributive and procedural justice of performance appraisal is social justice that defines the nondiscriminatory nature of the process between social groups (no gender or other similar discrimination). Here distributive justice refers to the fairness of compensation in the light of an employee's performance and procedural justice to the accuracy and suitability of appraisal procedures (Boyd, Kyle 2004, see also Brown, Benson 2003).

Performance appraisal has also been viewed as a "painful annual event' when the manager evaluates the employees' performance; it rarely had close links with the overall mission and program of the organization that were designed to maximize human effort. Appraisal results have a very important role in the HRM activities of the organization. A well-established appraisal system helps to make justified decisions and avoid litigation by terminated employees (Mani 2002).

Yet, in the ideal case, a performance appraisal system should establish a connection between the organizational and personal goals as well as shape and change organizational culture towards a result-driven climate (Grote 2000). The modern appraisal process is an essential part of organizational life, for it helps to justify, besides compensation differentiation, for example, promotions, demotions, selection validations and terminations (Longenecker, Fink 1999). A well-established performance appraisal system should render enough information for determining a fair compensation.

Although performance-based compensation has been traditionally quite common, and sometimes complemented by experience-based compensation, more contemporary compensation systems are based upon an employee's skills and competence. Interestingly, a comparative study of the HRM functions showed that private-sector companies tend to use skills-based or competency-based systems, while public organizations prefer more traditional compensation systems (Budhwar, Boyne 2004). This allows us to conclude that compensation policies develop faster in the private sector, which shows that private universities tend to be more dynamic and adapt faster to changes in the education market.

3. Performance appraisal and basics of compensation in the higher education system

Performance in higher education is not necessarily related to academic standards--universities (colleges) must establish procedures to monitor the quality of their graduates. This can be done through formal survey processes or informal feedback. For example, the evaluation of the education provided by different universities and colleges does not clarify the reasons why some companies prefer particular graduates. It may be because certain companies need to hire individuals who have received training in a particular academic field. Improvement in the quality of graduates begins with the recognition of their position on the labour market and also the requirements of possible employers.

The higher education system is rather conservative by its very nature and management style. This is mainly due to traditions and academic freedom, and therefore the payment-by-result system is still looked upon as a rather new approach. However, state budget difficulties have called for better management and more efficient motivation of lecturers and researchers in the public higher education institutions.

Appraisal and management of performance have recently attracted much attention in European universities and colleges. With an increase in the number of students, total costs have risen and, with limited state funding, there is a strong competition for money among various social services, therefore much more attention has to be paid to the quality of performance. Gatfield, Barker, and Graham (1999) claim that in the last decade the issue of quality has become a significant subject and will continue to be one of the predominant points of debate in higher education. The drive to quality is dictated by consumer demands for increased standards and performance, and by the need for organizational excellence. Admittedly, in recent years there has been a rising interest in quality as perceived and determined by the consumer. Sinclair (2003) considers private for-profit universities to be providers of the best quality credentials accepted by end-users at lower than public university prices.

Some authors (e.g. Stilwell 2003) question the suitability of commercial criteria and economic incentives, which have been popular political choices, in the setting of higher education. They may lead to the corporate managerial model that lays too much stress on the economic rationale, seeing competition and markets as the most appropriate means for achieving high quality in teaching and research. Scott (1999) in turn argues that the 'student as customer' view is often rejected because of its implicit reference to the marketing view that 'the customer is always right'. In order to be aware of customer interests, universities should monitor more closely their customers' expectations. Thus, awareness about these expectations is important even if the customer aspect is only one of several performance appraisal criteria.

Elliott and Shin (2002) suggest that to determine student satisfaction, the multiple-item weighted gap score analysis approach should be used as the diagnostic method. This method outlines the gaps between the ideal and actual scores, and consequently those areas which need more attention. Regarding producer-consumer relationships in higher education, Houston and Rees (1999) describe postgraduate students as having variable roles that range from co-producer to consumer. They also analyze the complex process of developing a quality management system for postgraduate education which incorporates appraisal aspects.

The three key functions of tertiary establishments are teaching, research and service. The establishments continually need to re-evaluate course offerings, grading procedures, admission requirements, student services, and the employee skills and personal traits required by hiring firms. The present article mainly examines the problems surrounding the appraisal processes of teaching and research work.

The quality of performance in teaching requires that the higher educational institutions prepare the students for their first position as well as provide the basis for performance in future positions. Part of the quality of performance is to maintain an awareness of the needs of the customer and to have the ability to build on strengths and eliminate weaknesses. Understanding what kind of personnel needs business employers have is necessary because it will enable assessment and raise the quality of college (university) graduates. The challenge to universities is to produce graduates who meet the requirements of the employers. The quality of performance in teaching at tertiary institutions would include measures such as alumni feedback that consists of several questions, for example: What were the most helpful courses? (Mergen et al. 2000, Willis, Taylor 1999). Teaching does not include only what is done, but also how it is done.

The quality of teaching depends on the qualifications and research potential of the academic staff. Research outputs, as well as successful teaching, are expected of everyone; so additionally they help to keep one's employment. This is also important for the future success of a university, as it helps to attract students of different levels. Hence the following new performance targets have become important to universities: the number of doctoral, graduate and MBA students; the number of research contracts; and most naturally the quality of research and publications. Furthermore, it is important that the academic staff should believe in the necessity of research and higher degrees to get promotion and they know that adequate support will be available for research (Pratt, Margaritis 1999).

In academic placement, monetary compensation and physical working conditions form only one, though important, aspect of the reward. Studies have shown that the academic staff values autonomy and flexibility as job characteristics so highly that they tend to remain in the academic sector even when their working conditions deteriorate (Bellamy et al. 2003). This autonomy is further reinforced by tenure systems, which may even make it difficult to agree on system-wide changes between universities (Chevaillier 2001). This leads to the suggestion that in academic work non-monetary compensation in terms of greater autonomy and flexibility retain their importance.

4. The survey of performance appraisal and pay-for-performance of the academic staff in Estonian universities: general characteristics of systems

Next we will analyze the survey results outlining the differences in academic staff job performance appraisal systems in Estonia's leading universities, including four public universities and two private universities. The universities were selected on the basis of their importance in Estonian society, especially in teaching social sciences. Several of them, or more specifically, some faculties in these universities have implemented particular appraisal systems and improved them over time. In order to investigate the appraisal and compensation systems used by Estonian universities, the authors compiled a survey questionnaire and in 2005 carried out 'The survey of performance appraisal in Estonian universities' in six Estonian universities. The questionnaire involved 19 questions, including both multiple choice and open answer questions. The majority of the questions were opinion-based and respondents had a four-point scale to use.

The questionnaires were distributed among the personnel managers or other experts of the universities, who engage in and are responsible for the management, appraisal and remuneration of the academic staff. Altogether 41 questionnaires were sent out via traditional mail. A total of 25 people from six universities responded to them: the University of Tartu (5 respondents), the Estonian Agricultural University (3), Tallinn University of Technology (4), Tallinn University, (3), Audentes University (4) and the Estonian Business School (6). The questionnaire was delivered to the personnel managers and other experts, which explains the relatively small number of respondents, as there are not many experts in the field of performance appraisal and compensation. During the research project, the respondents were asked, through an open question, to describe their appraisal system. In addition, the information on the university home pages was studied. The following representation of the main appraisal principles was derived from these two sources.

The majority of the questions were opinion-based and used a four-point scale. The first three questions involved the main principles of staff appraisal, through which it was possible to determine the main principles and appraisal basis of a particular university. Questions 4 and 17 asked the respondents' opinions about appraisal and appraisal-development interviews and whether they thought these were necessary. Questions 5-9 established specific aspects of academic staff appraisal in universities through multiple choice and open answer questions. Questions 10-12 dealt with the implementation of student questionnaires and their appraisal criteria. Questions 13-15 looked at the implementation issues of appraisal and development interviews. Question 16 studied the problems related to the publication of appraisal results, and the last question requested the respondents to state the pluses and minuses of appraisal in an open answer.

The Survey showed that there are academic staff appraisal systems in place that apply either to an entire university or to a particular faculty. In universities where appraisals are carried out on various structural levels, the appraisal methods, forms and frequency depend on the structural levels involved and vary considerably. Appraisals are carried out also on lower levels, such as institutes or departments. 90% of the respondents claim that lecturers and researchers are evaluated during the period between the faculty elections. Many faculties evaluate their lecturers regularly. However, this is still done rather superficially and without sufficient regulation. Yet, in several universities appraisal systems go through dynamic adjustments which link them more closely with pay-for-performance compensation systems. Often a more unified appraisal system still needs to be developed. The main identified appraisal methods were: 1)work programs and annual reports; 2) development interviews and 3) student (and alumni) questionnaires.

We can say that Estonian universities do not have a unified appraisal system. The universities and their faculties adopt various appraisal systems in accordance with their specific needs. The university councils usually approve the procedures and the main rules for the implementation of the academic staff appraisal, but in several universities the faculty councils establish more detailed systems on the basis of these rules. The features of appraisal systems in Estonian universities and their links to compensation are summarized in Table 1.

Although the principles of appraisal vary in universities and their faculties, there are still some common features. The similarities involve the use of teaching loads in the form of lectures and supervision of papers, scientific research and teaching material publications and results of student surveys, which all contribute to the appraisal of the quality of teaching. Relatively less value is attributed to administrative workloads, negotiated and fulfilled contracts. However, dynamic developments towards integrating academy-society cooperation into appraisal policies might lead to somewhat higher importance of research contracts.

The analysis of the effects of performance appraisal on the organizations involved indicated all respondents' agreement that appraisal of the academic staff is necessary or rather necessary, and that it is possible to evaluate the work results of the academic staff. Almost all the respondents agreed that appraisals would help universities to reach their goals. At the same time, the negative effect of appraisal on teamwork was noted--45% of the respondents believe that regular appraisals will not/rather not enhance cooperation. It was indicated that appraisal-based bonuses should form 20-30% of the total compensation package.

Attitudes towards the appraisal and development interviews were generally positive and the majority of the respondents believed it was necessary to have a link between appraisal and compensation. Heads of units consider it useful to evaluate lecturers and researchers, and to apply appraisal-development interviews at the end of appraisals. At the same time, almost half of the respondents think that the interviews do not have to be official, after which an official form has to be filled (see Table 2).

One question in the questionnaire explored the determinants of pay-for-performance bonuses from nine different angles. 82% of the respondents said that bonuses were directly or considerably related to academic qualifications (especially academic degrees). The second most important determinant is the size of the student groups. More than a half claim that bonuses are directly or considerably related to group size. The rest of the work components, including the level of teaching and specifics of a student group, teaching language, preparation and marking of test papers and exams, defending of papers and theses, work at admissions boards and advanced training courses are the basis for bonus payments to a lesser degree.

The majority of the universities carry out student questionnaires for particular courses in order to evaluate the tuition quality. The Estonian Agricultural University also carries out questionnaires among their alumni, which increases the reliability of results regarding particular lecturers. Student questionnaires are very popular in the majority of the universities and are one of the most important information sources for academic staff appraisals. It is important to carry out questionnaires among alumni more often, as this would enable the university to determine the influence of the academic staff on the development and careers of the alumni.

The student questionnaires usually contain two types of questions--multiple-choice and open answer questions. Opinions about which appraisal criteria to use were rather different. (See the second column of Table 5 in the next section) We can see from the table that the majority of the criteria used in student questionnaires were considered relevant. Only two of the criteria used (co-operation between the lecturer and students outside course hours and the level of difficulty of the subject) were considered irrelevant by almost half of the respondents.

There are different opinions about the necessity and form of development interviews. Their usefulness is accepted and they are conducted, but it is mostly done unofficially and no official form is filled. Development interviews of the academic staff are usually made by direct supervisors (heads of departments, heads of institutes, and deans). Less than half of the academic staff in universities takes part in development interviews and the interviews are often regarded as overly time- and work-consuming. The majority of the respondents believe that appraisal results should be communicated to the staff in private, without involving departments, institutes or faculties. However, it was also suggested that the results should not remain a secret as then they would not be motivating and the staff would not develop sufficiently.

In the framework of the questionnaire, the respondents were also asked via an open-ended question to point out the advantages and disadvantages of academic staff appraisals (see Table 3).

According to the answers, performance appraisal of the academic staff has several important pluses, including a rise in the motivation of the staff through feedback and acknowledgement. This all will guarantee employee development, effectiveness of their work and improved work quality. The main minuses, however, are the complexity and time consumption of the systems. Also, the job performance appraisal does not enhance teamwork and causes tensions and problems in departments and institutes.

On the basis of the questionnaire we can conclude that the need to appraise the academic staff is widely accepted. At the same time, appraisal systems in the universities are still rather basic and as no unified appraisal systems exist, each faculty uses its own appraisal system.

5. The survey of performance appraisal and pay-for-performance of the academic staff in Estonian universities: the differences between public and private universities

In this section we investigate to what extent the appraisal and compensation views introduced in theoretical section are used differently in public and private universities. Although the connections to theory might not always be explicit and straightforward, the following comparison implicitly relies on the recent dynamic developments in appraisal and compensation systems highlighted in theory.

As described earlier, the survey included four public and two private universities. In terms of the appraisal level there are no major differences by ownership type. However, one of the two private universities, Audentes University, does not use university-wide appraisal methods at all, the staff being evaluated only on faculty level. On the other hand, even though in the Estonian Business School the university-level appraisal activities do exist, faculty level appraisals tend to dominate. In public universities appraisals of lower levels are very important too, but university-wide appraisals were reported as well.

In public as well as in private universities the appraisal system is not related solely to election to positions, but takes place also between the elections. Only three respondents from public institutions expressed the opinion that there is no regular appraisal in between elections. Regarding the selection of appraisal criteria, private universities are more unified in valuing the feedback from student questionnaires. Yet, negotiated and fulfilled contracts are not viewed as the basic factor in the appraisal system in Audentes University. This criterion was likewise least mentioned by the respondents from the Estonian Business School. However, two public institutions did not consider the contracts to be important indicators either.

Respondents from private universities found the appraisal of teaching and research staff definitely important in 9 out of 10 cases, while only 63% of public university representatives were absolutely sure of its importance. However, the remaining 37% still considered it important rather than unimportant. This result indicates that private universities are somewhat more interested in appraisal-based feedback. Private universities are on average also slightly more convinced that students evaluations should be used as a component of the appraisal system.

Public universities in turn were more convinced that appraisal results should be summarized during the development interview (average scores in 4-point scale 3.74 for public and 3.30 for private universities), whereas the responses of respondents from private universities had also a much higher variability (standard deviations 0.42 and 1.25, respectively). Furthermore, the private sector considered it marginally more important that the conducted development interviews should be official and well recorded. 60% of private university respondents found that performance appraisal and compensation should be definitely directly related and yet another 20% found that they should be rather related than unrelated. In the public sector, about 37% of the respondents definitely supported this interlink, while 53% (rather) tended to support the idea as well. Thus, private universities are somewhat more convinced of the benefits of appraisal-based compensation.

Comparison of the scores attributed to the selected compensation criteria revealed that both university types consider employee qualifications (degree, practical experience) to be the most important criterion used when assigning pay-for-performance. However, on the 4-point scale the average score was 3.90 for private universities and merely 3.00 for public universities, which indicates that performance-based pay depends more on staff qualifications in private education. The same trend in responses characterized the dependency of scores of other pay-for-performance criteria, because private universities considered them to be rather important parts of their systems, while the respondents from public universities deemed several of them to be rather unimportant (see Table 4).

An especially large difference characterizes the use of a foreign language as the language of instruction as the compensation criterion. Two private universities use this as an inherent part of their compensation system, while pay-for-performance systems in public universities do not depend on that aspect to any considerable degree. One compensation aspect that is more prominent in the public sector is the specific form of instruction (full-time, distant learning or open university). The general importance of this criterion, however, remains below the average Likert score (2).

In most universities, the determination of appraisal principles takes place on the level of university councils. There are no clear-cut differences between public and private sector, because in Audentes University standards are set by the rector, while the Estonian Business School involves all academic levels in the standard-setting process. However, from the responses it became evident that in the Estonian Business School the appraisal system is established jointly by the academic as well as non-academic management. If this is true, it suggests an important feature that might differentiate private educational organizations from public institutions, but further evidence is needed to generalize this case.

The frequency of appraisal is also somewhat more unified in the private sector (in Audentes once a year; in the EBS twice a year), while answers by respondents from public universities vary from 'as needed' to 'once per election period', although including some more regular options as well.

The comparison of the universities' attitudes about the content of student feedback again showed higher average positive scores from private university respondents (see Table 5). Although the difference in averages could be partially put down to the small sample sizes, some tendencies could still be discussed. The ability to demand maximum deployment of ability from students is viewed as a very important aspect by private universities, whereas for public universities this aspect is less important.

Public universities tend to see cooperation between lecturer and student outside course hours as a slightly more important performance quality indicator than private universities. The same tendency characterizes the attitudes towards the difficulty of the subject as appraisal criteria. It has to be said, however, that both these aspects failed to be seen as the most appropriate components of student questionnaires. Nevertheless, these differences can be partially explained by the nature of the cohort, where private universities offer more evening classes for working students, which make contacts outside course more complicated.

On average, private universities seem to put marginally more importance on the interaction with students during the course, as indicated by average scores to 'considering student reactions, ability to create rapport' and 'readiness for discussions with students', and on illustrative aspects of the lectures (see again Table 5). They set higher value also on keeping to the planned work schedule and on the overall evaluations given by students.

There were no major differences in the percentages of staff participating in development interviews, except that the participation ratios of 80-90% reported by Audentes University exceeded considerably those reported by the other respondents. Again, the answers of private universities were in general less diversified than those of their public counterparts. The results about the preference of official or unofficial interviews as well as about the need to record the interview results with signed protocols did not reveal any pattern that would set the private or public universities apart. If usually development interviews are conducted by the professors of departments or faculty deans, then in the Estonian Business School certain interviews are made jointly by the management and professors. Both public and private universities prefer personal disclosure of appraisal results to more public disclosure.

More detailed responses about the role of the appraisal process show that the opinions in public and private universities do not differ dramatically in questions about the need for appraisal (both groups find it very important), about the visibility of academic staff appraisal (both find it possible), and about the interrelationship of the appraisal function and the organization's objectives (both groups find that appraisal facilitates realization of objectives). The only difference in that segment of questions concerns the interrelationship of the appraisal with cooperation between employees. The answers indicated that neither group has a very strong impression that a regular appraisal would facilitate employee cooperation, whereas private university respondents were somewhat more optimistic (average scores on 4 point scale were 2.47 (public) and 2.80 (private)). Although, due to the small number of respondents, the differences identified are not very reliable, it can be cautiously concluded that Estonian private universities have slightly more feedback-based, customer-oriented, and organization-wide appraisal systems than public educational institutions. This might be partially attributed also to the differences in size and profile of these institutions (the two private universities included are oriented to teaching business administration). Nevertheless, some signs of stricter management control and governance were identified.

6. Conclusions and implications

The performance appraisal and compensation process has gone through several important phases of development. From the simple evaluation of output produced by blue-collar workers, it has developed into a sophisticated management function characterized by a close relationship between individuals' goals and organizational objectives. In the higher education sector, appraisal systems have been implemented at the organization-wide level mostly since the 1990s. University staff usually accepts appraisal if it is oriented towards personal and organizational development and not towards stricter control. There is also a discussion going on about how extensively staff appraisal in universities should be oriented to student evaluation questionnaires, and thus to customer-oriented performance quality measures. This is also partially related to public university funding systems that range from enrolment-based financing to performance-based funding. Faculty compensation systems should strive for procedural, distributive and social justice as well as facilitate not only individual efforts, but also cooperation and teamwork.

From survey analysis we can conclude that:

* Estonian universities use several types of appraisal systems. Different faculties in large universities have their own appraisal systems that vary considerably.

* On the positive side, these appraisal systems give feedback about the performance (including the opinions of students), support the individual development of staff, increase motivation, and help to achieve the quality goals of the university. Thus, the consumer-oriented and performance-based elements discussed in theory are clearly incorporated.

* On the negative side, the existing systems do not facilitate teamwork, provide possibly biased student feedback, might create tension between departments, and, if improper procedures are applied, even cause more HRM problems.

* The appraisal-compensation interlink has yet to be improved, although in some faculties the pay-for-performance system is already in place. Yet, much like in the case of appraisal systems, there are often no unified university-wide compensation rules that would incorporate the entire compensation package.

* Although staff attitudes towards appraisal are positive, the systems are often underdeveloped and fail to encompass the whole organization. Thus, this is still the most dynamic aspect of these systems to be refined further.

We can thus conclude that the general features of the consumer-oriented and performance-driven appraisal-compensation aspects discussed in theory are already adopted in Estonian universities. However, the initial adoption has yet to be followed by improvements in the student feedback systems, in the pay-for performance systems and in the organization-wide recognition of highly-similar appraisal-compensation principles.

The comparison of Estonian public and private institutions did not yield very large differences. However, private universities seem to set more store by student feedback in the appraisal process and value the appraisal function somewhat more highly than their public counterparts. Public universities, on the other hand, see development interviews as a more valuable tool for summarizing the appraisal results. The appraisal-compensation connection is again more straightforward in the private sector. Unlike public universities, private institutions find that teaching in a foreign language should be used as an important determinant of the pay-for-performance. Private universities also involve their management more actively in the elaboration of the appraisal methods and their appraisal processes are reportedly taking place frequently. In general, appraisal systems in private universities tend to be more direct feedback-based, student-oriented, and university-wide. This is in part made possible by the smaller size of these institutions, which allows them to be more flexible. However, this result should be viewed with caution because the present study has several limitations.

The first limitation is related to the sample size. The survey of the universities unfortunately yielded only 25 usable responses. The low number of observations is likely to provide unstable results which should be verified by other surveys. Although the respondents were in most cases experts of HRM aspects, the dataset is still too small for making any conclusive generalizations. The second limitation concerns the specific profile of private institutions included in the survey. Both universities are teaching economics and management, which makes them inherently more conscious about appraisal management. Hence the results can probably not be generalized to all private universities. The third limitation involves the dynamic nature of appraisal systems. Because appraisal procedures are still being developed and evolve constantly, the described systems might no longer represent the status quo of all the aspects of appraisal. This is even more so in the case of the compensation aspect.

The future research in the field should devote more attention to the compensation systems that have close connections to appraisal results. Pay-for-performance solutions have found usage in both public and private sector alike. Yet, it is important to define performance and to determine performance indicators that are measurable, objective, and support the achievement of organizational objectives.

Acknowledgements

The authors are very grateful to Tiina-Liina Lepasepp and Mari Nomm for their contribution to the preparation of dataset and to the preliminary data analysis. This research has been partially financed by the Estonian Science Foundation grant project No 7018.

References

Annual Report of The University of Tartu 2002 (2003, 2004). Tartu.

Bellamy, Sheila, Clive Morley, and Kim Watty (2003) "Why business academics remain in Australian universities despite deteriorating working conditions and reduced job satisfaction: an intellectual puzzle". Journal of Higher Education Policy & Management 25, 1, 13-28.

Boyd, Neil M. and Ken Kyle (2004) "Expanding the view of performance appraisal by introducing social justice concerns". Administrative Theory & Praxis 26, 3, 249-277.

Brown, Michelle and John Benson (2003) "Rated to exhaustion? Reactions to performance appraisal processes". Industrial Relations Journal 34, 1, 67-81.

Budhwar, Pawan S. and George Boyne (2004) "Human resource management in the Indian public and private sectors: an empirical comparison". International Journal of Human Resource Management 15, 2, 346-370.

Chevaillier, Therry (2001) "French academics: between the professions and the civil service". Higher Education 41, 1/2, 49-76.

Elliott, Kevin M. and Dooyoung Shin (2002) "Student satisfaction: an alternative approach to assessing this important concept". Journal of Higher Education Policy & Management 24, 2, 197-209.

Fletcher, Clive (2001) "Performance appraisal and management: the developing research agenda". Journal of Occupational & Organizational Psychology 74, 4, 473-487.

Gatfield, Terry, Michelle Barker, and Peter Graham (1999) "Measuring student quality variables and the implications for management practices in higher education institutions: an Australian and international student perspective". Journal of Higher Education Policy & Management 21, 2, 239-260.

Grote, Dick (2000) "Public sector organizations". Public Personnel Management 29, 1, 1-20.

Houston, Don and Malcolm Rees (1999) "Developing a quality management system for a postgraduate education programme: a case study". Journal of Higher Education Policy & Management 21, 2, 227-238.

Longenecker, Clinton O. and L. Fink (1999) "Creating effective performance appraisal". Industrial Management 41, 5, 18-26.

Mani, Bonnie G. (2002) "Performance appraisal systems, productivity, and motivation: a case study". Public Personnel Management 31, 2, 141-59.

Mergen, E., D. Grant, and S. Widrick (2000) "Quality management applied to higher education". Total Quality Management 11, 3, 345-353.

Pratt, M. and D. Margaritis (1999) "Developing a research culture in a university faculty". Journal of Higher Education Policy & Management 21, 1, 43-57.

Scott, S. V. (1999) "The academic as service provider: is the customer 'always right'?". Journal of Higher Education Policy & Management 21, 2, 193-202.

Sinclair, Mark (2003) "Three futures for university provision: the social justice market, state capitalism and private for-profit universities". Journal of Higher Education Policy & Management 25, 2, 161-171.

Smith, Brien N., Jeffrey S. Hornsby, and Roslyn Shirmeyer (1996) "Current trends in performance appraisal: an examination of managerial practice". SAM Advanced Management Journal 61, 3, 10-15.

Stilwell, Frank (2003) "Higher education, commercial criteria and economic incentives". Journal of Higher Education Policy & Management 25, 1, 51-61.

The Survey of Performance Appraisal in Estonian Universities, 2005 (survey data).

The Survey of Performance Appraisal in the Tartu FEBA, 2004 (survey data).

Willis, T. Hillmann and Albert J. Taylor (1999) "Total quality management and higher education: the employers' perspective". Total Quality Management 10, 7, 997-1000.

Addresses:

Turk

Faculty of Economics and Business Administration

University of Tartu

Narva mnt 4-A218

51009 Tartu

Estonia

E-mail: Kulno.Turk@mtk.ut.ee

Tonu Roolaht

Faculty of Economics and Business Administration

University of Tartu

Narva mnt 4-A219

51009 Tartu

Estonia

E-mail: Tonu.Roolaht@mtk.ut.ee

Kulno Turk, Tonu Roolaht University of Tartu

Table 1. The features of appraisal systems in Estonian universities
and links with compensation Name of the university

Name of the university
(form: public or private) Appraisal system:

The University of Tartu no unified and compulsory
(public) appraisal system, but unified
 yearly reports

Tallinn University of appraisal system based on a work
Technology (public) program-report and work-program
 implementation interviews

Tallinn University annual appraisal system based on
(public) teaching, research and
 development activities

The Estonian Agricul- academic staff appraisals mainly
tural University (public) in the framework of faculty
 elections

Audentes University appraisal is based on individual
(private) reports and development
 interviews

The Estonian Business appraisal based on work program-
School (private) report regarding teaching and
 student questionnaires

Name of the university Links with compensation:
(form: public or private)

The University of Tartu direct links between appraisal
(public) and remuneration in some
 faculties and certain link at the
 university level

Tallinn University of direct links to be outlined with
Technology (public) new improved appraisal system

Tallinn University direct links between appraisal
(public) and remuneration

The Estonian Agricul- several links based on teaching
tural University (public) and resource performance

Audentes University only general effect on salaries
(private) and renewal of employment
 contracts

The Estonian Business no systematic links between the
School (private) appraisal and salaries, but
 appraisal results considered

Source: The survey of performance appraisal in Estonian universities,
2005; Note: n = 25.

Table 2. The general importance of appraisal and its characteristics
(% of respondents agreeing with statement)

Propositions Right/
 Rather right

Appraisal of lecturers and researchers is necessary 100%

The results of student questionnaires have to be taken 96.5%
into consideration at appraisals

Appraisal results should be discussed and conclusions 96.5%
drawn during appraisal- development interviews

Appraisal-development interviews should be official, 62%
after which an official form is filled

Job performance appraisal should be directly linked to 86.2%
remuneration

Source: The survey of performance appraisal in Estonian universities,
2005; Note: n = 25.

Table 3. Advantages and disadvantages of appraisal of the academic
staff in universities (in random order)

Advantages Disadvantages

Helps to fulfil the strategy and Difficult to administer and
goals of university; record;

Gives an overview of the quality Time-consuming administrative
of lecturers; side;

Feedback about your work and May create tensions between
enables self-analysis; departments;

Employees have a better understanding Does not enhance team work;
of what is expected of them;

Directs lecturers towards results Student feedback depends upon
and achievements; subject--interactive courses
 get higher marks;

Rise in motivation and discipline; Results of student
 questionnaires are not
 trustworthy where there are
 only a few respondents;

Stimulates training and development. Unsystematic appraisals might
 bring forth more negative than
 positive results.

Source: The survey of performance appraisal in Estonian universities,
2005; Note: n = 25.

Table 4. Comparison of selected pay-for-performance criteria in public
and private universities

Pay-for-performance criterion Public institution Private
(pay-for-performance in institutions
university: 1--does not
depend; 2--rather Mean Standard Mean Standard
does not depend; 3--rather deviation deviation
depends; 4--depends on...)

Level of study (bachelor,
master, doctor) 2.42 1.30 2.7 1.25
The qualifications (degree,
practical experience) 3.00 * 1.11 3.90 * 0.32

Language of instruction
(foreign language) 1.47 * 0.84 3.10 * 0.88

Preparation and marking
of tests/ exams 1.26 * 0.87 2.30 * 1.25

Defence in front of a board
(board membership, reviewing) 2.21 1.23 2.70 0.95

Participation in admission
boards 1.63 0.90 1.90 0.57

Number of students
enrolled on the course 2.32 1.34 2.80 0.63

Specifics of the group
(full-time or distant
learning/ open university) 1.95 1.27 1.60 1.07

Continuing education 1.84 1.34 1.90 1.37

* Difference statistically significant at alpha = 0.05 level.

Source: The survey of performance appraisal in Estonian universities,
2005; Note: n = 25.

Table 5. Comparison of the estimations of suitability of the evaluation
criteria in student questionnaires

Evaluation criterion (criterion in Right/ Public
student questionnaires is: 1--wrong rather institutions
2--rather wrong 3--rather right right
4--right) Mean Std Dev.

Subject matter of lectures 93% 3.42 0.69

Clarity and logical presentation
of lectures 100% 3.74 0.45

Competence of lecturer 86% 3.21 * 0.71

Level of preparation for lectures 86% 3.21 0.71

Responding to student reactions,
ability to create rapport 100% 3.32 * 0.48

Readiness for discussions
with students 93% 3.32 0.95

Ability to demand maximum
deployment of abilities from
students 75% 2.74 * 1.05

Ability to promote students' active
participation in the course 86% 3.32 0.67

Visual aids were informative and
helpful for learning 100% 3.53 0.51

Subject matter of written teaching
material 89% 3.26 0.65

Relationship between the amount
of written teaching material and
the need for it 68% 2.84 0.69

Illustrations and examples to
support lectures 100% 3.26 * 0.45

Lecturer associates the subject
with other subjects and practical
life 93% 3.63 0.68

Lecturer treated students equally
and fairly during the course 93% 3.63 0.68

Cooperation between lecturer and
student outside course hours 58% 2.74 0.73

Keeping to schedule 100% 3.42 * 0.51

Level of difficulty of the subject 55% 2.58 0.77

Overall evaluation to lecturer 96% 3.26 * 0.56

Evaluation criterion (criterion in Private
student questionnaires is: 1--wrong institutions
2--rather wrong 3--rather right 4--right)
 Mean Std Dev.

Subject matter of lectures 3.60 0.52

Clarity and logical presentation of lectures 3.80 0.42

Competence of lecturer 3.60 * 0.70

Level of preparation for lectures 3.40 0.70

Responding to student reactions, ability to
create rapport 3.70 * 0.48

Readiness for discussions with students 3.60 0.70

Ability to demand maximum deployment of
abilities from students 3.50 * 0.71

Ability to promote students' active
participate the course 3.30 0.82

Visual aids were informative and helpful for
learning 3.60 0.52

Subject matter of written teaching material 3.50 0.71

Relationship between the amount of written
teaching material and the need for it 2.80 0.92

Illustrations and examples to support lecture 3.60 * 0.52

Lecturer associates the subject with other 3.70 0.48
subjects and practical life

Lecturer treated students equally and fairly
during the course 3.70 0.48

Cooperation between lecturer and student
outside course hours 2.50 0.71

Keeping to schedule 3.80 * 0.42

Level of difficulty of the subject 2.40 0.84

Overall evaluation to lecturer 3.80 * 0.42

* Difference statistically significant at alpha = 0.05 level.

Source: The survey of performance appraisal in Estonian universities,
2005; Note: n = 25.