Rater Issues & Performance Management
PSYC 601 – Performance Assessment
Who can provide performance feedback?
What influences rater behavior?
Types of Bias
Self-Serving, Leniency, Centrality…
A unique example of bias in the workplace
How can we prevent rater distortion?
Case Study 6-4
Importance/Implications of PM Ratings
There are many benefits to a well implemented PM system
Increased motivation and self-esteem
Job criteria are clarified
Employees become more competent; misconduct is minimized
Org. change is facilitated
Employee engagement enhanced
When one link is broken, the whole system fails.
If raters do not provide accurate information, this is a failure in the system less support for fairness of administrative actions
Who should provide performance information?
Employees should be involved in deciding which of the above sources will rate the various dimensions of their performance.
Without going into too much detail, just sum this up
WHY DO WE CARE?
“REGARDLESS OF WHO RATES PERFORMANCE, THE RATER IS LIKELY TO BE AFFECTED BY BIASES THAT DISTORT THE RESULTING RATINGS” (AGUINIS, 2013, P. 150)
Rater Behaviors Influenced by:
Motivation to provide accurate ratings
Are there consequences for being inaccurate?
Are there rewards for having accurate ratings?
Motivation to distort the ratings
As a means toward achieving other goals
Supervisor May Inflate Ratings:
Maximize the merit raise/reward
Avoid creating a written record
Avoid confrontation with employees
Promote undesired employees out of unit
Make manager look good to his/her supv.
Supervisor May Deflate Ratings:
To shock an employee into action
Teach a rebellious employee a lesson
Imply that an employee should leave
Build a strongly documented, written record of poor performance
Types of Rater Bias
Do you think a supervisor would be more likely to rate a specific trait as important if they themselves also possessed that trait?
Workers were asked to self report their level of competency on a given measure and also report the value of that competency to the job through a worker-oriented job analysis survey.
Analysis of this historical data of government employed clerical workers (N = 26, 682) found statistically significant positive correlations across all competencies between self-rated performance and importance ratings of competencies.
(Cucina et al., 2012)
While this study did not examine performance appraisal specifically, the findings are still applicable in making the point that it is human nature to rate those qualities which are inherent in oneself as important over those which one does not possess.
Correlations at each competency, standardized within each occupation
Additional analysis was run to rule out common source variance.
Standardizing across occupations also helped to account for false strong correlations due to a competency not applying to a certain job title.
The results show that importance ratings could be somewhat bias upward or downward depending on incumbent standing on a certain competency.
The tendency to rate performance as better than it actually is, this correlation is most noticeable when looking at poor performing employees.
Bol (2011) supported the claim that manager tend to display both leniency and centrality bias. Leniency was related to the managers’ desire to avoid confrontation with poor performers.
When the employee-manager relationship is weaker, rater’s tend to display less leniency bias (Bol, 2011).
The clustering of ratings toward a middle value score; not providing any extreme value ratings.
Bol (2011) supported the claim that managers tend to display both leniency and centrality bias. Centrality was related to the opportunity-cost of obtaining relevant performance information.
Bias & Opportunity Cost
Collecting performance information can be costly for a manager, this will vary based on location of employees, how similar job duties are between supervisor and incumbent, and how often the supervisor works directly with the incumbent.
When there are limited natural opportunities to collect this data, supervisors lack the complete picture. Bol (2011) also found evidence to support that this lack of complete information will lead to centrality bias.
Bias & Impact on Future Performance
Even if an objective rating is not a valid benchmark of performance, employees will use this benchmark for comparison and deriving perceptions (Bol, 2011).
Centrality bias has a negative influence on performance improvement for below-average performers
Centrality bias has a negative effect on all employees’ incentives
Employee performance over time increases when leniency bias is present
There is also likely a stronger employee-incumbent relationship when leniency bias is present.
Another example of Bias
Employee start times, supervisor beliefs, and impact on performance appraisal.
Start times – stereotypic belief held by many supervisors that employees whom start early are better employees and more conscientious.
This did have an impact on performance ratings, but only when the supervisor believed in this stereotype that late starting employees viewed as being less ‘conscientiousness,’
When supervisors hold to this stereotype, employees are more likely to be rated poorly
Even though flexible work schedules are established to help retain good employees through increasing work/life balance, employees with late start times are putting themselves at risk of receiving lower performance scores (all other factors being equal).
(Yam, Fehr, & Barnes, 2013)
Methods for preventing rater distortion
Managers should understand the reasons for implementing the performance management system.
How to Identify & Rank Job Activities
How to Observe, Record, & Measure Performance
The appraisal form and system mechanics
How to minimize rating errors (be aware of bias)
How to conduct an appraisal interview
How to train, counsel, & coach
Case Study / Activity
6-4, p. 165
Provide a detailed discussion of the unintentional rating distortion factors that may come into play in this situation?
Evaluate the kinds of training programs that could minimize the factors you have described? What do you recommend and why?
Summary & Review
What we covered
What we learned
We covered motivation factors for providing accurate or inaccurate performance ratings
We reviewed three types of bias: Self-Serving, Centrality, & Leniency
We also saw a unique example of bias
We learned that various individuals can provide performance information
We learned methods for preventing rater distortion
We came up with ideas to trim down bias in organizations through the case study activity
Aguinis, H. (2013). Performance management. Upper Saddle River, NJ: Pearson Prentice Hall.
Bol, J. C. (2011). The determinants and performance effects of managers’ performance evaluation bias. The Accounting Review, 86(5), 1549-1675.
Cucina, J.M., Martin, N.R., Vasilopoulos, N.L. & Thibodeuax, H.F. (2012). Self- serving bias effects on job analysis ratings. The Journal of Psychology, 146(5), 511-531.
Yam, K.C., Fehr, R., & Barnes, C.M. (2014). Morning employees are perceived as better employees: Employees’ start times influence supervisor performance ratings. Journal of Applied Psychology, 99(6), 1288-1299.