360-degree feedback:

Cornerstone for sustained development

360° feedback has grown in popularity for more than a quarter of a century. At a time when collaboration is critical, coworkers work remotely, and employees are more autonomous – obtaining objective feedback from peers, direct reports, and others is more important than ever. Still 360° feedback is not without its critics. Controversy has grown in recent years about the effectiveness of these programs, with some claiming that the process may actually decrease employee performance. That said, after 30 years of research, we know how the process works best and what we can do to maximize its impact on skill development.

Strategic feedback for growth

360° feedback (aka multisource feedback) is a process allowing employees to receive anonymous performance feedback from multiple perspectives such as direct reports, peers, internal customers and supervisors. Its popularity has grown since the mid 1980s because of the increased emphasis within organizations on teamwork and decentralized reporting relationships. With the more recent trends in remote officing and increasingly autonomous work requirements, today’s managers have less opportunity to observe specific work behaviors directly and/or provide feedback to their employees.

The multisource feedback process allows individuals to receive feedback from those who observe them and/or those who depend on their performance on a daily basis. The variety of perspectives provides an opportunity for insight into both consistency and diversity of behavior. In other words, individuals not only learn how others perceive their strengths and weaknesses, but how their behavior may be perceived differently across different situations.

Typically recommended as a tool for employee development, 360° feedback is used for a variety of purposes across different organizations. In organizations where it’s used strictly for employee development, it is often integrated into a leadership development curriculum or used among select audiences such as high potential employees or executive leadership teams. In these cases it may be incorporated into a classroom experience, an assessment center experience or a coaching or mentoring relationship. Other organizations use 360° feedback on a wider scale – often as input into performance appraisal or as an integral part of an annual development planning process.

Beyond providing direct feedback to participants, 360° feedback also sends a message to the organization about the value of collaboration and respect. Today’s employees are often accountable to individuals other than their direct managers, and the people who depend most on their performance may or may not be their boss. By implementing a 360° feedback process the organization sends a clear message that these interactions matter.  

While continuing to grow in popularity, 360° feedback is not without its critics. With an increased interest in positive psychology and “strength-based” employee development, critics of the process claim that 360° feedback decreases rather than improves performance because it causes participants to focus on their weaknesses rather than their strengths. Some research has even gone so far as to assert that companies that implement 360° feedback programs see a decrease in company performance. Others argue that the process institutionalizes a “popularity contest” rewarding employees for being liked rather by achieving results. While some research has demonstrated the impact of 360° feedback on managerial change, other research indicates most programs have little if any impact on performance. While these research findings may seem to contradict, there are key differences between those programs that result in positive change and those that are less successful.

The purpose behind the tool

360° feedback provides an important benefit for leadership development for several reasons. Clearly feedback is a fundamental component of learning. Early in an employee’s career lifecycle, that feedback is provided by a supervisor or by the task itself. As employees move into leadership roles, they may continue to receive feedback from their supervisor, but may or may not receive feedback from their employees.

As an employee takes on more responsibility, autonomy also increases. The leader may or may not be receiving the feedback that he/she needs to improve his/her performance. What many leaders do not understand is that as they move up the organizational hierarchy, individual employees become less likely to provide feedback about the impact of specific leader behaviors. Even leaders who see themselves as being very approachable, often find that employees are intimidated about raising concerns and providing feedback.

In addition to increased autonomy and decreased likelihood for feedback, there is a fundamental shift in the skills required for success as a manger moves up the hierarchical ladder (Goldsmith & Reiter, 2007; McCall & Lombardo, 1983). For example, while first line managers focus mostly on managing downward – making sure employees understand what they need to do and providing feedback about how they are doing – senior leaders must lead much more strategically. The higher in the organization an individual is, the more important it is to collaborate, to understand implications on other parts of the business, and to get others on board before making decisions. While command and control may have worked in the first line supervisor role, the same behaviors become derailers in the senior leadership role. 360° feedback, therefore, provides a channel for guidance on the more subtle interpersonal behaviors that impact performance employees and coworkers on a daily basis.

Building sustainable growth

It is true that 360° feedback can cause a great deal of damage if implemented inappropriately. One legitimate criticism of the process is that 360° feedback can cause leaders to shift focus away from what’s working toward what’s not working. And focusing development primarily on weaknesses has been shown to be associated with decreases in performance. That’s why it’s so critical to understand the factors that determine whether the tool will benefit or harm performance. When implemented correctly, 360° feedback can prove to be a powerful force for developing leaders.

 Aligning change.  Although there are a number of high quality “off the shelf” multi-rater feedback programs, many organizations find a custom built tool designed to align with corporate competencies, values and culture more appropriate. Before you build your own tool, keep in mind that homemade surveys can be confusing and even harmful if they are not well designed.

In an article published in the Consulting Psychology Journal, Church (2000) raises legitimate concerns about the content of “homegrown” 360° feedback surveys. Although his research demonstrates the relationship between 360° feedback scores and managerial performance, he also raises questions about the likelihood of that relationship if survey content is not appropriate:

“It is a common (though probably not often articulated) assumption among many consultants and feedback practitioners that higher ratings on any given multirater assessment instrument—almost regardless of its purported content—will directly translate to or correlate with some degree of improved individual performance… Of course, this belief can also represent a serious problem for those practitioners and consultants who are less than skilled at item-writing and item-response theory, particularly given the various types of common mistakes made by novices when developing survey questions.” (Church, 2000, pp. 100-101).

If you don’t have a solid competency model, you’ll want to start there. Your leadership competency model should align with your organization’s business strategy, culture and values. Keep in mind that a good competency model should be designed to move the organization toward its goals, and therefore should delineate not only what is needed for successful performance today, but also what will be needed for the organization to reach its long term strategic goals. As business strategies change, competency models should be updated to reflect those changes (Schippmann, 1999; Spencer & Spencer, 1993).

Most organizations find it’s best to hire an expert to build their competency model as well as their 360° feedback content. Look for someone with an advanced degree (Ph.D. or master’s degree) in Industrial and Organizational Psychology or a related field and experience building competency models. Typically this individual will spend some time learning about your business, strategy and objectives. Interviews with key executives to understand current and future challenges for the business are critical. A good consultant will integrate what they learn from your executive team with current research on leadership to create a model most effective for the organization.

The 360° feedback content should contain items which are behavioral (rather than trait based), observable, and appropriate for the target audience. It’s important that content is tested to ensure that participants have an opportunity to perform the behaviors that are included in the survey. Because expectations vary across organizational levels, most organizations find it appropriate to create different survey content for different levels within the organization. For example, executive leaders would have a different set of behaviors than would front line managers.

Most organizations use an expert to create their 360° feedback content. It’s imperative that the tool be well designed. In fact, research by Fletcher, Baldry, & Cunningham-Snell (1998) suggests that surveys that are not psychometrically sound often deliver feedback that is confusing and/or misleading. Again, look for someone with an advanced degree in Industrial and Organizational Psychology to help you design a survey that will work for your organization.

Optimizing survey length.  One of the most common questions that we receive when helping an organization develop survey content is, “How long should our survey be?” The answer to this is always – it depends. As a general rule, a 40 to 60 item survey will typically take less than 7 minutes to complete. We find this is about right for most surveys. But the correct length depends on how the survey will be used.

In general, the more frequently the survey will be used, the shorter it should be. For surveys that will be used for in-depth feedback among a smaller, more select group of individuals, a longer survey would be appropriate. For example, it’s not uncommon to see surveys containing more than 150 items used as part of an assessment center experience or a special orientation at the beginning of or upon completion of a ‘stretch assignment’ for high potential employees. On the flip side, instruments implemented on a wider scale basis – such as a tool used annually for by all organizational managers – will need to be much shorter – perhaps only 30 to 40 items.   

Rater anonymity.  Rater anonymity is one of the most important factors influencing participation as well as accuracy of ratings. There are several steps that companies can take to ensure participating raters that their ratings will be anonymous. Using an external vendor can help to alleviate concern.

Research shows that direct reports tend to be much more lenient in their ratings of their supervisors than are other raters. This could be because mangers tend to focus much of their time and energy on working directly with their employees. However, in some cases, direct reports may be afraid of retaliation that may result from giving their boss a low rating. This is one more reason why communication about anonymity as well as about how the data will be used is so important. Employees who understand 1) what steps will be taken to protect their anonymity and 2) that the data will be used for development purposes only are more likely to provide accurate ratings that are not inflated.

It’s important to be explicit in guidelines for appropriate follow-up. Sometimes the best intentions turn into very inappropriate behavior. While leading a training program, we once had a manager approach us with concerns about the accuracy of the report. He had a “friend” who had received lower ratings than he expected on a few of the items. This friend had approached each of his direct reports to try to find out who had provided the low ratings. All employees denied providing low ratings. So the manager called the survey  vendor. Because the vendor (appropriately) refused to provide information about who it was that gave the low ratings, this 360° feedback participant concluded the ratings were in fact incorrect. Somehow the reason for anonymity escaped this individual – who was unable to take the ratings as well intentioned feedback, and instead went on a “witch hunt.”

Working through resistance.  Feedback is hard. And it’s important not to lose sight of that fact as you implement the process throughout your organization.

Participants need to trust the process, believe that results are confidential, understand that the feedback will be used for development – not for evaluation – in order for them to embrace the feedback and use it to drive change.

The nature of the feedback received can also have a huge impact on what managers do with their results. For example, in an article published in the Journal of Applied Psychology, Smither and Walker (2004) found that the number and focus of comments received as part of a 360° feedback process influenced motivation and likelihood to change. Behavioral and/or task based comments were much more likely to be acted on than comments that focused on traits and/or more permanent characteristics. Number of comments also made a difference – with more limited numbers of comments being more likely to encourage change. 360° feedback participants who received a small number of behaviorally or task-focused comments improved more, while participants receiving a large number of behavior/task-focused comments actually showed decreases in performance.

Perceptions of rating accuracy are also important. Rating accuracy depends on a number of factors
including familiarity of the raters to the participants, quality of the survey items, and communication about the survey purpose to raters as well as participants. It’s important that 360° feedback participants select their own raters – people they trust enough to ask for honest feedback. It’s also important that raters understand the purpose of the feedback and feel confident that their ratings will remain anonymous and that there will be no retaliation for negative ratings.

Keep in mind that the process may be designed perfectly, but inappropriate use of the tool can derail trust and future effectiveness of the process. When 360° feedback is used as a hammer or when trust in anonymity is compromised, the most perfectly designed program can become virtually useless. A manager once told me that he felt so uncomfortable with the constant questioning that always followed the feedback process, that he now only provided high ratings. This reduced the chances that he would be cornered with follow up questions about the results. But it also decreased the usefulness of the process.

Avoiding a bottom up approach to action planning.  When managers receive feedback reports without appropriate training, too often they focus on their weaknesses first. Just like a school child receiving his report card, a natural tendency is to look to the bottom scores first. What were the competencies and/or behaviors that were rated lowest? Too often managers pull those items that they scored the lowest on and create action plans around those lowest scoring items. These are – after all – the areas that need to be “fixed” – right?

Yet research shows that this weaknesses-based approach to development may actually decrease rather than improve performance. Effective development requires a more thoughtful approach. 360° feedback participants should be interpreting their feedback in terms of their long and short term career goals. .

• Thinking first about the feedback itself, participants should reflect on who rated them, their relationship with their raters and the reasons they asked these raters for feedback.  Are there specific types of interactions or projects that are most difficult?  If so, how do these impact the ratings?
• Next, participants should think about their current job and the challenges they are facing in that job.  How does the feedback inform those challenges?  How will the strengths identified help them accomplish their current work objectives?  How might issues raised in the more negative feedback impact their ability to accomplish their goals?
• Long term goals should also be taken into account.  What are the next roles that participants are hoping to obtain?  Where do the participants hope to take their careers over the next five to ten years?  How will the skills required change when compared with their current position?  Will their strengths become more or less important?  How about their weaknesses?
• Finally, participants should think about the feedback in terms of their understanding of their own more permanent strengths and weaknesses.  Are the areas in need of improvement behaviors that are easy to change?  Habits that will change with some effort? Or are they more permanent trait-based issues?  This will help participants understand where it makes sense to focus development efforts and where it makes sense to rely on others through delegation and/or partnerships.

Most organizations find that enabling managers to work with a trained and qualified coach is the most efficient way to help managers focus their development efforts appropriately. Coaching sessions may be scheduled over the phone – making them much more efficient to carry out, and need not be long-term contracts. We find that a 45-minute telephone session with an external coach scheduled to coincide with report delivery can go a long way toward jumpstarting an effective development plan.

A follow up call with the coach, participant, and participant’s manager may be helpful in some situations. In addition, research shows that training before and after survey administration leads to more effective development among 360° feedback participants (Hazucha, Hezlett, & Schneider, 1993; Wilson, O’Hare, & Shipper, 1990).

Preparing the organization for feedback.  For 360° feedback to be effective, it’s important that it’s part of a larger program. As mentioned earlier, this tool implemented carelessly can do more harm than good. Companywide training is often the method chosen to clearly communicate the purpose of the program. How will the results be used? How will individuals be expected to use this as input into their development? Will the participant’s manager receive a copy of the feedback report? Who else will have access to the results? Will the feedback in some way inform performance appraisals?

This type of training can be important to ensuring that the process will be used appropriately. For example, we know of one company that very consciously chose to provide reports only to individual participants. However, when the process was rolled out to the field, mangers asked their direct reports to bring a copy of their 360° feedback results to their next one-on-one meeting. It is highly likely that most managers requesting this did not believe that they were doing anything wrong. However, the communication was not adequately disseminated informing participants or their managers of the appropriate way to use the reports. Keep in mind that inconsistencies between what raters and participants are told and what actually happens with the results not only decreases trust in the process, but violates ethical guidelines for this type of process.

Increasing rater accuracy.  Research shows that the accuracy and utility of feedback depend on a number of factors. For example, ratings tend to be less inflated – and more accurate – when the rater believes that the participant truly wants honest feedback, when only the participant will receive a copy of the feedback report, and when raters believe their ratings will be anonymous.

Organizations can do all the right things in terms of 360-degree feedback administration, but if raters don’t understand and/or believe these things to be true, ratings will not be accurate. For this reason, rater and participant training is critical to successful implementation. Pre-survey kickoff training should include the following components:

• Program purpose.  What is the primary purpose of the 360-degree feedback program?  Is it purely developmental?  Who will see the results/report/data?  And how will the information be used?  Be careful what you promise and be sure you can follow through to enforce what you promise.  Trust in the system is the most important factors in success in the program.
• Protection of anonymity.  Raters will need to know how their anonymity will be protected.  What is the minimum level of raters required before scores will be reported?  Typically this is 3.  Who will have access to the raw data?  Is the survey administered by a third party or internally?  Who has permission to contact that party to request additional information?
• Survey process.  What steps can participants and raters expect throughout the process?  What are the timelines for the process?  How will the administration process guard anonymity of raters and confidentiality of survey results?
• Survey content.  What behaviors will be included in the survey?  And how should raters use the rating scale? How does the survey content align with other organizational processes, including long term organizational objectives and individual performance expectations?
• Follow up.  What can raters expect from participants in terms of follow up? What should they do if a participant acts inappropriately with regards to the survey process (for example, if they try to identify who rated what, if they try to influence ratings via intimidation, etc.)?  Is there an ethics hotline that they can call that will be prepared to handle these types of concerns?  Or should they contact their HR generalist?  Or the survey help desk?
• Rater selection.  Participants (and raters) should be provided with some guidance regarding who they select as raters.  Research shows that raters who have known the participant from 1 – 3 years tend to be the most accurate in their ratings.  The number of raters is also important.  Too few and the data is not stable or robust.  Yet if the participant extends the rater list to include people they do not know well, the accuracy and relevance of the feedback declines.  For most people, about 5 raters for each perspective (peer, direct report, etc) seems to be about right.  
• Content and rating scale.  The best training for raters includes a comprehensive description of the survey content, including how the behaviors included support the goals for the organization.  It should also include practice exercises designed to calibrate raters and build consistency around rating individual performance.

Providing support for development.  Once raters have provided their input and reports are generated – now what?  Raters have provided their input and reports are generated – now what? After the survey is over it’s important to provide support so that participating managers use the data appropriately. Research by Facteau, et al (1998) and by Hazucha et al (1993) indicate that a supportive climate for behavior change is critical to facilitate development among 360° feedback participants. Our own research also supports this finding. In research with managers participating in a 360° feedback program with administrations spaced 6-months apart, we found the greatest improvement in scores among participants who stated that their manager was supportive of the process (see graph below).

Be patient.  Don’t expect huge changes right away. Change takes time. And even when change happens, it may be difficult for those we work most closely with to recognize that change. We tend to see each other and ourselves as more constant, more consistent, than we actually are. It’s part of the way we cope with and integrate the massive amounts of information that we receive every day. Research by Smither, et al (1995) suggests that after rating their managers, employees may align future perceptions with those ratings, paying more attention to behavior that is consistent with those already formed perceptions. No wonder change is hard – people tend not to notice even when we do change.

A research study by Walker and Smither (1999) reinforces the idea that change is slow. These researchers saw little improvement in ratings between the first and second years of feedback. However, they found steady improvements from the second through the fourth years of feedback.

For these reasons, many organizations implement pulse surveys or “mini-surveys” to help participants track their progress. These instruments are designed to allow participants to select and receive feedback on the items they have been focusing on as part of their development plan. The survey serves two purposes. The first is to provide feedback to the participant about progress toward their goals. Just like a dieter needs a scale or a general manager needs financial reports, it’s important to provide periodic feedback about progress toward leadership development goals. At the same time, the mini-survey serves as a communication tool to direct reports and coworkers. “Remember – I’m working on this – how am I doing?” Use of pulse surveys can greatly impact successful behavior change among managers.

 About the Questar:

For 25 years, Questar has been partnering with companies to help them understand what makes their customers and employees tick—and help them achieve even greater levels of success. Our clients have used our techonolgy to power their human capital solutions across 25 countries and in 40 languages. Questar has collected over 150 million surveys that have helped organiztions improve the way they 
operate.

To learn more about Questar visit www.questarweb.com or call 
Steve Mars, Vice President, at 651.688.1906.

References:

Church, A. H. (2000). Do higher performing managers actually receive better ratings? A validation of multirater assessment methodology. Consulting Psychology Journal: Practice and Research, 52, 99-116.

Church, A. H., & Waclawski, J. (2001). A five-phase framework for designing a successful multisource feedback system. Consulting Psychology Journal: Practice and Research, 53, 82-95.

Facteau, C. L., Facteau, J. D., Russell, J. E. A., & Poteet, M. L. (1998). Reactions of leaders to 360-degree feedback from subordinates and peers. Leadership Quarterly, 9, 427–448.

Fletcher, C., Baldry, C., & Cunningham-Snell, N. (1998). The psychometric properties of 360 degree feedback: An empirical study and a cautionary tale. International Journal of Selection and Assessment, 6, 19-25. 

Goldsmith, M. & Reiter, M. (2007). What Got You Here Won't Get You There. New York: Hyperion.

Hazucha, J. F., Hezlett, S. A., & Schneider, R. J. (1993). The impact of 360-degree feedback on management skills development. Human Resource Management, 32, 325–351.

Hellervik, L. W., Hazucha, J. F., & Schneider, J. (1992). Behavior change: Models, methods, and a review of the evidence. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 3, pp. 821–895). Palo Alto, CA: Consulting Psychologists Press.

McCall, M.W. Jr. & Lombardo, M.M. (1983). Off the track: Why and how successful executives get derailed. Greenboro, NC: Centre for Creative Leadership.

Phau, B. & Kay, I. (2002).  Does 360-degree feedback negatively affect company performance? HR Magazine, 47, 54–60.  

Schippmann, J.S. (1999) Strategic Job Modelling: Working at the Core of Integrated Human Resources. Mahwah, NJ: Lawrence Erlbaum Associates.

Smither, J. W., London, M., & Reilly, R. R. (2005). Does performance improve following multisource feedback? A theoretical model, meta-analysis, and review of empirical findings. Personnel Psychology, 58, 33-66. 

Smither, J., London, M., Reilly, R., Flautt, R., Vargas, Y., & Kucine, I. (2004). Discussing multisource feedback with raters and performance improvement. Journal of Management Development, 23, 456-468. 

Smither, J. W., & Walker, A. G. (2004). Are the Characteristics of Narrative Comments Related to Improvement in Multirater Feedback Ratings Over Time? Journal of Applied Psychology, 89, 575-581.

Spencer, L.M. & Spencer, S.M (1993).  Competence at Work.  John Wiley and Sons, New
York. 

Walker, A. G., & Smither, J. W. (1999). A five-year study of upward feedback: What managers do with their results matters. Personnel Psychology, 52, 393-423.

Wilson, C. L., O’Hare, D., & Shipper, F. (1990). Task cycle theory: The processes of influence. In K. E. Clark & M. B. Clark (Eds.), Measures of Leadership (pp. 185–204). West Orange, NJ: Leadership Library of America.

 PDF version