Benchmarking Competitive Performance

Today’s world is awash in data, which has the potential to dramatically impact the way we make decisions. Consider, for example, the world of competitive sports; whereas a generation ago the simple win-loss record may have been the primary focus, the rise of detailed metrics and statistics has revolutionized the way we evaluate the competitive performance of athletes and teams. Performance data has changed the game, so to speak. When it comes to evaluating competitive athletic performance, we have a better understanding of what to pay attention to.

What about business? How do managers and organizations benchmark competitive performance in the business world? What competitive metrics influence the decision-making of corporate leaders?

Darden Professor Jared D. Harris’ research took a close look at the underpinnings of that question to determine what kinds of performance feedback managers pay attention to and how they pay attention to it.

“Based on some classic work in behavioral theory, we already have a broad sense of how organizations respond to signals about competitive performance. But the findings of this study add some nuance and give us a subtler understanding of how organizations respond to performance feedback,” he says.

The findings, in a paper written with Philip Bromiley of the Paul Merage School of Business at the University of California, Irvine, were published in a recent issue of Strategic Management Journal.

The study takes as its jumping-off point the idea that managers have traditionally benchmarked their company’s performance with respect to two different reference points: self-referential performance — checking how this year compared to last year — and a comparison to competitor performance.

The three dominant models often employed to describe how managers evaluate performance have many similarities, but it’s the details that differ. One model argues that managers first pay attention to industry benchmarks, and only switch attention to their own firm’s past performance once they are performing above the industry average. Another model — the one most closely associated with traditional theory — suggests that managers constantly “average out” industry and self-referent benchmarks into one aggregate performance target. A third model suggests that self-referent and socially-referent benchmarks are always relevant to managers, but have independent influence.

Harris wanted to see which of these models most accurately described what managers actually do. “All three models typically predict the various outcomes in a statistically meaningful way,” Harris says. “But we wanted to know which model most accurately represented the way managers pay attention to performance feedback. So we ran a horserace between these three well-known models of organizational benchmarking. How do organizations really evaluate their own performance? That was the question.”

To find the answer, Harris used public data on thousands of firms to run the comparison. This resulted in a comprehensive comparison of different benchmarking models, different performance measures and different corporate outcomes, using a common set of data: the first study of its kind.

Surprisingly, the study found that the “averaging” model was least effective at explaining corporate action, though it’s the model most strongly associated with traditional organizational theory. And the winner of the horserace? The model in which social and self-referent benchmarks have independent influence. Which, according to Harris, makes sense: “Individually speaking, this is, of course, how we tend to live our lives; we have different performance targets for different aspects of our lives, and we are constantly juggling. This study shows that organizations do it, too.”

A second surprise was in store. Though it wasn’t the primary objective of the study, the research also shed some light on a long debated question of which performance metric provides the “best” measure of a firm’s performance. In the academic field of business strategy, more and more focus has been placed on sophisticated, unbiased performance measures. Would these complicated metrics prove more influential? Surprisingly, the study found otherwise.

Harris found that “raw, unscaled income was the most influential performance measure in the way these managers evaluate their performance. It isn’t the most accurate measure of a firm’s true performance, of course, but the study shows that managers don’t pay as much attention to that more sophisticated stuff. They look at the net income and say, let’s up our game. From a research standpoint, it’s surprising the managers favored net income, because it’s the worst, noisiest performance measure.” But it’s the metric that was most influential from a behavioral standpoint. The choice, in part, may be encouraged by external pressure from stock analysts and the press.

“The study was about better understanding how managers and organizations respond to and interpret performance feedback, and what we found is that managers think about different benchmarks independent of each other. In addition, the analysis demonstrated that managers tend to focus on simpler, blunter measures of performance when benchmarking,” Harris says. “That’s just what people in organizations do.”

“Managers are capable of striving for multiple criteria,” Harris continues. “The findings open the door to thinking more carefully about how organizations actually set targets and evaluate their performance, specifically with respect to multiple decision criteria. Better theoretical models of organizational decision-making can lead to better organizational decision-making.” In other words, how we keep score can influence which play we might call next.

Jared D. Harris co-authored “A Comparison of Alternative Measures of Organizational Aspirations,” which appeared in Strategic Management Journal, with Philip Bromiley of the Paul Merage School of Business at the University of California, Irvine.