Harvard Business Review (print edition): What happens when people draw conclusions from incomplete data? Suppose you are investigating the relationship between corporate performance and a particular risky business practice, such as using cross-functional teams. If you plotted the results of all companies that engaged in this practice, you would find that the more widespread the practice, the more volatile company performance, and your graph would look like this:
On average, as the trend line indicates, any such risky practice is correlated somewhat negatively with performance. But suppose that you looked only at existing companies and excluded all those that had gone out of business while engaged in this practice. Then your graph would look like this (* surprise, surprise *):
Bias, Bias Everywhere:
Many of the popular theories on performance are riddled with selection bias. One of the most enduring ideas in management, for instance, is the notion that successful firms are those that focus most of their resources on the area or technology rather than diversifying. Books such as In Search of Excellence, Built to Last, and Profit from the Core all recommend that managers "stick to their knitting" and "focus on the core." Typically, the research studies behind these books look only at existing companies or--even more narrowly--only at highly successful companies. As a result, their authors overestimate the benefits of focus. Consider, for example, Chris Zook and James Allen's finding in Profit from the Core that 78% of all high-performance firms focused on one set of core activities while only 22% of lower-performance firms did. The study comprised some 1,854 companies, judging high-performance according to share price returns, sales, and profit ratios, but it included only businesses that survived throughout the study period. It did not, therefore, consider any company that started with a focused strategy but then failed. Including those failures would have changed the picture substantially. According to Zook and Allen, 13% of all firms achieved high performance, of which 78% -- or 188 firms -- focused on the core. If in that period just 200 companies with focused strategies that had gone out of business had been included in the sample, then the true relationship between focus and performance would be the precise opposite of the one Zook and Allen infer.
On average, as the trend line indicates, any such risky practice is correlated somewhat negatively with performance. But suppose that you looked only at existing companies and excluded all those that had gone out of business while engaged in this practice. Then your graph would look like this (* surprise, surprise *):
Bias, Bias Everywhere:
Many of the popular theories on performance are riddled with selection bias. One of the most enduring ideas in management, for instance, is the notion that successful firms are those that focus most of their resources on the area or technology rather than diversifying. Books such as In Search of Excellence, Built to Last, and Profit from the Core all recommend that managers "stick to their knitting" and "focus on the core." Typically, the research studies behind these books look only at existing companies or--even more narrowly--only at highly successful companies. As a result, their authors overestimate the benefits of focus. Consider, for example, Chris Zook and James Allen's finding in Profit from the Core that 78% of all high-performance firms focused on one set of core activities while only 22% of lower-performance firms did. The study comprised some 1,854 companies, judging high-performance according to share price returns, sales, and profit ratios, but it included only businesses that survived throughout the study period. It did not, therefore, consider any company that started with a focused strategy but then failed. Including those failures would have changed the picture substantially. According to Zook and Allen, 13% of all firms achieved high performance, of which 78% -- or 188 firms -- focused on the core. If in that period just 200 companies with focused strategies that had gone out of business had been included in the sample, then the true relationship between focus and performance would be the precise opposite of the one Zook and Allen infer.
Mahalanobis - am 2005-05-11 01:43 - Rubrik: mathstat
