The Innovator's Dilemma and the Future of Silicon Valley

Junfu ZHANG

Perspectives, Vol. 3, No. 1

"Throughout the decades, …, the Valley has constantly remade itself just in time for the Next Big Thing…. Silicon Valley has shown time and again that it can reinvent itself-a quality that will be key to its survival."
-- Business Week, Aug 18-25, 1997

A. Introduction

Silicon Valley, the center of the high-tech industry, has become the biggest industrial cluster in the U.S., followed by the banking industry on Wall Street, the automobile industry in Detroit, and the entertainment cluster in Hollywood. As a successful model, Silicon Valley faces competitors and imitators from all over the U.S., from Washington, DC to Seattle, Washington, from Austin, Texas to Boston, Massachusetts. Can the Valley keep its leading position in the future?

More than anywhere else, the economy in Silicon Valley is driven by continuous innovations. Firms in the Valley compete fiercely by introducing innovations rather than cutting prices. It has been recognized that Silicon Valley has the ability to reinvent itself over time. This ability is not only an important determinant in the Valley's past success but also the key to whether it will remain a big success in the future.

By reinvention, we mean that whenever a big technological breakthrough happens, Silicon Valley can always "catch the wave" and reorganize itself around the new technology. In the 1960s, the Valley hosted a big collection of firms in the semi-conductor industry; from the late 1970s to the 1980s, the Valley transformed itself into a major provider of hardware and software in the course of the PC revolution; in the 1990s, the Valley became a leading player in the Internet economy. What's the "Next Big Thing?" Can Silicon Valley catch it as it always did?

Professor Christensen (1997) at Harvard B-school wrote a national bestseller that popularized his concept of "innovator's dilemma." The book investigates why successful big companies are often defeated by new comers and lose their market dominance. While it is clear that the author is addressing to managers in successful companies, regional economists can learn a lot from his insightful analysis.

Following Christensen (1997), we emphasize the distinction between sustaining innovations and disruptive innovations in this article. Disruptive innovations refer to those big technological or organizational breakthroughs that revolutionize the business in a big market or the whole industry. A region reluctant to accommodate disruptive innovations is more likely to lag behind in Schumpeterian competition (competition by innovation). We further argue that big successful firms face the "innovator's dilemma": their success in the existing market tends to prevent them from implementing or adopting disruptive innovations. Startup firms targeting at a niche or an emerging market are most likely to adopt and promote disruptive innovations. We believe that Silicon Valley was able to reinvent itself in the past mainly because it provided a relatively favorable environment for the formation of new firms. Likewise, the future of Silicon Valley will be determined by its birth rate of new firms. The higher rate at which the Valley generates new firms, the more likely it will catch the next wave and reinvent itself around the next big thing. Policy implications of this argument are presented at the end.

B. Sustaining Innovations vs. Disruptive Innovations

An innovation refers to a new product, a new technology to produce a product, a new way to distribute products, a new way to provide services, or any combination of those new things. By this definition, innovations include both technological advances and organizational improvements.

Bower and Christensen (1995) and Christensen (1997) distinguish between sustaining and disruptive technological changes. Although they limit their discussion to "technological" innovations, they define "technology" so broadly that we feel it is possible and appropriate to extend this argument to organizational innovations.

Sustaining innovations add incremental improvement to the performance of a product or an organization. For example, Microsoft introduced Windows 98 to improve upon Windows 95 and replaced Word 97 with a better Word 2000. Both Windows 98 and Word 2000 embody some sustaining innovations. When the marketing group of a company introduces a new statistical model to better utilize information about their customers, it is a sustaining innovation. When Barnes & Noble finds a cost-saving channel to ship books, it is another sustaining innovation. In fact, sustaining innovations take place every day throughout the economy.

Disruptive innovations emerge as a completely new product that brings new dimensions to the existing product, or as a completely new way to produce or distribute existing products, or as a completely new way to provide services. Usually there is no existing market awaiting disruptive innovations. In the short run, disruptive innovations bring little profit or perhaps even lose money. Professor Christensen's favorite example is the evolution of the hard disk drive from 14 inches diameter to 8 inches, and then to 5.25 inches, and finally to 3.5 inches, each step representing a disruptive innovation. "Bigger" examples are the introduction of personal computers by Apple and many technologies coming along with the Internet, which are so disruptive that they have changed the whole high-tech industry. Amazon.com sets a specific example by introducing a new way to sell books and music, which is completely different from what Barnes & Noble was doing traditionally.

Sustaining innovations take place frequently; disruptive innovations emerge occasionally. Sustaining innovations push forward along a continuous path; disruptive innovations make a quantum jump. All firms welcome sustaining innovations, because those innovations tend to bring more profit; successful big companies often turn away from disruptive innovations, because those innovations are often in conflict with their current priorities. As Christensen (1997) points out, most successful companies failed when they confronted with disruptive innovations. Likewise, disruptive innovations pose big challenges to a successful high-tech industrial center like Silicon Valley. Silicon Valley could fail if it misses the next big thing, which, by definition, must be a big disruptive innovation.

C. The Innovator's Dilemma

Christensen (1997) writes about the failure of companies to stay atop their industries when they confront certain types of market and technological changes. As the author emphasizes, "[the book is] not about the failure of simply any company, but of good companies - the kinds that many managers have admired and tried to emulate, the companies known for their abilities to innovate and execute." His conclusion is that successful companies often fail because of the very management practices that have allowed them to become industry leaders. Those practices make it extremely difficult for them to develop or adopt the disruptive technologies that ultimately steal away their markets. It is a dilemma because companies fail for the same reason they succeeded. This is, in spirit, similar to Schumpeter's famous thesis that capitalism will fail because of its success. While there has not been any concrete case to prove Schumpeter's theory, numerous failures of great companies have exemplified Christensen's dilemma.

Take the computer industry as an example. IBM once dominated the mainframe market but lagged behind for years in the minicomputer market, although the latter is technologically simpler than mainframes. Digital Equipment Corporation (DEC) pioneered in the minicomputer market, closely followed by Data General, Hewlett-Packard, Nixdorf, Prime and Wang. However, each of those missed the emergence of the desktop personal computer market. In this case, it was another new comer Apple Computer that took the lead. When Apple brought its portable PC to the market, however, it was already six years behind Compaq. Similarly, the workstation market was created by some other rookie players at the time, namely, Apollo, Silicon Graphics and Sun.

A similar story is found in the hard-disk-drive industry, the example that Christensen has referred to again and again (Christensen, 1997). In that industry, "no single disk-drive manufacturer has been able to dominate the industry for more than a few years. A series of companies have entered the business and risen to prominence, only to be toppled by newcomers who pursued technologies that at first did not meet the needs of mainstream customers. As a result, not one of the independent disk-drive companies that existed in 1976 survives today." (Bower and Christensen, 1995)

Why do disruptive technologies cause great firms to fail? Christensen (1997) argues that great companies are managed in the way that makes them excellent at developing or adopting sustaining innovations and hence succeed. However, the same set of practices make them miss disruptive innovations and hence fail.

1. Successful firms listen to their customers and invest aggressively in technologies that give those customers what they say they want. This helps those firms to attain their market dominance. However, at the same time, this practice prevents them from getting the right information about disruptive innovations. A firm's current customers will naturally demand a product that performs better than the one they are buying. A disruptive technology usually represents a very different product that does not provide better performance but only add more dimensions to the existing product. For example, laptop notebooks are not as powerful as desktop PCs; a 3.5-inch disk drive does not have more capacity than a 5.25-inch disk drive. For this reason, firms trying hard to serve their current customers fail to see the importance of disruptive innovations.

2. Successful firms seek high margins and target at large markets rather than small ones. However, disruptive innovations usually fit into a niche market or a market that does not exist at all for the time being. Moreover, disruptive innovations, although have a bright future, usually bring little or no profit in the short term. When Steve Jobs and Steve Wozniak brought forward their Apple I to the market in 1976, only 200 units were sold to hobbyists and few people took it seriously. They were lucky enough for not losing money, not to mention collecting big profits. Inevitably, a giant like IBM with an annual profit of millions would and should ignore it at the early stage until the potential of PC was fully recognized. In fact, IBM's stand-alone PC division later did successfully grab a substantial piece of the pie in the emerging PC market, which is rather an exception in the world of Innovator's dilemma. It is worth noting that many great firms missed disruptive innovations not because they did not have the technology, but because they were too eager to seek something big. Seagate Technology once was a great success in disk drive industry, whose revenue grew to more than $700 million dollars in six years since its inception. It was the pioneer and a big player in the 5-inch hard-disk-drive market. Seagate had developed their own 3.5-inch disk drive but had chosen to put it on the shelf, because it could not bring the big profit they expected from a new product. In the end, Seagate became only a minor supplier of 3.5-inch disk drive when that market boomed.

Christensen may have been indulging himself too much in the paradoxical observation that great firms fail for the reasons they succeeded. He neglects some inherent problems with big firms that may be crucial in accounting for their failures to catch disruptive innovations. These include:

3. Disruptive innovations usually bring a new product that will compete with the current well-marketed product, or a new organization that will turn the existing corporate structure upside down. Over the past two decades, perhaps there has been no high-tech firm as successful as Microsoft. It is a common knowledge that Microsoft's recent success is built around its dominant operating system Windows. Is Microsoft an innovative firm? Sure, it is. But all their innovations are sustaining in the sense that they enhance Microsoft's Windows and its software based on it. Any platform-independent technology such as Internet protocols and Java software protocols will be disruptive to Microsoft. If Microsoft develops those technologies, it is committing suicide and choosing to be reborn. It is Microsoft's right rather than wrong decisions to show little enthusiasm to Internet at its early stage and to fight against Sun Microsystems for its Java. Microsoft's huge market share prohibits itself from developing disruptive technologies and competing with itself. Its destiny is to be dethroned by other firms with disruptive innovations.

The telecommunication industry tells a long story about AT&T's hostility to disruptive innovations. No one has doubted that AT&T, in its early years, was doing a great job of improving the efficiency and effectiveness of the Bell system, which transfers voice communication over long distances through copper lines. Research on radar during World War II made breakthroughs in microwave transmission. To AT&T's Bell system, the microwave technology is a radical and more efficient alternative that employs tall towers with antennas to relay microwave messages. Although AT&T was working on its own microwave system, it took its time in spreading the technology over the mass market. At the same time, AT&T lobbied Federal Communications Commission (FCC) to keep other microwave innovators out of the transmission market. It took a new comer Microwave Communications, Inc. (better known as MCI now) a long fight against AT&T to finally get FCC's authorization to provide long distance service in 1971. Yet history repeated itself when there came the cellular phone. The birth of the cell phone technology traces back to AT&T's Bell Labs, but AT&T again failed to make the cellular business. It was Seattle's Craig McCaw who created America's first nationwide commercial wireless network. Ironically, AT&T bought McCaw's network in 1993, which becomes today's AT&T Wireless (Norton, 2001).

Another example is that Barnes & Noble failed to pioneer in on-line books and music retailing. Again, it is a "right" decision, because if Barnes & Noble had opened that market, it would be competing with its own physical stores. Nowadays both Barnes & Noble and Borders have their own on-line store, which is only a defensive measure against Amazon.com. One feature of the high-tech industry is that the first mover usually enjoys a big advantage over followers. Barnes & Noble has spent a lot of money advertising its own on-line bookstore, but is still trailing Amazon from far behind.

4. Another reason that great firms often missed disruptive innovations is that it is often too risky to pursue those innovations. The triumph of disruptive innovations usually hinges on an emerging market. It is easy to recognize an emerging market ex post, but not ex ante. In the late 1970s, who could anticipate the prevalence of personal computers today? A disruptive innovation, never tested on the market, has a much higher chance of ending up as a failed project. Even legendary venture capitalists in Silicon Valley have to live with the cruel reality that one in every three of their investments produces a total write-off. Few established firms are willing to face dead programs so frequently. So they choose to avoid such projects. For this reason, new startups are most suitable for experimenting disruptive innovations, because they are usually backed by venture capitals or banks, institutions that have better ways to neutralize risks. Think about it, the successes of Netscape and Yahoo had covered how much loss incurred by hundreds of dead startups in Silicon Valley?

D. The Importance of New Firms

After building a failure framework to explain the dilemma, Christensen (1997) spends the rest of his book advising firms on how to avoid the dilemma. Christensen emphasizes the aspect of the dilemma that great firms fail for the same practices leading them to success. Therefore he sees the dilemma as curable: firms may get around it by managing disruptive innovations properly. I see it differently. I understand the dilemma mainly as inherent problems in big firms, which cause them to miss disruptive innovations due to their right (not wrong) decisions. In this sense, I stress the role of small startups in the development of an innovation-driven regional economy such as Silicon Valley.

Small startups face no problems that constitute an innovator's dilemma. A startup has no footing on any market, existing or emerging. In many cases, the whole purpose of a new firm is to target at a niche market or create a new market. Because of its small size, a startup feels happy for even a small profit. Startups, especially in the high-tech industry, usually start with a completely new product, because there is no need for another firm to provide mature products like Microsoft's Windows. For this reason, a startup is usually tied with a disruptive innovation from the very beginning. Startups have established nearly nothing, hence they have no self-competing problems. Startups are usually backed by risk-sharing financial institutions, so they can live with the high risk associated with disruptive innovations.

Professor Christensen sees the advantage of small startups clearly. One of the suggestions he gives to managers in successful big firms is to "set up a separate organization small enough to get excited by small gains." Christensen (1997) obviously has also noticed that disruptive innovations are often in conflict with a firm's current establishment, so he suggests that the small organization a firm creates to develop disruptive innovations should be separate and independent of the firm. I wonder whether this is feasible. I don't see how Microsoft would like to set up a small firm to overthrow its own market dominance. It is like to exercise your left arm so that it can be strong enough to tear off your right leg.

Startups adopt disruptive innovations and grow up to challenge successful big firms' dominance. This is observed over and over again in the history of capitalist economies. In fact, this characterizes the exact dynamics of capitalism, which was termed as "creative destruction" by Schumpeter. In the high-tech industry that is driven by innovations, this dynamics takes a faster pace than elsewhere.

A successful firm is going to lose its market dominance if it misses a disruptive innovation. Likewise, a high-tech industrial center like Silicon Valley may lose its leading position if firms in the Valley miss a wave of disruptive innovations. As the innovator's dilemma suggests, established big firms are slow in adopting disruptive innovations, so Silicon Valley should depend on continuous formation of small startups to reinvent itself. High birth rate of startups explains the Valley's success in the past. From the earliest Hewlett-Packard to Fairchild, to Intel and Apple, to Sun, Oracle and Silicon Graphics, to Cisco, Netscape and Yahoo, small startups grew up into successful big firms in Silicon Valley during every wave of disruptive innovations in the high-tech industry. Notice that, the reinvention of Silicon Valley was not a result of big firms like Hewlett-Packard reinventing themselves over and over again (Hewlett-Packard might indeed have done better in reinventing itself than many other giants); rather, it was a result of generations of successful startups. Behind the generations of successes are thousands of failed startups, through which Silicon Valley paid its tuition.

In contrast is the industrial center along Route 128. The Boston area, especially along Route 128, used to be the top high-tech center led by Digital Equipment and Wang. Since they failed to pioneer in the wave of innovations associated with personal computer, the area has lagged far behind Silicon Valley now. The failure of Route 128 to remake itself is different from the fading-out of Digital Equipment and Wang. Route 128 failed because it had no startups to grow up as leaders to replace firms like Digital Equipment and Wang. If this can happen to Route 128, why can't it happen to Silicon Valley?

Silicon Valley's future will be determined by its birth rate of startups. By continuous formation of startups, the Valley will have a bigger chance to create the next big thing and throw other high-tech regions further behind, or at least, to be able to catch the next big thing and not to be overtaken by other regions.

E. Policy Implications for Silicon Valley

There is no doubt that local public policies play an important role in the formation and development of an industrial cluster. As standard economics suggests, local government should focus on providing public goods and facilitating coordination between firms, activities that individual firms have no incentive to do or capability of doing. Due to local government's limited resources, the priorities of its policies have to be well defined. Our analysis here suggests that public policies should be in favor of the creation of startups in order to secure the future of Silicon Valley as the leading high-tech center. What the local government can do include:

1. Help identify and promote the Silicon Valley culture. Since the Gold Rush era, the San Francisco-San Jose area has had a long tradition of entrepreneurial activities. As a consequence, this area perhaps has more legendary entrepreneurs than any other area in the country. The success of Hewlett-Packard and Jobs-Wozniak has been an inspiration for "garage tinkers." The "Traitorous Eight"[*] of Shockley Semiconductor Labs have encouraged many spin-offs from great firms. In addition, the Valley has a unique culture that risk-taking is the norm and the stigma is not for failure but for not trying. Local government can identify and promote this culture by erecting statues, opening museums and historical sites, which will help foster entrepreneurship and encourage entrepreneurial activities.

2. Improve quality of life. Cooper and Folta (2000) report that one of the most important factors determining where a firm is located is where its founder likes to live. The pleasant climate in Silicon Valley gives the area a big advantage in attracting entrepreneurs. Local government should have high-tech people's needs in mind. It should improve public schools and libraries; it should encourage a vibrant, diverse setting for cultural activities, eating and drinking, shopping, and entertainment; it should streamline permitting, planning, and other government services such as fire fighting and garbage collecting. As a result of its recent development, Silicon Valley has established a bad reputation for expensive housing, road congestion and high energy prices. These can scare entrepreneurs away. Local government should try to solve those problems and prepare for the next boom.

3. Foster Entrepreneurship. Cooper and Folta (2000) also report that, at the time when a person starts a firm, he or she seldom moves. This is especially true in the high-tech industry. Entrepreneurs do not move because they feel more familiar with the local environment, because their spouses are holding a job locally, or simply because it is too costly, in time and money, to search for a right location. For this reason, local government should expect local residents to start their firms; it should attract potential entrepreneurs from other places to reside in the Bay area rather than attract people to bring their startups from elsewhere to the Valley. High-tech firms are often founded by engineers who are not necessarily good managers. Local government should sponsor training programs that teach engineers basic knowledge they need to start a firm. It should also support local universities to attract talented people to the Valley.

4. Adapt local regulations and laws. The administrative process for starting a firm should be reduced and simplified to its minimum. Local regulations and especially tax policies should favor high-tech startups.

5. Provide and help seek venture capital. The availability of financial support is the most important determinant of whether and when a person chooses to start a firm. Hellmann and Manju (2000) find that venture capitals are more likely to finance "innovator firms" than "imitator firms." Silicon Valley has been the most successful area in attracting venture capital in the past decade. Retaining this leading position is important for the Valley's future. Depending on political feasibility, local government may consider establishing special funds to invest in startups; it can help organize an "angel investor" network among local people who have an interest in investing some of their wealth in the high-tech industry.

F. Conclusion

Following Schumpeter, we perceive the economy, especially the high-tech industry, as an evolutionary process driven by innovations and entrepreneurship. In light of the "innovator's dilemma," we recognize that successful big firms are excellent at developing and adopting sustaining innovations, but are likely to ignore disruptive innovations. Moreover, disruptive innovations are extremely important to a specialized regional economy because those innovations bring radical and fundamental changes to an industry. We argue that the success of Silicon Valley in the past is achieved by its generations of startups that have not missed any wave of disruptive innovations. We also believe that the future of Silicon Valley hinges on its birth rate of startups and hence suggest policies in favor of the formation of new firms.

It is particularly worth noting that disruptive innovations are hard to identify ex ante. Professor Christensen and his partner launched a mutual fund in 2000. Based on Christensen's theory, they select stocks of companies that are considered as disruptive. The fund, which was called the "Disruptive Growth Fund," was closed before its first birthday with 64% of its value lost. It is a vivid example that nobody but the market decides which technology is able to cause disruption. Nobody can pick winners. If a regional economy such as Silicon Valley wants to win the game on the market, it has to have more players. That is, to encourage the creation of new firms.

Endnotes:

* Eight young engineers resigned from Shockley Semiconductor in 1957 and founded Fairchild. Many of them left Fairchild later again to start other successful firms. Among them are Gordon Moore and Robert Noyce who later co-founded Intel, and Eugene Kleiner who launched the Valley's premier venture-capital firm, Kleiner Perkins Caufield & Byers.

(The author is a research fellow at the Public Policy Institute of California, a non-partisan think tank located in San Francisco. The author appreciates many invaluable comments by an editor of Perspectives.)

References
1. J. L. Bower and C. M. Christensen, 1995. "Disruptive Technologies: Catching the Wave." Harvard Business Review, Jan.-Feb., 43-53.
2. Business Week, 1997. "Silicon Valley: How It Really Works." August 18-25, 64-147.
3. Christensen, C. M., 1997. The Innovator's Dilemma. Cambridge, MA: Harvard Business School Press.
4. Cooper, A and T. Folta, 2000. "Entrepreneurship and High-technology Clusters." In The Blackwell Handbook of Entrepreneurship, edited by D. L. Sexton and H. Landstrom. Malden, MA: Blackwell Business.
5. Hellmann, T. and P. Manju, 2000. "The Interaction between Product Market and Financing Strategy: The Role of Venture Capital." Review of Financial Studies 13(4), 959-84.
6. Norton, R. D., 2001. "The Geography of Radical Entrepreneurship in U.S. Telecommunications After 1960." Unpublished paper draft.
7. Saxenian, A., 1994. Regional Advantage: Culture and Competition in Silicon Valley and Route 128. Cambridge, MA: Harvard University Press.
8. Schumpeter, J. A., 1934. The Theory of Economic Development: An Inquiry into Profits, Capital, Credit, Interest, and the Business Cycle. Cambridge, MA: Harvard University Press.
9. Schumpeter, J. A., 1950. Capitalism, Socialism and Democracy, 3rd ed. New York: Harper & Brothers.