The Efficiency Paradox

1

From Mill to Platform:
How the Nineteenth Century Redefined Efficiency and the Twenty-First Has Transformed It

We are living in a second age of efficiency. Journalists and entrepreneurs do not use that word as often as they used to. We’ll see synonyms later. But never far from our minds is consciousness of the value of getting the greatest possible output from available inputs, whether increasing production or profits, or reducing time.

My claim that preoccupation with efficiency in the short term may harm efficiency in the long run risks being considered a heresy by some and a truism by others. I hope to show that it is an obvious proposition when one reflects on it. It is also obvious, as I shall suggest in succeeding chapters and in the Conclusion, that combining efficient algorithms with holistic analog understanding can produce far better results than using either strategy alone. But it is not always simple to defend the obvious. It is helpful to see efficiency as a concept that has developed over the past two hundred years or so, and as a set of practices that are much older. The idea of efficiency, as we shall see, emerged in the age of the steam engine and was best expressed not by the eighteenth-century metaphor of a shop’s division of labor (essential as that remained) but by the substitution of continuous production for the fabrication of one unit at a time. The greatest enterprises invested vast capital and employed up to a hundred thousand workers or more to keep it in operation. Both classical economic liberalism and rival doctrines like Marxism reflected this model; it should not be so surprising that even communist governments admired Western mass production.

The importance of technologies of uninterrupted (“continuous process”) as opposed to batch production was first underscored by the Swiss architect and critic Siegfried Giedion and the American historian Daniel J. Boorstin in the mid-twentieth century. Rollers, belts, and other devices changed the nature of consumption as well as production. Cable television programs like How It’s Made reveal how much of today’s industrial processes are already automated, especially when compared to episodes of the Industry on Parade series broadcast on network television in the 1950s. Today’s programs will probably look equally quaint in even less time. But further reducing labor costs on assembly lines is not the kind of efficiency that interests us here. It is a new kind of enterprise that has—unforeseen by even the boldest futurists—taken over what Vladimir Lenin called the commanding heights of the economy, dominating its agendas. “Silicon Valley” evokes the mixture of admiration, fear, and scorn once inspired by the grimy industrial metropolises of the Northeast and Midwest, but while an approach to Chicago or Detroit or Pittsburgh by automobile or train can still be a visually striking experience, nothing on the peninsula south of San Francisco is tall enough to inspire awe, stupendous as its wealth has become. The server complexes of its companies are scattered as inconspicuously as possible around the globe. Yet the giants of Silicon Valley have ideas about social organization as radical in their own way as Lenin’s, and they share with classical communism a passionate faith in efficiency.

This chapter will investigate the contrast between continuous process efficiency (which fascinated painters as well as photographers and filmmakers in its monumentality and awesome concreteness) and platform efficiency, which is far more profitable but concealed and evanescent and that takes a leap of artistic imagination to dramatize. It will suggest how matchmaking by electronic networks takes advantagenot only of the steady if recently slowed improvement of the efficiency of integrated circuits, but also of the ability of ingenious computational techniques—algorithms—to multiply the speed of these circuits manyfold.

This efficiency raises a profound question, the chapter will argue. Why have these platforms apparently had such little effect on the self-perceived satisfaction of the United States and other nations in which they are most advanced? Why are citizens around the world so unhappy with their governments, so ready to look to extreme solutions? One reason may be that the platform revolution has been diverting talent and capital from other technological projects that could be more transformative. I cannot identify them, nor rule out that they are already well advanced and may flower soon. After all, the U.S. boom after the Second World War was in part based on innovations like broadcast television and dry photocopying that actually were under development during the darkest years of the Great Depression, along with Alan Turing’s theoretical work that helped make the platform economy possible.

The question, which I don’t pretend to resolve, is why the platform corporation, so profitable for its investors so far (especially the early ones), has been such an underachiever. Enthusiasts will insist that major innovations commonly have troughs of disappointment; the best is yet to come. This is especially the viewpoint of Facebook and its founder, Mark Zuckerberg, who in early 2017 published a manifesto acknowledging mistakes and vowing to build better communities and a better planet with the help of Facebook’s users. To many adversaries such promises have long been “silicon snake oil” and “future hype”—to quote the titles of 1990s and early 2000s books by disillusioned technologists. To critics on the left in particular, the new bosses are not so different from the old bosses, just equipped with state-of-the-art surveillance and manipulation in place of the goon squads of yore. Some wary journalists see an existential threat to their own profession in declarations like those of Zuckerberg. I am not sure any organization really has such power. I will suggest at the end of this chapter that the most serious unintended consequence of platform efficiency may be its opportunity cost, its claim on resources that would in the long run do more to promote real efficiency.

One paradox of the movement for efficiency is that innovations that have promoted efficiency and rationality have arisen in spite of discouraging data, driven by intuition and emotion. That does not mean that gut feelings alone are a more reliable guide than data-based analysis, but only that data, and tools for analyzing it, never can take the place of the imagination in foreseeing future patterns of human behavior. Most such intuitions fail. The exceptions fill inspirational and business books. Venture investment has a high failure rate built into it. Yet out of the inefficient maelstrom emerged some of the world’s most efficient technology.

***

The history of efficiency should rightly start with nature itself. As biophysicists have discovered, DNA stores energy far more densely than the most advanced technological systems. The control of gene expression allows complex and robust organisms to develop with stunning speed. Tiny variations in the genomes of fruit flies can produce strikingly different behaviors. Evolution has been prodigiously successful in optimizing the flow of information. Leveraging limited resources is our biological heritage.

The quest for efficiency seems to be built into human biology as well, as revealed by anthropological and archaeological evidence. There have been tens of thousands of years of innovations in tool making that sometimes reached dead ends but occasionally produced masterpieces of functionality. Think of the Australian Aborigines’ boomerang, or the Central Asian steppe nomads’ composite bow. Is any cutting tool more efficient than traditionally forged Japanese blades, or sharper than the obsidian knives flaked expertly by pre-Columbian Native Americans?

Turning to the West, many ancient Roman medical instruments were so well adapted to their purpose that similar ones are used today, and their quality was not surpassed until modern times. Roman troops were famous for their ability to assemble bridges and fortifications with a speed that dazzled their adversaries. There was even a kind of mass production of oil lamps, stamped and marketed with early trademarks.

Recent archaeology has revealed more dynamism and technologicalinnovation in the ancient world than historians of fifty years ago acknowledged. The slave economy, for example, did not rule out labor-saving machines like water wheels, just as steam engines were used on slavery-era sugar plantations in the early nineteenth century. There was a great deal of efficiency in practice. But the concept of efficiency as we know it had no clear place in ancient life. The ancient Greeks and Romans (and other Mediterranean and Near Eastern societies, including Egypt), had administrative and recordkeeping systems that worked for centuries. But they had no doctrine of systematic improvement of output. The classical historian Peter Thonemann has underlined that Roman society in particular was based on principles of patronage, loyalty, and obligation. There was no theory of wages, interest, or productivity. Prestige was often more important than functionality. Books were written and read as rolls that were stored together in chests. Writing was scriptura continua, no space between words, which space would have increased papyrus and parchment use slightly but made reading and education far easier. The difficulties of reading—manipulating the scroll, looking ahead to determine word breaks—were part of the performance skills of an educated person. That kind of inefficiency was a feature, not a bug in today’s terms.

Europe of the Middle Ages and the early modern era was a time of growing practical efficiency—but also without an underlying theory. The black letter handwriting that seems so quaint and old-fashioned today was actually a relatively rapid and legible style of writing for those accustomed to it. The Romans had the optical knowledge and the glassblowing and metallurgical skills to make eyeglasses, but there was no market for them. Aging literate people had educated slaves to read to them. The Romans made excellent cloth presses (one of which survives at Herculaneum) and could cast bronze letters, but they felt no need for printing.

By the eighteenth century, Denis Diderot’s Encyclopédie and its Scots imitator, the Encyclopaedia Britannica, summarized the knowledge and improvement in dozens of trades. In The Wealth of Nations, Adam Smith showed how the separation of the making of pins into distinct operations by specialists could multiply the number of pins each worker could make per day. There was an even finer division of labor in the manufacture of needles in medieval Persia.

Still, Smith was an exceptional pioneer. The nineteenth-and twentieth-century sense of efficiency was not quite present. A nineteenth-century political economist, whether laissez-faire or socialist, would be deeply interested in measuring just how much more productive a pin workshop would be than a traditional one. Many products were still made according to artisanal tradition and style rather than after systematic study of customer needs. The French technology theorist Jacques Ellul has pointed out that the armorers who made swords for late medieval mercenaries each followed a craft tradition and decorative style without studying the ergonomics of combat. Every soldier had to adapt his fighting style to the instrument.

No eighteenth-century figure was more celebrated than Benjamin Franklin for his union of practical ingenuity with investigations of scientific theory, despite or because of the limits of his formal education. The designs of Franklin and his contemporaries—he never patented his inventions and encouraged further adaptation—for fireplace linings significantly improved the efficiency of wasteful conventional fireplaces. But late-eighteenth-century inventors still had no scientific way to quantify savings in heat produced per unit of wood. Only in the mid-nineteenth century did thinkers like the brewer and scientist James Joule develop consistent units to measure heat production: the British thermal unit and the SI (metric) Joule.

The two inventions that introduced modern efficiency were the work of other geniuses of the early nineteenth century, now known mainly to specialists: the millwright Oliver Evans and the paper manufacturer Henry Fourdrinier. If we look at plates of Diderot’s encyclopedia as edited by Charles Gillispie, many of the workshops were not so different from those of the ages of Leonardo da Vinci or Galileo. Masters, assisted by journeymen and apprentices, made each product, though Smith’s principle of the division of labor was beginning to spread. Goods were still fashioned individually or in small batches.

Oliver Evans was the founder of continuous process efficiency. He is less well known than Franklin, Eli Whitney, Samuel Morse, or Thomas Edison, but for two centuries he was at least as influential as any of these. As Siegfried Giedion wrote in his classic Mechanization Takes Command, before there was any real American industry, “a solitary and prophetic mind set about devising a system wherein mechanical conveyance from one operation to another might eliminate the labor of human hands.” Grain was raised to the top of the mill by a chain of buckets and conveyed by gravity through each of the stages of milling with belts, screws, and other continuous conveyances. Individually these were not entirely new; some had existed since antiquity. The idea of an integrated system that processed raw materialsand semifinished products was still a breathtaking step in efficiency. Evans’s system seemed shaky and he lacked Franklin’s persuasive powers, but in “the power of his vision,” Giedion rightly concluded, “Oliver Evans’ invention opens a new chapter in the history of mankind.”

The second of the landmarks of classic modern efficiency was the Fourdrinier paper mill. Ever since its introduction in China, and to this day in the production of Japanese artisanal papers like washi, paper was made from fibers in individual sheets. Papermakers were highly skilled workers who formed powerful guilds; books and newspapers were still costly. A French printer named Nicolas-Louis Robert was the first to understand the potential of continuous paper production. As the historian Mark Kurlansky has pointed out, Robert’s invention of a wire framework used the principle of today’s conveyor belts, but actually preceded their invention. (The first use wasby the Royal Navy for the manufacture of ship’s biscuits in 1804.) In his machine, a moving screen received the wet fibers and agitated the pulp laterally to distribute it evenly, as sheet paper artisans did. After the water was removed, the semifinished paper was rolled on to a series of drums, the final ones heated, for drying. The paper manufacturers Henry and David Fourdrinier made technical improvements in the Robert process, but not enough to make it practical, and they were forced to declare bankruptcy. It was the engineer Bryan Donkin who finally made usable continuous papermaking machines on the basis of Robert’s idea. This complex parentage reveals an important feature of continuous process efficiency: even more than other innovations, it is the drawn-out result of failure, collaboration, and competition.