One of the key insights coming out of the progress studies movement seems like a simple idea on the surface, but it’s an important core thesis: that progress is not an inevitability. We don’t see new inventions, innovations, and improvements to quality of life by accident. It’s the result of deliberate effort by people in searching for new life improvements. Using names like “Moore’s Law” perhaps makes it sound like computer chip improvements “just happen,” but researchers at Intel or TSMC would beg to differ on how automatic those developments were.

For at least the last 150 years, steady, expansive progress has been the default. Since the Industrial Revolution, scientific discovery has marched forward, and since the days of the Enlightenment, science and progress have been generally accepted as net benefits to humanity¹.

I think what we see today isn’t so much a reversal on this position, but perhaps a sense of passivity and taking progress for granted. That scientific advancement just happens to us without deliberate effort.

Jason Crawford had some interesting thoughts on this subject on Roots of Progress, first posing a question on where progress comes from:

Do you think the last 200+ years of technological/industrial progress were…

… a trend with substantive causes, that we can expect to continue?

… a fluke, a stroke of luck, not to be repeated?

And why?

(Not a poll because I'm more interested in your reasons than your votes)

— Jason Crawford (@jasoncrawford) September 10, 2020

Most of the arguments in response supporting the second case fell into two categories:

• Failure of imagination — ”I can’t imagine any big breakthroughs, so they must be flukes or strokes of luck” or
• Materialism — ”Progress happens through exploitation of resources, so will peter out as we run out of physical materiel”

Exploring innovation’s inner workings is helpful in understanding the issues with these two arguments.

In his phenomenal book How Innovation Works, Matt Ridley describes innovation as a gradual, bottom-up, evolutionary process of thousands of small steps forward. We tend to look back on the history of progress and point to pillar breakthroughs like Orville and Wilbur at Kitty Hawk, Edison’s lightbulb, Marconi’s radio, or Pasteur’s vaccines, treating each as a Big Bang moment of inspiration that happened in one fell swoop. Ridley tells us this is a flaw in human reasoning; we love narratives and stories, so we spice up the reality of how these inventions came to be. The truth on the ground was much more gradual and dispersed in each of these cases. Hundreds of precursor steps had to happen, proffered by hundreds of different individuals dispersed around the globe. The Big Bang invention story takes as given the source branches lower on the tree that sprouted these successor innovations.

It’s not as if innovators aren’t actively pursuing discovery, that solutions just fell in their lap while sitting in their living rooms. The Wright Brothers knew that they were trying to get a flying machine off the ground. What they didn’t predict, though, was the impact that flight would have on global economics, war, trade, recreation, and every other dimension of modern life. The same is typically true of other instances of progress. From our position in the 21st century, it seems obvious that flight would have massive ramifications for the global order. But this is easy too see in hindsight. As Ridley puts it:

Technology is absurdly predictable in retrospect, wholly unpredictable in prospect.

I think Jason raises a good point on why we have trouble imagining where the next breakthrough will come from:

The historical reason is that the big breakthroughs of the past were not easy to imagine or predict before they happened. In a different context, Eliezer Yudkowsky points out that even the creators of inventions such as the airplane or the nuclear reactor felt that their breakthroughs were fifty years out, or even impossible, shortly before they happened. Now is no different.

One of the biggest factors to this trick of hindsight is that innovation in situ is a gradual phenomenon. Only in hindsight do we look back on it as a “eureka” process of going from 0 to 1 in a flash of inspiration. I think if you reframe your understanding of progress and innovation around a steady, gradual march of deliberate advances, you begin to see why waiting around for it to happen in big bursts is an incorrect model.

Another insight from Ridley is that innovation is rooted in trial and error. In a world where we’ve become hyper-concerned with risk and protecting against downsides (just look at our expanding regulatory complex — only growing, never shrinking), we slow ourselves down from making the errors necessary for progress. Edison, a man that turned innovation into a product in itself, had this to say on his process, acknowledging error as a baked-in prerequisite for discovery:

I’ve not failed, I’ve just found 10,000 ways it won’t work.

Regarding the point on material constraints, it’s worth reading into the concept of dematerialization. The best work on the topic I know of is Andrew McAfee’s More From Less, which dives deep on this topic of how much modern progress is able to not only continue but in most cases accelerate, all while using fewer resources than once required. Take the simple aluminum can: the first ones weighed 85 grams. Modern refinement and manufacturing processes have reduced that to 11. The book is filled with cases like this of dematerialization outpacing our increase in consumption. It’s not a universal law of innovation, but rather a pattern that we see with continued progress².

Getting back to Jason’s original question — I believe that progress does have substantive causes. But between innovation’s gradual nature and the sea of trial and error, it’s hard to notice it while it’s happening. While you’re sitting in the present without the clarity of hindsight, it can feel like progress comes from flukes and strokes of luck³. But the deliberate effort and small victories add up (and compound) over time to enormous progress.

One of the goals of the progress studies movement is to expose what the sources of innovation are, to teach people how innovations came to be. And it’s important to recognize that innovation is intentional. It happens because we choose to work on making our lives better.

A final quote from the beginning of How Innovation Works (emphasis mine):

Innovation, like evolution, is a process of constantly discovering ways of rearranging the world into forms that are unlikely to arise by chance – and that happen to be useful.