My notes on aroraChangingStructureAmerican2020.
(*) - will note my own comments
The authors point out that you can divide the history of the
American innovation system into roughly four periods: 1850-1900, 1900-1940, 1940-1980, and 1980 -> present. There are four threads to track in each period is the relationship to producing new and impactful technology-based goods and services (which I will begrudgingly refer to as innovations although we need a better word): universities, large incumbent corporations , startups/inventors, and the government.
The period from 1850-1900 could be described as the age of the inventor. During this period inventors were the main mechanism for creating innovations. These folks would usually sell these patents directly to large companies like the telegraph operators, railroads, or large chemical companies. The companies themselves did little R&D and primarily operated labs to test the patents that inventors brought them. There was a healthy market in patents. My hunch is that this world was enabled at least in part by the nature of the inventions.* The cutting-edge inventions were for the most part based on tinkering and trial-and-error that an individual could do in their garage. This was the era of the Wright Brothers building an airplane in their garage, Charles Goodyear creating vulcanized rubber on his stove, and Alexander Graham Bell creating the telephone in his house. Even Thomas Edison’s Menlo Park was less of a corporate lab working on big projects and more of a bunch of individual inventors all working in the same place for the same person. Over time, ‘inventor’ became a more specialized profession as indicated by fewer people creating more of the patents.
My hunch is also that the nature of the inventions is why you see the direct sale of patents instead of the creating a company around it and then aim for being acquired later. The inventions were simple enough that I literally could hand you the patent and you could figure out how to build and operate it. Draw a contrast to complex technologies that need human capital to be packaged with the patent in order to be useful for the acquirer. In a way modern tech startups are packages of human capital and IP. (This will become important later.) Additionally, the patent was often issued after someone got the thing to work, rather than before it is actually implemented, as is the case today. Companies understandably prefer to buy things that already work, so the shift from selling patents to selling companies may reflect the changing relationship of patents to the technologies they describe as well.
American Universities were mostly focused on practical arts, especially agriculture - Harvard and other older schools were the exception and emulated a more European model. Many universities were started during this period specifically to train people in useful skills and discover better ways of doing agriculture and manufacturing. That was was the whole point of the land-grant universities like Cornell, which to this day has one of the best agriculture departments in the world. MIT was explicitly founded to teach a combination of professional and liberal arts.
The government was almost uninvolved in R&D directly except during wartime when they were legally only allowed to spend money on short-term military-need research and in the creation of the Land Grant Colleges.
Small companies don’t come into the picture during this period much, though clearly many relevant companies were started:
A piece of complete speculation on my part but perhaps there was less of a distinction between big companies and small companies in this period. Beyond the railroads, steel companies, an oil companies did any companies actually reach massive scale?
As Anton Howes points out in Arts and Minds, the common ad
Inventor seeks capitalist from this period comes to mind .
Innovations during this period shifted to chemicals, and complex systems like cars and planes. The nature of these innovations made it harder to simply sell a piece of paper with an explanation of how the thing worked to a company and expect them to be able to replicate it. Just having the piece of literature doesn’t solve the problem. DuPont bought the patent for creating viscose rayon (processed cellulose used for creating artificial silk and other fibers.) However, Dupont was unable to replicate viscose rayon successfully and eventually had to partner with the original inventors to get it to work. () Context is important and underrated for knowledge transfer. Additionally, these innovations started to require more specialized and expensive equipment which was out of the range of individuals, small companies, and universities as well. There are many pieces of equipment or tacit knowledge that only exist in one or two places in the world.
The federal government increased anti-trust enforcement increased after a massive spike of M&A around 1900 so large corporations felt more pressure to create their own products. Additionally, the Hatch (1887) and Adams acts (1903) provided states $15,000 and then $30,000/year for agricultural research (which was presumably distributed to the state/land grant colleges.)
Most industrial labs were started in this period, partially inspired by German chemical companies. Note that the germans were the first ones to step into the domain of complex/chemical innovations and first ones to build industrial labs. The implied reasons for this were the complexity of new innovations and the fear of antitrust preventing M&A as a way of expanding. The German precedent suggests that the nature of the technology itself was a causal factor.
Universities relied on state and industrial funding. Saying “universities did” anything is a bit anachronistic because before WWII, individual professors brought funds to their own labs with very little university overhead. (See bushPiecesAction1970.) The need for industrial funding pushed many professors towards mission-oriented research. This direction irked some professors, especially those influenced by European academic philosophy that emphasized pure discipline-oriented research. Caltech was founded by these dissidents. Industrial labs attracted professors away from universities because they had equipment that only large companies could afford.
The paper actually skips the period of WWII and its immediate precedent and aftermath. It does note that WWII saw Federal R&D go from $83.2M -> $1313.6M in 1945 (2 orders of magnitude!) Most of it went to universities.
I have a hypothesis that this massive injection may have been line Bane in batman.
The technology story starts to shift towards drugs, materials, and even more complex systems (like early computers and jet engines.) Drugs arguably have the most science-based bottleneck. Development of these complex systems require
complementary engineering skills and a lot of tacit knowledge. For example, the transistor required a combination of physicists, metallurgists, and chemists all working with a specific mission-focus. A notable fact about the technology of this era is that many scientific discoveries had clear uses - the maser is a good example. As soon as it was discovered, people realized it would be useful.
The federal government continued to pump money into university research because it realized how coupled technological superiority and military superiority would be in a conflict with Russia. This period also saw peak antitrust cases in the US. It’s only briefly mentioned in the paper but this was also the period during which DOE National Labs were created.
The combination of M&A-focused antitrust enforcement and the technology frontier’s need for complementary skills drove more corporate lab work. There was a lot of discovery-focused science going on at corporate labs. Corporate labs were able to act on new discoveries faster than academia.
The 1970’s marked a discontinuity near the end of this period as well. The Vietnam war and associated cultural shifts and protests pushed universities to divest from aiding the ‘military industrial complex’ - so mission focused Federal R&D and R&D with large corporations. Universities as a whole turned towards more discipline-focused research.
Venture capital and small companies also started taking off near the end of this period. Genetic engineering-based pharmaceutical development and Semiconductor-based computing both have high upfront capital costs and low unit costs which makes them ideal for the venture capital model.
The innovations of this period are concentrated in computing, the internet, software, and pharmaceuticals. Note that except for pharmaceuticals (which have their own issues - see Science Business - Book), these have lower and lower capital costs outside of the human capital. Many of the innovations were less ‘technology’ and more ‘tech’ - business models that take advantage of computers and the internet, shifting businesses online.
Andrew Odlyzko argues (and the authors support) that there’s an increasingly small window of time between even cutting edge groups discovering something. If the that gap is large, a company can use in-house research to build out a patent portfolio (Aircraft Carrier investing). The example cited is how Xerography was invented in 1937 and not patented by Xerox until 1950. The implied assertion (which I agree with) is that systems are exponentially more valuable than single technologies. Or more extreme - a really useful technology is made up of a bunch of smaller inventions. Innovation requires multiple heuretics. In a world where all the pieces of an innovation are rapidly patented by multiple entities it makes more sense to spend your money buying those patents or entities themselves and the returns from in house research are lower because even if you get to the first patent, it’s not that valuable if other people get the others. If this argument holds, slightly more Slack - concept around new inventions is important.
It also raises the question - Why is there so little slack in mission-focused research?There are definitely more scientists so it could be that the number of researchers has gone up faster than the number of areas of exploration and you’re just seeing the effect of more researchers per area. This saturation would be exacerbated the number of areas for cutting edge patentable research has gone down. The other two possibilities are cultural. There may be more of a cultural equilibrium of patenting everything that seems vaguely valuable as soon as possible instead of waiting for a higher value patent or group of patents because you know that everybody else will do the same thing. This may be exacerbated by both Bayh-Dole but also the cultural expectation of how valuable science should be. This cultural expectation didn’t really come about until after WWII and perhaps took years to saturate the system. The other cultural explanation for the missing slack might be conservatism around areas of exploration. If there are more incentives for groups to do research on known good ideas then naturally there will be fewer discoveries that nobody else was working on.
Modern tech companies at least in part solve the problem of patents building out valuable systems by having war chests of defensive patents creating a Mexican standoff where any tech company has the ability to sue any other company so they all just cooperate to not sue at all. This situation doesn’t keep the patents from blocking development (as they do in other domains) but it doesn’t enable any companies to capture value from patents.
The idea of a division of labor between academia and industry (which by the way is deeply embedded in the language used in academia) was strong. This separation can only be helped by the fact that the majority of university research money continues to come from the federal government. The authors argue that the division of labor removed forcing functions on universities which allowed research to sub-divide problems into chunks that are easy to research and publish but then are harder to translate to something useful. This specialization may in part be a cause for the fact that discoveries feel less useful today. The more specialized, discipline-focused nature of university research means that it requires more work to become a useful product. Antedisciplinary Science describes how truly new fields come from people in different disciplines coming together.
It’s possible that the large amount of federal funding for universities also helps prop up this division of labor because university professors are able to afford expensive equipment or access federally owned equipment where previously they were incentivized to join corporate labs to access more resources. *
The work to translate research into product is more often being done by startups. Universities are more incentivized to be involved in startups and corporations are less incentivized to do translational research.
Universities are incentivized by the Bayh-Dole act to be directly involved in startup activity. The Bayh-Dole act gives universities (not the professors) ownership over IP funded by federal money. Since the most profitable path for IP in this period is to have a startup built around it, universities find themselves incentivized to get into the startup business. Other factors encouraging university spinoffs are perceptions of increasing institutional calcification in large companies and funding both from VCs and the government in the form of SBIRs. However, since 2005 university patents have continued to climb while their licensing fees have flattened out.
In addition to spending lot of money on university research, the federal government hasn’t been prosecuting anti trust cases and has been encouraging small business activity. The Bayh-Dole act incentivized universities to create more spinoffs. SBIR grants directly subsidize small businesses working on technology.
Large corporations find it easier to expand through M&A. (Large companies use startups as outsourced R+D) M&A activity has gone through the roof (see Merger graph 1880-2020) , probably caused at least in part by the absence of anti trust. If it’s easy to acquire products, there is presumably less incentive to do the translational work themselves. The nature of the innovations also lend themselves to acquisition. The paper calls these
disembodied knowledge products - fabless chip designs, software, and drug formulas. It raises the question of whether these products suffer from the same problems that Dupont experienced in translation? The story works if the answer is “yes” because it fits with M&A being the primary mode of acquisition. If not, you would expect to see a lot of organizations just focused on selling IP products. ARM is one example, but are there others?
Large corporations also experience more pressure to specialize both from investors and from more competition - especially from overseas. Competition and lack of slack creates specialization. This specialization makes corporate labs less valuable because you don’t always know what you’re going to get out of discovery-type research and there’s a higher chance it will be ‘out of scope’ at a more focused company.
AI research is an exception to the decline of corporate R+D
AI research bucks the trend of the decline of corporate labs. Like other areas that enticed professors away from universities, AI research (can) require massive resources - in this case it’s just thousands of dollars of compute for training models and the datasets to train on. AI research benefits from having multiple disciplines in the same place (and the people who will eventually produce it.) So you actually see healthy and productive corporate AI labs: Google Brain, DeepMind, FAIR, Microsoft Research. These places are highly regarded and collaborate closely with academia in the same way that previous corporate labs did. They advance fundamental knowledge
The existence of successful corporate AI Labs suggest a couple of uncomfortable possibilities. Clearly the ability to acquire technology through M&A is not the only reason that corporate labs have declined. Tech companies can and do acquire AI companies, and yet they still labs. AI labs suggest that successful corporate labs can only exist in domains where there are large corporations with monopoly-like power. The decline of corporate labs in chemistry and physics-related domains may have been caused by commoditization of those products. GE, Dupont, Kodak, and others’ share prices suggest that they at least are no longer perceived as monopolies. However, that story isn’t complete either because Boeing has a near-monopoly and doesn’t have an impressive corporate research lab. Corporate labs might be able to pursue AI In the modern environment while other labs cannot because AI development does happen on a faster timescale than most atom-based research.
AI is also just perceived as being incredibly valuable, while new atom-based products are not. The key question is whether this perception is accurate or not.
Almost by definition corporate labs can only operate successful in areas where corporations perceive them creating value. The decline of corporate labs means corporations see their expected value as negative. As mentioned previously this negative value could be due to specialized firms being unable to capture the value of broader technologies, the timescales on which corporations need to capture value shrinking, no longer needing to create peace offerings to avoid being broken up by the government,* and M&A being a more attractive option for acquiring technology.
A terrifying possibility is that we’ve actually exhausted the physical phenomena that can be tapped to create valuable products. In the past corporations clearly saw a connection between tapping new phenomena and their bottom line. Most of the arguments around the decline of corporate labs focus on the second half of that sentence, how businesses and the culture around them has changed. I hope I’m wrong, but what if new phenomena are just less exploitable? What if we’ve pushed the knowledge frontier into realms that are too big or too small to be relevant on human scale? Just the other day the ISS crew created a Bose-Einstein condensate that lasted almost a second! The only use the article suggested for literally a fifth phase of matter was for prospecting for minerals. Are Bose-Einstein Condensates useful for anything? People have said for years that being in space should enable you to build materials and drugs you never could on earth. So why isn’t there a corporate research lab in space? Either 1) Companies don’t believe those things are valuable and are right , 2) Companies don’t believe those things are valuable and are wrong 3) Companies don’t believe those things would be valuable on the timescales they care about. I really hope it’s not # 1.
The other terrifying possibility is that the era from 1900 to 1980 was a weird outlier and we’re not regressing to the mean. Or that the massive spending and focus in WWII + the Cold War was like Bane injecting himself with Venom to make himself really strong for a short amount of time, but wrecking himself afterwards. Low Hanging fruit theory of stagnation
Corporate labs create an environment where you have a diverse group of skillsets working on mission-focused research with a long timescale in the same organization that has the capabilities to implement it.
No institution in the modern ecosystem outside of AI labs has all of those characteristics at once.