A pattern starts to emerge when you start to ask “how are ‘golden age’ corporate R&D orgs” different than one today, which no longer seem to have the impressive output or cache that they once did? Corporate labs no longer fill the niche they did in the early-mid twentieth century.
It seems that all of them have a trifecta of conditions
They were run by a monopoly
More specifically, they were run by companies that were extracting monopoly rents on some high-margin product. Xerox was the only game in town for copying machines (which, believe it or not were core to many businesses), Dupont controlled well-known materials like Teflon or Lycra, Corning was Pyrex and Silicone, and of course AT&T controlled the telephone networks.
They were working on a clearly high potential technology
The technologies were the sort of thing where they had strong conviction that if they could actually pull it off technically, they would be able to sell it. Of course if you can replace a vacuum tube with something that uses 1000x power people would use it. Of course if you can create glass that won’t shatter when you heat it up and cool it down people will use it.
What’s not clears why so many fewer technologies seem to fall under that umbrella now. It could be that we picked the low hanging fruit in atom land. Low Hanging fruit theory of stagnation. It could be that people are just more pessimistic about what we can build. It could be that timelines have shrunk, and there is no high-potential technology on those timelines. It could be that people think in terms of products instead of technology. Or it could be that companies have scoped down.
Xerox PARC is an illustrative example through the different dynamics of the laser printer, the personal computer, and the lab itself. While PARC is legendary because of the impact of its personal computing work, structurally most of that impact shouldn’t have happened. PARC’s personal computing work wasn’t tied to Xerox’s core business. The only reason it was impactful was because of a crazy set of contingencies - Steve Jobs’ ‘raiding party’ and Bill Gates quickly following suit. Xerox wasn’t set up to scale up and build a business around personal computing. Different technologies require different organizations.
The laser printer, on the other hand was directly in line with Xerox’s core business - they already sold printers and copiers and this was just a better, faster version.
Through this lens, AT&T almost cheated on this front because its core business was the full stack of “communications” so everything from chemistry that kept telephone poles from rotting to information theory were actually tied to the company’s core business.
Transistors had the potential to integrate directly into AT&T’s system and enable faster, cheaper service. Faster, cheaper service was an existential priority for AT&T because if they didn’t continue to improve, the US government would have an excuse to break them up. Regulation drove many of Bell Labs’ actions.
I suspect that the existential threat criterion in a big factor in the difference between Google Brain and Google X. Better machine learning technology directly ties into google cloud, search, etc. While drone delivery and autonomous cars can potentially be big businesses they are basically orthogonal to google’s core business.
^1: Condition #3 was originally “1+2 are insufficient if the technology the lab works on isn’t tied into the company’s core business.” However, this isn’t quite right. There are situations where technology work on the core product can have nothing to do with existential risks to the business (for example Salesforce.) Similarly there are situations where technology work has little to do with the core product but addresses an existential threat (for example flashy research at Bell Labs keeping regulators from breaking up the monopoly.)