Beyond a certain size, research organizations become drastically less effective and arguably become a waste of money. More subtlety, research organizations don’t scale because trying to dominate a niche is actively bad for that niche and the organization as a whole.
This attribute is important to note and dissect because one wants to take very different actions and adopt different assumptions when building or supporting organizations that can scale vs. ones that can.
Research org unscalability is also an important contrast to both startups and most government policy both because it helps frame the difference and because there’s a strong overlap between people trained in some combination of “startup thinking” and “government thinking” who are interested in building or supporting new innovation organizations.
You could imagine Stripe having a monopoly on internet payments, or Uber having a monopoly or duopoly on ride sharing. Arguably, every high-growth startup should be striving to becoming a monopoly - “competition is for losers.” Whether you buy the Thielian concept that every startup should try to become a monopoly or not, it’s pretty clear that the idea of growing massive is core to the idea of modern startups. This size requires processes that scale well - that is, they work just as well or better in a massive organization as a small one. Yes, startups act like research organizations early on, but this corresponds to “do things that don’t scale.”
Historically, research monopolies are nonexistent. Even Bell Labs did not come anywhere near monopolizing research. While the organization was huge, Bell labs was primarily not researchers and The famed bell labs research was a fraction of their activity.
DARPA is more ideas limited than money limited. The most stunning example of this was that former DARPA Director Verne Lynne actually lobbied congress to reduce DARPA’s budget in the mid 90’s. Talk about putting your money where your mouth is about innovation organization not scaling.
You just don’t see any single universities or labs monopolizing a field long term. Temporary clusters where all the action is happening will occur for a bit but they are neither absolute nor permanent.
Of course, just because it’s never happened doesn’t mean that it can never happen. However, it means that the burden of proof weighs heavily on the person who thinks that a research org can scale.
There are no scientific way to “prove” why research orgs don’t scale (at the moment.) So like Smith’s inviable hand, these are going to be suspicions and stories that I flesh out with logic. However, I think this form of knowledge is still useful.
Research requires more trust than other disciplines and Trust is hard to scale. At a certain size an organization can no longer depend on trust to make decisions and instead needs to switch to metrics. Metrics aren’t necessarily detrimental to business processes but are detrimental to research. I believe this is the most important but also the most nebulous reason why research orgs don’t scale in the same way that startups do.
You need to break rules to create new paradigms. Any organization has a set of rules, no matter how permissive. So an organization monopolizing a niche will prevent paradigm shifts from happening in that niche. If ‘supporting paradigm shifts’ is a crude success metric for research organizations, for any niche a research organization should want at least one other organization in that niche with a different set of rules so that ideas can always find a new home.^1Federated systems make it so disruptive changes cannot be completely damped out].
Operating at the knowledge frontier requires a significant level of specialization both in fields and types of activities. Some areas of specialization may actually preclude other areas - for example you might get extremely good at externalized research, which precludes working on things that require internalized research, or you get very good at finding and funding research with extremely fuzzy goals and applying that process to research that requires tight goals and a lot of coordination would lead to completely failure. It feels vaguely analogous to how multiples decrease as both hedge funds and VC firms increase in size.
The natural timescale of research might also play into the tendency for effective organizations to remain small because The fewer people there are the easier it is to align on goals and the longer those goals can remain aligned.
^1: This argument feels right if a bit funky