Each sort of institution is good at enabling a certain set of activities: academia is good at generating novel ideas; startups are great at pushing high-potential products into new markets; corporate R+D is great at incremental improvements to existing product lines, etc. You could think of each institution ‘covering’ some area on a map of all possible activities. The specific area on the page that an institution covers and the limits of that area are dictated in a complex way by the incentives within that institution. We could call these boundaries constraints on institutional activity.
Abstractly you could imagine some massively multidimensional “constraint space” that describes the characteristics of different activities that eventually result in awesome stuff.
You could think of all the institutions together as the ‘innovation ecosystem.’ ^1 Some activities are covered by multiple institutions, but some aren’t covered by any institutions. These activities are ‘constrained out’ of happening by existing institutional structures. For example, unsexy >5 year projects that aren’t focused on a product that don’t produce novel papers are simultaneously beyond startups/VC (which could support either a shorter timeline or more product focus) , Corporate R+D (which would want them to be sexy or product focused), Philanthropy (which would want them to be sexy), and Academia (which would push for novel papers.) Of course there are counterexamples of projects with those characteristics that have been supported by existing institutional structures! It will always be possible to make arguments like “SpaceX exists, so the innovation ecosystem is fine.” Antibiotics are pretty great, even though people can sometimes survive infections without them. Similarly, it’s worthwhile to try to enable more constrained activities even if a few projects make it through.
Institutionally constrained activity is a more useful and precise way to think about the Nebulous bundle of things that people label ‘The Great Stagnation’ or ‘research/academia/science/physical innovation is broken’ - the vibe that the world could be on a more wonderful trajectory than it’s on
right now.^2 Thinking in terms of institutional constraints helps us talk about specifically which activities we expect to be happening that aren’t supported which in turn can (hopefully!) lead to addressing those gaps by adjusting incentives in existing institutions or creating new institutions with different sets of incentives. Looking at the innovation ecosystem this way leads me to conclude that Contemporary research has multiple failure modes: what I would call Einstein would have been stuck in a patent office - failures of paradigm-shifting science and We have no flying cars - failures of paradigm-shifting engineering.^3 Addressing each of these areas is important but requires incentivizing different sorts of activities and thus different institutional structures.
Thinking precisely about which set of disincentivized activities you’re trying to enable is helpful in several ways. It allows you to ask “what incentives are preventing institutions from enabling these activities? Should I be shifting incentives within an existing institution or building a new one? How do I keep a new institution from falling into the same incentive traps?” You can know what not to work on despite temptations to do all the things. ‘Fixing research’ is going to require many new institutions (that I suspect will not scale Research orgs don’t scale) to cover different parts of constraint space. Legible niches can help prevent mimetic infighting over who is going to ‘save science.’
^1: I hate the term ‘innovation ecosystem’ because it’s usually used as a Suitcase Handle Word, but it is useful to have a shorthand for “all the institutions that enable activities that collectively cover the process of an idea becoming a new impactful idea or technology.”
^2: My unsubstantiated personal opinion is that analyses on high-level metrics like total factor productivity contribute to this unhelpful lumping.
^3: I suspect many readers will take umbrage at my fast and loose use of the oh-so-tricky words “research” “science” “engineering” “paradigm-shifting” - here is where I would beg some temporary trust that I take the nebulosity of these words very seriously and have concrete meanings behind my use of them and getting into precise definitions feels like a boondoggle for this particular piece.
^4: This isn’t to suggest that this division is either the only way you can break down the problem space or that these are clean breaks.
^5:And this is completely ignoring non-science and technology focused creative work!