Is it more impactful to find and train undiscovered diamonds or to unlock well-positioned but constrained researchers?

Is it more impactful to find and train undiscovered diamonds or to unlock well-positioned but constrained researchers

Both situations are a question of counterfactuals, and Counterfactuals are hard. On the one hand, there’s the question of how many people have the raw horsepower and outside perspective to do great work but are unable to get into the system (or do great work outside the system.) Call this the ‘discovery problem.’ On the other hand there’s the question of how many people already in the system would have done great work if the system were different (or there was an alternative system). Call this the ‘enablement problem.’

I’m going to assume that the goal here is “more potential paradigm-shifting blue sky research.” There are many other worthy goals like equality and preventing talent from being wasted that are often conflated.

To start, some important questions to ask are:

  • Do you think that at the highest levels of performance, paradigm shifting work is a numbers game? That is, is the Binding Constraint on the dominant system’s output the number of high-quality people in it?
  • What do you expect to happen to undiscovered people once they’re discovered? Either they will need to feed into the system or you will need to build a parallel system for them.
  • At what point (both in terms of age and education) do you think the ability or mindset to do great Blue Sky Research is pure discovery work that asks extremely open-ended questions disappears? The relative importance of the two problems shifts depending on whether you believe that the dominant system beats the ability to do great research out of people by the time they’re done with a PhD or, like Braben, you believe that experienced professors can do great work by shifting to an entirely new field.
  • It’s also important to be a bit more precise about the discovery problem. Are we talking about kids in Africa who don’t even know what research is? College students who aren’t considering grad school or can’t get in? Graduate students in lower-tier universities? Professors or post-docs in lower-tier universities?
  • How much apprenticeship/hands-on training do you believe researchers need? There are examples of people with almost no formal training coming “out of nowhere” and making massive contributions like Ramanujan in Mathematics or Chris Olah in machine learning. How replicable is their pattern? It’s getting ahead of asking questions but my hunch is that this varies between different disciplines. Disciplines that have relatively little Tacit Knowledge like mathematics or involve a lot of tinkering and have relatively low equipment requirements like deep learning research require little training.
  • If training matters, do you believe that you could create a better way to train people than the current system?
  • How big do you believe the ‘output’ difference is between the top 0.01% and 0.001% of researchers is? 0.001% and 0.0001%? Put less abstractly, if Einstein had died of pneumonia as a child, would someone else have come up with General Relativity in 1915? Would it have taken until 1925? 1985? Or would it have never been proposed. This is another unanswerable counterfactual question where opinions vary wildly.
  • How hard do you believe it is for people who would be good researchers to get into the system and from where? Are we talking about an elementary student who lives in rural Africa or India and would otherwise become a farmer? A high school student in inner-city America who would drop out and deal drugs?

Looking at these questions, a series of cruxy questions emerge: do you believe that the binding constraint on great blue sky research is the quality and mindset of people going into the top-tier universities or is it a systemic constraint on the people already in the top-tier universities? If it’s the former, the discovery problem is absolutely the right one to focus on. If it’s the latter, you need to work on the enablement problem and the question becomes: do you believe that the people who have never been in the system are better than the people already in the system? This difference could be either because the system systemically selects for people less suited to do great work or actively destroys people’s ability to do great work.

My belief is that the binding constraint is on the system itself. There are definitely many people who could be great researchers who never get the opportunity to do so. However, even if you could identify them and get them into the system, they would be as constrained out of doing great work as everybody else. In this situation, the only reason to work on the discovery problem (if what you care about is purely incredible blue sky research) is if you think it will work better to build an entirely parallel system. I actually believe that the system is pretty good at training researchers and doesn’t somehow ruin them, so I am inclined to believe that building an entirely parallel system will just be more work for possibly worse results.

Thoughts that inform my answers to these questions

The massive list of §Academia Constraints paints a pretty clear picture of the heavy constraints even on amazing researchers in top-tier universities. It’s tempting to say that once a researcher has made it into a top-tier research program perhaps as a grad student and definitely as a post-doc or professor, they’re well-positioned to do Blue Sky Research is pure discovery work that asks extremely open-ended questions. While there are certainly some examples of success, I would argue that they are the exception and not the rule.

From a professor friend who just got a grant proposal rejected:
some of the reviews are extremely glowing. the ones that were less glowing mostly suggest that proposal was "too ambitious" and "this is a new field to the PI, will be less risky once they can show that they have instrumentation set up

The fact that even success stories (Boyden, mRNA, etc.) happen by the skin of their teeth suggests that counterfactually there are many more examples on the other side of the survival line.

brabenScientificFreedomElixir2008 has several examples of well-regarded professors who couldn’t get funding to pivot out of their niche, but did great work once that pivot was enabled because of their prior experience. Of course, BP research suffers from another counterfactual problem because they only sponsored people with relatively robust track records.

odlyzkoDeclineUnfetteredResearch1995 strongly biases me to believe that science is not a numbers game and that “more talent” could actually have the opposite of intended effect.


Web URL for this note

Comment on this note