There are many heuretics (discoveries/inventions/innovations/ideas etc.) that don’t fall within the constraints of the institutions we expect to support them. “Things that are not paper-worthy enough for academia and not-product focused enough for a startup” is shorthand for one class of these heuretics. Note that there are many others!
First, let’s address some misconceptions about what falls into this category. Wacky ideas and long-term projects are two things that many people assume don’t get supported in universities or startups. This is not true.
- Long-term projects get support if they hit consistent milestones and they’ll be able to capture a lot of the value they create if they succeed.
- Wacky ideas get support as long as they don’t threaten paradigms too badly or someone thinks they can make money off of them.
- Research projects that require a lot of people or coordination as long as the final product is a paper they can all put their names on. See the LHC.
- Research projects that require some engineering effort as long as the engineering effort resembles something that has been done before: building a bigger telescope/particle collider/linear accelerator/space probe, etc.
The reasons that ideas are not paper-worthy or product-worthy make sense given the incentive frameworks in their relative institutions. Most people are not malicious or stupid.
‘Paper-worthy’ is meant as shorthand for ’things academics are incentivized to do.’ So things that are not paper worthy are those that don’t obey §Academia Constraints. It’s important to explicitly call those implications out because that call-out can shine a light on where to dig for hidden gold.
A non-exhaustive list of what gets shot down by peer review:
I would argue yes, valuable projects that wouldn’t get many citations include:
- Projects that are intermediate steps to a larger vision without a strong conclusion besides “we did a thing!”
- Engineering work in a theory-heavy field or theory work in an engineering-heavy field. More generally, work in a field that the field doesn’t think is valuable.
- Ideas that are far enough out of mainstream won’t get many citations because people don’t have the bandwidth to dig in and see if it’s worthwhile and the default answer is “no” (see #3 below)
It’s hard to publish ideas that are too outside of the approved way of doing things. Paradigms outline a set of questions and the rules around answering them. Ideas can go too far outside of accepted paradigms when when you’re asking different questions, or measures success differently than standard ways. Antedisciplinary Science often falls into this trap as well, when it’s too much like discipline X to be published in a journal for discipline Y but it’s too much like discipline Y to be published in a journal for discipline X. This situation is of course tricky because many (most?) ideas that go outside of paradigms are legitimate crackpottery. Anyone on the knowledge frontier is a bit of a crackpot.
It’s hard to write a paper if you can’t even get the money to do the research to write the paper.
The requirement to align with current funding priorities is particularly pernicious because The government grant process depends on politics and committees. Additionally, one reason to write papers is to continue playing the game, so if a paper doesn’t hint at your continued ability to do work that aligns with funding priorities, you’re incentivized not to work on it. People giving out grants try to derisk them as much as possible.
A project is unlikely to lead to a paper if it’s not within scope for a PhD student or untenured professor.
Graduate students are the labor in academic science and engineering. In exchange for cheap labor, grad students expect to produce research within a 4-7 year time frame that can help them move on to the next rung of the academic career ladder. (Despite the fact that There is an increasingly wide gap between the number of graduate students training for PhDs and academic positions for them to fill.) While a tenured professor might be willing to work on something crazy that will take 10 years and might be a complete dud, no grad student would rationally work on it. The incentive for PhD students to produce high-impact papers for the sake of their careers serve to amplify all the other constraints, and add the timescale constraint. Additionally, PhD students in science get most of their funding via research grants, so research they work on is especially susceptible to #4.
Projects whose major that produce a lot of structured data or reusable code don’t led themselves to papers. Academic papers are built around the paradigm of scientific inquiry, where collecting data and writing code are only in service to producing an abstract model or theory - the more generalizable the better. There is little reward to producing high quality, reusable datasets or code, despite the value they can create.
A project is unlikely to lead to a paper if it involves a lot of coordination. Academia emphasizes individual agency. This is a good thing, but if a project cannot be effectively modularized it will end up requiring a lot of grungy coordination work that doesn’t count as a contribution unless you’re in charge of the project. A big reason to go into academia in the first place is to avoid having a boss and because you like doing your own thing, your own way. Obviously there are exceptions like the LHC, but that is a situation where the experimental particle physicists are locked into a paradigm and have no other option.
Coordination aversion hits hard projects that don’t fit into a particular disciplinary bucket or require integration between components built in different labs. Projects of a particular scale require coordination by their very nature. SeeA new structure for scalable research for a much deeper dive.
‘Product-focused’ is meant as shorthand for ’things startups are incentivized to do.’ So things that are not paper worthy are those that don’t obey §Startup Constraints. It’s important to explicitly call those implications out because that call-out can shine a light on where to dig for hidden gold.
A more subtle issue than the question of value capture itself is that trying to capture the value created by an innovation can severely hamstring the total value it creates. Consider the Graphical User Interface and other personal computing innovations created at Xerox PARC. Arguably they’ve created trillions of dollars of value for the world. Would that be the case if Apple had a patent on the GUI and mouse? Probably not. This situation is not unique.
Obviously, the change needs to happen eventually, but many paradigm shifts (especially in complex systems) initially lower performance, at least on traditional metrics. Metrics can cause paradigm lock-in. Change happens either when the new paradigm gets enough reps to catch up to the old way of doing things or people see the new paradigm in action and accept that the traditional metrics weren’t capturing all the important features of a system. Both of these situations are hard for a traditional startup to achieve because both approaches take a lot of time and resources before showing ‘traction’ and the potential customers (the people running the systems or consuming their outputs) will assert that the idea is dumb up until its not.
In these situations, the vast majority of the value generally accrues to the end-product producer or consumers. The naive advice would be “start a firm that’s doing the end-to-end thing” and outcompete the incumbents. While that should theoretically work, it often runs into the reality of extremely complex products that you would need to reinvent from the ground up (like if you had a better way of making passenger jets), heavily regulated industries where it would take massive lobbying just to use a new process, and other scenarios. A startup doesn’t usually have the time or resources to tackle these scenarios. The alternative approach is to take on a change-management consulting role, which usually doesn’t capture enough value to be VC-fundable so it needs to be profitable early on. Additionally consulting runs head-long into the issues in point #4 above.
In some ways this point is intertwined with #4.
It’s common wisdom among VCs that it’s not a good idea to invest in companies that are taking on two, let alone three of these risks. This heuristic isn’t misguided. The chance of failure goes up dramatically when an organization tries to tackle tow, let alone three of those risks. Startups should only tackle one (or maybe two) of these. The reason therapeutics focused startups can exist is that they have almost no market or channel risk - there is a well established pipeline to being acquired by a pharma company and insurance companies are guaranteed to pay for drugs that go through FDA approval and target big conditions.
At the end of the day, the work you need to do to drive adoption of a technology is often very different from getting it to a certain performance level. The work priorities drive the type of organization you create. As much as we like to think that startups are about building technology, they are actually about selling technology and adopting it for specific markets so that they can do that better.