Things that are not paper-worthy enough for academia and not product-focused enough for a startup

There are many heuretics (discoveries/inventions/innovations/ideas etc.) that don’t fall within the constraints of the institutions we expect to support them. “Things that are not paper-worthy enough for academia and not-product focused enough for a startup” is shorthand for one class of these heuretics. Note that there are many others!

First, let’s address some misconceptions about what falls into this category. Wacky ideas and long-term projects are two things that many people assume don’t get supported in universities or startups. This is not true.
- Long-term projects get support if they hit consistent milestones and they’ll be able to capture a lot of the value they create if they succeed.
- Wacky ideas get support as long as they don’t threaten paradigms too badly or someone thinks they can make money off of them.
- Research projects that require a lot of people or coordination as long as the final product is a paper they can all put their names on. See the LHC.
- Research projects that require some engineering effort as long as the engineering effort resembles something that has been done before: building a bigger telescope/particle collider/linear accelerator/space probe, etc.

The reasons that ideas are not paper-worthy or product-worthy make sense given the incentive frameworks in their relative institutions. Most people are not malicious or stupid.

What makes something not paper-worthy?

‘Paper-worthy’ is meant as shorthand for ’things academics are incentivized to do.’ So things that are not paper worthy are those that don’t obey §Academia Constraints. It’s important to explicitly call those implications out because that call-out can shine a light on where to dig for hidden gold.

  1. An idea isn’t paper worthy if not considered “novel enough” or “not a contribution to the literature” by peer reviewers. The word ‘novel’ is used as an idea bludgeon in academia and weeds out many classes of ideas.

A non-exhaustive list of what gets shot down by peer review:

  • New approaches to old “solved” problems,
  • Better implementations of an existing idea that’s already been implemented,
  • Efficiency improvements along a different metric than the ones the field normally cares about,
  • Scaling existing techniques,
  • Creating something for a very specific application as opposed to a general principle/purpose,
  • Further work on things that have already gotten to proofs of concept
  • System integration of several already-existing ideas.

For more see: Academia is not a good environment for systems engineering. Academia incentivizes novelty, not focus. Academic research is often just the concept half of creating a heuretic.

  1. An idea generally isn’t worth writing a paper about if it’s not expected to get a lot of citations. Even if it’s published, low-citation ideas languish in obscurity. It’s valid to ask ‘are there things that are valuable that don’t get a lot of citations?’

I would argue yes, valuable projects that wouldn’t get many citations include:
- Projects that are intermediate steps to a larger vision without a strong conclusion besides “we did a thing!”
- Engineering work in a theory-heavy field or theory work in an engineering-heavy field. More generally, work in a field that the field doesn’t think is valuable.
- Ideas that are far enough out of mainstream won’t get many citations because people don’t have the bandwidth to dig in and see if it’s worthwhile and the default answer is “no” (see #3 below)

  1. It’s hard to publish ideas that are too outside of the approved way of doing things. Paradigms outline a set of questions and the rules around answering them. Ideas can go too far outside of accepted paradigms when when you’re asking different questions, or measures success differently than standard ways. Antedisciplinary Science often falls into this trap as well, when it’s too much like discipline X to be published in a journal for discipline Y but it’s too much like discipline Y to be published in a journal for discipline X. This situation is of course tricky because many (most?) ideas that go outside of paradigms are legitimate crackpottery. Anyone on the knowledge frontier is a bit of a crackpot.

  2. It’s hard to write a paper if you can’t even get the money to do the research to write the paper.
    The requirement to align with current funding priorities is particularly pernicious because The government grant process depends on politics and committees. Additionally, one reason to write papers is to continue playing the game, so if a paper doesn’t hint at your continued ability to do work that aligns with funding priorities, you’re incentivized not to work on it. People giving out grants try to derisk them as much as possible.

  3. A project is unlikely to lead to a paper if it’s not within scope for a PhD student or untenured professor.
    Graduate students are the labor in academic science and engineering. In exchange for cheap labor, grad students expect to produce research within a 4-7 year time frame that can help them move on to the next rung of the academic career ladder. (Despite the fact that There is an increasingly wide gap between the number of graduate students training for PhDs and academic positions for them to fill.) While a tenured professor might be willing to work on something crazy that will take 10 years and might be a complete dud, no grad student would rationally work on it. The incentive for PhD students to produce high-impact papers for the sake of their careers serve to amplify all the other constraints, and add the timescale constraint. Additionally, PhD students in science get most of their funding via research grants, so research they work on is especially susceptible to #4.

  4. Projects that produce a lot of structured data or reusable code don’t led themselves to papers. Academic papers are built around the paradigm of scientific inquiry, where collecting data and writing code are only in service to producing an abstract model or theory - the more generalizable the better. There is little reward to producing high quality, reusable datasets or code, despite the value they can create.

  5. A project is unlikely to lead to a paper if it involves a lot of coordination. Academia emphasizes individual agency. This is a good thing, but if a project cannot be effectively modularized it will end up requiring a lot of grungy coordination work that doesn’t count as a contribution unless you’re in charge of the project. A big reason to go into academia in the first place is to avoid having a boss and because you like doing your own thing, your own way. Obviously there are exceptions like the LHC, but that is a situation where the experimental particle physicists are locked into a paradigm and have no other option.

Coordination aversion hits hard projects that don’t fit into a particular disciplinary bucket or require integration between components built in different labs. Projects of a particular scale require coordination by their very nature. SeeA new structure for scalable research for a much deeper dive.

What makes something not product-focused enough for a startup

‘Product-focused’ is meant as shorthand for ’things startups are incentivized to do.’ So things that are not product-worthy are those that don’t obey §Startup Constraints. It’s important to explicitly call those implications out because that call-out can shine a light on where to dig for hidden gold.

  1. There’s no convincing story for how the project’s output produce a massive and competitive ROI on a reasonable timescale for a VC-funded startup
    Massive ROI Venture capitalists need to believe that any investment in their portfolio could potentially return the fund.
    Competitive ROI Investing isn’t just about absolute returns - any investment is in leu of another investment. So if the stock market is absolutely crushing it, there’s less incentive to invest in higher risk technology. Even within the VC realm, many atom-based technologies have the same risk or higher than a software startup but expected return is lower (especially when you take timeline into account) or farther in the future. From the cold hard numbers SaaS is often just a better investment.
    Timescale J.C.R. Licklider started work on the ARPA project to build what we would consider “modern computing” in 1962 and perhaps the first real ‘value capture’ event based on it was Apple’s IPO in 1980. Venture capitalists need to fund things that increase valuation in a few years and return in 5-7 years so this trajectory would have been untenable as a startup. Venture timescales often lead to projects being pushed to focus on a specific application or being acquired before achieving the technical goals they set out to hit. There is nothing wrong with either of these routes per se, but they can lead to technologies failing to live up to their potential.
  2. It will take a lot of development work to sell anything for a bootstrapped startup
    Startups do not need to raise money from VCs. However, that does mean that if the founders are not extremely wealthy (SpaceX, Blue Origin) the startup needs revenue early on. If the project will take a lot of development work before it can sell anything the startup needs to get revenue somewhere. Some bootstrapped startups use consulting work to create revenue to fund development. Unless it’s wildly profitable, the consulting approach severely limits the amount of time and money that can be spent on development, possibly dragging it out forever. Some projects need effort above a certain threshold to move forward. SBIR grants can provide another option, but they are relatively small (order $1m over several years.)
  3. It’s hard to capture the value the project creates
    Markets are great, but there is no law of economics that people can always capture the value they create if they are clever enough. (See Schumpeterian Profits in the American Economy - Theory and Measurement.) Research often falls into this trap because Innovation requires multiple people. Often the person who figures out how to make a lot of money off of a discovery or invention is not the person who created it, nor are they even in the same organization. “But what about intellectual property ?” There are many reasons intellectual property does not ensure everybody involved in an innovation capture the value they create but a short list includes: many people in many organizations contribute to most innovations, patents only work for a narrow set of innovations, and patents have massive downsides - including potentially capping how much value an innovation can create.

A more subtle issue than the question of value capture itself is that trying to capture the value created by an innovation can severely hamstring the total value it creates. Consider the Graphical User Interface and other personal computing innovations created at Xerox PARC. Arguably they’ve created trillions of dollars of value for the world. Would that be the case if Apple had a patent on the GUI and mouse? Probably not. This situation is not unique.

It is hard to capture value from research

  1. The project is changing one or more components of a bigger system or process
    A startup that is trying to improve or change a component of a bigger system needs to either convince whoever runs that system to adopt the change (which is especially hard when no particular entity runs the system) or the startup needs to build the entire system themselves with the change incorporated.

Obviously, the change needs to happen eventually, but many paradigm shifts (especially in complex systems) initially lower performance, at least on traditional metrics. Metrics can cause paradigm lock-in. Change happens either when the new paradigm gets enough reps to catch up to the old way of doing things or people see the new paradigm in action and accept that the traditional metrics weren’t capturing all the important features of a system. Both of these situations are hard for a traditional startup to achieve because both approaches take a lot of time and resources before showing ‘traction’ and the potential customers (the people running the systems or consuming their outputs) will assert that the idea is dumb up until its not.

See Fundamental Manufacturing Process Innovation Changes the World for more.

  1. Innovations that can’t be encapsulated into a product
    There are some innovations that it’s just hard to productize. Process innovations often fall into this category - especially those that don’t depend on new technology per se, but on a new way of doing the same thing with basically the same tools. The Bessemer Process, for example, used basically the same crucibles but you put different things in them. Social process changes (like Kaizen^1) are even harder to productize.

In these situations, the vast majority of the value generally accrues to the end-product producer or consumers. The naive advice would be “start a firm that’s doing the end-to-end thing” and outcompete the incumbents. While that should theoretically work, it often runs into the reality of extremely complex products that you would need to reinvent from the ground up (like if you had a better way of making passenger jets), heavily regulated industries where it would take massive lobbying just to use a new process, and other scenarios. A startup doesn’t usually have the time or resources to tackle these scenarios. The alternative approach is to take on a change-management consulting role, which usually doesn’t capture enough value to be VC-fundable so it needs to be profitable early on. Additionally consulting runs head-long into the issues in point #4 above.

In some ways this point is intertwined with #4.

  1. Startups shouldn’t tackle market, channel, AND technical risk
    New technological paradigms usually come with all three of technical, market, and channel risk. You simultaneously need a lot of work to get the technology to a point where it’s competitive with alternatives, to figure out who wants to use it, and to establish how they’re going to buy it.

It’s common wisdom among VCs that it’s not a good idea to invest in companies that are taking on two, let alone three of these risks. This heuristic isn’t misguided. The chance of failure goes up dramatically when an organization tries to tackle tow, let alone three of those risks. Startups should only tackle one (or maybe two) of these. The reason therapeutics focused startups can exist is that they have almost no market or channel risk - there is a well established pipeline to being acquired by a pharma company and insurance companies are guaranteed to pay for drugs that go through FDA approval and target big conditions.

At the end of the day, the work you need to do to drive adoption of a technology is often very different from getting it to a certain performance level. The work priorities drive the type of organization you create. As much as we like to think that startups are about building technology, they are actually about selling technology and adopting it for specific markets so that they can do that better.


^1: Kaizen is Toyotas process for continuous business process improvement

Web URL for this note

Comment on this note