Systems must take performance hits to get out of local optima

Lock-in at Local Optima is often caused by Structural Constraints. There are many systems where interlocking dependencies mean that you can’t improve the system much by changing one thing at a time (point fixes) but instead you need to ~simultaneously change multiple things, in the same way that you can only pull off certain moves by hitting multiple buttons at the same time in a video game.

Even if you are able to change multiple pieces of the system simultaneously, it usually results in an initial performance hit. Most people will not accept performance hits for the sake of long term benefits. People are more averse to short term losses than they are attracted to long term gain.

The cost in money and time of getting out of a local optimum combined with the inevitable performance hit is a major reason why Getting to new optima is hard.

So how do you break this chicken and egg situation?

  1. Heuretics can develop in niches where the performance of the current system is not as good. A niche can sustain a lower-performing but higher potential technology while it grows into a form that can take on the current paradigm. This is why Frontier technologies need to start in niche markets where they are especially valuable. This niche-and-expand is how disruptive innovation happens - see the classic examples from Clayton Christensen: PCs vs Mainframes, Minimills vs Steam Mills. The niche-and-expand method of development works when there is a fog of war around a heuretic’s potential. The phenomenon of seemingly trivial technologies becoming incredibly important happens because there is fog of war on the tech tree.
  2. Heuretics can develop while decoupled from the the market system. There are many ways this can happen - slow tinkering in an academic lab, as the secret side project in a company, as a rich person’s company (Blue Origin), etc. Every successful example of this strategy eventually circles back to something that the market actually likes. This strategy often leads to making something that nobody wants (Magic Leap) but at the same time can enable things that don’t have an immediate niche. In a way keeping promising technologies alive independent of the market is part of Publishing feels more like hardware development than software development’s role. DARPA provides a derisking role for people in other organizations - UAVs and PC technology, for example. If you’re going to decouple from market discipline you better have a plan. Technological roadmapping can give an idea of where technological evolution can go and how to get there. Decoupling from market discipline is like cave diving.

In reality there is a mix between no.1 and no.2. Many technologies spend a lot of time in development and then move into a niche. SpaceX took six years to launch its first rocket

There are tons of examples of this in history:


Web URL for this note

Comment on this note