Most roadmaps suck

As the name suggests, roadmaps should have the properties of a map (What are the properties of maps?) Maps are multi-scale, Maps help you understand the state of something, Maps help you figure out possibilities and explicitly point out those possibilities and associated tradeoffs.

Most roadmaps do a poor job on many of these points. Instead, they feel like a mush of keywords and key phrases that someone evaluating a grant could check against and say “aha! This grant is legitimate because they’re trying to do this this thing that’s mentioned in the roadmap.”

Of course, it’s possible that creating a good roadmap is an impossible task. It may be impossible to create a roadmap that actually enables coordination and progress that would not happen otherwise. However, the assertion that most roadmaps suck is based on a list of points that are very possible to fix.

• Most roadmaps are primarily text, which makes it hard for them to be multi-scale. You should be able to glance at a roadmap at at least get a sense of what you need to do. For most of these roadmaps you need to do a deep reading to actually get a full picture. The only thing you can get at a glance from a full-text roadmap is an isolated chunk of whatever you’re looking at.
• None of the roadmaps I’ve seen talk about either who is most suited for a task and who is going to do it. This people/organizational decoupling might work on for the semiconductor industry, where everything is highly modularized, interfaces are standardized, and each organization has a clear responsibility. For less mature industries this isn’t the case and there’s still no pointers from jobs to be done to people or organizations. Roadmaps need to be coupled to people.
• Most roadmaps don’t have precise ideas of what would enable a capability step change. All aspects of a technology can be improved, but improving some metrics can actually make the difference between a tool being almost useless and super useful Continuous changes can lead to discrete differences. Sometimes these changes are going to be improved metrics, but I’m being careful not to over-focused on metrics because often the key blocker keeping a technology from being useful is more fuzzy. At the same time, sometimes numbers are necessary to enforce precision.
• Most roadmaps don’t go into the precision needed to actually decide whether a project is on a critical path towards enabling something important. Following on the previous point, most of the intermediate steps in roadmaps are incredibly fuzzy “Step 5: build improved planning algorithms.” What does it mean to improve a planning? Calculating paths faster? Being able to recalculate smoothly on the fly? etc.
• Most roadmaps don’t precisely present the state of an area. A question I have constantly in response to “In five years, we hope to be able to X, Y, Z” is “Ok, so why can’t we do those things today?” I suspect the way to answer that question is to work backwards from where you hope to be to where you are today. ie. Good roadmaps work backwards from a goal. In order to do that you need a clear idea of where you are on the things that matter.
• Most roadmaps don’t make it clear which constraints are major blockers and which are nice-to-haves from an application perspective. For example, do we need robots to have five fingered hands that are as dextrous as a human’s in order to take care of older adults? Probably not! Do they need to be able to locomote over carpet? Absolutely.
• Most roadmaps don’t make it clear which metrics matter and which metrics don’t matter. Different metrics are important to different applications.
• All the roadmaps I’ve looked at don’t distinguish between exogenous changes that they’re counting on and endogenous changes in the field that someone reading the roadmap needs to go do. Instead, the roadmaps just use the blanket statement “X should be possible or true in five years.” The litmus test is whether X would still happen if everybody reading the roadmap went and drank mai tais on the beach for five years. In the context of robotics, for example, Moore’s Law chugging away or shifts in the ratio of older adults to working-age people are exogenous changes. Faster path-panning algorithms are endogenous. It’s important to distinguish between endogenous and exogenous predictions because exogenous predictions can be core assumptions, while endogenous predictions need to be broken down into “X will happen in five years IF A, B, and C” so that if A is behind schedule you can update your expectations for X.
• Most roadmaps don’t talk about different scenarios. It’s absurd to think that there will simultaneously be no exogenous changes in the world and you are absolutely right about each step of the plan. Plans can change in a definite worldview. I would hope to see statements like “The primary path to success looks like A -> B -> C, but if B fails to achieve X, an alternative path looks like E -> F -> G.” This sort of thinking allows you to assign some sort of weight to the chance that different scenarios would play out and spend effort effectively. Scenario planning has been a discipline since at least the 1970’s, so there isn’t much excuse. What role does scenario planning have in technological program design?
• Most roadmaps don’t break down why a timeline is what it is. Most timelines are in the form of “in the next five years, we could expect to get to this point.” That sort of estimate doesn’t give you any room to come to your own conclusion. A bulk time estimate also doesn’t let you
• Most roadmaps are not execution oriented. Concretely, most roadmaps don’t address jobs to be done. There’s a description of applications (like elder care) and ‘tools’ (like path planning) ::need a better word here:: but not “these are the challenges in path planning that need to be done in order to enable elder care.”
• Most roadmaps are dead documents. Make living documents is almost universally true but is doubly so for roadmaps. The whole point is to try to shift the frontier, so hopefully within a few months something is different about the world than when the roadmap was created - whether it’s an exogenous assumption, an endogenous capability, or even just different information about the requirements on a subsystem thanks to a failed experiment.
• Most roadmaps do not acknowledge that Program design is a coordination tool.

Examples

THIS IS NOT A ROADMAP
The Rejuvenation Roadmap | Lifespan.io - reasonable
This can’t help you make any decision https://nowandnext.com/PDF/Mega%20Trends%20and%20Technologies%202017-2050%20(Web).png

Examples of roadmaps that don’t suck

The International Roadmap for Devices and Systems is actually useful for guiding the semiconductor industry. I suspect this is because there are well-defined interfaces, most progress will be incremental, entities are fairly fungible, and there’s strong incentive for new entities to take on different pieces of the roadmap (like in the semiconductor industry .

Related

• The returns to good technical planning have decreased
• Instead of planning not working as well as people in the past thought as a society we’ve become worse at technological planning
• List of Roadmapping Examples
• Most predictions suck
• What are the properties of maps?

Web URL for this note

Comment on this note