The best DARPA program managers are the ones who can look at an entire literature in an area and notice a systemic bias

Exploratory Program Organization

DARPA Program managers pull control and risk away from both researchers and directors

DARPA PMs need to think for themselves, be curious, and have low ego

It is hard for a private ARPA to capture value without hamstringing impact

Where do ideas come from?

By definition outlier analysis can’t rely on data as comparison to other organizations

Who is the ARPA Model for and what do I hope to achieve

DeepMind combines bottom-up research work with top-down strike teams

Lessons learned from publishing ARPA Model

ARPA Model Conclusions

Startups and Academia are bad at addressing structural constraints

Why try to emulate DARPA and not Bell Labs

Why doesn’t ARPA use prizes instead of grants?

ARPA-E faces different industry incentives than DARPA

DARPA works on a mix of Big H and Little H hard problems

# §ARPA model
This is not meant to be a primer on *how* the ARPA model works or a history of DARPA. See references for plenty of good writing on that. It is *not* meant to be an unbiased presentation of the facts or argue for a general thesis about the world (though it hints at some.) The explicit goal of this analysis is to acknowledge that DARPA is a massive outlier and to figure out which of the attributes of DARPA contributed to its outlier results with the explicit intention of creating more organizations that can enable more outlier results.  [[§How do we get more awesome sci-fi shit?]]

The nature of outliers means that you can’t do a data-based analysis. Perhaps controversially, I think it’s foolish to try to find patterns among outliers because outliers are outliers in *different* ways except in the most broad strokes. Instead, I dig into the things that seem distinctive and ask why they might lead to outlier results.  At the end of the day, this is storytelling. But stories are powerful.  Of course, it’s easy to take this approach too far and create a story about how  *anything* weird about an outlier contributed to their success. “Steve Jobs only wore one outfit which is important because it allowed him to have EXTREME FOCUS.”  I try to avoid this by focusing on why the pieces lead to outlier performance *in combination* - many people who try to replicate outliers fail by picking and choosing the pieces they want to replicate.

There is some relevant history to deciding what to pay attention to without [[Cargo cult]]ing.

- [[The definition of the ARPA model has changed over time]]
- [[ARPA became DARPA in 1972 because of the increased scrutiny on military spending both in the government and outside of it]]
- [[Most of DARPA’s outlier output happened before 1972 when it was ARPA]]
- [[Was the shift from ARPA to DARPA a focus change or a process change?]]
Upshot:
- [[Pay attention to DARPA’s informal process and ignore formal process]]

### Program Managers
Anything written about DARPA agrees that it’s all about the program managers.
* [[DARPA Program managers pull control and risk away from both researchers and directors]]
* [[The best DARPA program managers are the ones who can look at an entire literature in an area and notice a systemic bias]]
*  [[DARPA is incredibly flexible with who it hires to be program managers]]
* [[DARPA Program managers have a tenure of four to five years]] 
* [[Why do people become DARPA Program managers?]]
* [[A large part of a DARPA program manager’s job is focused network building]] 
	* [[Maverick work is lonely and DARPA provides peers and community]] ::new::
* [[DARPA PMs need to think for themselves, be curious, and have low ego]]
*  [[The dependence of DARPA on high quality program managers mirrors the obsession with “talent” in other disciplines]] ::Updated::

### Incentives and Structure
* [[The government’s influence on DARPA is buffered by opacity and the director]]
* [[Opacity is important to DARPA’s outlier success]]
* [[DARPAs director is important]] ::needs work::
* [[DARPA does multiple levels of top-down problem generation and bottom-up solution generation]]
* [[The transient nature of most people at DARPA enables ideas to be revisited]]
* [[DARPA is relatively tiny and flat]] 
* [[DARPA Employees aren’t paid very much compared to what they could be]]
* [[DARPA’s aversion to people with a web presence may be how they avoid asymmetric career risk]]
* [[DARPA doesn’t do any research in house]] ::Updated::
* [[DARPA works on programs that go against the established paradigm]] ::new:: ::needs work::
* [[The ARPA Model is an unbuffered system]]::new::
* [[DARPA Staff are Important]] ::new:: ::needs work::
* [[Negative results are important to DARPA]]

### Funding
* [[DARPA is ~0.5% of the Department of Defense Budget and ~12% of that is basic research]]
* [[DARPA is more ideas limited than money limited]] 
* [[DARPA funds wacky things that go nowhere]]
* [[It is relatively easy for DARPA PMs to re-deploy funding]]
*  [[Program Managers have the ability to deploy money without much overhead]] 

### Process 
* [[The DARPA execution framework boils down to showing that thing is not impossible, showing that thing is possible, and then making that thing possible]] 
* [[There are many tight feedback loops built into the ARPA model]] 
*  [[The initial exploratory tranche of a DARPA program is approximately $1.5m]]
*  [[DARPA PMs use seedling programs to ‘acid test’ the riskiest pieces of a program idea]]
* [[The ‘framework’ DARPA PMs use to create a program is by modeling the tech council]]
* [[Every program at DARPA is intensely technically scrutinized by the tech council]] 
* [[DARPA facilitates idea sex both between PMs and Performers]]
* [[DARPA provides a derisking role for people in other organizations]]
* [[Decoupling from market discipline is like cave diving]] ::new:: ::Needs work::
* [[DARPA works where markets dont]] ::New::

### Riffing on ARPA
- [[A handful of organizations have riffed on the ARPA Model with mixed success]]::New::
	- Answers:  [[How does IARPA work? What are its successes?]]
* [[It’s easy to set out to build a DARPA but end up building a Skunkworks]]
* [[Most DARPA clones start too big or with heavy process]]
* [[Can the ARPA model exist outside of the government?]]

### Closed Questions
-  [[How successful did DARPA look ten years in?]]
	- Pretty successful

## Open Questions
Because a lot of this is just storytelling there are a lot of open questions about different knobs, what happens when you turn them and what they mean in different domains.  

* [[When is externalized research better than internalized research and vice versa?]]
*  [[Is it essential that a research org is a small fraction of the money factory’s budget?]]
*  [[Why doesn’t ARPA use prizes instead of grants?]]
* [[Is there a minimum effective budget to generate outlier results?]]
- [[How could the ARPA Model take advantage of the internet?]]

## Conclusions
* [[ARPA Model Conclusions]]
* [[Things about ARPA that can be improved]]

## Related

## References
* [[Funding Breakthrough Research]]
* [[“Special Forces” Innovation: How DARPA Attacks Problems]]
* [[Sam Moleneux convo 16/1/20]]
* [[Sam Moleneux Conversation 31/10/2019]]
* [[Know When to Fold ‘Em: An Empirical Description of Risk Management in Public Research Funding]]
* [[What makes DARPA tick]]
* [[What else makes DARPA tick]]
* [[Cloning DARPA Successfully - Article]]
* [[Rethinking the role of the state in technology development: DARPA and the case for embedded network governance]]
* [[Mark Micire Podcast]]

 
          
[Web URL for this note](http://notes.benjaminreinhardt.com/§ARPA_model)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/§ARPA_model)




DARPA does multiple levels of top-down problem generation and bottom-up solution generation

How does IARPA work? What are its successes?

# Email w Daniel Windham Part 2
- Assertion: government is the organization with the best long term incentives 
- Assertion: 20 year out thinking can be bad
- The null hypothesis for an impactful org is to grow the orgs that work
- Questions: which orgs work? Why?
- Would growing an org destroy them
- What does it mean to “make more things like DARPA”
Related to [[Conversation with Daniel Windham 31/10/2019]]
[[Daniel Windham]]

                            

[Web URL for this note](http://notes.benjaminreinhardt.com/Email_w_Daniel_Windham_Part_2)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Email_w_Daniel_Windham_Part_2)




§ARPA model

# How does IARPA work? What are its successes?
## Related
[[IARPA Process]]
## References
[[Email w Daniel Windham Part 2]]
             



[Web URL for this note](http://notes.benjaminreinhardt.com/How_does_IARPA_work-_What_are_its_successes)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/How_does_IARPA_work-_What_are_its_successes)




# IARPA Process
IARPA follows [[The Heilmeier Catechism]]
One way that an IARPA program manager fleshes out an idea is to hold a 1-2 day workshop bringing together potential researchers and users - advisory and informal ([Workshops](https://www.iarpa.gov/index.php/working-with-iarpa/workshops))
Phase one contracts can be $5.2m (see [[Raytheon BBN gets 1.7M federal deal to develop automation system]])
Programs can be estimated at $10-15M per year
           


[Web URL for this note](http://notes.benjaminreinhardt.com/IARPA_Process)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/IARPA_Process)




Ben's working notes

How does IARPA work? What are its successes?

Related

References

Links to this note