Scott Alexander

Low Hanging fruit theory of stagnation

# Ars Longa Vita Brevis - SSC
`“Then we will talk this over, though rightfully it should be an equation. The first term is the speed at which a student can absorb already-discovered architectural knowledge. The second term is the speed at which a master can discover new knowledge. The third term represents the degree to which one must already be on the frontier of knowledge to make new discoveries; at zero, everyone discovers equally regardless of what they already know; at one, one must have mastered every previously-discovered fact before one can discover anything new. The fourth term represents potential for specialization; at one, it is impossible to understand any part without understanding the whole; at zero, it can be subdivided freely. The fifth…”`

`“The first student has no master, and must discover everything himself. He researches for 70 years, then writes his wisdom into a book before he dies. The second student reads the book, and in 7 years, he has learned 70 years of research. Then he does his own original research for 63 years and writes a book containing 133 years of research. The third student reads for 13.3 years, then does his own research for 66.7 years, ending up with 200 years. Imagine going further and further. After many generations, 690 years of research have been done, and it takes a student 69 years to master them. The student only has one year left of life to research further, leaving the world with 691 years of research total. So the cycle creeps onward, always approaching but never quite reaching 700 years of architectural research.”`

Even with people becoming experts at teaching and teaching teachers and writing textbooks, fields can get so far that people can only truly do a tiny amount of original work in them. 
### Published
11/2017
## Questions
## Thoughts
- The reason the world hopefully won’t asymptote out is that you can keep creating new disciplines 
## Related
- [[Specialists create knowledge within fields]]
- [[Stagnation and Scientific Incentives]]
- [[Forecasting S-Curves is hard]]
- [[How do you identify where a technology is on an S-Curve?]]
## Authors
- [[Scott Alexander]]
## Link
[Ars Longa, Vita Brevis | Slate Star Codex](https://slatestarcodex.com/2017/11/09/ars-longa-vita-brevis/)



[Web URL for this note](http://notes.benjaminreinhardt.com/Ars_Longa_Vita_Brevis_-_SSC)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Ars_Longa_Vita_Brevis_-_SSC)




Anna Goldstein

How do you identify where a technology is on an S-Curve?

§ARPA model

Michael Kearney

Technological S curves

The Effects of Prize Structures on Innovative Performance

Erica Fuchs

Enabling Mission Impact - Funding Strategies for High-Risk High-Reward Innovation

How to invent the future Recreating ARPA, the most successful research agency in history

A specific focus for PARPA

# Funding Breakthrough Research
P1
- ARPA launched in 1958
- DARPA budget ~$3B
- ~100 program managers in darpa
- ARPA case studies
	- Missile defense
	- Stealth technology
	- GPS
	- The internet
	- Autnomous navigation
	- Personal computer
	- The laser
	- Microsoft Windows
- Other ARPA-modeled agencies are HSAPRA, IARPA, and ARPA-E
P2
- Hard to evaluate ARPA because it specifically shoots for high-risk high return outcomes
	- Patents and even commercialization don’t work
`While DARPA has been lauded for significant technological successes ex post (e.g. the internet, GPS, etc.), ARPA-E, a relatively young agency, has yet to be associated with similar outsized wins, making the challenge of intermediate evaluation more difficult. `
The ARPA model has four characteristics
1. The flexible nature of the organization
2. The identification of “Technological whitespace” and designing programs to fill the void
3. The freedom of program managers to choose their projects
4. The active management of each project using specific milestones and timelines
P3
- Darpa has changed focus over time in terms of timescale and domains
- Sputnik inspired mission was “prevent technological surprise”
- Focus on the best people since the 60’s
- DARPA does take unsolicited proposals
P4 
- Vietnam war shifted DARPA focus towards more near term and military applications - “direct and apparent relationship to a specific military function” 
- 1987-1992 DARPA has a focus on industrial competitiveness and dual-use technology
- Over time different directors changed focus and tightness 
- Presidents affected who was director
P6
- Roland and Shiman empirically studied the Strategic Computing Initiative
- Roland and Shiman point out that DARPA’s role was to get research into the limelight that would otherwise be neglected
- Roland and Shiman point out that DARPA played a role as “connectors” in a research ecosystem
- Colatat implies that DARPA pushes people to have novel collaborations
P7
- DARPA is isolated from normal operations
- Independent program managers are key to DARPA success
- Famous program managers
	- Kradeep Khosla
	- Zacharay Lemnos
	- Arati Prabhakar
	- Robert Taylor
P9
- Program managers are responsible for 2-4 of the four attributes of the program (see P2)
- Fewer layers of beaurocracy (ie fewer people with veto power) enable DARPA to be able to flexibly respond
- DARPA has 3 layers, ARPA-E has 2 layers
- Focus on external research enables fewer layers
- Flexibility on hiring both who and how much they’re paid - don’t have to go through normal federal hiring channels and restrictions 
P10
- Fixed term for program managers pulls new ideas into the org and seeds the world with alumni
- Flexibility on how they fund and pay for things
P11
- There is a level of pitching from the program
- Coming up with the program can take up to 18 months of a 36 month contract
- Programs are typically 3 years
- DARPA program managers talk to both military stakeholders and scientists
- They fly groups of scientists in to brainstorm together 
- DARPA program managers attempt to address bottlenecks in the innovation systems when designing programs
- Program approval often requires specific technical targets which create concrete benchmarks
P12
`It enables program managers to deliberately connect `
 `“You get communities together that don’t naturally talk and you give them some latitude and some life, and you push them forward and see what comes out of it” `
- Program managers build a goal focused community
P13
- Peer review has a bias against novelty
- APRA model avoids the peer review bias against novelty by giving program directors discretion on who to fund
- The gateway in the DARPA program is on the program idea itself and then it’s blackboxes
- ARPA-E funding is under more scrutiny than DARPA because of politics - they have a slightly more rule-based system
P14
- Fund competing ways of doing the same thing
- Program directors connect researchers who would never have met in the hopes that they will submit good proposals
- The discretion of program managers to build teams and fund unconventional proposals is illustrated by the case study of Robert Taylor and ARPANET
P15
- Active project management is “best practice” in industry R&D
- Active project management is created by having “real options.” These include:
	- Delay
	- Abandon
	- Contract
	- Expand
	- Switch
(Huchzermeir and Loch)
- Most grant-giving agencies give researchers independence once they have funds
	- counter Example: HHMI gives tons of independence which in fact gets better results than NIH researchers with tighter restrictions
P16
- ARPA model moves the creativity and independence from the researchers to the Program director/managers
- 1960’s and 1990’s had less tight control and 1970’s and 2010’s had more tight control 
- Killing projects that are performing poorly statistically enables arpa to avoid throwing good money after bad and therefore take on riskier projects
P17 
- You can’t just take one of the practices apart from the rest
- The bundling of different practices is called coherence
- The coherence of the ARPA model revolves around the empowered talented independent program managers.
P18
- ARPA models require 
	- Clear mission
	- friction in the markets for ideas and technology
	- Nascent S curve research areas
P19
- Program design process needs to be able to start with the mission
- The mission needs to be associated with quantifiable goals
- Mowery argues that the single customer is important to DARPA
P20
- Defines applied research as performance improvement of existing technologies
- Existing technologies = existing technological paradigm
- ARPA-E awards have higher rates of publishing and patenting
- Foster* introduced the concept of technology S-curves in 1986 where performance follows a sigmoidal function of research effort
P21
- ARPA model works best at the beginning of the S curve: the curve exists, but hasn’t hit stride
- ARPA model does best in part of [[Edison’s Quadrant]] where the use is nascent
P22
- Claims private sector underinvests in R&D because firms cannot capture all the benefits of an innovation
- Contrasts pharmeceutical and energy market in terms of existing pathways and not despite similar costs
P23
- It’s hard to measure the impact of ARPA model programs because the timescales are so long to actually transform an industry
- Paying different research orgs is hard
- Independence means that outcomes are strongly tied to talent
24
- Program managers don’t have as much skin in the game
- Program design requires researchers to share - need to build trust with them
- It is hard to maintain culture with a high turnover rate
	- One culture mechanism is alumni community helping review proposals
P26
- Two axes of innovation management strategies
	- Idea Generation
	- Project Execution
- Idea generation can either be investigator initiated or a mission-inspired solicitation
- Project execution can be driven by Investigator freedom or empowered program staff
- Examples of investigator initiation+investigator freedom
	- [[Howard Hughes Medical Institute (HHMI)]]
	- [[National Institute of Health]]
	- [[National Science Foundation]]
- Examples of mission-inspired solicitation + investigator freedom
	- [[Gates Foundation]]
	- [[Chan Zuckerberg Initiative]]
- Examples of empowered program staff + Investigator initiation
	- 
## Questions
- Quadrants of Innovation management strategies seem to very much align with sources of funding - is there a fundamental reason why that is?
- Are 
## Authors
- [[Anna Goldstein]]
- [[Michael Kearney]]
- [[Pierre Azoulay]]
- [[Erica Fuchs]]
## Link
[Dropbox - Funding Breakthrough Research.pdf - Simplify your life](https://www.dropbox.com/s/hyh0awvpuh9fn76/Funding%20Breakthrough%20Research.pdf?dl=0)

                          

[Web URL for this note](http://notes.benjaminreinhardt.com/Funding_Breakthrough_Research)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Funding_Breakthrough_Research)




Stagnation and Scientific Incentives

Why did funding breakthrough research only focus on DARPA and ARPA-E?

Ars Longa Vita Brevis - SSC

Several technology trends make a reasonable argument that there is a why now for telerobotics

The math and Economics of S-Curves

# How do you identify where a technology is on an S-Curve?	

[[Funding Breakthrough Research]]



[Web URL for this note](http://notes.benjaminreinhardt.com/How_do_you_identify_where_a_technology_is_on_an_S-Curve)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/How_do_you_identify_where_a_technology_is_on_an_S-Curve)




Stagnation

neumannOneProcess2020

§Tech Tree Model of Heuretics

Graduate students are the labor in academic science and engineering

Is the rate of scientific progress slowing down?

Academia incentivizes novelty, not focus

Einstein would have been stuck in a patent office - failures of paradigm-shifting science

Human capital theory of stagnation

Mikko Packalen

Stagnation and Scientific Incentives argues against the low hanging fruit hypothesis by historical analogy

Lee Ricketson conversation 6/5/20

Jay Bhattacharya

Stagnation, scientific incentives, and funding models - Roots of Progress

People are incentivized to work on nodes higher up the tech tree

Notes on the Endless Frontier Act

Notes on the changing structure of American Innovation

Heuretics cannot exist separate from humans\*

# Stagnation and Scientific Incentives
### On Low Hanging Fruit
[[Low Hanging fruit theory of stagnation]] is `convenient as it absolves scientists, institutions, and analysts from any responsibility for the slowdown, it is likely wrong. One problem is that this explanation is not unique to our era. `

### Core Hypothesis
`In this paper, we argue that the root of the slowdown in scientific progress lies in the more quotidian fact that scientific work on truly novel ideas that have the potential to develop into groundbreaking advances is no longer rewarded in the same way it once was. `

### Theories for problem
`if also novelty metrics were utilized in scientist evaluation, scientists might pursue more innovative, riskier, projects.`
Changes in scientific incentives→ slowing science innovation→ economy
Citations shifted behavior towards incremental science
- Science requires exploration and play
- Lifecycle of scientific ideas look like S curves
	- But the height of the S curve is variable, and it’s hard to know what its parameters are
- Citations make scientists more conservative
	- Eugene Garfield developed the idea of citations as a measure of science productivity
		- ::How did it work before?::
	- `The citation revolution started to gather steam in the 1970s.`
		- ::[[WTF happened in 1971? - Article]]::
	- - `In recent decades the largest scientific funding agency, the National Institutes of Health (NIH), has also become less likely to support novel work despite its best efforts to the contrary.`
![](BearImages/DE686EC7-350A-4C31-BBA9-ADB71B8D9B25-68756-0005E4EDFA984E25/AF030A33-7420-4241-ABFB-F2DEAD21CB5A.png)

- Academic Search Engines solidify citation obsession
	- Only see citations, not what sort of exploration it was
	- ‘winners win harder’
	- ::The internet helps the tail *if search is good* and in this case it’s not::
	- Just counting papers is a possibility that wouldn’t help either
	- Science administrators are the ones focusing on the citations
	- Possible solution is to measure and present exploration level
	- Baseball players are evaluated on more dimensions than scientists: see [[Sabermetrics]]
- Measureing exploration in science
	- Pushback that scientific stagnation is inevitable 
	- `Of course, to reward scientific novelty, we must first measure it` 😭
		- Suggest a textual analysis of research publications
			- [[Undiscovered public knowledge discovery overindexes on things that have good databases]]
			- [[Most human knowledge is not encoded]]
	- Idea of Edge factor + Impact factor
`The third possible path forward could put science back on the path envisioned by Vannevar Bush by broadening how scientific productivity is measured and evaluated. On this path, in addition to scientific impact, we start measuring and evaluating research based on what kind of science it represents—whether the work is novel in that it tries out new ideas or more conventional in that it seeks to advance well-established ideas.`

### Published
2/2020
## Questions
- Which science friends care about citation counts - in some fields it’s much more about who knows you. Might be similar to startups in that if the right people you know you but there’s a dominant game otherwise.
- Could citation counts be like customers?
## Thoughts
- Funded by a grant from [[National Institute on Aging]]
- The administration of science by people who don’t get the science is a problem
- [[Stagnation and Scientific Incentives argues against the low hanging fruit hypothesis by historical analogy]]
## Related
- [[Technological S curves]]
- [[How do you identify where a technology is on an S-Curve?]]
- [[There is an increasingly wide gap between the number of graduate students training for PhDs and academic positions for them to fill]]

#### Cites
- [[Are Ideas Getting Harder to Find?]]
- [[Is the Force Awakening?]]
- [[Science is getting less Bang for its Buck]]
- [[Is the rate of scientific progress slowing down?]]
- [[Quantifying the Ease of Scientometric Discovery]]

#### Non-cited refs
- [[Is Science Slowing Down? - SSC]]

## Authors
- [[Jay Bhattacharya]]
- [[Mikko Packalen]]
## Link
https://www.nber.org/papers/w26752

[Web URL for this note](http://notes.benjaminreinhardt.com/Stagnation_and_Scientific_Incentives)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Stagnation_and_Scientific_Incentives)






19 Jul 2020

# The math and Economics of S-Curves
### Related
- [[Forecasting S-Curves is hard]]
- [[How do you identify where a technology is on an S-Curve?]]
- [[Sigmoid functions occur when a process has a rate proportional to the amount of a consumable or reversible and autocatalytic]]
- [[Sigmoid Functions: Some Approximation, and Modeling Aspects]]
- [[Could you get S-Curves to pop out of a tech tree and human perception?]]



[Web URL for this note](http://notes.benjaminreinhardt.com/The_math_and_Economics_of_S-Curves)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/The_math_and_Economics_of_S-Curves)




The ability to pull funding is a clear reason to ‘black box projects’

Blue Sky vs White Space

Each additional mission in the government allows another person to stick their nose in the process

Triumphant innovation is unexpected and surprising

What if you could think of PARPA as Skunkworks for VC?

What if your mission was “create technological surprise?”

What would VC financing look like if they could take their money back from companies that weren’t doing well?

Why is “advance technologies that could be critical to national security” quantifiable but “revolutionizing our understanding of the human brain” is not?

# Why did funding breakthrough research only focus on DARPA and ARPA-E?
[[How do you identify where a technology is on an S-Curve?]]



[Web URL for this note](http://notes.benjaminreinhardt.com/Why_did_funding_breakthrough_research_only_focus_on_DARPA_and_ARPA-E)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Why_did_funding_breakthrough_research_only_focus_on_DARPA_and_ARPA-E)




Ben's working notes

How do you identify where a technology is on an S-Curve?

Links to this note