Graduate students are the labor in academic science and engineering

People giving out grants try to derisk them as much as possible

Manufacturing is important and often ignored

The word ‘novel’ is used as an idea bludgeon in academia

All the incentives in research drive towards conservatism

Peer review doesn’t catch duplications

Bell Labs is one of the few examples of an innovation org that was aligned with its money factory

A new professor needs about $1.5m to get going

Academic culture prizes individual recognition

Younger professors have to spend more time getting money

Academics are incentivized to do peer-reviewed publishable work

Things that are not paper-worthy enough for academia and not product-focused enough for a startup

Pay-it-forward tithing could resolve the tension between the potential public-good and potential profit in research.

The idea that academia is for crazy exploration and companies are for low risk exploration is bullshit

Three myths about scientific peer review

A PI needs to provide evidence that they have access to a lab facility

Different axes of where research gets done in the US

What was the real incentive for people to work on the DARPA Grand Challenge?

It is easier to have a multipoint mutation in technology than in evolution

NS Notes on Officers Techniques

Necrobotics illustrates how academia forces engineering

The boundaries between industry government and academia are fuzzy

Startups and Academia are bad at addressing structural constraints

The Architected Innov􏰂ation Institute - Engineering Scientific Revolutions

There are very few senior individual contributors in academia

Is it more impactful to find and train undiscovered diamonds or to unlock well-positioned but constrained researchers?

Wolfgang Merkt conversation 12 Oct 2021

An institution riffing on the ARPA Model could enable paradigm shifts that are not happening

Academic administrators and professors are playing very different games

An Institute for Creative Technology - Riffing on ARPA

Funding structures lead to several layers of indirection on what even gets proposed

Is it possible that we’re actually getting less research because we have more researchers?

Running list of relevant science news?

# §Academia Constraints
- [[Academia is the game where you gain status by getting attention for new knowledge]]

- [[Universities have multiple conflicting missions]]
- [[Universities have been pushed more towards applied research since the end of WWII]]
- [[Academic credit is a first-to-the post system]]
-  [[Graduate students are the labor in academic science and engineering]]
- [[People giving out grants try to derisk them as much as possible]]
- [[Professors do three or more jobs]]
- [[Successful professors are not necessarily the best researchers]]
- [[University professorships are the highest status jobs in academia]]
- [[National labs and other government research centers are a research alternative to universities]]
- [[The game in national labs is roughly the same as academia but with different constraints]]
- [[In academia more grant money leads to more power]]
- [[Academic research is often just the concept half of creating a heuretic]]
- [[Professors are constantly doing the calculation of whether a grant is worth the time to apply for it]]
- [[Most grant money is tagged so unrestricted money is like gold for professors]]
- [[Universities can take more than half of grant money as overhead]]
- [[A PI needs to provide evidence that they have access to a lab facility]]
- [[Physical experiments are expensive]]
- [[Younger professors have to spend more time getting money]]
- [[The government grant process depends on politics and committees]]
- [[Any technology that comes out of a university needs to go through their tech transfer office]]
- [[Months of grant writing is one of the things that makes people think twice about academia]]
- [[Academia incentivizes novelty, not focus]]
- [[Academia is not a good environment for systems engineering]]
- [[Academics are incentivized to do peer-reviewed publishable work]]
- [[Going outside of the approved process can be a black mark against people in academia]]
- [[People proposing work that they have already done is a sign that feedback loops are broken]]
- [[Necrobotics illustrates how academia forces engineering]]
- [[Labs implicitly judge research through the sense of dollars per paper]]

### Related
- [[§Startup Constraints]]
- [[§Corporate R&D Constraints]]
- [[§Philanthropy and Non-Profit Constraints]]
- [[Market Failures in Science]]

[Web URL for this note](http://notes.benjaminreinhardt.com/§Academia_Constraints)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/§Academia_Constraints)






§Academia Constraints

Asymmetric career risk

The definition of the ARPA model has changed over time

Current value capture mechanisms are crude

How does money work in PARPA?

“The System” of AT&T

Problems that require system-level solutions often require massive teams

Top of mind

National labs and other government research centers are a research alternative to universities

Many innovations are not valuable on their own but can be combined to be valuable

Healthy R&D orgs feel like they have very smooth project rampups with high ceilings

AI research is an exception to the decline of corporate R+D

We need new institutional structures to fill the role that corporate labs once occupied

Government research is corporate R&D for United States Inc

Profit-maximizing organizational structures can hamstring technologies whose impact depends on them

R+D labs require companies that can win based on Technology

Research programs that are constrained from existing

# §Corporate R&D Constraints
 Follow innovation through bell labs
	* Run into problem with “the system” - can’t fit enough calls into a single cable. More cables are expensive. Want people to make more, better, calls. Problems constantly coming down from the top and 
	* 
* Follow Innovation through GoFaAmSoft
* Politics 
	- [[Internal incentive structures within a large organization work against high quality R&D labs]]
	* [[R&D orgs are always on the side of an org chart - The lab is always separated from the mothership]]
	* [[Rewards in a big company operate on one timescale and R&D happens on another timescale]]
* The most valuable research for a company tends to be systems-level improvements. But [[Problems that require system-level solutions often require massive teams]] while [[Research tends to work best when it can be done by small teams]].
* Risk aversion
	* [[Asymmetric career risk]] 
	* [[Asymmetric career risk impacts corporate R&D]] 
* Incentives
	* [[The incentives inherent to corporate R&D mean that researchers motivated by money leave and those who are motivated by glory are not the best in the world]]
	* [[At the organizational level corporate R&D labs either cause brain drain or don’t have the best people]] 
* [[R&D orgs are expensive]]
* [[R&D orgs are answerable to shareholders both in scope and timescale]]
* [[Companies want R&D to be off of their balance sheets]]
* [[The siloization that tends to happen in large corporations hamstrings applicability and serendipity]]
* [[Big companies cannot start small things]] 
* [[R&D efforts need to be connected to a core product to maintain support]]
* [[There are several macro-economic analyses of both why and whether there has been a decline in corporate R&D]]
* [[Corporate R-D orgs experience a tension between Research and Development]]
* [[Corporate R&D orgs have a lot of momentum]]
* Trends
[[Corporate Innovation has four main options: partner, R&D, corporate venture capital, and venture building]]
<draw 2x2 box>
	* Externalize R&D through startups and universities 
	* Internal startups via Red Box at Swisscomm or Area120 at Google
	* “Innovation Arms”
	* Grants
## Related
- [[§Startup Constraints]]

                            

[Web URL for this note](http://notes.benjaminreinhardt.com/§Corporate_R-D_Constraints)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/§Corporate_R-D_Constraints)




Charities raise money on a yearly or sub-yearly basis

Most people give money to philanthropy to make themselves look and feel good

The traditional non-profit structure is a bad idea for an organization that requires a significant chunk of capital and produces uncertain results on a long timescale

What if there was a clean philanthropy-to-government policy pipeline?

Alex Rider Convo 17-1-2020

The C3SOP structure appears to allow non-profit employees to get stock options

Nadia Eghbal conversation 7 Jul 2021

Lots of philanthropy is based on small checks even from big donors

There’s lots of money out there … for good investments

# §Philanthropy and Non-Profit Constraints
Philanthropy is the funding mechanism and Non-Profits are the Organization in an Organization-Funding Diad. ([[Organizations and their funding mechanisms are coupled]])

- Profit in non profits goes back towards the mission
	- Could this involve investing it?
- Non-profits must spend all of their income every year 
	- This can lead to very cyclic activity 
	- Cyclic hiring harms employee’s ability to think long term because there is no buffer to pay them in income draught periods
	- Are universities non-profits? They invest their excess income. 
- Non-profits need to constantly be fundraising 
	- Is this because they need to spend down every year?
	- This means that they can’t do anything that looks bad in the short term 
- What’s the difference between a foundation and a non-profit? 
	- A foundation is a *type* of non-profit. The other type is Public charities.
	- The money for foundations comes from an initial gift from an individual or business and spends out of interest. 
	- Foundations can only spend in ways that adhere with 501(c)(3) of the Internal revenue code
	- Foundations can be operating or non operating
		- A non operating foundation give money to orgs
		- An operating foundation distributes funds to its own purposes
	- Public Charities
		- A charity must receive 1/3 of its contributions from the general public OR at least 10% from general public + subjective test of “support”
		- Public charities have higher donor tax deductibles
			- How much?
		- [[Only charities can receive tax-deductible donations]]
	- 501(c)(3)s all are exempt from paying income tax, but only charities exempts donors from income tax
		- [What rules do charities have to follow? | HowStuffWorks](https://money.howstuffworks.com/economics/volunteer/starting-a-charity/rules-charities-follow.htm)

- [[Funders want to follow other funders]]
- It’s easier for foundations to pay out grants than anything else ([A “Watch-Out” for Private Foundations Offering Prizes](https://ssir.org/articles/entry/a_watch_out_for_private_foundations_offering_prizes))  but  [[People giving out grants try to derisk them as much as possible]]
- [[Charities raise money on a yearly or sub-yearly basis]]
- [[Most people give money to philanthropy to make themselves look and feel good]]
- [[Philanthropies can easily become misaligned with their money factories]]
- [[Philanthropies are forced to show consistent short term progress]]
- [[Most philanthropists give to specific causes]]
- [[Non-profits must meet several requirements to Maintain 503c Status]]
- [[People have different and unknown caring money thresholds]]
- A possible tweak: [[Pay-it-forward tithing could resolve the tension between the potential public-good and potential profit in research.]]

See also: [[§Misalignment between funders and nonprofits in research]]

[Web URL for this note](http://notes.benjaminreinhardt.com/§Philanthropy_and_Non-Profit_Constraints)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/§Philanthropy_and_Non-Profit_Constraints)






Decoupling from market discipline is like cave diving

When should an idea that smells like research be a startup?

§Corporate R+D Constraints

Startups are good at point changes

Systemic decay theory of stagnation

Ideally corporate R+D enables work on a general purpose technology before it’s specialized

There are many tight feedback loops built into the ARPA model

One function of a private ARPA is to fill in the gap once filled by corporate R+D

There are many important innovations that are just bad investments

Startup Constraints - essay

The market in for-profit organizations is fairly efficient

PARPA could walk the line between profit and non profit

Startups  need to spend effort on sales, marketing, and productization

# §Startup Constraints
- [[Startups need to make a lot of money or have a lot of users]]

- [[Startups need to conform to investor timescales]]
- [[Venture capital doesn’t work for things that only realize their value on long timescales]]
- [[Startups need an exit strategy]]
- [[Startups need a great story]]
- [[Startups need to produce a scalable product]] 
- [[Startups need to break development into correctly sized stages]]
- [[Startups need specific growth curves]]
- [[If a project has massive possible upside and low enough risk it is venture fundable]]
- [[Scalability and venture fundability happens when you turn a process innovation into a tool innovation]]
- [[Startups are good at point changes]]

- [[Startup Constraints - essay]]

[Web URL for this note](http://notes.benjaminreinhardt.com/§Startup_Constraints)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/§Startup_Constraints)






Timesharing could be extended from digital infrastructure to physical infrastructure

Marissa Convo 1/5/2020

# A PI needs to provide evidence that they have access to a lab facility
This creates a slight chicken and egg problem and makes scientists more at the mercy of the [[Distributed Innovation Suppression Complex]] because if the institution doesn’t like you, you’re less likely to have a lab which makes it harder to show you’re right. 

### Related
- [[Younger professors have to spend more time getting money]]
- [[Decisions made by committees lead to median results]] 
- [[Professors do three or more jobs]]
- [[Successful professors are not necessarily the best researchers]]
- [[§Academia Constraints]]



[Web URL for this note](http://notes.benjaminreinhardt.com/A_PI_needs_to_provide_evidence_that_they_have_access_to_a_lab_facility)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/A_PI_needs_to_provide_evidence_that_they_have_access_to_a_lab_facility)




A large part of a DARPA program manager’s job is focused network building

Trust-building timescale and budgets tend to be similar in different domains

# A new professor needs about $1.5m to get going
# A new professor needs about -1-5m to get going
This money is meant to help set up a lab and buy equipment

According to this list of grants [New Faculty/Young Investigators | FSU Office of Research](https://www.research.fsu.edu/research-offices/ord/finding-funding/new-facultyyoung-investigators/) 
- NSF Early career - $400k + over five years
- NASA - $240k over three years
- Air Force - $360k over three years
- Navy - $510k over three years
### Related
- [[§Academia Constraints]]





[Web URL for this note](http://notes.benjaminreinhardt.com/A_new_professor_needs_about_-1-5m_to_get_going)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/A_new_professor_needs_about_-1-5m_to_get_going)




A new structure for scalable research

storrshallWhereMyFlying2021

Academia is not a good environment for systems engineering

EA is a luddite mind virus

In defense of academia outline

Systems research requires a lot of work that is expensive and uninteresting

Academics want to do timeless work

Human capital theory of stagnation

In the past, corporate labs filled a particular niche in the innovation ecosystem

Week of 11 Jan 2021

ICT Distillation

ICT Outline

Legible novelty is the worst

# Academia incentivizes novelty, not focus
 `Innovation is one of the key criteria by which grant applications and academic publications are judged. To fulfill the innovation criterion, academics are incentivized to distinguish themselves from others and to pursue novel research. In the academic sciences, novelty comes in the form of new discoveries, whereas in academic engineering it comes in the form of proofs of concept. `  [^5]

This novelty prioritization manifests in two problematic ways. First, it is one of the reasons that [[Academia is not a good environment for systems engineering]]. Second, it leads to many promising technologies failing to reach their potential like baby dinosaurs that are fossilized right as they’re hatching.::Need better analogy:: 

It’s impossible to count[^1] the number of times that someone embedded in the academic way of thinking[^2] has dismissed a project because ‘there was nothing new about it.’ This completely serious dismissal has included proposals to build a useful system that had never been built before, but each of its components had been. If your experience building things is anything like mine, you realize that there is a huge gap between “it should work” and “it does work.” And unlike putting together lego sets, there is a lot of creative and sometimes research-level work to be done in order to assemble a set of working components into a single system. 

People rarely get grants or academic accolades for doing the work to turn a one-off prototype into a technology that is actually useful. And for context, most academic prototypes are not simply the full system but custom-built by hand. They’re a duct-taped mess that requires the person who built it to poke it in just the right place and then connect a wire after waiting exactly 3.2 seconds (or the chemical, biological, software equivalent.) While demoing the prototype system that laid the groundwork for modern personal computing, Douglas Engelbart `would spend the entire presentation listening to the murmur of impending disaster: “Bill, we’ve lost XYZ!” or “Bill, we’re not going to be ready in time!”`[^3] Turning a prototype into something that people who didn’t make it themselves can use requires increased consistency, robustness, generalization, and legibility, all of which can end up requiring additional conceptual leaps. Note that some of the things Engelbart demoed in 1968 didn’t become widespread until twenty years later. Prototypes are also fraught with tacit knowledge[^4] so if their original creators don’t have an incentive to help, the chances of them becoming something more decrease. 

The charitable view is that academic rejection of building systems and going beyond prototypes is simply specialization and division of labor — they’re important activities but not academia’s institutional job. 

### Related
- [[The word ‘novel’ is used as an idea bludgeon in academia]]
- [[bhattacharyaStagnationScientificIncentives2020]]

[^5]:From [[A new structure for scalable research]]
[^1]: Partially because it has happened so many times and partially because I have a terrible memory
[^2]: It’s worthwhile to point out that the novelty-prioritizing way of thinking bleeds out into many domains but its origin is squarely in academic culture
[^3]:<sidenote> From *The Dream Machine* by M. Mitchell Waldrop
[^4]: See [[Tacit Knowledge, Trust, and the Q of Sapphire]] for an amazing exposition on this point.



[Web URL for this note](http://notes.benjaminreinhardt.com/Academia_incentivizes_novelty,_not_focus)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Academia_incentivizes_novelty,_not_focus)




Academia incentivizes novelty, not focus

Prototyping needs manufacturing in the room

Innovation requires multiple heuretics

Scientific inquiry is an abstracting process while engineering design is a narrowing process

Optimizing individual components of a system is often at odds with the system itself

Mark Johnson the Elder Conversation 4 Jan 2021

drexlerRadicalAbundanceHow2013

Healthy R+D orgs feel like they have very smooth project rampups with high ceilings

The Manhattan project was primarily grungy engineering improvements

# Academia is not a good environment for systems engineering
The academic model is set up around scientific inquiry, in the [[Eric Drexler]] sense. That means that people are rewarded for moving information *up* [[The ladder of abstraction]]. The more general your theory or technique, the more you are praised.  
![](../BearImages/7B55ED64-9596-4A57-9FC7-AC2B1A11E73C-1435-0003F270A51A6206/AB2482A3-D5A1-4559-BF55-545994CDEFE3.png)
In the Drexlerian sense, Engineering moves information *down* the ladder of abstraction. Moving down the ladder of abstraction clashes with academic incentives, leading to academic engineers creating proofs of concept for the sole purpose of validating general engineering principles or design concepts. The whole working system is left as an exercise to the reader. Proofs of concept, general principles, and designs are important! But they don’t push technology beyond [[TRL3 - Proof of Concept]].  Hence, [[The (idea) valley of death]].

Because academic engineering focuses on the top of the ladder of abstraction, there are rarely feedback loops on the implementation of a design concept. A design can check all the boxes on a list of requirements (and hence be great for a paper) but completely fail to be a useful system. My quintessential example of this is how perfect right-angle corners at the bottom of a cut-out are completely free (and the default!) in a CAD model, but are *extremely* hard to machine correctly. These weirdly important implementation details are usually [[Tacit Knowledge]] which is why [[Prototyping needs manufacturing in the room]]. 

 [[Academia incentivizes novelty, not focus]]. Academic work is always judged by “what are the new ideas here?” Much more than “does it work?” In an ideal engineering situation, you would use as few new ideas as possible to get the system to work. This tension means that academic engineers will not work on some ideas or bolt on unnecessary fancy techniques because [[The word ‘novel’ is used as an idea bludgeon in academia]]. Bolting together a bunch of old ideas to get something work is great engineering practice, while it doesn’t create any general academic insight. 

[[Academic culture prizes individual recognition]] via its use of citations as currency, etc. Many engineering projects require ‘teamwork’ in the sense of just doing the not-sexy work that needs to be done without remembering who did what. This isn’t to say that academics won’t do not-sexy work, but it does need to culminate in something paper worthy. There *are* an increasing number of academic papers with large numbers of authors, but as [this article](https://www.natureindex.com/news-blog/paper-authorship-goes-hyper#:~:text=The%20average%20number%20of%20authors,driven%20by%20the%20physical%20sciences.)notes, there is a roughly constant amount of “esteem currency” per paper so the career capital per author decreases as the number of authors increase.  Contrast to professional engineers, where reward is usually invariant to the number of people on the project and possibly tied to how successful it is, in which case the incentive is to have as many people working on it as necessary to make it work. 

![](../BearImages/B465BAA6-D9D0-4F71-9818-D1D8B1D6D494-1435-0003F44BD89092BB/15146F03-7C43-4BE1-966D-1EB5638F0DE1.png)

[[Academics want to do timeless work]] and engineered systems go obsolete! 

### Related
- [[A new structure for scalable research]]
- [[Things that are not paper-worthy enough for academia and not-product focused enough for a startup]]
- [[Research has many orthogonal and non orthogonal classification axes]]
-



[Web URL for this note](http://notes.benjaminreinhardt.com/Academia_is_not_a_good_environment_for_systems_engineering)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Academia_is_not_a_good_environment_for_systems_engineering)




People and organizations are all playing some game that has different ways of gaining status and power

The peer review and citation system incentivizes people to work on things that other people think is interesting

§Institutions and Games

Successful professors are not necessarily the best researchers

Academic research is often just the concept half of creating a heuretic

All institutions are coupled to at least one game

Motivation is not context-free

Research requires intrinsic motivation

Bottling Lightning

Universities have multiple conflicting missions

Researchers distend the meaning of discovery because of academic incentives

# Academia is the game where you gain status by getting attention for new knowledge
People keep track of their ‘score’ in citations, published papers, grant money, awards.

### Related
- [[People and organizations are all playing some game that has different ways of gaining status and power]]
- [[Academia 



[Web URL for this note](http://notes.benjaminreinhardt.com/Academia_is_the_game_where_you_gain_status_by_getting_attention_for_new_knowledge)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Academia_is_the_game_where_you_gain_status_by_getting_attention_for_new_knowledge)




Notes and statements should be bloody

Is peer review a good idea - paper

DARPA facilitates idea sex both between PMs and Performers

# Academic credit is a first-to-the post system
Everybody in academia pays attention to ‘who was the first person to discover this thing.’ The way that it counts is by publishing something about it.  

The first-to-the-post system and the existence of preprint servers means encourages people to create a lot of false positives. This would not be a problem if there was a good way of alerting the world that something was a false positive but there is no good system for that. 

The first-to-the-post system makes everybody obsessed with being “scooped” because the first person to publish an idea gets the credit even if someone comes along afterwards and explains it better. Of course, it doesn’t always happen like this and you also get into fights where there is a better known expression of an idea and the person who originally came up with it spends an inordinate amount of time yelling that they came up with it *cough* Schmidthuber *cough* 

This is why [[The word ‘novel’ is used as an idea bludgeon in academia]] - extending an existing idea is not in high esteem even if it’s an important reframing that can unlock something important. 

The first-to-the-post system doesn’t encourage dispersing a concept beyond making sure everybody knows that you came up with it and will cite you as such. ([[Dispersion is the process of a heuretic impacting the world]]) But [[Just having the piece of literature doesn’t solve the problem]]. 

### Related
- [[weinsteinBretWeinsteinPrediction]]
-  [[Manage the Transfer not the Technology]]



[Web URL for this note](http://notes.benjaminreinhardt.com/Academic_credit_is_a_first-to-the_post_system)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Academic_credit_is_a_first-to-the_post_system)




Roadmapping could be thought of as backwards-chaining together a series of conception-only heuretics

gertnerIdeaFactoryBell2012

Just having the piece of literature doesn’t solve the problem

The creation of a heuretic has two steps - conception and implementation

Conception is the imaginative design of a heuretic

One of the most failure-prone points in technological development is when a technology needs to move from an organization that focuses on R+D to one that focuses on marketing and manufacturing

We can cleanly attribute philosophical ideas to people but we cannot do the same with science or technology

# Academic research is often just the concept half of creating a heuretic
Is there a mapping between concept and implementation halves of heuretics and TRLs? [[Conception lives in the realm of TRL 1-3]]

Since [[Academia is the game where you gain status by getting attention for new knowledge]] the work necessary to make the knowledge impactful is devalued.  Those priorities are not necessarily ‘bad’ but it does require explicit acknowledgement that there is a lot more work necessary to make a full heuretic. [[The creation of a heuretic has two steps - conception and implementation]].  The focus on conception means that *academia does not create complete heuretics.* Incomplete heuretics make transferring the technology out of academia deceptively hard ([[Manage the Transfer not the Technology]].) Taking over an incomplete heuretic requires a lot more context.  [[Context is important and underrated for knowledge transfer]]. 

The need for context may suggest that it’s important to have whomever is taking over involved in the process *during the conception.* Organizations that successfully do transfers seem to do this:
- [[TandemLaunch]] talks to people in the lab before
-  [[Calvin Nguyen]]’s entrepreneur-in-lab idea is trying to embed the person who would do the transfer 
- Most successful companies that spin out of labs either have someone who did the research as a cofounder 
	- Are there counterexamples to this? [[Are there examples of successful technology transfer out of a lab without close collaboration between someone in the lab and the external organization?]]

A corollary to the increased need for context around a concept-only heuretic is that copying an incomplete heuretic is much harder than copying a complete one. Throwing things over the wall doesn’t work unless you have clean interfaces and [[only complete heuretics have clean interfaces]] and therefore the conception half of a heuretic can’t be thrown over a wall. This absence of clean interfaces may be a big reason why [[Just having the piece of literature doesn’t solve the problem]]! 🤯

Another corollary is that chaining together conception-only-heuretics will *absolutely require* talking to people in order to get enough context and that it *can’t* be done automatically by reading papers. ([[Roadmapping could be thought of as backwards-chaining together a series of conception-only heuretics]]) 

### Related
- [[Technology Readiness and the Valley of Death]] - specifically the “ilities" 



[Web URL for this note](http://notes.benjaminreinhardt.com/Academic_research_is_often_just_the_concept_half_of_creating_a_heuretic)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Academic_research_is_often_just_the_concept_half_of_creating_a_heuretic)




Peer review has failed as an institution

People are incentivized to work on nodes higher up the tech tree

Peer review gives peers veto power over each other

# Academics are incentivized to do peer-reviewed publishable work
Peer review published papers both determine an academics livelihood and status. Therefore, they are incentivized to spend their time doing work they think can be published. 

On top of this [[Academic credit is a first-to-the post system]]

[[People and organizations are all playing some game that has different ways of gaining status and power]] In Academia there are a couple of long term games:
- Gain [[Kleos]] through contribution to the [[Knowledge frontier]]
- Just get to scratch your itch to investigate certain things
- ….

[[People create proxy games for long term or hard-to-measure games]] and in academia the entwined proxy games are:
- Peer reviewed Published papers
- Citations on those papers
- Grants (which are usually based on papers) 
- Awards (which are usually based on who knows you which comes from papers)

It’s a safer strategy to publish lots of papers than to try to write one super-cited paper because it’s very hard to know which papers will be highly cited. Additionally everybody is impressed when you say “I’ve published a bajillion papers” while only a subgroup might know your super-cited paper by title unless you’ve [[hit the top of the columnade]]. 

Tenure is based on the number of papers you’ve published and the amount of grant money you’ve brought in.  Without tenure, academia is extremely rough. People who don’t think they can get tenure leave universities. [[National labs and other government research centers are a research alternative to universities]].

[[The peer review and citation system incentivizes people to work on things that other people think is interesting]]

### Related
- [[§Academia Constraints]]



[Web URL for this note](http://notes.benjaminreinhardt.com/Academics_are_incentivized_to_do_peer-reviewed_publishable_work)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Academics_are_incentivized_to_do_peer-reviewed_publishable_work)




Technology transfer is a mystifying abstraction

# Any technology that comes out of a university needs to go through their tech transfer office
Technically any IP created at the university belongs to the university. This was originally intended to *lower* friction for professors (see [[bushPiecesAction1970]]) Apparently the one place this is not true is in Denmark, where professors own IP they create directly. 

As a consequence, if you are going to aggregate technology from multiple universities, you need to deal with multiple tech transfer offices.

### Related 
- [[Freely available data resources to track US university technology transfer]]
- [[Mick Stadler writes a letter in 1976 on effective transfer mechanisms]]
- [[Most tech transfer offices are not profitable]]



[Web URL for this note](http://notes.benjaminreinhardt.com/Any_technology_that_comes_out_of_a_university_needs_to_go_through_their_tech_transfer_office)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Any_technology_that_comes_out_of_a_university_needs_to_go_through_their_tech_transfer_office)




Innovation orgs need to be aligned with their money factory

Bell Labs

Alignment requires existential threats

Bell Labs was only profitable in conjunction with AT+T

The incentives inherent to corporate R&D mean that researchers motivated by money leave and those who are motivated by glory are not the best in the world

Why was Bell Labs Different?

§Bell Labs Model

Labs Document Outline

# Bell Labs is one of the few examples of an innovation org that was aligned with its money factory
 [[Innovation orgs need to be aligned with their money factory]] and arguably one of the reasons behind [[Bell Labs]]’s outlier success was that it managed to have a broad scope and long time scale that were both *still aligned* with AT&T.

At the end of the day AT&T’s goal was to maintain its monopoly and increase its business. In order to do that, it needed to enable people to make more, better calls. 

[[“The System” of AT&T]]  had such an expansive scope that it actually benefitted from the whole spectrum of activity happening at Bell Labs. Everything from information theory to novel materials directly fed back into AT&Ts core business. More cables are expensive so information theory actually helps people make more calls. The faster you can switch, the more calls you can route so semiconductor switchers *directly* help with the goals. 

Additionally, [[Bell Labs and AT&T had timescale alignment]] thanks at least in part to AT&T’s monopoly. The monopoly enabled Bell Labs to plan on multi-decade timescales because they didn’t have to worry about a competitor coming along tomorrow. 

AT&T maintained their monopoly only with tacit approval from the government. This approval rested heavily on Bell Labs’ output - both continuous improvements to A&+Ts offerings and open licenses from the lab. [[Regulation drove many of Bell Labs’ actions]]. 

Between the cost reductions and monopoly maintenance, Bell Labs was actually addressing existential risks to AT&T. [[Alignment requires existential threats]]. Even R&D labs that are building new product lines within the scope of a company’s actives rarely address existential threats to that extent.

This alignment in both timescale and focus-scope between an innovation org an its money factory. See [[§Startup Constraints]], [[§Corporate R+D Constraints]], [[§Philanthropy and Non-Profit Constraints]], and [[§Academia Constraints]] for all the lovely ways the misalignment can happen.

## Related
- [[gertnerIdeaFactoryBell2012]]



[Web URL for this note](http://notes.benjaminreinhardt.com/Bell_Labs_is_one_of_the_few_examples_of_an_innovation_org_that_was_aligned_with_its_money_factory)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Bell_Labs_is_one_of_the_few_examples_of_an_innovation_org_that_was_aligned_with_its_money_factory)




The number of researchers has exploded

Universities have been pushed more towards applied research since the end of WWII

How and Why Government, Universities, and Industry Create Domestic Labor Shortages of Scientists and High-Tech Workers

Nobody will admit that a field is stuck

The university is being unbundled but nobody is taking on the research part

What is the history of Universities with respect to research?

Second Lieutenants at AFRL are like graduate students in universities

Research scout network

Another downside to importing graduate students to work in labs is that it makes papers less intelligible

Michael Filler Conversation 25 Aug 2022

There is a group of people who are capable interested in science and the current institutions are not working for them

# Graduate students are the labor in academic science and engineering
Modern science needs a lot of skilled labor to do not particularly exciting jobs. Pipetting, building devices, coding, running tests. Arguably they are creating a lot of value for the PIs they work for: progressing their careers, bringing grant money to the lab and the university, creating  knowledge for society. 

Grad students provide this labor much more cheaply than the value they create. Graduate students are paid around $40k a year but it’s much much less than the same smart ambitious people would be making in a real job. The explicit deal is that you get non-monetary compensation. You do get a credential at the end of grad school. The assumption is that this credential is incredibly valuable but that deal has broken down.

This system is problematic for several reasons. 
- Imagine trying to run a company building a problem that demands a ton of [[Tacit Knowledge]], but you could only hire people with almost no experience who would never stay more than five or six years. This constant retraining is an unescapable tension if your labor is students.
- The amount of money you need to support a grad student is actually quite similar to the amount of money you need to support a high-skilled employee at a company, but about half of it goes to the university for administrative overhead and student fees. The university is actually getting a student’s worth of money from grad students to “teach” them but it comes out of grants instead of undergrads pockets. 
- Most graduate students’ incentive is to publish as many high impact papers as possible and generally do work that sounds good to a faculty hiring committee. These incentives are often aligned with their professor and lab. However, if a project requires many people’s worth of work a grad student either won’t make it onto the paper or will be last in a massive list of authors which nobody wants because a finite amount of credit is distributed among all the authors. It’s also not appealing for a grad student to work on a project that won’t produce paper-worthy results on a timescale that aligns with graduating. There’s the obvious case where the timescale is longer than the grad student expects to be in grad school. The more subtle case is that the project takes a few years but is risky enough that it’s not clear that it will produce a paper-worthy result, possibly leaving a PhD student high and dry several years in. This incentive selects against long-term engineering-heavy projects. 
- Hiring committees care about potential professors being able to do novel work. [[The word ‘novel’ is used as an idea bludgeon in academia]]. This criteria means that grad students need to start doing work that stands out from work that has been done before in the lab if they want to get a faculty job. This incentive puts pressure on projects that need a lot of focus over a long time.  [[Academia incentivizes novelty, not focus]]
- Graduate students are primarily funded through grants. Grants are typically renewed on a yearly basis (with exceptions.) So even regardless of what the grad student is ok with working on, there’s a tight coupling between grad students and doing work that can hit paper-worthy milestones on a yearly basis. Additionally, funding grad students through grants effectively ties them to a specific project. Project-coupled grad students make it near impossible to shift effort away from projects that are going poorly towards projects that are going well. Imagine if a startup couldn’t abandon things that weren’t working in favor of things that were.  [[Most grant money is tagged so unrestricted money is like gold for professors]]. 
- You can’t really fire grad students or hire them whenever you need them. For the most part, new graduate students join labs at the beginning of the academic year so if a project needs a lot of labor halfway through the year you either need to plan that out and have a grad student underutilized for half a year or you need to postpone that project until the beginning of the next year. Once you’ve taken on a grad student you’re stuck with them until they graduate or give up - this is a terrible situation for both sides.

There are several theories about how this system came about. Like any [[Complex System]], the real answer is probably “a bit of each of them.”
- It is easier to get grants (especially government ones) that are primarily earmarked for grad students as opposed to lab technicians or labor-saving automation. Why? Some combination of tradition, incentives on the part of the university because they get more fees from grad students than other spending, and the idea that money towards grad students is building up the country’s STEM workforce. 
- [[How and Why Government, Universities, and Industry Create Domestic Labor Shortages of Scientists and High-Tech Workers]] actually argues that the system was set up on purpose to keep wages for high skilled scientific workers low. 
	- Universities get more money when professors bring in larger grants. Because its easier to get big grants for grad-student-heavy projects universities are incentivized to hire professors who do more grad-student-heavy projects, perpetuating the cycle. [[In academia more grant money leads to more power]]. 

### Related
- [[§Academia Constraints]]
- 
- [[Stagnation]]
- [[bhattacharyaStagnationScientificIncentives2020]]
- [[The NSF earmarks a lot of money for training]]



[Web URL for this note](http://notes.benjaminreinhardt.com/Graduate_students_are_the_labor_in_academic_science_and_engineering)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Graduate_students_are_the_labor_in_academic_science_and_engineering)




Lee Ricketson conversation 6/5/20

# In academia more grant money leads to more power
Because there’s a cap on profit, more grant money goes to either hiring more people or getting better equipment. Lab size and equipment is absolutely how professors flaunt.  

Weirdly, this indirect power accumulation (as opposed to direct money accumulation) resembles both communism and startups. In communism governments luxury and agency are coupled to a position and in the world of startups, you can’t spend money from a massive funding round directly but you can get a luxurious office, perks, business travel, etc. 

So while the accumulation is not as direct as a 

This is only true in certain fields where having more grad students in useful. 





[Web URL for this note](http://notes.benjaminreinhardt.com/In_academia_more_grant_money_leads_to_more_power)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/In_academia_more_grant_money_leads_to_more_power)




Untapped technical people near the end of their careers may have more potential than young people

Jade Leung Conversation 26 May 2021

# Is it more impactful to find and train undiscovered diamonds or to unlock well-positioned but constrained researchers?
# Is it more impactful to find and train undiscovered diamonds or to unlock well-positioned but constrained researchers
Both situations are a question of counterfactuals, and [[Counterfactuals are hard]]. On the one hand, there’s the question of how many people have the raw horsepower and outside perspective to do great work but are unable to get into the system (or do great work outside the system.) Call this the ‘discovery problem.’ On the other hand there’s the question of how many people already in the system would have done great work if the system were different (or there was an alternative system). Call this the ‘enablement problem.’ 

I’m going to assume that the goal here is “more potential paradigm-shifting blue sky research.” There are many other worthy goals like equality and preventing talent from being wasted that are often conflated. 

To start, some important questions to ask are:
- Do you think that at the highest levels of performance, paradigm shifting work is a numbers game? That is, is the [[Binding Constraint]] on the dominant system’s output the number of high-quality people in it? 
- What do you expect to happen to undiscovered people once they’re discovered? Either they will need to feed into the system or you will need to build a parallel system for them. 
- At what point (both in terms of age and education) do you think the ability or mindset to do great [[Blue Sky Research is pure discovery work that asks extremely open-ended questions]] disappears? The relative importance of the two problems shifts depending on whether you believe that the dominant system beats the ability to do great research out of people by the time they’re done with a PhD or, like Braben, you believe that experienced professors can do great work by shifting to an entirely new field.  
- It’s also important to be a bit more precise about the discovery problem. Are we talking about kids in Africa who don’t even know what research is? College students who aren’t considering grad school or can’t get in? Graduate students in lower-tier universities? Professors or post-docs in lower-tier universities?
- How much apprenticeship/hands-on training do you believe researchers need?  There are examples of people with almost no formal training coming “out of nowhere” and making massive contributions like [Ramanujan](https://en.wikipedia.org/wiki/Srinivasa_Ramanujan) in Mathematics or [Chris Olah](https://twitter.com/ch402) in machine learning. How replicable is their pattern? It’s getting ahead of asking questions but my hunch is that this varies between different disciplines. Disciplines that have relatively little [[Tacit Knowledge]] like mathematics or involve a lot of tinkering and have relatively low equipment requirements like deep learning research require little training. 
- If training matters, do you believe that you could create a better way to train people than the current system?
- How big do you believe the ‘output’ difference is between the top 0.01% and 0.001% of researchers is? 0.001% and 0.0001%? Put less abstractly, if Einstein had died of pneumonia as a child, would someone else have come up with [[General Relativity]] in 1915? Would it have taken until 1925? 1985? Or would it have never been proposed. This is another unanswerable counterfactual question where opinions vary wildly. 
- How hard do you believe it is for people who would be good researchers to get into the system and from where? Are we talking about an elementary student who lives in rural Africa or India and would otherwise become a farmer? A high school student in inner-city America who would drop out and deal drugs? 

Looking at these questions, a series of cruxy questions emerge: do you believe that the binding constraint on great blue sky research is the quality and mindset of people going into the top-tier universities or is it a systemic constraint on the people already in the top-tier universities? If it’s the former, the discovery problem is absolutely the right one to focus on. If it’s the latter, you need to work on the enablement problem and the question becomes: do you believe that the people who have never been in the system are better than the people already in the system? This difference could be either because the system systemically selects for people less suited to do great work or actively destroys people’s ability to do great work. 

My belief is that the binding constraint is on the system itself. There are definitely many people who could be great researchers who never get the opportunity to do so. However, even if you could identify them and get them into the system, they would be as constrained out of doing great work as everybody else. In this situation, the only reason to work on the discovery problem (if what you care about is purely incredible blue sky research) is if you think it will work better to build an entirely parallel system. I actually believe that the system is pretty good at training researchers and doesn’t somehow ruin them, so I am inclined to believe that building an entirely parallel system will just be more work for possibly worse results. 

### Thoughts that inform my answers to these questions

The massive list of [[§Academia Constraints]] paints a pretty clear picture of the heavy constraints even on amazing researchers in top-tier universities. It’s tempting to say that once a researcher has made it into a top-tier research program perhaps as a grad student and definitely as a post-doc or professor, they’re well-positioned to do [[Blue Sky Research is pure discovery work that asks extremely open-ended questions]]. While there are certainly some examples of success, I would argue that they are the exception and not the rule.   

From a professor friend who just got a grant proposal rejected:
`some of the reviews are extremely glowing. the ones that were less glowing mostly suggest that proposal was "too ambitious" and "this is a new field to the PI, will be less risky once they can show that they have instrumentation set up`

The fact that even success stories (Boyden, mRNA, etc.) happen by the skin of their teeth suggests that counterfactually there are many more examples on the other side of the survival line.

[[brabenScientificFreedomElixir2008]] has several examples of well-regarded professors who couldn’t get funding to pivot out of their niche, but did great work once that pivot was enabled *because* of their prior experience. Of course, BP research suffers from another counterfactual problem because they *only* sponsored people with relatively robust track records. 

[[odlyzkoDeclineUnfetteredResearch1995]] strongly biases me to believe that science is not a numbers game and that “more talent” could actually have the opposite of intended effect. 

### Related
- [[Untapped technical people near the end of their careers may have more potential than young people]] 
- [[People who have done a thing should be in charge of a thing]]
- [[Understanding a thing has to do with intuiting the affordances of a thing]] — when there’s tacit knowledge at play, it takes experience just to get the affordances of a field, so complete outsiders might be less useful. 
- [[Einstein would have been stuck in a patent office - failures of paradigm-shifting science]]

[Web URL for this note](http://notes.benjaminreinhardt.com/Is_it_more_impactful_to_find_and_train_undiscovered_diamonds_or_to_unlock_well-positioned_but_constrained_researchers)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Is_it_more_impactful_to_find_and_train_undiscovered_diamonds_or_to_unlock_well-positioned_but_constrained_researchers)






# Is it possible that we’re actually getting less research because we have more researchers?
# Is it possible that we’re actually getting less research because we have more researchers
[[The number of researchers has exploded]] When you have more researchers, it’s harder to judge them directly on their process or quality of thinking. Instead, because [[People create proxy games for long term or hard-to-measure games]] there are a ton of academic proxy games: number of peer-reviewed papers, citations, prizes etc. as proxy games for research quality.  [[§Academia Constraints]]. For each of these, there is an argument why they get in the way of doing good research. In the extreme case, standardized grant application processes could push the world towards the [[Szilard point]].

Separately, the proxy games can also be explained by universities being run by administrators who themselves are not researchers. [[People who have done a thing should be in charge of a thing]]. The 

### Related
- [[The peer review and citation system incentivizes people to work on things that other people think is interesting]]
- [[What is the difference between administering, managing, operating, and staffing an organization?]]



[Web URL for this note](http://notes.benjaminreinhardt.com/Is_it_possible_that_we’re_actually_getting_less_research_because_we_have_more_researchers)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Is_it_possible_that_we’re_actually_getting_less_research_because_we_have_more_researchers)




Brains are wombs for heuretics

# It is easier to have a multipoint mutation in technology than in evolution
A multipoint mutation is analogous to [[Systems must take performance hits to get out of local optima]] - in the tech tree model this means having one or more low impact precursor nodes.  

Almost by definition you need multipoint mutations to fix [[Structural Constraints]] because structural constraints mean that there are two or more interlocking dependencies. 

In biology, it’s (impossible?) to have a precursor with a single mutation that renders it inviable give rise to a precursor that has a second mutation that addresses issues from the first and builds on it. We’ve used genetic engineering and mutation breeding to get around this fact. ([[What does technological genetic engineering look like?]] and [[What does technological mutation breeding look like?]])

In technology a non-viable (non impactful) precursor can be kept alive in a lab or in an obscure paper somewhere. These are unexploited [[Leonardo Ideas]]! Which may give a hint that the process of finding unexploited Leonardo ideas ([[Leonardo Ideas can be systematically identified]]) may be part of the answer to [[What does technological genetic engineering look like?]]

 - It is hard for startups to address multipoint mutations because it takes a long time and its hard to sell as a product [[§Startup Constraints]]
 - It is hard for corporate R&D to address multipoint mutations because people often have vested interests in the old system. [[§Corporate R+D Constraints]]
 - Academic research often creates proposals for multipoint mutations but because [[Academics are incentivized to do peer-reviewed publishable work]] they have no incentive to do implementation. [[§Academia Constraints]].
 - Why does 

### Related
- 



[Web URL for this note](http://notes.benjaminreinhardt.com/It_is_easier_to_have_a_multipoint_mutation_in_technology_than_in_evolution)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/It_is_easier_to_have_a_multipoint_mutation_in_technology_than_in_evolution)




The way to get around journals is to focus on non-paper artifacts

Week of 29 Nov 2020

# Labs implicitly judge research through the sense of dollars per paper 
# Labs implicitly judge research through the sense of dollars per paper 
Because papers are the metric of a lab’s outputs, it makes sense for a lab to want any grant money they spend to be as “paper efficient” as possible. You can maximize your paper efficiency (ie. maximizing papers/$ or minimizing $/paper) in one of two ways. Either you just do cheap experiments or you spend the money on one piece of equipment that can be used for many many experiments. This drive may also be a partial explanation for the “salami slicing” phenomenon where researchers slice their research into absolutely minuscule pieces per paper. 

While it is difficult to conclusively demonstrate ([[Counterfactuals are hard]]) the drive towards paper efficiency would suggest that there is a class of experiment that isn’t happening often. These experiments would take a lot of money per paper either in the form of just taking a lot of time or requiring a piece of equipment that would only be used for one or two experiments.

The lens of paper efficiency gives another explanation for the grad student explosion. [[There is an increasingly wide gap between the number of graduate students training for PhDs and academic positions for them to fill]].  Besides the fact that many grants have large carveouts explicitly for grad students, grad students themselves are the only thing you (as a PI) can spend money on that is self-motivated to generate papers. It’s conceivable that you could spend money on equipment that yields zero papers. Not the case with grad students!

### Related
- In addition to labs adopting the “$/paper” metric, funders adopt it as well [[riconFundPeopleNot2020]]

[Web URL for this note](http://notes.benjaminreinhardt.com/Labs_implicitly_judge_research_through_the_sense_of_dollars_per_paper)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Labs_implicitly_judge_research_through_the_sense_of_dollars_per_paper)






Materials and manufacturing underpin civilization

Physical experiments are expensive

Implementation is the actualization of the heuretic in the real world

olsonManagingModesEffort2020

Bell labs brought many disciplines together under the same roof and broke down barriers between them

Parney Albright conversation 17 Jan 2023

New manufacturing paradigms are more likely than other technologies to generate positive second-order effects

Smart people no longer do base-layer work

Jeremy Glick conversation 18 May 2022

Tying between technology and the underpinnings of civilization

# Manufacturing is important and often ignored
Physical things need to be manufactured if you want to have more than one of them. There is a whole set of properties that make something more or less manufacturable. 

[[Skunkworks podcast]]  mentioned several times how the key to moving fast was to have people who were going to manufacture the thing sitting in the room to point out where a design might be manufacturable. 

Things can fail in manufacturing in a bajillion different ways. You could build a completely functional prototype of something that is entirely unmanufacturable.   (Can things fail in scaling in a bajillion different ways?) 

The people who handle and have heuristics around manufacturing are usually not the same people who do the prototyping and original building. The informal interaction that ‘having the person in the room’ during the prototyping process enables is a [[Transaction costs]] that is reduced if the people are all part of the same firm. (Thanks [[Robert Coase]]!) The whole point of design reviews is when the people who are manufacturing and designing are not in the same firm (or not in the same org within the same firm.) 

The gap between people who design and prototype and people who manufacture is especially bad in academia. ([[§Academia Constraints]].) From my experience, people doing university research literally do not think about manufacturability at all. This missing perspective is one of the gaps left by [[aroraChangingStructureAmerican2020]]. 

Manufacturing is the atomland instantiation of software scalability in bitland. ([[§Atomland, Bitland, and Peopleland]]  - what is the people land version? Replicability? Systemization?) The difference is that you don’t have to worry about scalability until you increase use by orders of magnitude, manufacturing needs to happen from day 1 for dispersion. Manufacturing is the gap between [[TRL7 - prototype version in real environment]] and [[TRL8 - production version tested in real environment]]. A “production” version has been manufactured in the same way that every other version would be.

 In software when going from TRL7 to TRL8 you can get away with doing almost nothing - maybe changing a few more settings, adding more tests, etc. There are two reasons for this tiny gap. The first reasons is that “Production” versions of software don’t need to scale because the cost to disperse a new production version that does scale later is nothing. The only cost is the up-front cost of building it in the first place. The other reason is that you can develop and prototype using roughly the same ‘tools’ as you would use to produce it thanks to standardized software environments, containerization, etc. In Atomland there are gaps both between the CAD/simulation design, the prototype, and the manufactured product that can be uncrossable if you didn’t take into account where it the thing needs to go. 

In the past, manufacturability was less of an issue because the gap between prototyping tools and manufacturing tools was smaller. Everything was made by hand or with simple tools anyway. 

Manufacturing is an important place where [[Learning by doing]] is key. 

Manufacturing is a failure point in DARPA’s transitions. [[DARPA has a mixed record on transitioning]].  From what I can tell DARPA doesn’t really consider manufacturability during programs. 

Constraints on manufacturing:
- Tooling
- Time
- Materials
- Accessibility
- Number of parts

### Related
- [[List of things that affect dispersion]]
- [[Manage the Transfer not the Technology]]
- [[Skunkworks podcast]]
- [[Process Knowledge]]
- [[Dan Wang 2019 Letter]]
- [[Technology transfer is a mystifying abstraction]]
- [[New manufacturing paradigms can remove bottlenecks for other paradigms]]
- [[Materials and manufacturing underpin civilization]]
- [[Designing something to be manufactured can make a drastic difference in its success]]



[Web URL for this note](http://notes.benjaminreinhardt.com/Manufacturing_is_important_and_often_ignored)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Manufacturing_is_important_and_often_ignored)




# Market Failures in Science
#### Published
#### Authors
- [[Milan Cvitkovic]]
#### Link
[Milan Cvitkovic](https://milan.cvitkovic.net/writing/market_failures_in_science/)
### Related

[Web URL for this note](http://notes.benjaminreinhardt.com/Market_Failures_in_Science)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Market_Failures_in_Science)






Fair games are self-limiting

Professors are constantly doing the calculation of whether a grant is worth the time to apply for it

Example of grant system failure for COVID19 tweet thread

Notes on the Endless Frontier Act

The Endless Frontier Act

# Months of grant writing is one of the things that makes people think twice about academia
[[Professors are constantly doing the calculation of whether a grant is worth the time to apply for it]]

But then researchers in [[Federally funded research and development centers - FFRDC]] like [[Lawrence Livermore National Lab]] need to write grant proposals anyway. 

### Related
- [[Venkatesh Narayanamurti Podcast 20 May 2020]]
- [[40 percent of researcher time is spent writing and administering grants]]



[Web URL for this note](http://notes.benjaminreinhardt.com/Months_of_grant_writing_is_one_of_the_things_that_makes_people_think_twice_about_academia)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Months_of_grant_writing_is_one_of_the_things_that_makes_people_think_twice_about_academia)




Why would people want to work with PARPA?

Project-based funding reduces organizational agency

Accounting can be a tool to unlock innovation

# Most grant money is tagged so unrestricted money is like gold for professors
### Related
- [[The government grant process depends on politics and committees]]



[Web URL for this note](http://notes.benjaminreinhardt.com/Most_grant_money_is_tagged_so_unrestricted_money_is_like_gold_for_professors)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Most_grant_money_is_tagged_so_unrestricted_money_is_like_gold_for_professors)




Josh Greenberg Conversation 26 April 2021

Hesitation as a useful heuristic

# NS Notes on Officers Techniques
- ::Most of the piece is concerned with human interactions::
![](BearImages/01A22DC7-894B-4707-B0CE-66A28CC801F0-461-0002FE4DB1CC8AB7/778BF54B-5AFD-4430-B743-0FC929863D0B.png)
- ::Unwise and naughty are both underrated words::
![](BearImages/C1E3CFC2-EA55-42C0-A7D3-5FD7CF05AB64-461-0002FE93585F11A4/DC47C7FE-E6B1-46F9-9CCA-3EA335EEC2E3.png)
- ::This is the tricky thing about figuring out a program before the program has funding::
	- [[Development Tranches and Tranched Funding]]
![](BearImages/A539AF36-3316-4FE9-ACE7-EAF5088ED2B2-461-0002FE9F3EB2A549/F3AA2F2B-BBBE-4392-AC9B-D6385DE0E480.png)
- `A few scholars are very good salesmen for their projects. Some are very effective and also very reasonable in their proposals. Some are unpleasantly insistent and pushing, whether the request is for so little that we should not be concerned, or for so ex- travagantly much that they should be ashamed. Other scholars are very poor salesmen for their own ideas and needs - shy, or modest, or reticent, or uninterested in money and inexperienced as to v/hat it can do for them, or in extreme cases even sullen and pugnacious over the idea that some person with a vulgar lot of money is presuming to take up a scholar’s time and tell him what to do.`
	- [[If science depends on markets scientists become salesmen]]
	- [[There is a fundamental tension around the causality between salesmanship and good work]]
	- ::note also that this is in tension with the note below to not fund people who nobody likes working with::
- `The RF officer must train himself to disregard both good and bad salesmanship, except for those cases where the same personality traits that explain the salesmanship would handicap the project. `
	- This is different from startups in that good salesmanship is necessary for a startup to succeed, while I suspect it’s less necessary for a research project.
	- [[There is a fundamental tension around the causality between salesmanship and good work]]
- `When Rob Millikan or Jim Angell used to show up, in their prime, we would lock  the safe, throw the key out the window, and put cotton in our ears: and then afterwards study the written proposals very carefully and try to do the right thing.`
	- What is the correlation between great scientists and being great fundraisers? A specific kind of science? Is this just an artifact of the system?
- `This requirement [to get written approval from administrators] is based on experience. The RF must meticulously avoid any act which could be interpreted or even misinterpreted as an attempt on our part to force support on an institution in order to accomplish purposes which we seek, but which are not part of the genuine internal interests of the institution. This rule is not merely protective; it has a positive aspect. It is essential in helping assure that our activities serve the broad, integrated, purposes of institutions.` 
	- Academia creates an awkward dance with multiple stakeholders
	- [[§Academia Constraints]]
	- [[Academic administrators and professors are playing very different games]]
- `And explained declinations always bounce back.`
	- [[Fundamental Tensions]] around whether to explain rejections or not
- `If you decide to leave Chllombia, the present grant cannot be transferred.`
	- Raises a historical reason why funding went to projects not people — politics. Multiple stakeholders make it awkward to fund people who could use that funding to leave the institution. How does that change with everybody being a free agent now?
	- [[Academic administrators and professors are playing very different games]]
- *Diaries of interviews and Visits* `It is an old, proven, and treasured tradition of the RF that officers write diary accoimts of all interviews of any importance, and of trips. It is not necessary that these observe all the niceties of elegant grammar. Succinctness, clarity, accuracy, and completeness of essentials are more important than Johnsonian sentences. `
	- [[Written institutional cultures]]
- `Ask a man what he is interested in and give him a real chance to answer. Don’t tell him. `
	- [[Life Style Guide]]
- `*The coat of arms of the Foundation officer bears the motto “Solvitur Ambulando.” `
- `It took years really to convince the public that the founders and officers of this institution treat the available money as a serious public trust. ` ::Need to go out of your way to show that you’re doing publicly useful things::
- `If it can be arranged tactfully, get a chance to talk with the younger investigators alone - that is, with the Herr Geheimrat not present.` ::Importance of tenuretrackless folks:: 
	- [[The people who would want to do early program work tend to be post-docs, grad students, and technically trained people who have left academia]]
- `In other words, the average person is neither very critical nor very frank about written recommendations. He assumes that the backers of other candidates will keep still about faults and limitations, and will use pretty rosy adjectives. So, in fairness to his candidate, he does the same. `
- `The remedy for this involves a lot of time and patience and energy. It involves slowly building up a relationship of close confidence with a large number of shrewd and hardheaded scientists; `
	- [[A large part of a DARPA program manager’s job is focused network building]]
	- [[Trust-building timescale and budgets tend to be similar in different domains]]
	- [[Trust takes time]]
- RF interest lies in propagating a field, and less in individual genius researchers
- `This same general point explains why we think it is necessary and proper to be interested in the personal qualifications (as contrasted with the intellectual qualifications) of fellowship candidates. I know of a man who is almost surely the best expert on the genetics of oak trees in the world; but he doesn’t take baths, and he swears so much and so violently that most persons just won’t work with him. As a result he is living a frustrated and defeated life; and he not only doesn’t have any students - he doesn’t even have a job. ` ::This seems like a clutch decision point - people or systems::
	-  [[There is a fundamental tension around the causality between salesmanship and good work]]
	- 
- `We also, less frequently, appoint men to “training fellowships,” these characteristically being used to add biological training to previous training in the physical sciences, or vice versa. `
	- This seems like an interesting way to do “externalized” bringing disciplines together like how [[Bell labs brought many disciplines together under the same roof and broke down barriers between them]]
- `Where, in making our selections, should we draw the line in this intermediate group?” We have always claimed that the mere asking of this question reveals an unfortunate philosophy concerning fellowship selection. For we would say: “Deal exclusively with the first group of unquestioned excellence. Draw the line before you ever get to the intermediate group. That is, the question’ May I have this fellowship?’ ought to be like the questions ‘Is this egg fresh?’ or ‘Do you love me?’ .” If you need to hesitate at all, the answer is surely no.` ::This is one argument against pouring more money into the NSF — same pool of people::
	- [[Adverse Selection Theory]]
	- [[Hesitation as a useful heuristic]]
- `But there are, of course, other and more serious difficulties connected with the concept of a permanent endowment. It is essentially impossible to forecast, for any long period, either interest rates or costs- Even more disruptive, however, are the irregularly occurring “but apparently inevitable crashes which wipe out or at least decimate capital values. `
- `. Were it not for the periodic leveling of values by catastrophic financial events, a single dollar invested at only compound interest in say 500 A.D. would now amount to nearly three thousand billion dollars ` ::We don’t talk or think about this very much anymore::
- ::Interest rates can discretely change strategy considerations::
- `At regular intervals someone is sure to ask (and it is a perfectly proper and important question): Why should the RF, which has almost unique power to do great things, “bother with chicken feed? What justification is there for officers to take up valuable time investigating and studying small grants? HS thinks that these questions have wholly convincing answers. Great things are very seldom - like an unbroken hen’s egg - perfect in completeness essentially indivisible, and wholly developed at first appearance. Great things are usually formed of small parts: great things usually grow from small things. If our thinking and investigation does not keep us in accurate and realistic contact with the actual operative details of science, we are not likely to be very wise about the larger plans. `::Seedling experiments::
	- [[It’s possible to get people to undertake seedling experiments]]
- `Our answer is: “If you feel that it is to your own advantage or interest to do so, we will not object to a brief and simple statement; but as far as we are concerned, we would somewhat prefer that such statements not be made.” ` ::this feels archaic::
	- ::Try to minimize fanfare::
- `nevertheless we are actually junior, ` ::Always remember to be humble and secondary::
- `This organization is under no compulsion to show/quick results. It doesn't want to be lethargic, or stodgy; but it doesn’t like “being hurried.`::A research management organization should never be late, nor should it be early; it should arrive exactly when it means to, Frodo Baggins::
	- [[Anybody attempting to emulate DARPA needs to be willing to sit with a lot of discomfort for a long time]]
- `In one large and important case ~ Biological Abstracts ~ we helped solve the problem by kicking the child out of our warm bright living room, into the dark snowy night outside. The child braced up, threw away its fancy clothes, got a good suit of jeans and a job, and proceeded to earn an honest living. `
- `Remember that a new journal is desired by almost every scientist whose interests have become very narrow and special`
- `Once in a while the opposite of specialization operates - a man wants a journal which will be about everything in general. `
	- [[§Specialists vs. Generalists]]
- `Many institutions are almost compelled “by circumstances to minimize, if not indeed to cover up, their mistakes. A State university president is not very likely to include in his report to the state officials, a section which “begins “It must be agreed that the money spent over the last ten years in an attempt to “build up a Department of Polish has “been wasted.” And yet educational institutions (and foundations) do make mistakes, and ought to profit “by them.`
- ` Indeed it can “be persuasively argued that the RF ought to make mistakes. If it does not, it has almost surely not “been as imaginative and as adventuresome as it should have “been in ferreting out hard and important problems and in attempting their solution. Although we have certainly made some mistakes which we never should have made, it seems probable that we have, on the whole, made too few mistakes - or at least, too few good mistakes. `
`A “good” mistake, I take it, is one which is recognized and profited by; and which represents a bad outcome of a situation which (though somewhat risky) contained really important promise. The moral is that an officer should be alert, imaginative, and flexible; and should not be too afraid - too desperately and paralyzingly afraid - of maiking a mistake.* And when one is made, we should do a good thorough autopsy, and find out what the patient died of. `
	- [[More Uncorrelated Mistakes lead to more success]]
	- How does this square with [[Opacity is important because it allows people to be open about failure]]

#### Published
1946
#### Authors
- [[Warren Weaver]]
#### Link
https://www.dropbox.com/s/oo6p2uza5zitge7/ns%20notes%20on%20officers%27%20techniques_2021-01-13%2016.24.04.pdf?dl=0
[N.S. notes on officers’ techniques - Digital Library Listing - The Rockefeller Foundation: A Digital History](https://rockfound.rockarch.org/digital-library-listing/-/asset_publisher/yYxpQfeI4W8N/content/n-s-notes-on-officers-techniques)
### Related
- [[List of people who need biographies written about them]]

[Web URL for this note](http://notes.benjaminreinhardt.com/NS_Notes_on_Officers_Techniques)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/NS_Notes_on_Officers_Techniques)






Different institutions have created monopolies on various enterprises

Marissa Weichman Conversation 2020-04-10

# National labs and other government research centers are a research alternative to universities
Government researchers are still in the ‘academia’ game in that they want to publish papers and discover and invent cool shit.  [[People and organizations are all playing some game that has different ways of gaining status and power]]

Government researchers have both less freedom and more freedom than university researchers. If they can justify how their research is aligned with the higher-level missions of the organization, they don’t need to spend time writing grants or teaching like professors in universities. [[Professors do three or more jobs]]. The need to spend the majority of their time on organizationally aligned research means that they can’t do super crazy things. Additionally [[The people setting research priorities at AFRL are often not technical]], which is another constraint that may kill promising projects.

Research at national labs is perceived by people in academia as being lower impact than at universities. [[University professorships are the highest status jobs in academia]]. The lower impact is because their research needs to align with the department they are part of. ([[Innovation orgs need to be aligned with their money factory]]. This creates a feedback loop where the really good researchers don’t work at government labs and the phenomenon of [[A players hire A players and B players hire C players]]  happens.

This whole story is so similar to [[§Corporate R+D Constraints]], I would go so far as to say that government labs are basically tax-payer funded Corporate R&D.  [[Government research is corporate R&D for United States Inc]].

Questions about the orgs:
- [[How are research agendas set at National Labs?]]
- Are the people who work there technically government or university employees?
- How is the money allocated
- How does your career progress?

### Related
- [[Federally funded research and development centers - FFRDC]]
-  [[Frederick National Laboratory for Cancer Research]]
- [[Air Force Research Laboratory - AFRL]]
- [[Naval Research Laboratory]]
- [[Jeff Graham Conversation 4-17-2020]]

[^1]:// According to [[Marissa Weichman]] :



[Web URL for this note](http://notes.benjaminreinhardt.com/National_labs_and_other_government_research_centers_are_a_research_alternative_to_universities)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/National_labs_and_other_government_research_centers_are_a_research_alternative_to_universities)




§Philanthropy and Non-Profit Constraints

Pay-it-forward tithing is underrated

Kevin Fischer conversation 28 Jun 2022

# Pay-it-forward tithing could resolve the tension between the potential public-good and potential profit in research.
# Pay-it-forward tithing could resolve the tension between the potential public-good and potential profit in research
Companies and individuals who benefitted from public-good should pledge to tithe 10% of their profit to research - especially that which falls outside of traditional institutions. [[It is nebulous a priori which research should produce public goods vs private goods]], especially if it isn’t focused on creating either patents or papers. Pay-it-forward tithing could enable people to embark on research projects focused purely on reaching whatever the goal of the research is without worrying about either needing to create specific ROI *or* making sure to adhere to the optics demanded by open source projects or charities. 

On top of enabling projects that don’t fit well into traditional boxes of non-profit charity or for-profit investment, a culture of pay-it-forward tithing could decorrelate research funding. The vast majority of research funding comes in the form of grants from either the government or large foundations. Putting aside the question of their ability to allocate money, money coming from a small number of similar organizations with a similar set of constraints on that money ([[§Philanthropy and Non-Profit Constraints]], [[§Academia Constraints]]) will lead to more correlated lines of inquiry. Mechanisms to decorrelate research are good because [[More Uncorrelated Mistakes lead to more success]].[^1]

### Related
- [[Decentralized systems are the ideal way to enable many uncorrelated mistakes]]
- [[Uncorrelated samples of an idea space means that some people are working on something that other people would find absurd]]
- [[There are many questions and issues with the idea of pay-it-forward tithing]]

[^1]: More, different, funding sources aren’t a magic decorrelation bullet, of course. There will still be trends and hype cycles.



[Web URL for this note](http://notes.benjaminreinhardt.com/Pay-it-forward_tithing_could_resolve_the_tension_between_the_potential_public-good_and_potential_profit_in_research)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Pay-it-forward_tithing_could_resolve_the_tension_between_the_potential_public-good_and_potential_profit_in_research)




How many undetected frauds in science

# Peer review doesn’t catch duplications
[Elisabeth #StayingAtHome #StayingAlive Bik on Twitter: “Update!This is a set of (at least) EIGHT papers, from different authors, different hospitals, different cancers, different protein expressions. But with identical Kaplan-Meier curves, tables values, line graphs. See e.g. https://t.co/SmHEMPpEm5… https://t.co/grcaEE5tEI”](https://twitter.com/MicrobiomDigest/status/1266140721716719616)

### Related
- [[When you see a position being advocated with no pushback but no action - it means there are a bunch of little frictions in the system]]
- [[§Academia Constraints]]
- [[Academics are incentivized to do peer-reviewed publishable work]]
- [[The peer review and citation system incentivizes people to work on things that other people think is interesting]]
- [[Is peer review a good idea - paper]]
- [[Problems With Peer Review And Alternatives]]
- [[Three myths about scientific peer review]]
- [[Peer Review- Rodney Brooks]]
- [[Lets just get rid of peer review - article]]
- [[The Philosophical Basis of Peer Review and the Suppression of Innovation]]
- [[The fact that large-scale automated machine reading discovers new cancer-driving mechanisms doesn’t cite big mechanism is a clear failing of peer review]]
- [[Unlock ways to share data on peer review]]



[Web URL for this note](http://notes.benjaminreinhardt.com/Peer_review_doesn’t_catch_duplications)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Peer_review_doesn’t_catch_duplications)




Benʼs working notes

§Academia Constraints

Related

Links to this note