The ability to say ‘huh that’s funny’ is important for discovering new phenomena

The government grant process depends on politics and committees

Formal process lets people outside the organization trust in the process instead of the people

Legibility poisoning

Workshops could be a way to start small building a private ARPA

PARPA needs a constitution with checks and balances

Fast decisions involve few people

Every program at DARPA is intensely technically scrutinized by the tech council

A PI needs to provide evidence that they have access to a lab facility

Trusted hierarchies are not always the best way to do centralized coordination

The only time you can get philanthropies to do interesting things is at the beginning

People often confuse whether they are or should be creating prescriptive vs descriptive work

Productive Nanosystems - A Technology Roadmap

The good idea fairy is the failure mode of a single decision maker

Unaccountability leads to progress

NSF and DARPA as Models for Research Funding - An Institutional Analysis

National Labs don’t foster efficient or long term research

Analyze by who can tap the breaks and who can open the throttle

Coauthors are committees

Paradigm shifts require extraordinary individuals

Peer review is a great system for preventing waste

Alexey Guzey conversation 17 Aug 2022

# Decisions made by committees lead to median results
Something something game theory

https://twitter.com/patrickc/status/1408219906319937536?s=20



[Web URL for this note](http://notes.benjaminreinhardt.com/Decisions_made_by_committees_lead_to_median_results)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Decisions_made_by_committees_lead_to_median_results)




§Tech Tree Model of Heuretics

Heuretics come out of people’s brains (TECHNOLOGY IS PEOPLE)

Designed Serendipity

In order to extend a heuretic it must be legible to you

When is push or pull appropriate for innovation orgs?

Human capital theory of stagnation

Innovation requires multiple people

Legibility forces you to fall back on conventional framings which causes you to think about the thing in a conventional way

Science and technology advance through the same fundamental mechanism

Untapped technical people near the end of their careers may have more potential than young people

Each node on the tech tree can recursively be decomposed into more nodes

Learning by doing and idea sex happen through the same mechanism

Multidisciplinary teams need to leverage different paradigmatic lenses

Luke Durant Conversation 29 Sep 2020

Heuretics cannot exist separate from humans\*

# New ideas need to come out of a single mind
At the end of the day *there can’t exist an idea that nobody has thought of.* 

When multiple people seem to come up with an idea together the process looks like passing a bundle of nodes back and forth and adding incremental pieces. But the fundamental action of addition or subtraction happens in one person’s mind.  [[Connections create new knowledge]].

A heuretic isn’t ‘viable’ until it can exist separate from the mind it came from. [[Brains are wombs for heuretics]].

The fact that new combinations need to come out of a single brain at the end of the day is why [[Naming things is Powerful]] - names allow you to hold more things in your brain at once. 

### Computers do count as minds
 Does it count as an idea that nobody has thought of if a computer helped? At the end of the day the assertion of the idea needs to come from the computer or the person.  So in this case a combination of [[Using computers to simulate possible experiment targets is still primitive]] and [[Using robotics to carry out automated experiments]] does not challenge this assertion. It’s just the computer acting more and more like another mind.

Computers can act as minds without ‘general AI.’ 

 I have a hunch that there is a set of parameters that solves the problem. I do an optimization problem and the computer finds the parameters. I could have asked a person who knew how to find those parameters to do the same thing. In this case, did the idea come from both of us?  I don’t think so. The final idea came from the person who did the calculation. The idea of the idea came from me. 

Another example is that  [[Algorithmic knowledge generation]] has already found new drug targets from an existing database.  

*This note was originally titled ‘New ideas need to come out of a single brain’ but there is no reason why a new idea couldn’t come out of a computer. 

### Related
- [[Heuretics come out of people’s brains (TECHNOLOGY IS PEOPLE)]]
- [[There are many pieces of equipment or tacit knowledge that only exist  in one or two places in the world]]





[Web URL for this note](http://notes.benjaminreinhardt.com/New_ideas_need_to_come_out_of_a_single_mind)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/New_ideas_need_to_come_out_of_a_single_mind)




Podcast List

# Peer review is a great system for preventing waste
Peer review is good at preventing waste because it gives many people the ability to say “no” < [[Analyze by who can tap the breaks and who can open the throttle]]> and therefore you have multiple independent checks on whether a project will be wasteful. If each of m reviewers has an n% chance of noticing a wasteful proposal, then then a wasteful proposal has a (1-n)^m chance of making it through. 

Peer review is effectively a distributed committee and thus has most of the properties of committee based decision making. Thinking about it, I’m tempted to ask whether anything would be different if we replaced the words “peer review” with “committee” everywhere they appear.  <[[Decisions made by committees lead to median results]]> 

Of course, I don’t think we culturally have a good handle on what wasteful means in the context of research. ‘Wasteful’ is implicitly tied to efficiency which requires a metric on the input and output of a process. Assuming efficiency is the thing you want to maximize[^1] what are the metrics on research? Money is probably a reasonable metric on input at an organizational level but output? Citations or number of papers with a lot of citations is a common output metric, but do most people or funding agencies *actually* care about citations - no, especially organizations like the military. <[[Using citations to measure science is insidious]]>

### Related
- [[The government grant process depends on politics and committees]]
- [[The word ‘novel’ is used as an idea bludgeon in academia]]
- [[New ideas need to come out of a single mind]]
- [[Fast decisions involve few people]]
- [[All the incentives in research drive towards conservatism]]
- [[The ability to say ‘huh that’s funny’ is important for discovering new phenomena]] and peer reviewed grants generally do not allow room for saying huh that’s funny. 

[Web URL for this note](http://notes.benjaminreinhardt.com/Peer_review_is_a_great_system_for_preventing_waste)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Peer_review_is_a_great_system_for_preventing_waste)



[^1]:  Which shouldn’t be a given! Some wonderful things in the world are wildly inefficient! [[Efficiency is overrated]]




Heat cycles for democratization and cutting edge experiments innovation and discovery

For sure the stars outline v2

Phenomena-based cycles

arthurNatureTechnologyWhat2009

Research has many orthogonal and non orthogonal classification axes

gertnerIdeaFactoryBell2012

X-punk imagines a world dominated by X

Focused play enables serendipity

Computers are not great at saying ‘huh that’s funny’

Phenomena are things that happen without a human consciously making them happen

Timesharing could be extended from digital infrastructure to physical infrastructure

Introducing Speculative Technologies Outline

Ideally corporate R+D enables targeted dicking around

Bell Labs did not give researchers complete freedom

Telerobotic technology can unlock collaboration and serendipity

Bell labs asked people to do useful things but left them room to say huh that’s funny

Architecture matters for creativity

Unexplainable phenomena are not just indistinguishable from magic, but are magic

Tool accessibility is underrated for heuretic development

# The ability to say ‘huh that’s funny’ is important for discovering new phenomena
Many phenomena are not predicted by theory, and in the [[Thomas Kuhn]]sian framework, the only way we get new theory is exactly *by* observing phenomena that don’t fit theory.  Put in a [[Thomas Kuhn]]sian framework, the ability to say ‘huh that’s funny’ is ability to notice something unpredicted by existing hypotheses.

 [[Theory can suggest where to dig for new phenomena]]  - but even then it requires the ability to recognize when a theory is implying something unexpected about the world. 

Saying “huh that’s funny” (and not just brushing it off) is actually *really hard.* How many natural phenomena weren’t investigated not because we didn’t have the tools to do so, but because people just didn’t engage with the “oh that’s funny” instinct - for example the fact that two balls of unequal mass will fall at the same rate. Whoah! One theory of how our brains work is that they literally spend all their time trying to minimize the mismatch between what we expect to observe and what we actually observe. 

Repeated observations and the ability to ‘look at things from a different angle’ both help engage with saying ‘huh that’s funny.’ If you see an anomaly only once, the rational prior is that it’s a mistake in the observation rather than a new phenomenon. The importance of repeated observations is why instruments that cost a ton per observation (like the LHC) are suboptimal. ‘Looking at things from a different angle’ is a broad term that effectively means running a slightly different experiment intended to highlight the potential new phenomenon. The importance of observation tweaking is why instruments with fixed parameters like a mars probe with only a camera and a mass spectrometer are suboptimal.

 ‘huh that’s funny’ type observations can come about in two broad ways. New observational tools can get into the hands of a few people (like the Hubble telescope, or LHC) or old tools can into the hands of many people, like telescopes becoming cheap enough that amateur astronomers can proliferate. You could call these two modes “frontier-pushing” observations and “democratized” observations. Both have been historically important. 

“Frontier-pushing” observation tools are often expensive, custom made, and finicky: requiring dedicated skills to operate and interpret the results of. As a consequence, they are usually part of well funded, ‘centralized’ research efforts tied to institutions and led by incumbents (or at least people who are good at sales.) Powerful telescopes, Scanning Electron microscopes, test reactors, early computers all match this pattern. 

As better frontier-pushing tools are created, two things happen: the processes for making them become more well-known and standardized, driving down the price; and the original users get better ones, throwing away the old tools (or selling them for cheap.) “Science advances when people can buy rich people’s garbage toys” - [[Gabriel Licina]]. This price decrease and supply increase allows more people to get their hands on the observation tools. In addition to simply more eyeballs increasing the probability that someone will see something that makes them go “huh, that’s funny”, the people using more democratized tools are more likely to be outside of a traditional institutional structure, whether it’s a specific field, a university, etc. and so they will have different rules constraining their behavior ([[Institutions shape how individuals interact]]) and so might think about what they’re seeing differently and realize “huh, that’s funny” about something the original frontier-pushers overlooked.

In an ideal world, this loop of frontier->democratization will observe new phenomena that feed back and enable new generations of frontier tools. [[Heat cycles for democratization and cutting edge experiments innovation and discovery]]. 

There are several trends that worry me about a continued ability go ‘huh that’s funny.’ 

The cost of frontier-pushing tools seems to have outpaced GDP so that only governments or consortia of governments can afford things like particle accelerators or space telescopes. Government-dependent frontier-pushing tools mean that they are inherently coupled to heavy bureaucracy, are under the control of “government approved” institutions, and if politicians no longer think they’re valuable, they will not be built. The cost also means that very few of the are built, so there is massive demand relative to the supply. This demand means that there will be gatekeepers who need to sanction experiments so it’s harder to do something “outside of the paradigm” and further explorations into ‘huh that’s funny’ observations often are “out of paradigm.” Additionally the demand makes it hard to spend time ‘dicking around’ with the tools - another common pathway to discovering new phenomena.   

There’s a class of modern frontier-pushing observation tools that aren’t dropping in price and enabling democratization the way they have in the past. Even though they’ve been around for more than half a century, there are no community owned particle accelerators. The concern is a widening gap between the cutting edge and democratized modes of phenomena discovery. However, there is a lot of hope on this front. The internet enables easier time and cost fractionation for large pieces of equipment, decreasing launch costs are starting to put space telescopes in the cost range of amateurs, and you can literally buy scanning electron microscopes on eBay. 

Modern frontier-pushing tools also require large teams to operate them - particle accelerators, space telescopes, mars rovers, etc. These teams introduce coordination costs that kill the ability to go “huh, that’s funny.”  Large teams require experiment plans that are hard to deviate from on a hunch. Group decisions are less likely to lead to high-variance observations. [[Decisions made by committees lead to median results]] And, as noted before, groups of people are more easily locked into certain paradigm lenses. [[Paradigms are lenses for looking at the world]].

Perhaps most controversially, the digitization and computerization of observations has made it harder to notice things that make you go “huh, that’s funny.” Note that on whole, digitization and computerization been on net a massive boon for research!

Until you get *very small* the world is continuous, while computers are discrete. This difference has two consequences. The first is that every input needs to be run through an analog-to-digital conversion which involves ‘bucketing’ continuous signals into discrete buckets. The choice of these buckets can make the difference between an anomaly being classified as signal or noise. It’s like a different take on [[Simpsons Paradox]]. 

 Digital measurements tend to capture a limited set of information channels - either specific wavelengths of light, only the mass spectrometer data, etc. It’s like observing the world through a little peephole with headphones on. The example that comes most to mind is mars rovers. Contrast to 19th century chemists who would look at, smell, touch, *and* lick things. 
Of course, this isn’t purely because of digitizations - pushing the limits of what can be observed requires more specialized instruments. 

Computers have enabled an entirely new way of observing the world through simulations.  However, simulations are built up from how we already believe the world works.  So while simulations can recreate emergent phenomena, they are totally blind to phenomena that don’t arise from components we don’t think are important to simulate, or don’t realize that they even exist.  [[Simulations and models often fail to capture important aspects of a real system]]. 

### Related
- [[Galaxy Zoo]]
- [[Kuhnsian knowledge creation and Popperian knowledge creation assign different first steps to the same loop]]
- [[Computers are not great at saying ‘huh that’s funny’]]
- [[Heat cycles for democratization and cutting edge experiments innovation and discovery]]
-  [[Paradigms are lenses for looking at the world]]. 
- [[Phenomena-based cycles]]
- [[Cosmic Microwave Background]]



[Web URL for this note](http://notes.benjaminreinhardt.com/The_ability_to_say_‘huh_that’s_funny’_is_important_for_discovering_new_phenomena)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/The_ability_to_say_‘huh_that’s_funny’_is_important_for_discovering_new_phenomena)




§Academia Constraints

Most grant money is tagged so unrestricted money is like gold for professors

NSF Program officers can have multi-decade tenures

Things that are not paper-worthy enough for academia and not product-focused enough for a startup

Governments are well suited to funding research but effective research management is in tension with other government mandates

Lee Ricketson conversation 6/5/20

Program Managers have the ability to deploy money without much overhead

The money from SBIR is barely worth the money

# The government grant process depends on politics and committees
Roughly, the [[National Science Foundation - NSF]] and [[National Institute of Health - NIH]]  get some of the ‘top people’ in a field together in a room and ask them “what programs should we pursue?” This process depends first on being able to identify the top people in a field and convince them to stay in a shitty hotel in Washington. So the agenda is being set by ‘the top people in a field who are willing to come to Washington and stay in a shitty hotel.’ Additionally, who entails the ‘top people’ is up to the program officers and since [[Professors have long term relationships with their NSF program officers]] chances are that the professors who have put the work into those relationships are going to be the most salient. 

All the incentives are set up so people probably push for their lines of research. Even if they are being completely intellectually honest, researchers probably think that their area of research is the most important thing to fund. If they are slightly less intellectually honest, they realize that they’re more likely to be funded if their priorities line up with the NSF’s. 

The process is inherently conservative [[All the incentives in research drive towards conservatism]] because if you’re consulting the top people in a field on what to fund, the chances they’ll suggest the weird new thing are low. This is another reason why [[Younger professors have to spend more time getting money]].

### Related
- [[Decisions made by committees lead to median results]]



[Web URL for this note](http://notes.benjaminreinhardt.com/The_government_grant_process_depends_on_politics_and_committees)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/The_government_grant_process_depends_on_politics_and_committees)




Graduate students are the labor in academic science and engineering

Academia is not a good environment for systems engineering

Academia incentivizes novelty, not focus

Academic credit is a first-to-the post system

We have no flying cars - failures of paradigm-shifting engineering

Academia judges via novelty because there are no metrics but there are no metrics when the thing is still research

People are incentivized to work on nodes higher up the tech tree

Legible novelty is the worst

# The word ‘novel’ is used as an idea bludgeon in academia
“Novelty” is one of the constant criteria by which academic research is judged, both in terms of funding and publication. 

Whether something is novel enough not purely about the magnitude of the contribution (the characterization of reactions that barely differ from other reactions can be considered novel) or the actual newness of the idea (If I argued that the transition between turbulent and laminar flow was mediated by conscious 11-dimensional elves it would be a new idea, but not in the good way.) Instead, the worthy novelty of a contribution revolves around its relationship to existing paradigms.  [[Paradigms outline a set of questions and the rules around answering them]].  

A contribution has massive headwinds if the paradigm judges the question being answered to be already answered or out-of-scope or the answer is not in an approved form (the 11-dimensional elves again.) 

It’s easy to say “argh! The system is full of risk aversion, politics, and narrow mindedness!” But [[Most people are not malicious or stupid]] and that goes for academics as well. Most ideas that answer old questions in new ways or have completely different answers are crackpot ideas (11-dimensional ideas!) so it’s not irrational to default dismiss them given limited time and bandwidth[^1]. 

If a question has existed unanswered for a long time, inevitably many people have tried to answer it. Every failed answer rationally increases the priors that the next answer will also fail. Therefore it’s perfectly reasonable prior that a new answer to an old question will also fail. It’s also technically correct that proposing a new answer to an old question is not “novel” creating a rational excuse for people not to waste their time with crackpot answers. However, the novelty criteria become a cudgel when people use it  to avoid spending the time to actually engaging with an idea. The problem is that it’s not actually clear where to set the filter between “take every crackpot idea seriously” and “reject every answer to long-standing problems.” 

The incentives around new answers to old questions also apply to the creation of new questions. [[Academia pushes against the creation of new questions]]. New questions cannot be measured by their performance on standard metrics or data sets. At least in early 21st century machine learning, it’s easier to get a paper accepted by improving performance on a standard dataset from 98.2% to 9.85% than it is to argue that your system is doing something important independent of an established test or *heaven forbid* has performance that can’t be measured numerically. Again, I suspect this situation at least in part arises from the number of people all clamoring for attention and everyone’s limited bandwidth [[Attention is the ultimate scarce resource]]. New questions can also be completely useless. One out of a thousand times “What if we ran it backwards?” Is the key to saving the universe, but most of the time it’s someone trying to be clever with a bad idea.

These two situations parallel startups heavily. It’s rational to have a prior that a company trying ideas that are littered with the bodies of other companies (memexes, new social networks, elder care, etc.) will fail. However, [[It’s rational to try something that failed in the past as long as you explicitly call out why you will succeed when they failed]] but that requires time and precision from both the founders and appraiser. In parallel with new ideas in science, startups trying to introduce a product that “nobody knew they wanted” usually fail as well.  

The upshot is there are good reasons to dismiss new ideas that are both novel and not novel. Therefore “novelty” is a relatively arbitrary idea discriminator and easy for people to use to dismiss ideas they don’t want to engage with. 

### Relevant
- [[§Academia Constraints]]
- [[Mysteries vs puzzles]]
- [[Peer review has failed as an institution]]
- [[Novelty and Heresy - PG]]

[^1]: The coupling between bandwidth and ability to address crackpot ideas on a case-by-case basis of course plays into my bias that insufficient [[Slack - concept]] is a core source of friction in the system. 



[Web URL for this note](http://notes.benjaminreinhardt.com/The_word_‘novel’_is_used_as_an_idea_bludgeon_in_academia)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/The_word_‘novel’_is_used_as_an_idea_bludgeon_in_academia)




# Using citations to measure science is insidious
#### Some arguments for using citations as a way to measure science
- Great work becomes highly cited eventually
	- Is this actually true?
	- Selection bias
- We don’t have anything better 

#### Comparing metrics across fields
Citations are not a normalized or consistent metric across fields. Citation traditions vary across fields - in some domains you only cite a paper if you are leaning directly on their arguments to build your own and in others citations are like a hat tip to people who have done similar work. The most cited paper ever is a biological method (I think an assay) just because biology has a tradition of citing where an experimental method originated. Is the most cited paper the most important paper ever? The answer is pretty inarguably no.[^1] 

The straightforward response to different citation traditions is to only use citations to compare work within a field. But then where do you draw the lines between fields? How can you compare fields? I would also worry that forcing citations to live within fields would also further reenforce the divisions between fields. [[Categories that are coupled to metrics tend to take on a life of their own]].

#### Great work is cited eventually
One of the most compelling arguments for citations as a metric is that “the most important papers get cited eventually.” 

Nobody cites Newton when they calculate force in a paper, so why should we expect other papers that have become part of the intellectual milieu to be highly cited. Instead of importance, citations capture “perceived importance” which is a different thing that also has to do with salience and novelty. Perceived importance is incredibly path dependent, favoring work that is recent enough to be salient in areas where people are publishing many other papers - this roughly describes well-known work done in an area just as it is beginning a hype cycle. 

“The most important papers get cited eventually” argument smells a little bit like “the best startups will win in the end.” 

Citations are subject to the Malcolm Gladwell effect — often the canonical, most-cited paper on a concept is the most useful/interesting/from a famous author formulation of it, not the original appearance of the idea. Call the former the ‘popularizing paper’ and the latter the ‘originating paper.’ There is nothing wrong with people citing the paper they actually read! However, there is something wrong with then turning around and using those citations as evidence that the funding for the originating paper was poorly spent and funding for the popularizing paper was well-spent. 
You could even make an argument that the ‘popularizing paper’ is truly much more important than the ‘originating paper.’ “Ideas are cheap!” However, that seems like an unsettled argument so secretly embedding that philosophical stance in every citation-based piece of meta science seems insidious indeed. 

This idea of a ‘popularizing’ vs. ‘originating’ paper suggests one mechanism that citations can be subject to [[Goodhart's Law]] by pushing everyone towards popularizing papers. If funding organizations are judged based on whether they fund highly cited papers then they will bias towards research and researchers that are likely to produce popularizing papers and researchers in turn will shift towards work and papers that could potentially become popularizing in order to get funded. This critique does assume that it is possible to know a priori which researchers and work would produce highly-cited papers. I suspect this is an area where people can have a non-evenly-distributed sense of prior probabilities. [[Some parts of science have much more uncertainty]] . The insidious part is that (I suspect) people are better at predicting which work is likely to be highly cited because the researcher is a rockstar or the research is riding a hype cycle than they are at predicting which work is likely to be highly cited because it is a significant discovery. If citations are quality metric, this will shift support towards the former category and away from the latter. 

Citations as a metric ignores the (fact?) that there are multiple mechanisms for affecting the way we see the world.  Some people are not impactful because many people listen to them but because a few powerful people listen to them. Similarly, some work is important because it influences a few pieces subsequent work that themselves are significant. [[We stand on the shoulders of too many giants to give them all credit]]. 

Citations as a metric screws up citations as a knowledge tool. Abstractly, citations would be like pointers to encapsulated arguments that someone has made elsewhere. 

I wonder how citations per paper have changed over time. My hunch is that as they became more important as a measurement tool, total citations per paper skyrocketed. Of course, over the same period of time, the number of researchers and papers that could possibly do citing also skyrocketed. It’s probably impossible to disentangle those effects. It does emphasize the fact that at best, citations are a relative measure of quality at a single point in time. However, it’s not as though you can only cite a fixed number of papers per paper. You could cite zero or you could cite a hundred. Citation printer go brrr. So if paper is cited it’s not really a tradeoff for the author citing it. The don’t even need to have read it. 

The upshot of all of this is that I believe the burden of proof is on people who want to use citations as evidence to make statements about science or science management to show why they are better than no metric at all. 

### Related
- [[The peer review and citation system incentivizes people to work on things that other people think is interesting]]
- [[Decreased trust is crippling research]]
- [[Book Review - Seeing Like a State]]

[^1]: I don’t know if it’s even possible to say what the most important paper is, but it is possible to say which ones aren’t it. 

[Web URL for this note](http://notes.benjaminreinhardt.com/Using_citations_to_measure_science_is_insidious)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Using_citations_to_measure_science_is_insidious)






Benʼs working notes

Peer review is a great system for preventing waste

Related

Links to this note