A large part of a DARPA program manager’s job is focused network building

R+D orgs lie on a spectrum between externalized and internalized execution and benefits

Focused play enables serendipity

Online communities could enable designed serendipity

Workshops could be a way to start small building a private ARPA

§Augmented Knowledge Generation

People should be more open to small discrete questions

PARPA H1 2022 Update Outline

A system that encoded papers as problems and solutions and then surfaced the papers and associated researchers based on the problem you had

Designed serendipity program

Annoying little conjecture problem

Bell Labs enabled free radicals

Architecture matters for creativity

PARPA H1 2022 Update Letter

It is exponentially easier to answer a tight well framed question than a big fuzzy one

What do online DARPA workshops look like?

Collective Intelligence

Joshua Elliott conversation 17 Jan 2023

Lunch 5 Jun 2022

Proof Discovery Tree - PDT

vingeRainbowsEnd2007

# Designed Serendipity 
# Designed Serendipity 
This approach to [[§Augmented Knowledge Generation]] is focused on how to most effectively deploy expert attention.  Expert [[Attention is the ultimate scarce resource]] and ideally you would be able to talk to the relevant expert at the exact moment you need to and they could spend 10 minutes explaining something that would have taken you hours to figure out.

If you accept the idea that for [[Discrete technology improvements (quantum leaps)]], [[New ideas need to come out of a single mind]] this is the approach with the most upside. It’s also the fuzziest. 

In a way, [[§Multidisciplinary Teams]] are a ham-fisted attempt to create designed serendipity. 

### Case Studies
- [[Innocentive]] is a marketplace where companies can post challenges
	- It’s at TRL 9 but with many constraints 
- The [[Polymath Project]] hinted that it was possible for experts to coordinate better than the traditional way
	- TRL 3
- Open source software projects (Linux,  Firefiox) and wikipedia are the gold standard for designed serendipity
	- TRL 9
- The [[Polyplexus]] project is attempting to make it easier to encode ideas and find the people with relevant ideas. 
	- TRL 4/5?
- [[A system that encoded papers as problems and solutions and then surfaced the papers and associated researchers based on the problem you had]]
	- TRL 2/3
- There are many attempts to surface relevant researchers (Google Scholar, Orchid) but without additional information it’s not that much more useful than just emailing people who appear to have written relevant papers.

### Principles/Elements of Success
Some pieces that you see consistently in more successful projects
- [[Successful approaches to designed serendipity focus on small information quanta]]

### Challenges
- Getting enough of a shared context between two people so that they can come up with new ideas often takes a lot of time and patience. 
- There’s no way to set prices in a knowledge marketplace
- [[How do you structure attention so that you connect to the right microexpertise at the right time?]]

### Possible Opportunities
- Twitter may be underutilized as a designed serendipity platform.  

### Related
- Designed serendipity is almost a subset of [[Expertise Finding (wikipedia)]] that needs to happen as real time as possible and with as little friction as possible.

### References
- The term originally came from [[Michael Nielsen characterizes institution building as making previously illegible things legible]] in [[Reinventing Discovery]]  where he paints a vision for an ‘attention marketplace’ and describes many case studies. 
- [[Can Twitter Save Science?]] makes the argument that Twitter might be the correct platform for designed serendipity.



[Web URL for this note](http://notes.benjaminreinhardt.com/Designed_Serendipity)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Designed_Serendipity)




Designed Serendipity

Reinventing Discovery

Attention is the ultimate scarce resource

A ‘related’ section can serve as kindling for an evergreen note

Restructuring expert attention is powerful and done poorly

# How do you structure attention so that you connect to the right microexpertise at the right time?
# How do you structure attention so that you connect to the right microexpertise at the right time
 I haven’t seen any compelling attempts to address this. [[Polyplexus]] gestures at it. 

[Web URL for this note](http://notes.benjaminreinhardt.com/How_do_you_structure_attention_so_that_you_connect_to_the_right_microexpertise_at_the_right_time)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/How_do_you_structure_attention_so_that_you_connect_to_the_right_microexpertise_at_the_right_time)






Problems where it is easy to know whether an answer is correct but hard to get to that answer

# Reinventing Discovery
- More than a dozen Polymath-like projects have been launched  (p2)
- Description of the “postdoc” problem (p8)
- Case study: Kasparov vs the World (p17)
- [[Innocentive]] case study (p23)
	- $20,000 from Rockefeller foundation to create a solar powered router
- Expert [[Attention is the ultimate scarce resource]] (p24)
- Innocentive creates value by *Restructuring expert attention*
- [[Irina Krush]]’s  micro expertise helped the world team win (p25)
- Successful online collaborations consistently locates people with the right expertise at the right time (p27)
- Successful online collaborations have *designed Serendipity*
- The term Manufactured Serendipity came from Jon Udell
- Collaborations need to reach *collaborative critical mass*(p30)
- Summary of amplifying collective intelligence
	- Create an *architecture of attention* (p32)
		- Modularize collaboration
		- Encouraging small contributions
		- Rich and well structured information commons 
- Divisions of labor can be *static* or *dynamic* static is the typical thing in the economy but knowledge generating collaborations need more dynamic roles (p35)
- Prices are useful because they aggregate hidden knowledge (Hayek) (p37)
- Prices enable more dynamic divisions of labor
- Online collaborations work when committees don’t for two reasons (p40)
	- People in online collaborations are volunteers
	- Unconstructive people can stop a committee but be ignored in a collaboration
- Claims that you can get answers in 10 minutes from world experts (p41)
- Case study for Linux (p44)
- Cites open architecture network (p48)
- Galaxy Zoo successfully divided the big task into many smaller tasks that could be done by non-experts (p51)
- Polymath project was hard to modularize and so you actually needed full context to contribute which meant it was limited to 27 people
- Asks question - is it possible to modularize collaborations?
	- Linux
	- Wikipedia
	- Firefox
- Wikis don’t work for writing novels because every sentence could related to any other sentence (p53) 
- Predicts online collaboration tool that lets people track relationships across a collaboration (p55)
- Modularity pattern - atoms of attention -> architecture of attention (p56)
	- `Modular units are atoms of attention out of which the architecture of attention is built`
- Collaborative projects create a rich information commons that people can use pieces of (p58)
- It is hard to create tiny incremental improvements in science in the same way as on wikipedia (p59)
- Case study of Mathworks Competition (p60)
	- Try to fit songs on CD with least space left over
	- Leaderboard
	- Everybody gets to see solutions once they’re posted
- Idea of microcontirbutions (p63)
- Scores can help coordinate expert attention (p64)
- Mathworks score more effective at directing attention than polymath project where it could take hours to figure out where to go (p66)
- Stasser-Titus experiment: good and bad candidates on paper, people voted well they each had full profiles but if they had pieces of information about the candidates they consistently did not put together the full picture of them. (P71)
- Successful collaborations tend to have a gap where it is hard it is to come up with a solution easy to recognize a solution as good.  (P73)
- Stasser-Titus might have had a different result if everybody had pooled their information in the beginning (p74)
- A *shared praxis* is a body of knowledge and techniques that is shared among collaborators (p75)
- Shared praxis is critical
- A vision for how collaborative science could be done (p82)
	- Filtered requests for help 
- It’s difficult to share data in a way that’s useful for others (p109)
	- Terminology
	- Formats
	- Calibration information
	- Processing
- AI seeps into every discussion of potentially using computers to find meaning in knowledge (p111)
- `This middle-ground future is conceptually rich, fascinating, and strangely under-discussed, perhaps because the dreams of artificial intelligence exert such a strong pull on the imaginations of the technologically curious.`
- Term *Data Driven Intelligence* to describe ability of computers to pull meaning from Data (p112)
- Potential of databases to be linked up (p121)
- In science, models and explanations are blurred (p127)
- Story about someone searching for papers about a cancer as argument for open access (p159)
- Cites Pharyngula blog as getting 100,000 visits per day (p167)
- Promising but failed tools for scientists (p176)
	- Science wikis - Caltech guy tried to create quantum wiki but people all wanted someone else to take lead to contribute. 
		- No incentive to contribute except to vanity pages
		- Competition to get faculty jobs is brutal so the grad students and post docs who would be best able to contribute to a wiki don’t have incentive to contribute (p178)
	- Open comment sites
		- People wanted to read comments but not put them (p179)
- Call moving information out of people’s heads and laboratories onto a network. (P183)
- Argues that Bayh-Dole act shifted incentives towards making research more secretive (p184)
	- [[The relationship of government, academia, and technology fundamentally changed in 1980 with the Bayh-Dole act]]
	- Bayh-Dole made it so governments and grant agencies see the pursuit of intellectual property as a reason to support basic research
- Case study of Tobias Osborne trying to do open science as part of graduate school and concluding it was too much work (p187)
- Attempted contributions by people outside the praxis can overwhelm people (p199)
	- Open systems without requiring that all participants get equal attention

## Questions
- Have any polymath like projects succeeded outside of math?
- When do prizes work well? When don’t they work well
- [[How do you structure attention so that you connect to the right microexpertise at the right time?]]
- How do you apply prices to attention?
	- Especially when people are volunteers
- Did open architecture network actually work? 
- Did p55 prediction play out?
- How do you create micro contributions outside of software?
- How can you create something like a “score” for a much more open ended problem?
- Why have we failed to execute on the vision of linking databases up?
- How many visits does Pharyngula get per day in 2020? (Hypothesis that blog readership has gone down)
- Does the Bayh-Dole example push back on the philosophical argument that trying to capture all value created is optimal?
- How do you create an open-but-filtered program that doesn’t fall prey to heuristics like followers, citations - status symbols?
- Don’t all heuristics for filtering become status symbols?

## Thoughts
- Is there something to the fact that the collaboration-type projects don’t require manipulating any atoms?
- Where are [[Irina Krush]] ’s essays online?
- [[Restructuring expert attention is powerful and done poorly]]
- Collaborative critical mass is important but then there can be too many people as well - maybe the way it works is that there’s an optimum and technology can shift that optimum based on the problem
- You can’t actually get answers quickly from experts - why how could we change that
	- Need ‘spam’ filter for requests 
- You need human expertise and context glue to go from information to useful things when it’s not directly on a computer - this is why open source tends to work but you don’t see much open architecture or hardware.
- Note that in 2020 Wikipedia is having trouble getting enough people to contribute - people want to volunteer for exciting things but not boring things like maintenance.
- Seems like many open source projects are not “novel” there were operating systems before linux, browsers before Mozilla - the end goal was well known
- Rich information commons turn into a search problem and a comprehension problem. Also require a lot of work on the part of the people putting them out
- Scores need the problem to be well-defined
- Problems are often not well-defined on the knowledge frontier 
	- [[Creating tight problem definitions is valuable]]
- Kaggle is a logical extension of the Mathworks competition
- Score design is a lot like objective design for reenforcement learning
- Stasser-Titus and polymath point to the fact that tools to create shared context are super important and poorly done
- At the core of the vision for collaborative science is an [[Expertise Finding (wikipedia)]] problem
- Purely data-driven models like deep learning are not good explanations
- Pushback to value of open access - even if you have access to all the papers in the world on a type of cancer you still might have no idea what’s going on 
- Some fuzzy notion of the combination of 
	- Umwelt from [[The Intellectual Wild South Is Open For Business - Eric Weinstein]]
	- [[Bridge Model of Knowledge]]
	- And context
- Call for extreme openness in science feels a bit like socialism - it sounds really good but is in practice untenable - “hasn’t been done right”
- It feels like most semi-open online communities are overwhelmed by attempted contributions by people outside the praxis 
- StackOverflow may do the “open without equal attention” well and 
- Need better ways of filtering attention

## Related
- [[Universities have been pushed more towards applied research since the end of WWII]]
- [[Bret Weinstein - The Prediction and the DISC - Portal Podcast]]
	- Specifically the grad school problem 
- [[Irina Krush]]
	- Specifically her essays 
- Cites [[bushWeMayThink1945]]
- Cites [[Designing Organizations for an information rich world]]
- Cites [[The Attention Economy and the Net]]
- Cites [[eScience: A Transformed Scientific Method]]
- Cites [[Sam's encounter with manufactured serendipity]]

## Published
2012
## Authors
- [[Michael Nielsen characterizes institution building as making previously illegible things legible]]
## Link
https://www.amazon.com/Reinventing-Discovery-New-Networked-Science/dp/0691160198

[Web URL for this note](http://notes.benjaminreinhardt.com/Reinventing_Discovery)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Reinventing_Discovery)






# Restructuring expert attention is powerful and done poorly

### Related
- [[Attention is the ultimate scarce resource]]
- [[Center for Humane Technology]]
	- What if you could use modern attention-directing tools to direct expert attention in a useful way?
- [[How do you structure attention so that you connect to the right microexpertise at the right time?]]


[Web URL for this note](http://notes.benjaminreinhardt.com/Restructuring_expert_attention_is_powerful_and_done_poorly)

[Comment on this note](http://via.hypothes.is/http://notes.benjaminreinhardt.com/Restructuring_expert_attention_is_powerful_and_done_poorly)




Benʼs working notes

How do you structure attention so that you connect to the right microexpertise at the right time?

How do you structure attention so that you connect to the right microexpertise at the right time

Links to this note