The barriers to creating a for-profit organization boil down to initial costs, talent, and long term funding. All of these barriers have become low enough that even crazy ideas that are good investments should tend to get funded.
The costs to starting a company are low - even if you need a lot of equipment because it is relatively easy to borrow lab space or equipment time. The startup ideology has pervaded society to such an extent that even People who should not be starting companies are starting companies.
Socially, entrepreneurs are revered and people have been so indoctrinated that ‘stupid’ ideas can become billion dollar companies that there isn’t much inhibition the talent front.
On the funding side, there are more early-stage VCs than ever and despite the fact that technically Venture capital doesn’t work for things that only realize their value on long timescales, the sexy success of SpaceX has encouraged at least some of them to bet on long term projects that could be wildly valuable. If a project has massive possible upside and low enough risk it is venture fundable. Additionally, angel investing has become a status symbol in some circles, pushing capital’s risk tolerance even farther.
In combination these factors mean that the for-profit ventures that would be good investments but are not being started or funded are rare and on the margin. Of course, There are many important innovations that are just bad investments.
Perhaps the place where they are inefficient is actually at the super capital-intensive end. The Long Term Stock Exchange (LTSE) will test this.
The counterargument to this assertion is that the success stories of impactful companies do seem to be incredibly contingent. Success depends on the right people being in the right place at the right time and being willing to doggedly persevere. Are there systemic ways to increase the rate of these contingent events?