Innovation & market failure

Maynard Keynes famously said that the job of investing should not be left to the market. But can the same be said for research and development? This is the question raised by a new paper (pdf here) by Daron Acemoglu.
He starts from a fact which is often trumpeted by admirers of the free market - that producers capture only a tiny fraction of the full benefits of innovative activity.
This, he says, means that innovation will be misdirected. Imagine two products: say, a petrol car and an electric one. An innovation that slightly improves the petrol car can be marketed immediately and thus is profitable. However, a small innovation in electric cars merely makes them slightly less useless and so is less monetizable - although such innovations might be a necessary  stepping stone to further innovations which would make the car feasible.
Innovative activity will therefore be directed towards the petrol car rather than the electric one. The result will be insufficient diversity in innovation:

The recognition that there will be further innovations will discourage research in areas that will generate new products or technologies for the future relative to improving currently used products, processes, or technologies. Consequently, in equilibrium, too much research will be devoted to currently successful product and technology lines.

Three things make me suspect that this problem could be especially pressing now:
1. If there is a danger of catastrophic climate change, then there is a need for new green technologies. But the danger that producers won’t get the fruits of such innovations - as they’ll be usurped by future better innovations - can lead to under-investment in them.
2. Capital spending and productivity growth have been weak for years. This might be a symptom of a decline in useful innovations. Maybe capitalism has picked the low-hanging fruit.
3. Many innovations (pdf) since the 1980s have had the effect of increasing inequality. They’ve benefited capitalists and/or bosses at the expense of workers.
So, what are the counterweights to insufficient diversity?
One, says Acemoglu, is simple human nature. People differ in their interests and competences, and so might research fields which are a low priority from capitalists’ viewpoint. We might add to this the use of tax credits for R&D activity, and also the fact that capitalists tend to be over-confident and hence willing to invest in areas where the objective probabilities of success are low.
In this context, there is a strong argument for universities’ research priorities being independent of commercial considerations; such independence is one way of promoting the diversity of research which pure market pressures might tend to reduce.
It would of course be foolish to dismiss market forces as a tool for producing innovation: markets have a better record than state intervention at producing diversity. But we should ask whether there are policies which might make the market better in this regard. And this question should perhaps be a higher priority than it is.

Comments

[FiscalSubvert]

An astute query: "Maybe capitalism has picked the low-hanging fruit." The consequences of this are terrifying. In 1950 this would not have mattered. Now it does. As the low fruit is picked blindness is multiplied. It fogs the importance of choosing for future sustainability. I am considering moving to Siberia.

[Cecily]

Innovation is important, especially in over saturated markets. But there is always a limit to what new things the market will accept.

[Frustrated]

Great points Chris. I'd add that government is generally hopeless at funding innovation directly. At work we have had to deal with the Technology Strategy Board which doles out public money to start ups. Our experience has been a load of clueless bureaucrats picking winners and everyone gaming the system. The biggest winner is a guy who earns his living writing largely fabricated reports to get money off the agency on behalf of candidate companies.

You are right that our biggest hope is universities getting on with it independently.

[BT]

"...for producing innovation: markets have a better record than state intervention at producing diversity."

Actually, this might not be true. Business always has the short-term goal of profit. R&D in companies is almost always very applied science with no real breakthroughs.

Only government-funded research (and private charity-funded research) can persue genuine blue-sky science - which is where the big discoveries that change humanity come from.

[CS Clark]

One disturbing thought is that it might be superlarge companies with monopolies or quasi-monopolies, from AT&T's Bell Labs to Google's, um, Google, that allow themselves to indulge in blue-sky science, simply because they are less worried about short- to medium-term profits. Disturbing because that would mean otherwise worthwhile efforts to create competition would force those companies to give up developing anything that isn't going to make money soon.

[guthrie]

Well, yes, that's why forcing universities to concentrate on marketisable research is a dead end.
Of course you've written it far better than I could have done.

[Straus]

There is one area of government or government-backed activity that has always heavily invested in innovative technologies: the military-industrial complex, including the space sector. And, what is more, it has been a fruitful source of spin-off technologies, from non-stick pans to the early form of the internet. The market, though, is best at making sure that these technologies are used in ways that bring the greatest benefit to the greatest number of people. The Soviet Union produced its far share of cutting-edge technologies and had an enviable record for scientific research and development, but that did not translate into all those clever little devices that make life more fun and our daily chores less onerous, especially for the housewife (as she used to be known).

The currently budgetary squeeze, though, is forcing the government out of some of its traditional areas of activity. NASA is drastically reducing its operations and is now commissioning private companies to design and build craft that it will hire out for its missions. They say that this will be both cheaper and will stimulate greater innovation. A recent business programme on Radio 4 suggested the fact that the companies concerned are looking to make space travel commercially viable (albeit with some of their business coming from the government) has brought about innovations in materials technology that have significantly reduced construction costs. I suspect these innovations may have taken longer to come about had space technology remained a state monopoly.

[Luis Enrique]

economists have a pretty crude idea of what innovation is, imho. There is pure technology / idea innovation and then there is turning ideas into commercially viable products (i.e. things where the value placed upon them buy consumers exceeds the cost of producing them). When Keynes said investing ought not be left to the market, which was he talking about?

[Brassk]

Very interesting post - especially the question of making innovation benefit everyone - How can you make an investor choose to invest in an innovation that is good for everyone, over an investment that is better for them. Its when faced with that comparison that the failure is most evident. People are often happy to invest in the right thing - but maybe not when there is a more profitable alternative (regardless of the environment and social impact of the alternative).

[Guy in the Veal Calf Office]

Why is that improvement to the car "insufficient diversity"? It seems that innovations to products nobody uses is wasteful diversity.