September 13, 2007
Increased Funding Did Not Increase Scientific Discovery Rate?

Writing in The Scientist Frederick Sachs argues that the large increase in funding for the US National Institutes of Health did not produce a commensurate increase in scientific productivity as measured by papers published.

Since the NIH budget doubled from $15 billion to $26.4 billion from 1999 to 2003 (Figure 1), I reasoned that there should have been a corresponding jump in productivity. The test was the simplest measure of productivity: the number of publications.

Here's what I found: The number of biomedical publications from US labs did in fact increase from 1999-2004. However, so did the number of publications from labs outside the US where the research budget did not double. Figure 2 shows a parametric plot of the number of scientific papers indexed in the ISI Web of Science database by the keyword "biology" that were published each year from US labs and non-US labs. There is no upward jump that you would expect to see with a sudden increase in productivity.

So are the brains in science tapped out already? Why didn't the rate of papers published increase?

I would have expected an increase in the rate of research papers getting published as a result of more powerful scientific instrumentation. But maybe the brains that used to try to figure out how to tease out smaller answers from lousier tools now publish the same number of papers but with bigger findings per paper on average? Certainly that happens with a lot of genetics papers which describe findings about hundreds and thousands of genes at a time. 10 or 20 years ago a single research paper couldn't report about many genes at once because the tools for sequencing and measuring genes were too primitive.

Any speculations on what is going on with the result Sachs reports?

Share |      Randall Parker, 2007 September 13 10:28 PM  Policy Science


Comments
rsilvetz said at September 13, 2007 10:43 PM:

Geez. An inefficient, ineffective, socialist, bureaucratic research system (as measured by its failure to cure most anything over 40 years) hits diminishing returns...

Am I the only one not surprised?

Sublimate it as rapidly as possible into a free-market, sink or swim, enterprise and maybe we will see something good come of it.

Mats-Erik Pistol said at September 14, 2007 1:34 AM:

I agree completely with rsilvetz. I work in a University department and I now firsthand the enormous waste of money as well as the mindset of (most) senior university scientists.
Applying for grants fill most of the time. Grants are given to the most sexy proposals with complete disregard to if the investigator is competent or not. After a project is finished there are no checks whether any results came out. Usually there are no or very weak results and a new sexy proposal is submitted which nevertheless is granted. Grants are given to projects that can never work. Even if a proposal is serious and can work it is only a three year proposal making Universities much more shortsighted than industries, in contrast to established wisdom.
One scientist here gave a talk in Beijing for a cost of 7000 dollars and he spent only one afternoon in China before going back. Anotherone spends half his grant money going to conferences. Equipment is usually bought without any tenders. Positions are typically offered only internally without any open announcements. Nepotism is rampant. Even if the people seeking a position are evaluated, which sometimes is required by law, the evaluators have no personal interest in doing a good job since they are only paid and will not suffer 20-40 years from a bad choice. People can only get fired if they break the law.

Sublimate it as rapidly as possible into a free-market, sink or swim, enterprise and we will see something good come of it.

If the state wants a result, buy it from companies using open tenders instead of using the University system. Do we want to find a molecule that clears atherosclerotic plaque by 50 % ? For a billion dollars we could probably get ten from big pharma. Do we want a Higgs boson ? Let companies submit proposals for a 1 TeV accelerator. Do we want a space station ? Buy it from Bigelow aerospace or Hilton space hotels, whichever is cheapest. Do we want to find the genes behind IQ and alcoholism ? Let DeCODE submit a quotation. They probably have the genes already.

It has been estimated that state enterprises have a productivity of 10 % of private enterprises. Checking the cost of sequencing the human genome by Celera and the state I find rough agreement. A private proposal to send a scientific satellite to an asteroid was similarly cheap. NASA still got the job ... .

Hopefully Anonymous said at September 14, 2007 7:13 AM:

We're empiricists, not foil-seekers, right? It seems some commenters relation to government funding of anything is a foil-seeking opportunity to attack socialism and praise free markets. I don't think that's giving primacy to empirically derived solutions to the challenges we face. I think the OP takes the right tack: let's figure out what's going on, and the best empirically derived solutions/optimizations that we can come up with.

Kurt9 said at September 14, 2007 7:23 AM:

So, we have discovered that bureaucracy does not work. Gee. Why is this no supprise to me?

David A. Young said at September 14, 2007 9:00 AM:

OK, so the obvious bookend study would be to compare increases/decreases in private Biotech spending versus increases/decreases in productivity. It's probably been done. Something to look into when I have the time.

rsilvetz said at September 14, 2007 9:08 AM:

Oh for Pete's Sake! "Hopefully Anonymous" misses the point: It IS socialism that is empirically a catastrophe. It IS socialism that is going on here-- in all of its incentive-destroying nonsensical meet-the-regs-screw-the-result -- that is the causal problem here. It IS socialism (and the regulatory environment) that make it impossible to get to life extension therapy, cure for cancer, cure for diabetes, cure for heart disease. It's amazing biologics advances at all and in spite of the barriers thrown in front of it.

Right now -- this very instant -- I know how to make most solid tumors dissolve and can give the cancer patient equivalent survival without chemo or radiation. I can't get the therapy out there because of patents, FDA, and last but not least -- the damn NIH which publicly praises immunotherapy but privately tells everyone within earshot that chemo is the only way to go -- imagine what that does to funding... We do not have a free-market in healthcare.

But perhaps the other posters might want to review the literature surrounding the constantly dropping number of actually useful therapies approved by the FDA or the ever-increasing price of the approval process. Or perhaps the symbiotic relationship that the FDA gives preference to NIH-based initiatives and that the NIH doesn't pursue matters unlikely to garner FDA approval. A double-negative feedback loop based on intentions not facts. The NIH's and FDA's follies are precisely the sort of actions that one expects a powerful government bureaucracy to take--self-serving, irresponsible, heedless of the injuries it causes so long as they are ignored by the news media, and vindictive against whistle-blowers.

Never mind that the existence of large public funding spaces displace and dry up the private funding for private enterprises and prevent dangerous newcomers from accumulating capital and reputation. NASA with the space shuttle essentially prevented the emergence of private space vehicles. SpaceShip One could have been built in 1980 for ten times the cost. The FDA prevents the existence of private third-party review which would be much faster and cutting edge. Why do I have to waste millions in dog trials when I have a stack of papers 6 feet high outlining the whole damn thing including papers documenting human experiments... oh right -- retrospective data is not accepted for approval or post-approval monitoring... How f'ing stupid is that people? The NIH preempts the existence of private labs with all the incentives to aggressively move to market with a working idea. The NIH defacto publishes papers -- therapy is few and far to come by. Note the BMP-7 fiasco where J&J has quietly walked away from the therapy leaving kidney disease sufferers stranded -- even though BMP-7 by every measure available should cause regression of both glomerular and tubulointerstitial disease. Unfortunately there are no small or medium-sized labs outside of NIH that could possible pick up BMP-7 and run with it.

I leave you with two paragraph quotes:

1)Worse, Miller notes, the FDA routinely denies approval to successful products already used in Europe, holding up use in the U.S. of items ranging from injectable antibiotics for resistant pathogens to a vaccine against meningitis. Economists calculate that the number of people who have died or suffered while waiting for useful drugs to be approved by the FDA may outnumber those saved by keeping bad drugs off the market. In a 2000 review of economists' reports on the FDA, Santa Clara University economist Daniel B. Klein reported that he was unable to find a single one who felt the agency approved drugs quickly enough. He found economists' opinions on what to do with the FDA "ranging in degrees from gradual decontrol to outright abolition of the agency."

2)Department of Eating Out -- a great new bureaucracy to improve the pizza experience:

What if we created a fictitious U.S. Department of Eating Out? What would a pizza order look like? Well, they’d ask you for each individual’s social security number, look through their records, and determine that they will send you two medium anchovy pizzas for everyone in your eleven-person party. You argue with them on the phone and you tell them that most of you don’t like anchovies, but it turns out that three of your friends didn’t register with the the department of Eating Out. That’s why you are only getting two medium pizzas. And that’s why they made with anchovies. But don’t worry… Your pizzas will arrive in 120-150 minutes and they are free. That is until you look at how much the government is paying for your two pizzas: each pizza costs 15 dollars, but the total isn’t 30 dollars. The total hidden cost is 570 dollars because the government pays minimum wage and it was just raised to 13.00. The person who made your pizza dropped a bucket of tomato sauce on the floor and his friend slipped and had to be sent to the emergency room.

But who cares right? If it wasn’t for the goverment, we wouldn’t have pizzas. Everyone would have to buy their own pizza, and who would deliver it? Does anybody outside the government even know how to make a pizza? No thanks. We’ll eat our cold and tiny anchovy pizzas and pay 570 dollars for them. We are scared of the consequences of abolishing the Department of Eating Out.

NOW IMAGINE RESEARCH AT THE NIH IN THAT CONTEXT AND YOU WONDER WHY THE PAPER RATE DIDN'T INCREASE.....


Mats-Erik Pistol said at September 14, 2007 12:16 PM:

rsilvetz,

Do you think the regulation problem will be solved by medical tourism? Medical tourism is growing very quickly but most treatments are standard and patients mostly go for the money and to save time. I do not know which, if any, country that allows non-standard medicines, i. e. not approved by any state run agency. Stem cell treatments are exploding in Thailand, China and so on, which I take as a very very hopeful sign. I bet that in a few years time there will be competition among the stem cell companies leading to far better results in, e. g. heart treatments. Theravitae claims that 70 % of heart patients improve. Amazingly they want to do clinical trials in the US. It is far better to let US patients take a trip to Bankok. So I am hopeful for stem cells, surgical techniques and the like, but I do not know what is the situation for novel drugs and why not immunotherapies. In the best of worlds Asia will become a medical non-regulated (or rather self-regulated) hub where any medication can be used. The market will quickly weed out charlatan companies.
Competition between legislations is happening in finance where London is overtaking New York, Singapore competing with Zurich, Cayman Islands vs Nauru so why not in the medical field.

Yours,

TTT said at September 14, 2007 4:14 PM:

Why do we even call it 'Socialism'. Even that is a supn euphemism relative to how sinister it actually is.

Call it what it is : Petty leftist inadequacy.

Randall Parker said at September 14, 2007 6:16 PM:

Fellow empiricists,

I want the rate of advance of biomedical research to accelerate because I have a strong vested interest in not dying and in becoming young again in body and mind.

So I have a question: How to accelerate the rate of biomedical advance?

We can whack socialism or promote free market capitalism. But can you make your recommendations more specific to the topic of biomedical advance?

I can think of some proposals that might help (and then again might not):

1) Allow terminally ill patients to take any drug regardless of whether the drug has been approved by a government. Or at least let them take any drug that has gotten thru stage 1 trials.

2) Make biomedical research grants for more years so that scientists think longer term?

3) Offer big cash prizes for achievement of assorted biomedical breakthroughs such as:

A) Faster DNA sequencing.

B) A way to take normal cells and make them pluripotent without using an egg.

C) Make a mouse live longer (the Methuselah Mouse Prize).

D) Come up with a way to grow various replacement organs that people need.

How about putting $50 million or $100 million or, heck, $500 million up as a prize for achieving each of several big biomedical research goals?

Mats-Erik Pistol said at September 15, 2007 3:41 AM:

Fat Man,
Most scientific papers are terribly bad beacuse our universities say quantity=quality and the fact that most scientists are not interested in finding out how things work. One scientist at my department wished very strongly to be coauthor on a paper published in a prestigious paper although he knew the paper was wrong. Even worse it was logically wrong, the data were OK. The data were mine but I refused to be on the paper.

Randall
1) Allow terminally ill patients to take any drug regardless of whether the drug has been approved by a government. Or at least let them take any drug that has gotten thru stage 1 trials.
I agree. Seriously ill but nondying patients should also be included. Alzheimer and Parkinsson comes to mind. It takes decades for them to kill and life is hell meanwhile. Let the patients choose if they can.
2) Make biomedical research grants for more years so that scientists think longer term?
Still a waste of money and talented graduate students. It is the selection that has to be improved and I see no way to do that in a university system. Instead buy the results. Why should universities find genes for e. g. longevity when deCODE most likely know which ones they are already? Additionally buy the research from Malaysia, India, Brasil etc, which have strong and cheap scientists. This will also develop these countries.
3) Offer big cash prizes for achievement of assorted biomedical breakthroughs
Probably works a bit. Remember though that big pharma gets very high rewards for breakthrough drugs already. The Methuselah mouse is a nice idea. Why big pharma does not pursue some ideas beat me. Why do only two companies work on compounds breaking down glycated proteins when it is clear that even the "non-sexy" cosmetic market for a successful molecule would be enormous. The cosmetic market is not highly regulated.

Do lobby as much as possible for medical tourism though. If that is allowed to flourish we will finally see Moores law, or exponential growth also in the biological field. I met an investor from London recently who was completely clear that any investments from him in stem cell companies would only be in those with off-shore treatments. He soon has 5 MPounds under his control. I have a few commercial ideas so maybe even I should talk to him. rsilvetz I am sure could take his immunotherapy to the off-shore clinic with his help. There are so much bio knowledge already that there is almost no need for more basic bio-research. Even applied bio-research these days is "basic research". Maybe government support for basic bio-research should decrease, releasing these highly qualified people from regulatory shackles and grant application writing to pursue bio-tech careers.
Also talk to rich people so they donate "correctly". I also recently met a dollar billionaire who is old, very sharp and willing to donate to e.g. the MPrize. We will soon see if he does so. He can change the world personally.

rsilvetz said at September 15, 2007 10:21 AM:

Folks, what do you think you are arguing for when you toss out your "empirical" approaches? Free-markets! Virtually all your choices involve an attempt to get around a third-party systems that have arbitrary fascistic or socialist control on the pipeline of therapy from ideation (grant system) to R&D (NIH) to commercial development (FDA-BigPharma symbiosis) and approval (FDA). If you all want to get to the promised land you have to free me and my equivalents (and betters!) from the stranglehold of this ridiculous system. In a country based on freedom it is absurd that I need to ask government permission to save someone's life! (What do you think "approval processes", medical licensing, etc, etc is? Consumer protection? Please...)

Thankfully the liberal medical regimes of India and China almost certainly guarantee the eventual demise of the FDA. We should do everything in our power to hasten this process, including referring as many individuals to medical vacations for treatment elsewhere as possible until they [FDA] get out of our way. I'm already half-considering moving out of the US to China to do something. When local .gov in China says "sure, you can run a 10,000 patient apheresis trial after you pay this bribe" thank you very much, that's faster and more efficient than spending $15 million to meet arbitrary FDA regs that came about at the beginning of the 20th century.

You can't "fix" the grant system -- it's a socialist ploy. Steal money from taxpayer Peter to pay researcher Paul. Only you don't get a vote in what the priorities are. Your elected socialist/fascist Demopublicans do. Don't you think you would be infinitely better off in a free-market where ever-increasing demand in life-extension will give you what you want? Do we not understand that dereged markets advance almost exponentially? Just compare computers and healthcare?

What's the point of having one reg allowing the dying to get any drug they want (and propping up the FDA in the process) as opposed to having a Wild West of experimentation where the FDA can't get in the way? Hell, FuturePundit itself went ballistic on the case of some young kid that was saved with his own stem cells when his heart got punctured, where the docs involved were FDA censured for their life-saving efforts. The FDA is the problem! It's not like the FDA does good by us, it is a net loss at all levels.

You want "empirical" suggestions:

1) Abolish the FDA in its totality.

1b) Carve up and sell off the NIH to private entreprenuers willing to take the risks of medical research.

2) By law, establish approval-by-private-third-party review with bonding of the therapy companies with considerable liability to be absorbed by insurance on the the third-party-review companies. No therapy company may own or participate in any third-party review effort.

3) By law, and possibly constitutional amendment, guarantee the right to medical contract -- Any therapy, experimental or approved, shall be made available to any patient upon request by the patient or doctor under the supervision of an Institutional Review Board to guarantee transparency and free will. The IRB is to guarantee the process and not veto it. The patient will assume responsibility for any experimental therapy.

There. Now you have a free market and the system will advance exponentially. At a doubling-rate in effectivness of therapy of 18 months, we will all have a shot at serious life extension and immortality.

Join me people. Don't underestimate our power as thought-leaders here. The current system cannot be "fixed" to get you there. It needs to be clean-sweeped dumped. Your life depends on it.

Pistol -- my best to Sperling....

rsilvetz said at September 15, 2007 11:26 AM:

Inevitably somebody will say I'm impractical.

I reply very simply. You can't get there from here. Period. You can't have an exponentially advancing market in biologics when every Tom, Dick and Harry in every FDA, NIH and pressure group thinks they can pass a reg, an approval process or a demand upon you, backed by the muzzle of the .gov gun.

Why does anyone think it is practical to negotiate for incremental change with a sociopathic bureaucracy whose only achieved result is control of the process so as to tighten the noose around your neck as patient and the pressure of the boot on the neck of healthcare value creators?

Doing the same thing with the same outcome and refusing to change is a form of madness. Remember that. (Never mind the supporting a protection racket... which should be reason alone for dumping the FDA.)

We are reaching the dead end for the road we are on. And we will all end up dead at 72+/- 8 years on this road.

I say again -- you're life depends on you waking upto the reality of this situation -- and taking the obvious clean-sweep solutions.

Chuck said at September 15, 2007 11:58 AM:

Publication depends on acceptance by a peer-reviewed journal. There has been an increase in the number of scientists seeking publication without a corresponding increase in the number of pages available. This makes it increasingly more difficult to publish your results. That makes a straightforward comparison of the number of publications (as a measure of productivity) misleading unless that number is corrected by the number of available journal pages (not done).

The criticism of methodology described above is also specious. There are no perfect studies because practical constraints marr all studies. You do what is possible, what resources permit, not what would be best from a design standpoint. Most researchers recognize the flaws in their studies and it is expected that the flaws will be discussed in the resulting article, including the impact on the results. Scientists who actually do research understand this and try to glean what they can from what is done. I am suspicious of someone who comes down too hard on working researchers because of "flaws" in their work. It sounds like someone who doesn't do much work himself. There is a point where striving for perfection, instead of minimizing the flaws that will impact findings, becomes constipating. Most fields disdain those who have become fixated upon methods for their own sake. The point is to increase knowledge, not produce perfect studies. Even the best done study is an approximation of truth because it inevitably relies on sampling and because measurement is not the same as the entity being measured.

Randall Parker said at September 15, 2007 2:19 PM:

Chuck,

Thanks for your comments.

As for a possible shortage in research publications: Not sure how to measure this. I do see a proliferation of new venues for publishing research. For example, the open access Public Library of Science journals site now has 8 different journals. My guess is that the more scientists working in a field the more that are available to referee papers and to start up new publications.

I agree about flawed research. The fact is that in many cases the less flawed methods are known but too expensive. It can be orders of magnitude more expensive to do prospective double blind studies. It makes more economic sense to conduct studies that are cheaper but which have methodological limitations. If those flawed studies provide evidence for a potential finding that can justify spending larger amounts of money to conduct more elaborately controlled studies.

Mats-Erik Pistol said at September 15, 2007 2:19 PM:

rsilvetz,
The billionaire I met has only one billion I believe and is not Sperling (who has more). Thanks anyhow for pointing him out, I did not know about him. Can't believe I missed him during my internet sessions. The one I met is younger and is new to the biology field. Please lobby Sperling to donate to the SENS project and the Methuselah mouse prize. Make him force the people to do the research in Asia and Armenia which is a favourite science country for me. This will leverage the donation several times.
Seems like a consensus will emerge that medical tourims is extremely important for our medical future.
Let us hope WTO does not screw this up. At least not quickly.
In the worst of scenarios, let us do treatments on a boat in international waters.
In case Elon Musk is successful with SpaceX we can do it in space in a Bigelow space hospital.

Cheers to that.

Randall Parker said at September 15, 2007 2:23 PM:

Mats-Erik Pistol,

While I think prizes and other cash-for-results approaches could be used more they only work for cases where we can define the results and where the results aren't huge leaps. A lot of science involves various scientists following hunches to try to look for phenomena that might or might not be there. Some useful results come about by accident. While trying to discover something about X a scientist may observe something about Y that is great. If he's been offered money to discover something about X he can find himself having done great work but without having earned a cash reward that can fund further research.

Mats-Erik Pistol said at September 15, 2007 5:49 PM:

Randall,

Every university proposal promises results. Often these results are phrased as "increased understanding of X" and mean nothing. European Community grants now typically includes "deliverables" which can be a certain device or something similar, to be produced at a certain time. Scientists do not follow hunches in what they promise in their grants applications. They might sometimes do so in practice, but this is undercover work, usually performed late at night when the prof is away. I know only six scientists that will work without a financial incentive and only one that did so when unemployed. The rest have had jobs continuously so I am guessing there. Everybody else have morphed into parasites after taking their Ph. D's. The romantic notion that scientists are non-selfish, work for the common good, advance knowledge due to their thirst for knowledge etc is just wrong.
Cash-for-results will work wonders. Of course it is impossible to get a quotation for a room temperature superconductor, equally impossible as to get one for a molecule that cures cancer, but it is possible to get quotations for genes coding for high IQ (or low IQ if wanted), it is possible to get quotations for molecules that shuts down almost any bio-pathway in the cell, it is possible to get a quotation for a moonbased optical telescope and it is possible to get a quotation for a 10 PFlop computer. We can define these results and machines but I think they are huge leaps or can lead to huge leaps. Note that if we buy a 10 TeV linear collider we do not know the results in advance, but there will be results, and we got the machine so much cheaper than by having a consortium of universities building it. The saved money can be used to build a primordial black hole detector.

Post a comment
Comments:
Name (not anon or anonymous):
Email Address:
URL:
Remember info?

                       
Go Read More Posts On FuturePundit
Site Traffic Info
The contents of this site are copyright ©