News:

NOTICE: Posting schedule is irregular. I hope to get back to a regular schedule as the day-job allows.


Monday, May 7, 2012

COMMENT: Who funds the watchers?

[Note for email recipients:  use this link: http://teddysratlab.blogspot.com for online version with embedded links.]

Who Funds the Watchers
 by Tedd Roberts

A recent article on PJ Media by Dr. Paul Hsieh describes recent trends with respect to "sloppy science and ethical misconduct in medical research" which could lead to even more dangerous trends in the practice of American healthcare. I applaud Dr. Hsieh in making the connection between bad science, bad health care decisions, and the risks of centralized healthcare. While I am not here to criticize or counter Dr. Hsieh's findings or conclusions, I'm afraid that the content of this article does not go quite far enough in describing the full extent of the problem. In addition, the title of his article and his concluding premise leaves much to be desired from the perspective of a researcher, engaged in basic medical science, and funded by government sources. In fact, I wrote in this blog on this similar subject just about a year ago.

The problem of sloppy science cited in the article is one that ties in quite closely with questions of the utility and practicality of peer review.  I wrote an article for an online discussion about a year ago that brought together several criticisms of the scientific peer review process. In 2005, an article in the Public Library of Science journal claimed that half of all published research findings were false due to statistical inadequacies.  There is a growing opinion that prepublication peer review serves as a gatekeeper only with respect to the publication capacity of journals, and bears no relationship to scientific quality; therefore why should we not publish everything and let the scientific community as a whole determine the validity of scientific findings, and not just the so-called "gatekeepers" of scientific publication? Thus, simultaneous with a crisis of accuracy in scientific communication, we also have a crisis in terms of content. While a world without "gatekeepers" would certainly lessen some of the issues raised by Dr. Hsieh with respect to "publish or perish" attitude within the research community, it introduces the parallel problem of how to judge the validity of research without editorial gatekeepers.  The problem is hardly unique to science publication, since this is exactly the situation being faced by fiction authors who are breaking free of traditional publishers, and making their way into independent electronic publication.

Unfortunately, the solution that is appropriate to publication of e-books is not appropriate to scientific publication. The validity and worth of science publication needs to be judged by scientists are capable of looking at methods, techniques, statistics, and judging results within the context of appropriate medical research. It would be all too easy for the public to latch onto a scientific finding that matches a current political or social narrative, for example, the fears, misinformation and outright fraud associated with childhood vaccinations. In addition, there still remains a necessity for educated scientific professionals to assess methodological, statistical, and ethical concerns that contribute to the acceptance of scientific findings. In contrast, the history of science is replete with instances in which the accepted scientific view is challenged, the challenge is rejected, but later reinstated with the result that the accepted "dogma" is overturned. Thus, as in many fields, true scientific validity can only be established over time.  [For more on the issues of Scientific Peer Review, I will run the article I wrote last year, in the Wednesday 5/9 blog at http://teddysratlab.blogspot.com]

However, this does not completely address the basic premise of Dr. Hsieh's article -- that government funding of research is the problem. To counter this premise, I would ask the good Dr. whom he thinks would be capable of funding scientific research to the level that it is today?  Funding sources for research in America break into a relatively few categories: (1) medical research (both basic and directed at specific diseases) funded by the U.S. National Institutes of Health (NIH); (2) basic science funding by the National Science Foundation (NSF); (3) applied science  (i.e. directed at a particular product or goal) funding by the U.S. Department of Defense (via the individual service research agencies, DARPA, and congressionally-directed medical research program - CDMRP); (4) private funding, whether through philanthropic foundations such as the W.M. Keck Foundation, advocacy/interest groups such as the Muscular Dystrophy Association and American Heart Association, etc., or corporate-sponsored research such as pharmaceutical or medical/commercial device development; and finally (5) university intramural research paid for out of the steadily dwindling pool of general university funds.  Please note, that of the agencies and types listed above, only NIH, NSF, university-funded, and a few private foundations sponsor "basic science" which is research on interesting problems for the general advancement of the field.  DoD, corporate and advocacy group funding is always directed to solving a particular problem or disease. 

In attempting to research actual funding policies and proportions, I started with a Wikipedia article, which appears to indicate that about 30% of US research funding is by the government, and just over 60% by the private sector. When I went to check the references for myself, I found a circular reference which points back to the information in the table, thus indicating that Wiki is referencing itself for its own justification of these numbers.  Therefore, I looked at the NIH website where I found that for 2011, the NIH budget was around $30 billion, of which more than $25 billion directly supported biomedical scientific research.  From there, the I looked at the second major government source of research funding, the NSF. In 2011, the NSF budget was almost $7 billion, of which $5.5 billion was directly applied to research.  The entire DOD medical research budget for 2011 was $1.2 billion, leading to a sum of just under $32 billion for U.S. federal government funding of research.  There are, of course, smaller research budgets buried in agency funding, such as crop research by the USDA and toxicology studies by the EPA.  However, even discounting those sources, the Wikipedia claim of government only providing around 30% of research funds implies over $70 billion in funding per year by nongovernmental sources!

Finding summary information for nongovernment scientific research sources is much more difficult. I know that in my institution, a medical school combined with teaching and research Hospital, NIH and NSF funding accounts for approximately 80% of our research budget. In the area of cancer research, in 2011, the American Cancer Society awarded just under $50 million in grants, compared to $30 million in NIH appropriation.  As an example of private foundation funding, the W.M. Keck Foundation provided $20-$40 million in grant funds over the past 5 years, however the total has been steadily declining since 2007.  These funding levels would require well over 1,500 associations and foundations to make up Wikipedia's projection of nongovernmental funding.  More importantly, replacing government with nongovernment dollars would require an additional 1,000 agencies to provide those funds.

Of course, the story is also told in the scope of the budgets that can be submitted by individual investigators for particular scientific project. In my field of neural prosthetics, the two largest funding sources are NIH and the Department of Defense.  The entire DOD medical research budget for 2011 was $1.2 billion, while the Neuroscience budgets of just two of the NIH institutes performing Neuroscience research (National Institute for Mental Health and National Institute of Neurological Disorders and Stroke) combined to a total of $3 billion.  However, a typical single-investigator NIH budget is $200-$250,000 and a typical NSF budget is $100-$150,000. Budgets for grant applications to private foundations are varied, and American Cancer Society Grant may have a $250-$500,000 budget for multi-investigator research program, while an American Heart Association or Muscular Dystrophy Association grant may be limited to $50-$100,000. DOD grants tend to be in the same budget range, with the notable exception of DARPA projects, which may have "price tags" in the range of $1 million or more. 

It may very well be the case that the US government funds only about half of the total scientific research that is occurring within the United States at any given time. However, private foundation, industry, and interest group funding of research tends to be in much smaller increments requiring many more grants, many more bosses, and a much higher obligation to "publish or perish" to maintain the same level of funding!  As an additional caution, about 75% of a grant budget of $100k or more goes to personnel salaries - not the "rich" professors, they only get about 10-20% of their total salary from a grant - but primarily the technicians, students and post-docs.  For many employees of a lab (including faculty!) research grants are their only salary sources.  University funds for individual investigator research are very scarce, and likely only total $25-$100k total - not per year - total per career!

This is not to say that Dr. Hsieh is wrong in sounding the alarm that government sponsorship of research could very well lead to government endorsement only of "accepted" research findings and translate such acceptance to centrally manage standards of healthcare. The problem however is that in the absence of government funding, how does the United States maintain a high standard of scientific research? The problem with scientific research is that it is expensive. Laboratory equipment, chemicals, research subjects, and personnel to perform experiments all require funding. The way to maintain scientific integrity and validity of research is to experiment, publish the results, invite others to repeat the experiment and subsequently publish their own results. This requires both time and money. The original research must be funded, and then any sources and who wish to repeat the study also need to be funded as well. 

The total NIH, NSF and DOD funding for research listed above for 2011 was under $40 billion.  The U.S. federal budget for that same time period was $3.8 trillion, so research funding was approximately 1% of total government spending.  By the way, healthcare funding in the form of Medicare and Medicaid was nearly $800 billion or 20 times the scientific research budget for 2011. Of course, this does not include the Affordable Care Act and other sources of government spending on health care. So would seem much more likely that the more heavily funded field -- healthcare -- would you be much more likely to drive policies in research, and research would drive policies for healthcare.

Again, I don't disagree with the caution that Dr. Hsieh is sounding, it's just that there is so much more to the issue, leaving the very important question: Who will fund science?

Science funding is a small part, an extremely small part, of the US federal budget. Whether it funds 30% or 80% of total US science research is immaterial. Removing the government as a source of research funding means removing research. The only problem with that, is that some repetition is needed to confirm and validate existing studies. The consequence of such an action is to discourage repetition of studies with emphasis on "new" science or science directed only at specific diseases or desired products.  The U.S. Government is the largest supporter of "basic science" research - i.e. research simply for the sake of advancing knowledge within the field.  Replication of important research simply for the sake of confirming results falls within the scope of Basic Science where confirmation and validation is an important part of maintaining professional scientific research and research ethics.

If we eliminate government funding of science, we will eliminate a large proportion of the science studies and research education in this country.  More importantly, we will eliminate an important quality control on the science that we do perform.

So if we eliminate government-funded science, Who will fund the Watchers?


---


Writing and blogging as "Speaker to Lab Animals", Dr. Tedd Roberts is committed to making brain science understandable to SF/F writers and readers via his blog (http://teddysratlab.blogspot.com) and "The Lab Rat's Guide to the Brain."  Dr. Roberts is a Neuroscientist who conducts research and publishes professionally in fields of Neurophysiology, Pharmacology and Neural Prosthetics.

9 comments:

  1. If research would not be conducted without government funding perhaps it serves government purposes and individuals or business see no benefit for that line of research.
    While the space program for example showed there is always some serendipity in unexpected applications I'd submit that this is just a variation on saying people don't really have the sense to spend their money wisely to their own benefit and government has to take it away to see it is spent well.
    With peer review the process is hidden and we have to basically trust the reviewers not to cut off competition and apply personal bias.
    Why could not a publication be posted and subject to a public review just like the e-books you compared them to?
    The reviewers could post their credentials and experience and rate the article for several qualities from one to ten.
    How can you argue against a wider peer review?
    We are in no danger of running out of ones and zeros to publish as many papers as people wish. I suspect there are plenty of qualified people who would be happy to spend time reviewing papers. The ones getting high scores will get a more thorough review and people will cut off the low score papers before too many man-hours are wasted on them.
    I personally would be happy to directly contribute funds to lines of research I wanted pursued. I'd send money to anyone that showed promise developing regeneration of hearing and polywell fusion reactors.

    ReplyDelete
    Replies
    1. Insightful comments Mac - I agree with you on many points, but I'm going to cherry-pick a few.

      Yes, there are ways to hide information in peer review. I will address this in part in my next blog, but call your attention to PLoS One (published by the Public Library of Science publishers) which is one such open review journal. It invites scientific papers from all disciplines, conducts a scientific review for accuracy, ethics and feasibility, but not for merit. The merit review is performed by the readers through open access and open commentary.

      However, I do want to single out your final point: "I personally would be happy to directly contribute funds to lines of research I wanted pursued." In fact, that is the problem with private funding. It is what others want to see pursued, and is not basic research. Stem cell research has resulted in the "skin sprayer" for regeneration without grafts; drug abuse research is leading to a better understanding of memory; cancer research is telling us important things about aging; while aging research is telling us important things about adolescent development. Each of these fields may not be what we "want" to fund, and yet that same person may gain an unforeseen benefit. In a way it's like saying *I* have no need to drive to Detroit, why should my tax money pay for an Interstate Highway that goes from Detroit to Myrtle Beach?

      There is a certain "commonwealth" both in infrastructure in research that needs to be funded. It may very well be the case that government should not take our money to fund if - but if they don't, who will step up to meet the need?

      Delete
    2. Many thanks for the PJ Tattler link (http://pjmedia.com/tatler/2012/05/07/in-defense-of-government-funded-research/)!

      For what it's worth, *I* don't 100% agree with me either! Thanks Patrick, your comment "if all those federal monies disappeared tomorrow what would replace them?" was my whole point.

      Thanks again, and welcome PJ Media readers!

      Delete
  2. When political entities (government) fund science, you get politicized science.

    ReplyDelete
    Replies
    1. Not always. Polticization comes when an applicant is told "No" because of their political stance. That typically does not happen in NIH/NSF funded research.

      However, it also happens when politicians pick up on a result and try to make policy based on one cherry-picked finding. Sadly, that does happen, but see the next comment as well.

      Delete
    2. Sadly, always.

      Politics allocates resources -- one of those resources being attention. Let's take something relatively benign -- AIDS v. obesity. Many more dollars have been channeled into retrovirus research v. SIRT 1, despite a relatively small number of AIDS patients and relatively large obese patients (I kid!)

      It's not that retrovirus studies are bad science. There's been great strides made in that area and researchers should be proud of the good work therein.....but, like Bastiat's broken windows, what is "unseen"?

      Political pressures sent money to fund AIDS-related research and not to obesity-related research, but there is nothing to say that the benefits to mankind as a whole are any better than random fumbling might have produced -- perhaps the real breakthroughs that would change the world were in something totally out there, like gene-swapping with intestinal fauna. To weigh improvement in the areas where all the money is spent against stagnation in other areas is falsely maintaining that "heads I win, tails you lose." Politicization isn't just parties -- it's looking west when gains are east.

      Recommended: http://bastiat.org/en/twisatwins.html -- it's from 1850, but describes a blind spot that has existed from the cro-magnons and will exist so long as humans do.

      Delete
    3. This is true, political pressure influences the call for special research proposals - I was researching the Congressional Record in 2003 regarding a move by certain Senators to eliminate several biology-based research programs from the defense appropriation. In it's place they wanted to strip billions of dollars from Defense and fund Aids research. I think we can guess which Senators might have been behind the move to divert Defense funds.

      So yes, there are political moves to influence what is researched.

      Still, NIH will always accept and review grant applications for *any* research project, unlike most private funding sources.

      Also, in the comments to my follow-up blog on peer review, we've been discussing public accessibility of published articles. NIH-funded, peer-reviewed articles since 2008 *must* be deposited with PubMedCentral (http://www.ncbi.nlm.nih.gov/pmc/) where they are available for free one year (or sooner) after publication).

      Private foundation or industry supported research has no such requirement, and is most often protected by a "paywall."

      Delete
  3. So it seems to me there are two problems here. The first is that there is a lot of not exactly useful research funded by the government and that research produces results that are frequently worthless because they cherry pick data, abuse statistics etc. etc. That's not limited to government funded research of course but because of the way government funded research tends to be run it is almost certainly easier to game the system because the people paying for the research - the taxpayers - don't get to see any of the product in most cases.

    The second is whether government should fund research or not. And despite my libertarian/minarchist leanings I tend to think that on the whole government funded research is probably a better option than private funding. However I do think that taxpayers should have some input into what research they want funded and how much they are willing to pay for it. Right now they don't really have that choice as the pot of money is provided by politicians (who are influenced by lobbyists) and the pot is divvied up by science administrators and bureaucrats who barely even need to report to the pols.

    ReplyDelete
    Replies
    1. Francis, thanks for the comments. Your dichotomy is the same as mine, but I just don't have a good answer for *who* picks the projects. As for allowing the public to choose, we first need a *much* better basis of science education, lest "basic science" research gets ignored when no one understands its value.

      Delete

Please add comment - no links, spammers will be banned.