Wednesday, February 3, 2010

Bias in research (and now a word from our sponsors)


Flickr Photo by sflovestory
When I think of scientific research, a picture comes to mind of a person (women or man) standing in a laboratory performing some sort of experiment and carefully recording empirical results documenting the outcome of the experiment. They then sit down with the data and examine it with an impartial eye to see if it supports their theory.

I know this is somewhat of a hokey image but to me it speaks of what a researcher should be - impartial, unbiased and disinterested. Science is meant to be a search for the truth (or as close to it as we can get) and we should not be so attached to an idea that we are unwilling to go in a new direction when the data takes up there.

And yet, I am seeing more and more that this process is being subverted by the groups that are sponsoring the scientists. I am not saying that this being done intentionally or maliciously, but rather it seems to be a byproduct of the system of the funding. Think about it, if a corporation gives you money to fund research and you publish research that is critical of their products or industry, how likely are they to give you additional funding in the future? And what is a corporation going to do with results that are not favorable?

Consider the two following reviews.

The first one1, published this month the Journal of Alzheimer's Disease, reviewed 43 published studies and looked at the relation between cigarette smoking and Alzheimer's. The review of each study not only considered the quality of the data and outcome but also the affiliations of the authors of those studies.

Of the 43 studies, 11 of them had affiliations with the tobacco industry. The studies that were affiliated with the tobacco industry showed that cigarette smoking did not increase the chance of developing Alzheimer's and might even help prevent it. On the other hand, the studies that were not affiliated with the tobacco industry showed that it was likely that smoking greatly increased the chance of developing Alzheimer's.

It is not unusual for studies to show conflicting results, that happens all the time. What is unusual (or perhaps not) is for the results to align so closely with a funding source. So when it comes to the risks of smoking, are you likely to believe a researcher receiving funding from the tobacco industry, or one that is not?

The second study2 (thanks Maria), published last year in BMJ, looked at 259 studies of the effectiveness of the flu shot. The researchers were looking for relations between study quality, take home message, and funding source.  Ignoring the conclusions about the flu shot's effectiveness (better studies said less effective), the researchers found a relationship between industry funding and higher quality journals.

If a study was funded by industry it had a better chance of being published in prestigious journals where it would, presumably, be cited more and have a larger impact on future work. Or in other words, work sponsored by industry is more visible.  I wonder what would have happened if any of the studies showed results that were not as favorable. I would be willing to bet that they would be published in a less favorable journal, if at all.

There have been countless other examples of misdeeds recently, from ghost writing to journal companies publishing entire journals that are little more than paid advertisements. When you put all of the facts together, I think it becomes clear that there is a problem.

The problem is not that corporations are attempting to achieve their goals - that is what they are supposed to do.  If they weren't trying to sell something and make money, they wouldn't be in business.  No, the problem is the mismatch between what science is supposed to achieve and what a company is trying to do.  Corporate sponsorship is tainting scientific research and biasing the results toward corporate goals.

So remember, the next time you hear about some exciting bit of science, consider whether it is a legitimate scientific result or a message from the sponsors.

1: Cataldo JK, Prochaska JJ, Glantz SA. Cigarette Smoking is a Risk Factor for Alzheimer's Disease: An Analysis Controlling for Tobacco Industry Affiliation. J Alzheimers Dis. 2010;19(2):465-80. PubMed PMID: 20110594. Journal link

2: Jefferson T, Di Pietrantonj C, Debalini MG, Rivetti A, Demicheli V. Relation of study quality, concordance, take home message, funding, and impact in studies of influenza vaccines: systematic review. BMJ. 2009 Feb 12;338:b354. doi:10.1136/bmj.b354. Review. PubMed PMID: 19213766; PubMed Central PMCID: PMC2643439. (Open Access)

No comments:

Post a Comment