Author: Tom Siegfried / Source: Science News
Everybody agrees that medical treatments should be based on sound evidence. Hardly anybody agrees on what sort of evidence counts as sound.
Sure, some people say the “gold standard” of medical evidence is the randomized controlled clinical trial. But such trials have their flaws, and translating their findings into sound real-world advice isn’t so straightforward. Besides, the best evidence rarely resides within any single study. Sound decisions come from considering the evidentiary database as a whole.
That’s why meta-analyses are also a popular candidate for best evidence. And in principle, meta-analyses make sense. By aggregating many studies and subjecting them to sophisticated statistical analysis, a meta-analysis can identify beneficial effects (or potential dangers) that escape detection in small studies. But those statistical techniques are justified only if all the studies done on the subject can be obtained and if they all use essential similar methods on sufficiently similar populations. Those criteria are seldom met. So it is usually not wise to accept a meta-analysis as the final word.
Still, meta-analysis is often a part of what some people consider to be the best way of evaluating medical evidence: the systematic review.
A systematic review entails using “a predetermined structured method to search, screen, select, appraise and summarize study findings to answer a narrowly focused research question,” physician and health care researcher Trisha Greenhalgh of the University of Oxford and colleagues write in a new paper. “Using an exhaustive search methodology, the reviewer extracts all possibly relevant primary studies, and then limits the dataset using explicit inclusion and exclusion criteria.
”Systematic reviews are highly focused; while hundreds or thousands of studies are initially identified, most are culled out so only a few are reviewed thoroughly with respect to the evidence they provide on a specific medical issue. The resulting published paper reaches a supposedly objective conclusion often from a quantitative analysis of the data.
Sounds good, right? And in fact, systematic reviews have gained a reputation as a superior form of medical evidence. In many quarters of medical practice and publishing, systematic reviews are considered the soundest evidence you can get.
But “systematic” is not synonymous with “high quality,” as Greenhalgh, Sally Thorne (University of British Columbia, Vancouver) and Kirsti Malterud (Uni Research Health, Bergen, Norway) point out in their paper, accepted for publication in the European Journal of Clinical Investigation. Sometimes systematic reviews are valuable, they acknowledge. “But sometimes, the term ‘systematic review’ allows a data aggregation to claim a more privileged position within the knowledge hierarchy than it actually deserves.”
Greenhalgh and colleagues question, for instance, why systematic reviews…
The post Informed wisdom trumps rigid rules when it comes to medical evidence appeared first on FeedBox.