Meta-analysis: Difference between revisions

Jump to navigation Jump to search
VisualWikitext
Line 45: Line 45:


A weakness of the method is that sources of bias are not controlled by the method. A good meta-analysis of badly designed studies will still result in bad statistics. Robert Slavin has argued that only methodologically sound studies should be included in a meta-analysis, a practice he calls 'best evidence meta-analysis'. Other meta-analysts would include weaker studies, and add a study-level predictor variable that reflects the methodological quality of the studies to examine the effect of study quality on the effect size. Another weakness of the method is the heavy reliance on published studies, which may increase the effect as it is very hard to publish studies that show no significant results. This [[publication bias]] or "file-drawer effect" (where non-significant studies end up in the desk drawer instead of in the public domain) should be seriously considered when interpreting the outcomes of a meta-analysis. Because of the risk of publication bias, many meta-analyses now include a "failsafe N" statistic that calculates the number of studies with null results that would need to be added to the meta-analysis in order for an effect to no longer be reliable.
A weakness of the method is that sources of bias are not controlled by the method. A good meta-analysis of badly designed studies will still result in bad statistics. Robert Slavin has argued that only methodologically sound studies should be included in a meta-analysis, a practice he calls 'best evidence meta-analysis'. Other meta-analysts would include weaker studies, and add a study-level predictor variable that reflects the methodological quality of the studies to examine the effect of study quality on the effect size. Another weakness of the method is the heavy reliance on published studies, which may increase the effect as it is very hard to publish studies that show no significant results. This [[publication bias]] or "file-drawer effect" (where non-significant studies end up in the desk drawer instead of in the public domain) should be seriously considered when interpreting the outcomes of a meta-analysis. Because of the risk of publication bias, many meta-analyses now include a "failsafe N" statistic that calculates the number of studies with null results that would need to be added to the meta-analysis in order for an effect to no longer be reliable.
===Small study effect and publication bias===
{{main|Publication bias}}
The small study effect is the observation that small studies tend to report more positive results.<ref name="pmid23616031">{{cite journal| author=Dechartres A, Trinquart L, Boutron I, Ravaud P| title=Influence of trial sample size on treatment effect estimates: meta-epidemiological study. | journal=BMJ | year= 2013 | volume= 346 | issue=  | pages= f2304 | pmid=23616031 | doi=10.1136/bmj.f2304 | pmc=PMC3634626 | url=http://www.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&tool=sumsearch.org/cite&retmode=ref&cmd=prlinks&id=23616031  }} </ref><ref name="pmid20639294">{{cite journal| author=Nüesch E, Trelle S, Reichenbach S, Rutjes AW, Tschannen B, Altman DG et al.| title=Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study. | journal=BMJ | year= 2010 | volume= 341 | issue=  | pages= c3515 | pmid=20639294 | url=http://www.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&tool=sumsearch.org/cite&retmode=ref&cmd=prlinks&id=20639294 | pmc=PMC2905513 | doi=10.1136/bmj.c3515 }} </ref><ref name="pmid11451790">{{cite journal| author=Sterne JA, Egger M, Smith GD| title=Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. | journal=BMJ | year= 2001 | volume= 323 | issue= 7304 | pages= 101-5 | pmid=11451790
| url=http://www.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&tool=clinical.uthscsa.edu/cite&retmode=ref&cmd=prlinks&id=11451790 | pmc=PMC1120714 }} </ref> This is especially a threat when the original studies in a meta-analysis are less than 50 patients in size.<ref>F. Richy, O. Ethgen, O. Bruyere, F. Deceulaer & J. Reginster : [http://www.ispub.com/ostia/index.php?xmlFilePath=journals/ije/vol1n2/sse.xml From Sample Size to Effect-Size: Small Study Effect Investigation (SSEi)] .  The Internet Journal of Epidemiology.  2004 Volume 1 Number 2</ref>


==Types of meta-analyses==
==Types of meta-analyses==

Revision as of 20:50, 6 July 2013

Editor-In-Chief: C. Michael Gibson, M.S., M.D. [1]


Overview

In statistics, a meta-analysis combines the results of several studies that address a set of related research hypotheses. The first meta-analysis was performed by Karl Pearson in 1904, in an attempt to overcome the problem of reduced statistical power in studies with small sample sizes; analyzing the results from a group of studies can allow more accurate data analysis.

Although meta-analysis is widely used in epidemiology and evidence-based medicine today, a meta-analysis of a medical treatment was not published until 1955. In the 1970s, more sophisticated analytical techniques were introduced in educational research, starting with the work of Gene V. Glass, Frank L. Schmidt, and John E. Hunter.

The online Oxford English Dictionary lists the first usage of the term in the statistical sense as 1976 by Glass. The statistical theory surrounding meta-analysis was greatly advanced by the work of Nambury S. Raju, Larry V. Hedges, Ingram Olkin, John E. Hunter, and Frank L. Schmidt.

Uses in modern science

Because the results from different studies investigating different independent variables are measured on different scales, the dependent variable in a meta-analysis is some standardized measure of effect size. To describe the results of comparative experiments the usual effect size indicator is the standardized mean difference (d) which is the standard score equivalent to the difference between means, or an odds ratio if the outcome of the experiments is a dichotomous variable (success versus failure). A meta-analysis can be performed on studies that describe their findings in correlation coefficients, as for example, studies of the correlation between familial relationships and intelligence. In these cases, the correlation itself is the indicator of the effect size.

The method is not restricted to situations in which one or more variables is defined as "dependent." For example, a meta-analysis could be performed on a collection of studies each of which attempts to estimate the incidence of left-handedness in various groups of people.

Researchers should be aware that variations in sampling schemes can introduce heterogeneity to the result, which is the presence of more than one intercept in the solution. For instance, if some studies used 30mg of a drug, and others used 50mg, then we would plausibly expect two clusters to be present in the data, each varying around the mean of one dosage or the other. This can be modelled using a "random effects model."

Results from studies are combined using different approaches. One approach frequently used in meta-analysis in health care research is termed 'inverse variance method'. The average effect size across all studies is computed as a weighted mean, whereby the weights are equal to the inverse variance of each studies' effect estimator. Larger studies and studies with less random variation are given greater weight than smaller studies. Other common approaches include the Mantel Haenszel method and the Peto method.

A free Excel-based calculator to perform Mantel Haenszel analysis is available at: http://www.pitt.edu/~super1/lecture/lec1171/014.htm.

They also have a free Excel-based Peto method calculator at: http://www.pitt.edu/~super1/lecture/lec1171/015.htm

Cochraine and other sources provide a useful discussion of the differences between these two approaches.

Question: Why not just add up all the results across studies ?

Answer: There is concern about Simpson's paradox.

Note, however that Mantel Haenszel analysis and Peto analysis introduce their own biases and distortions of the data results.

A recent approach to studying the influence that weighting schemes can have on results has been proposed through the construct of gravity, which is a special case of combinatorial meta analysis.

Modern meta-analysis does more than just combine the effect sizes of a set of studies. It can test if the studies' outcomes show more variation than the variation that is expected because of sampling different research participants. If that is the case, study characteristics such as measurement instrument used, population sampled, or aspects of the studies' design are coded. These characteristics are then used as predictor variables to analyze the excess variation in the effect sizes. Some methodological weaknesses in studies can be corrected statistically. For example, it is possible to correct effect sizes or correlations for the downward bias due to measurement error or restriction on score ranges.

Meta analysis leads to a shift of emphasis from single studies to multiple studies. It emphasises the practical importance of the effect size instead of the statistical significance of individual studies. This shift in thinking has been termed Metaanalytic thinking.

The results of a meta-analysis are often shown in a forest plot.

Trial quality

A weakness of the method is that sources of bias are not controlled by the method. A good meta-analysis of badly designed studies will still result in bad statistics. Robert Slavin has argued that only methodologically sound studies should be included in a meta-analysis, a practice he calls 'best evidence meta-analysis'. Other meta-analysts would include weaker studies, and add a study-level predictor variable that reflects the methodological quality of the studies to examine the effect of study quality on the effect size. Another weakness of the method is the heavy reliance on published studies, which may increase the effect as it is very hard to publish studies that show no significant results. This publication bias or "file-drawer effect" (where non-significant studies end up in the desk drawer instead of in the public domain) should be seriously considered when interpreting the outcomes of a meta-analysis. Because of the risk of publication bias, many meta-analyses now include a "failsafe N" statistic that calculates the number of studies with null results that would need to be added to the meta-analysis in order for an effect to no longer be reliable.

Small study effect and publication bias

Main article: Publication bias

The small study effect is the observation that small studies tend to report more positive results.[1][2][3] This is especially a threat when the original studies in a meta-analysis are less than 50 patients in size.[4]

Types of meta-analyses

Network meta-analysis

A network meta-analysis[5] and Bayesian hierarchical models[6] pool studies in order to compare to treatments that have not been directly compared.[7][8] Network meta-analyses are commonly not well performed.[9] Network meta-analyses of both randomized controlled trials[10][11][12] and diagnostic test assessments[13] can have misleading results. Network meta-analyses have been conducted by the Cochrane Collaboration.[14][15]

Network meta-analyses can be conducted with Bugs and OpenBugs software.

See also

External links

da:Metaanalyse de:Metaanalyse nl:Meta-analyse no:Metaanalyse su:Meta-analysis sv:Meta-analys


Template:WikiDoc Sources

  1. ↑ Dechartres A, Trinquart L, Boutron I, Ravaud P (2013). "Influence of trial sample size on treatment effect estimates: meta-epidemiological study". BMJ. 346: f2304. doi:10.1136/bmj.f2304. PMC 3634626. PMID 23616031.
  2. ↑ Nüesch E, Trelle S, Reichenbach S, Rutjes AW, Tschannen B, Altman DG; et al. (2010). "Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study". BMJ. 341: c3515. doi:10.1136/bmj.c3515. PMC 2905513. PMID 20639294.
  3. ↑ Sterne JA, Egger M, Smith GD (2001). "Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis". BMJ. 323 (7304): 101–5. PMC 1120714. PMID 11451790.
  4. ↑ F. Richy, O. Ethgen, O. Bruyere, F. Deceulaer & J. Reginster : From Sample Size to Effect-Size: Small Study Effect Investigation (SSEi) . The Internet Journal of Epidemiology. 2004 Volume 1 Number 2
  5. ↑ Lumley T (2002). "Network meta-analysis for indirect treatment comparisons". Stat Med. 21 (16): 2313–24. doi:10.1002/sim.1201. PMID 12210616. Unknown parameter |month= ignored (help)
  6. ↑ Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med. 2004 Oct 30;23(20):3105-24. PMID 15449338
  7. ↑ Salanti G, Kavvoura FK, Ioannidis JP (2008). "Exploring the geometry of treatment networks". Ann. Intern. Med. 148 (7): 544–53. PMID 18378949. Unknown parameter |month= ignored (help)
  8. ↑ Ioannidis JP (2009). "Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses". CMAJ. 181 (8): 488–93. doi:10.1503/cmaj.081086. PMC 2761440. PMID 19654195.
  9. ↑ Song F, Loke YK, Walsh T, Glenny AM, Eastwood AJ, Altman DG (2009). "Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews". BMJ. 338: b1147. PMC 2665205. PMID 19346285.
  10. ↑ Kent DM, Thaler DE (2008). "Stroke prevention--insights from incoherence". N. Engl. J. Med. 359 (12): 1287–9. doi:10.1056/NEJMe0806806. PMID 18753641. Unknown parameter |month= ignored (help)
  11. ↑ Thijs V, Lemmens R, Fieuws S. Network meta-analysis: simultaneous meta-analysis of common antiplatelet regimens after transient ischaemic attack or stroke. ur Heart J. 2008 May;29(9):1086-92. Epub 2008 Mar 17. PMID 18349026
  12. ↑ Chou, Roger (2009-02-01). "Gabapentin Versus Tricyclic Antidepressants for Diabetic Neuropathy and Post-Herpetic Neuralgia: Discrepancies Between Direct and Indirect Meta-Analyses of Randomized Controlled Trials". Journal of General Internal Medicine. 24 (2): 178–188. doi:10.1007/s11606-008-0877-5. PMID 19089502. Retrieved 2009-01-26. Unknown parameter |coauthors= ignored (help)
  13. ↑ Takwoingi, Yemisi (2013-04-02). "Empirical Evidence of the Importance of Comparative Studies of Diagnostic Test Accuracy". Annals of Internal Medicine. 158 (7): 544–554. doi:10.7326/0003-4819-158-7-201304020-00006. ISSN 0003-4819. Retrieved 2013-04-01. Unknown parameter |coauthors= ignored (help)
  14. ↑ Singh JA, Christensen R, Wells GA, Suarez-Almazor ME, Buchbinder R, Lopez-Olivo MA; et al. (2009). "A network meta-analysis of randomized controlled trials of biologics for rheumatoid arthritis: a Cochrane overview". CMAJ. 181 (11): 787–96. doi:10.1503/cmaj.091391. PMC 2780484. PMID 19884297.
  15. ↑ Singh JA, Christensen R, Wells GA, Suarez-Almazor ME, Buchbinder R, Lopez-Olivo MA; et al. (2009). "Biologics for rheumatoid arthritis: an overview of Cochrane reviews". Cochrane Database Syst Rev (4): CD007848. doi:10.1002/14651858.CD007848.pub2. PMID 19821440.
Retrieved from "https://www.wikidoc.org/index.php?title=Meta-analysis&oldid=884608"