Review of the Use of Statistics in Infection and Immunity

<h2>ERRORS IN ANALYSIS</h2> Errors in analysis may lead to misinterpretation of the data and faulty conclusions. The most common such error was failure to adjust or account for multiple comparisons. When more than two experimental groups are compared with each other or with a common control, it is usually necessary to adjust the significance level or P value to account for multiple comparisons. Two common scenarios are comparing multiple treatments to a control and comparing two treatments at several different time points. While 24 studies used an appropriate adjustment for multiple comparisons, 26 studies, nearly 1 in 5, reported multiple comparisons without adjusting P values. Typically, the authors calculated a separate t (or similar) test for each comparison, using a significance level of 0.05 for each comparison. The overall error rate in this case equals 1 − (1 − 0.05) k , where k is the number of comparisons ( 9 ). Therefore, if three such independent comparisons were made for a single experiment (e.g., three treatments to a common control or treatment versus control for three time points), the probability of finding a falsely significant result for any of the comparisons performed increases from 0.05 to 0.14. If more treatment groups or time points are involved, the probability of false significance increases even more. Decisions regarding whether and how to adjust for multiple comparisons depend on several factors, including whether the comparisons were planned before the study began, how many comparisons are being made, and the study design. A...