>>Return to Tell Me About Statistics!
Even if the difference is not meaningful, once the sample size is increased, the null hypothesis will be rejected at some point and there will be a “statistically significant difference.” We explain the areas of focus to avoid the pitfalls of such tests.
How to find the test statistic t-value
The t-value is the difference between the means, divided by the standard error (SE).
Standard error (SE) is obtained using the following formula for the paired t-test:
Standard error (SE)＝standard deviation÷√n
The t-value is compared with the critical value, which is the standard value determined by statistics; if the t-value is greater than the critical value, the null hypothesis is rejected, and the alternative hypothesis is adopted. In other words, the larger the t-value, the easier it is to reject the null hypothesis, and the easier it is to become significant.
Let us now examine the above formula and consider cases with large t-values.
Suppose that you have conducted several sample surveys. Assume that the difference and standard deviation of the means, compared in any sample survey, are constant and the sample sizes are different.
The standard error (SE) value of a survey with a large sample size does not change because of the numerator (standard deviation), but the denominator (√n) increases; thus, the standard error (SE) value decreases. The t-value is large because the standard error (SE) is small, and the difference between the means is constant.
As the t-value increases, the p-value (the probability that the t-value is observed by chance) decreases, making it easier to reject the null hypothesis and make it significant.
[Figure 1]
Does that mean that the larger the sample size, the better?
Certainly, if you think that “anyway, it is good to have a significant difference,” it means that it is better to increase the sample size. However, even a small difference can become statistically significant if the sample size increases.
The only factor that can be judged by statistics is whether or not it can be said that “the difference value observed in the sample survey has a significant difference even in the population.” Whether the difference in values is clinically meaningful is another matter.
As an example, assume that after 24 weeks of administration of a type 2 diabetes drug, fasting blood glucose change decreased by -5.0 mg/ dL from the baseline.
A large t-value, resulting in p < 0.001, is likely to cause a pitfall in a very good result. However, if we take a close look at the data that “change in fasting blood sugar decreased by -5.0 mg/dL from the baseline,” we can conclude that it is not very meaningful.
In other words, both, the results of the statistical analysis, and the value of the clinical trial results are important.
Focusing too much on the results of statistical analyses tends to neglect the value of clinical trial results. Determining whether a value is clinically meaningful is more important than statistical analysis.
>>Return to Tell Me About Statistics!
Comments are closed