>>Return to Tell Me About Statistics!
A cutoff value is a standard value used to separate quantitative data. In the medical field, this value distinguishes between positive and negative results in a test and is also called the “pathological identification value.”
In vol.67, we introduce the cutoff value calculated using Cramer’s coefficient of association for a 2 × 2 contingency table. Here, we explain the cutoff value calculated using the ROC analysis.
ROC analysis
ROC stands for “Receiver operating characteristic.” The ROC curve was developed to evaluate radar performance during World War II and is now used in various fields, including industry and medicine.
ROC analysis uses the sensitivity and value obtained by subtracting the specificity from 100% (1 – specificity) when the cutoff value is continuously changed.
Create a graph by plotting sensitivity and 1-specificity on a graph, with sensitivity on the vertical axis (y-axis) and 1-specificity on the horizontal axis (x-axis). The curve drawn in this way is the “ROC curve.”
[Table 1] ROC curves for sensitivity and 1-specificity
There are two ways to find the optimal cutoff value using the ROC curve.
【Method 1】
The test result with the smallest distance from the origin coordinates (0% and 100%) in the upper-left corner of the graph was the optimal cutoff value.
In the example, the distance to the point at 7.1% on the horizontal axis and 83.3% on the vertical axis was smallest at 18.1%. When the BMI was 26, the optimal cutoff value was 26.
[Table 2]
【Method 2】
Draw a line connecting the point coordinates (0%, 0%) and (100%, 100%). The distance from the point to the line was calculated, and the test result with the maximum distance was considered the optimal cutoff value.
In this example, the distance from the point at 7.1% on the horizontal axis and 83.3% on the vertical axis to the line is 53.9%—the maximum. When the BMI was 26, the optimal cutoff value was 26.
[Table 3]
The optimal cutoff value may differ depending on the method used. This choice is left to the analyst’s discretion.
Assessing the usefulness of the test
We explain how to examine the test’s usefulness using two cases.
【Case 1】
All 10 people with a BMI of 26 or more are positive, and all 10 people with a BMI of less than 26 are negative. (Table 4)
The maximum Cramer’s coefficient of association was 1.000, and the cutoff value was 26.
[Table 4] Test results for case 1
The area enclosed by the ROC curve for case 1 was 1 (100%). The area of an ideal test that perfectly separates the positive and negative results is 100%.
[Figure 1] ROC curve for case 1
【Case 2】
As shown in Table 5, the 10 negative cases had a BMI value between 21 and 30. The 10 positive cases also had a BMI value of 21–30, and no cutoff value could distinguish between the positive and negative cases.
[Table 5] Test results for case 2
The area enclosed by the ROC curve in case 2 was 0.5 (50%). The area of a test that could not distinguish between positive and negative results was set at 50%.
[Figure 2] ROC curve for case 2
AUC
The area Under the Curve (AUC) is the area under the ROC curve. The AUC is an index used to measure the usefulness of a test. AUC is 0.50 (50%) when positive and negative cannot be distinguished and 1 (100%) when positive and negative can be distinguished.
The AUC in Table 1 is 92.3%, which is close to 100%, making this a useful test.
[Figure 6] ROC curves in Table 1
The usefulness of this test for a population can be examined using a one-group population proportion test.
Null Hypothesis: The AUC is 0.5 (50%).
Alternative hypothesis: AUC is greater than 0.5 (50%) (one-sided test).
[Test statistics]
The p-value can be calculated using Excel functions.
= 1-NORMSDIST (test statistics)
= 0.0001
Consequently, “As the p-value is <0.05, the BMI test is useful.”
>>Return to Tell Me About Statistics!
Comments are closed