Present: Levi Bonnell, Jessica Clifton, Justine Dee, Steve
DeVoe, Nancy Gell, Juvena Hitt, Amanda Kennedy, Ben Littenberg, Jen Oshita, Bradley
Tompkins, Connie van Eeghen, Adam Sprouse-Blum, Mariana Wingood
1.
Warm Up: Justine got
scooped ☹
2.
Ben: Chalk talk on sensitivity and specificity, ROC curves, and more
a. Deck of cards,
with 37% “sick patients” denoted with red backs
i.
Picture card means positive test
ii.
Non-picture card means negative test
b. Threshold: minimum
value to call the test positive, with aces reporting negative and kings
reporting positive
c. Prevalence: those
with the disease/N
d. Those who have
the disease:
i.
True positive + false negative
e. Sensitivity:
ability to correctly identify the sick (true positive and false negative)
i.
True Positive Rate: true positive/(true positive + false negative)
f.
Specificity: ability to correctly identify the well (false
positive and true negative)
i.
True Negative Rate: true negative/(true negative + false positive)
g. Positive Predictive
Value: correctness of a positive test: D+|T+ = TP/(TP+FP)
h. Negative Predictive
Value: correctness of a negative test: D-|T- = TN/(TN+FN)
i.
Example with card deck
i.
Of the 19 reds (who truly have disease)
1. 11 have a
correct positive test
2. 8 have a false
positive test
ii.
Of the 33 blues (who truly do not have disease)
1. 1 has a false positive
test
2. 32 have a
correct positive test
iii.
Positivity: 12/52 Ã how many
positives will come from a testing process that will need further services. Also, “test positivity” IS NOT the accuracy
of the test or the prevalence
iv.
True Positive Rate: 11/19 = 57.9% - not very good at identify the
ill
v.
True Negative Rate: 32/33 = 97% - very good at identifying the well
vi.
Positive Predictive Value: 11/12 = 91.7%
vii.
Negative Predictive Value: 32/40 = 80%
j.
The placement of the threshold drives the above values
i.
Test manufacturers can set these with known data, to create a
numeric threshold; some thresholds are based on analogue variables or based on
subjective assessments
k. Receiver
Operating Characteristics Curve
i.
False Positive Rate on x axis
ii.
True Positive Rate on the y axis
1. Goal is to have
a high TPR and low FPR
iii.
AUC: area under the ROC
1. A diagonal ROC
(straight line diagonal) means the test is 50% likely to get a true positive
and 50% likely to get a true negative
2. A ROC below the
represents a bad test; a perfect test is represented by the axes
3. For covid, we
want high sensitivity at the cost of specificity – get the public off the
streets
a. For a painful, costly
treatment, we want high specificity – don’t hurt people needlessly
b. Outside the ROC
are the people who get missed by the test
l.
PowerPoint and Excel sheet coming to an inbox near you soon!
3.
Future sessions
a. Next week: Justine’s
topic and the all claims data base
b. Future topics:
Jessica’s article in the Cone of Silence
Posts
