Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Relationship between Obuchowski–Rockette–Hillis and Gallas methods for analyzing multi-reader diagnostic imaging data with empirical AUC as the reader performance measure

Relationship between Obuchowski–Rockette–Hillis and Gallas methods for analyzing multi-reader... For analyzing multireader multicase (MRMC) diagnostic imaging data when the reader performance measure of interest is the area under the receiver-operating-characteristic curve (AUC), two popular methods of analysis that allow conclusions to generalize to both the reader and case populations are the method developed by Obuchowski, Rockette and Hillis (ORH) and the method primarily developed by Gallas (Gallas). While the ORH method is a general method that is applicable to most reader performance metrics, the Gallas method is limited to those metrics for which an unbiased variance estimate exists. Previously it was not known if the ORH method could be adapted so as to produce the same variance estimate as the Gallas method. In this paper, I show that a recently proposed version of the OR method produces the same unconstrained variance statistic as the Gallas method. However, the two methods differ in their approaches to constraining the variance estimate to be nonnegative and in their degrees-of-freedom estimates. These two differences are discussed and recommendations given. In addition, several contributions to the development of the ORH method are made, including determining sufficient conditions for unbiased variance estimates and providing justification for the ORH variance constraints and covariance estimation method. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Biostatistics & Epidemiology Taylor & Francis

Relationship between Obuchowski–Rockette–Hillis and Gallas methods for analyzing multi-reader diagnostic imaging data with empirical AUC as the reader performance measure

Biostatistics & Epidemiology , Volume OnlineFirst: 38 – Jun 9, 2022

Relationship between Obuchowski–Rockette–Hillis and Gallas methods for analyzing multi-reader diagnostic imaging data with empirical AUC as the reader performance measure

Abstract

For analyzing multireader multicase (MRMC) diagnostic imaging data when the reader performance measure of interest is the area under the receiver-operating-characteristic curve (AUC), two popular methods of analysis that allow conclusions to generalize to both the reader and case populations are the method developed by Obuchowski, Rockette and Hillis (ORH) and the method primarily developed by Gallas (Gallas). While the ORH method is a general method that is applicable to most reader...
Loading next page...
 
/lp/taylor-francis/relationship-between-obuchowski-rockette-hillis-and-gallas-methods-for-3y74SjWseu
Publisher
Taylor & Francis
Copyright
© 2022 International Biometric Society – Chinese Region
ISSN
2470-9379
eISSN
2470-9360
DOI
10.1080/24709360.2022.2062115
Publisher site
See Article on Publisher Site

Abstract

For analyzing multireader multicase (MRMC) diagnostic imaging data when the reader performance measure of interest is the area under the receiver-operating-characteristic curve (AUC), two popular methods of analysis that allow conclusions to generalize to both the reader and case populations are the method developed by Obuchowski, Rockette and Hillis (ORH) and the method primarily developed by Gallas (Gallas). While the ORH method is a general method that is applicable to most reader performance metrics, the Gallas method is limited to those metrics for which an unbiased variance estimate exists. Previously it was not known if the ORH method could be adapted so as to produce the same variance estimate as the Gallas method. In this paper, I show that a recently proposed version of the OR method produces the same unconstrained variance statistic as the Gallas method. However, the two methods differ in their approaches to constraining the variance estimate to be nonnegative and in their degrees-of-freedom estimates. These two differences are discussed and recommendations given. In addition, several contributions to the development of the ORH method are made, including determining sufficient conditions for unbiased variance estimates and providing justification for the ORH variance constraints and covariance estimation method.

Journal

Biostatistics & EpidemiologyTaylor & Francis

Published: Jun 9, 2022

Keywords: Obuchowski–Rockette; receiver-operating-characteristic curve; area under the ROC curve; Gallas; diagnostic radiology

References