Access the full text.
Sign up today, get DeepDyve free for 14 days.
J. Spencer, R. Forstner, T. Cunha, K. Kinkel, on Sub-Committee (2009)
ESUR guidelines for MR imaging of the sonographically indeterminate adnexal mass: an algorithmic approachEuropean Radiology, 20
K. Marill, Yuchiao Chang, Kim Wong, Ari Friedman (2017)
Estimating negative likelihood ratio confidence when test sensitivity is 100%: A bootstrapping approachStatistical Methods in Medical Research, 26
I. Thomassin-Naggara, E. Aubert, A. Rockall, A. Jalaguier-Coudray, R. Rouzier, E. Daraï, M. Bazot (2013)
Adnexal masses: development and preliminary validation of an MR imaging scoring system.Radiology, 267 2
Douglas Brown, P. Doubilet, F. Miller, M. Frates, F. Laing, D. Disalvo, C. Benson, Mark Lerner (1998)
Benign and malignant ovarian masses: selection of the most discriminating gray-scale and Doppler sonographic features.Radiology, 208 1
JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses JAMA Network Open
C. Anton, F. Carvalho, Elci Oliveira, Gustavo ArantesRosaMaciel, E. Baracat, J. Carvalho (2012)
A comparison of CA125, HE4, risk ovarian malignancy algorithm (ROMA), and risk malignancy index (RMI) for the classification of ovarian massesClinics, 67
D. Timmerman, B. calster, A. Testa, L. Savelli, D. Fischerová, W. Froyman, L. Wynants, C. Holsbeke, E. Epstein, D. Franchi, J. Kaijser, A. Czekierdowski, S. Guerriero, R. Fruscio, F. Leone, A. Rossi, C. Landolfo, I. Vergote, T. Bourne, L. Valentin (2016)
Predicting the risk of malignancy in adnexal masses based on the Simple Rules from the International Ovarian Tumor Analysis group.American journal of obstetrics and gynecology, 214 4
C. Anthoulakis, N. Nikoloudis (2014)
Pelvic MRI as the "gold standard" in the subsequent evaluation of ultrasound-indeterminate adnexal lesions: a systematic review.Gynecologic oncology, 132 3
J. Yazbek, S. Raju, J. Ben-Nagi, T. Holland, K. Hillaby, D. Jurković (2008)
Effect of quality of gynaecological ultrasonography on management of patients with suspected ovarian cancer: a randomised controlled trial.The Lancet. Oncology, 9 2
J. Kaijser, V. Vandecaveye, C. Deroose, A. Rockall, I. Thomassin-Naggara, T. Bourne, D. Timmerman (2014)
Imaging techniques for the pre-surgical diagnosis of adnexal tumours.Best practice & research. Clinical obstetrics & gynaecology, 28 5
I. Thomassin-Naggara, E. Daraï, Charles Cuénod, R. Rouzier, P. Callard, M. Bazot (2008)
Dynamic contrast‐enhanced magnetic resonance imaging: A useful tool for characterizing ovarian epithelial tumorsJournal of Magnetic Resonance Imaging, 28
H. Hricak, Min Chen, F. Coakley, K. Kinkel, Kyle Yu, G. Sica, P. Bacchetti, C. Powell (2000)
Complex adnexal masses: detection and characterization with MR imaging--multivariate analysis.Radiology, 214 1
J. Kaijser, A. Sayasneh, K. Hoorde, S. Ghaem-Maghami, T. Bourne, D. Timmerman, B. calster (2014)
Presurgical diagnosis of adnexal tumours using mathematical models and scoring systems: a systematic review and meta-analysis.Human reproduction update, 20 3
B. Calster, B. Calster, D. Timmerman, Lil Valentin, A. Mcindoe, S. Ghaem-Maghami, A. Testa, I. Vergote, T. Bourne, T. Bourne (2012)
Triaging women with ovarian masses for surgery: observational diagnostic study to compare RCOG guidelines with an International Ovarian Tumour Analysis (IOTA) group protocolBJOG: An International Journal of Obstetrics & Gynaecology, 119
Thomassin-Naggara (2008)
Dynamic contrast-enhanced magnetic resonance imaging: a useful tool for characterizing ovarian epithelial tumors.J Magn Reson Imaging, 28
Ian Jacobs, U. Menon, Andy Ryan, A. Gentry-Maharaj, M. Burnell, Jatinderpal Kalsi, N. Amso, S. Apostolidou, E. Benjamin, D. Cruickshank, Danielle Crump, S. Davies, A. Dawnay, Stephen Dobbs, Gwendolen Fletcher, J. Ford, Keith Godfrey, Richard Gunu, M. Habib, Rachel Hallett, J. Herod, H. Jenkins, C. Karpinskyj, Simon Leeson, S. Lewis, William Liston, Alberto Lopes, T. Mould, John Murdoch, David Oram, D. Rabideau, K. Reynolds, Ian Scott, M. Seif, Aarti Sharma, Naveena Singh, Julie Taylor, Fiona Warburton, Martin Widschwendter, K. Williamson, R. Woolas, Lesley Fallowfield, Alistair McGuire, S. Campbell, Mahesh Parmar, S. Skates (2016)
Ovarian cancer screening and mortality in the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS): a randomised controlled trialLancet (London, England), 387
J. Alcázar, M. Pascual, B. Graupera, M. Aubá, T. Errasti, B. Olartecoechea, Á. Ruiz-Zambrana, L. Hereter, S. Ajossa, S. Guerriero (2016)
External validation of IOTA simple descriptors and simple rules for classifying adnexal massesUltrasound in Obstetrics & Gynecology, 48
D. Timmerman, A. Testa, T. Bourne, E. Ferrazzi, L. Ameye, M. Konstantinovic, B. calster, W. Collins, I. Vergote, S. Huffel, L. Valentin (2005)
Logistic regression model to distinguish between the benign and malignant adnexal mass before surgery: a multicenter study by the International Ovarian Tumor Analysis Group.Journal of clinical oncology : official journal of the American Society of Clinical Oncology, 23 34
J. Landis, G. Koch (1977)
The measurement of observer agreement for categorical data.Biometrics, 33 1
M. Basha, R. Refaat, S. Ibrahim, Nadia Madkour, A. Awad, E. Mohamed, A. Sammak, Mohamed Zaitoun, Hitham Dawoud, Mai Khamis, Heba Mohamed, A. El-Maghraby, A. Abdalla, M. Assy, M. Nada, A. Obaya, E. Abdelbary (2019)
Gynecology Imaging Reporting and Data System (GI-RADS): diagnostic performance and inter-reviewer agreementEuropean Radiology
Mireille Ruiz, P. Labauge, A. Louboutin, O. Limot, A. Fauconnier, C. Huchon (2016)
External validation of the MR imaging scoring system for the management of adnexal masses.European journal of obstetrics, gynecology, and reproductive biology, 205
J. Yazbek, L. Ameye, A. Testa, Lil Valentin, D. Timmerman, T. Holland, C. Holsbeke, D. Jurković (2010)
Confidence of expert ultrasound operators in making a diagnosis of adnexal tumor: effect on diagnostic accuracy and interobserver agreementUltrasound in Obstetrics and Gynecology, 35
K. Kinkel, Ying Lu, A. Mehdizade, M. Pelte, H. Hricak (2005)
Indeterminate ovarian mass at US: incremental value of second imaging test for characterization--meta-analysis and Bayesian analysis.Radiology, 236 1
JOHN Curtin (1994)
Management of the adnexal mass.Gynecologic oncology, 55 3 Pt 2
K. Sasaguri, K. Yamaguchi, T. Nakazono, M. Mizuguchi, S. Aishima, M. Yokoyama, H. Irie (2019)
External validation of ADNEX MR SCORING system: a single-centre retrospective study.Clinical radiology, 74 2
F. Faschingbauer, M. Benz, L. Häberle, T. Goecke, M. Beckmann, S. Renner, A. Müller, T. Wittenberg, C. Münzenmayer (2012)
Subjective assessment of ovarian masses using pattern recognition: the impact of experience on diagnostic performance and interobserver variabilityArchives of Gynecology and Obstetrics, 285
F. Hsieh, D. Bloch, M. Larsen (1998)
A simple method of sample size calculation for linear and logistic regression.Statistics in medicine, 17 14
J. Borley, C. Wilhelm-Benartzi, J. Yazbek, R. Williamson, R. Williamson, N. Bharwani, N. Bharwani, V. Stewart, V. Stewart, I. Carson, E. Hird, A. Mcindoe, A. Farthing, S. Blagden, S. Ghaem-Maghami (2015)
Radiological predictors of cytoreductive outcomes in patients with advanced ovarian cancerBJOG: An International Journal of Obstetrics & Gynaecology, 122
S. Buys, E. Partridge, A. Black, C. Johnson, L. Lamerato, C. Isaacs, D. Reding, R. Greenlee, L. Yokochi, B. Kessel, E. Crawford, T. Church, G. Andriole, J. Weissfeld, M. Fouad, D. Chia, B. O'Brien, L. Ragard, J. Clapp, J. Rathmell, T. Riley, P. Hartge, P. Pinsky, Claire Zhu, G. Izmirlian, B. Kramer, A. Miller, Jian‐Lun Xu, P. Prorok, J. Gohagan, C. Berg (2011)
Effect of screening on ovarian cancer mortality: the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Randomized Controlled Trial.JAMA, 305 22
I. Thomassin-Naggara, E. Daraï, C. Cuénod, L. Fournier, Irwin Toussaint, C. Marsault, M. Bazot (2009)
Contribution of diffusion-weighted MR imaging for predicting benignity of complex adnexal massesEuropean Radiology, 19
E. DeLong, D. DeLong, D. Clarke‐Pearson (1988)
Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.Biometrics, 44 3
A. Tsili, C. Tsampoulas, M. Argyropoulou, I. Navrozoglou, Y. Alamanos, E. Paraskevaidis, S. Efremidis (2008)
Comparative evaluation of multidetector CT and MR imaging in the differentiation of adnexal massesEuropean Radiology, 18
P. Pereira, L. Sarian, A. Yoshida, K. Araújo, R. Barros, Ana Baião, D. Parente, S. Derchain (2018)
Accuracy of the ADNEX MR scoring system based on a simplified MRI protocol for the assessment of adnexal masses.Diagnostic and interventional radiology, 24 2
J. Kaijser, T. Gorp, K. Hoorde, C. Holsbeke, A. Sayasneh, I. Vergote, T. Bourne, D. Timmerman, B. calster (2013)
A comparison between an ultrasound based prediction model (LR2) and the risk of ovarian malignancy algorithm (ROMA) to assess the risk of malignancy in women with an adnexal mass.Gynecologic oncology, 129 2
I. Thomassin-Naggara, Irwin Toussaint, N. Perrot, R. Rouzier, C. Cuénod, M. Bazot, E. Daraï (2011)
Characterization of complex adnexal masses: value of adding perfusion- and diffusion-weighted MR imaging to conventional MR imaging.Radiology, 258 3
L. Bernardin, P. Dilks, S. Liyanage, M. Miquel, A. Sahdev, A. Rockall (2012)
Effectiveness of semi-quantitative multiphase dynamic contrast-enhanced MRI as a predictor of malignancy in complex adnexal masses: radiological and pathological correlationEuropean Radiology, 22
E. Siegelman, E. Outwater (1999)
Tissue characterization in the female pelvis by means of MR imaging.Radiology, 212 1
P. Morice, C. Uzan, R. Fauvet, S. Gouy, P. Duvillard, E. Darai (2012)
Borderline ovarian tumour: pathological diagnostic dilemma and risk factors for invasive or lethal recurrence.The Lancet. Oncology, 13 3
A. Rieber, K. Nüssle, Iris Stöhr, D. Grab, S. Fenchel, R. Kreienberg, S. Reske, H. Brambs (2001)
Preoperative diagnosis of ovarian tumors with MR imaging: comparison with transvaginal sonography, positron emission tomography, and histologic findings.AJR. American journal of roentgenology, 177 1
S. Sohaib, A. Sahdev, P. Trappen, Ian Jacobs, R. Reznek (2003)
Characterization of adnexal mass lesions on MR imaging.AJR. American journal of roentgenology, 180 5
D. Levine, Douglas Brown, R. Andreotti, B. Benacerraf, C. Benson, W. Brewster, B. Coleman, P. DePriest, P. Doubilet, S. Goldstein, U. Hamper, J. Hecht, M. Horrow, H. Hur, M. Marnach, Maitray Patel, L. Platt, E. Puscheck, R. Smith-Bindman (2010)
Management of asymptomatic ovarian and other adnexal cysts imaged at US: Society of Radiologists in Ultrasound Consensus Conference Statement.Radiology, 256 3
R. Moszyński, S. Szubert, D. Szpurek, S. Michalak, J. Krygowska, S. Sajdak (2013)
Usefulness of the HE4 biomarker as a second-line test in the assessment of suspicious ovarian tumorsArchives of Gynecology and Obstetrics, 288
Natalie Nunes, Gareth Ambler, X. Foo, Martin Widschwendter, D. Jurković (2018)
Prospective evaluation of IOTA logistic regression models LR1 and LR2 in comparison with subjective pattern recognition for diagnosis of ovarian cancer in an outpatient settingUltrasound in Obstetrics & Gynecology, 51
C. Holsbeke, V. Belle, F. Leone, S. Guerriero, D. Paladini, G. Melis, S. Greggi, D. Fischerová, E. Jonge, P. Neven, T. Bourne, T. Bourne, Lil Valentin, S. Huffel, D. Timmerman (2010)
Prospective external validation of the ‘ovarian crescent sign’ as a single ultrasound parameter to distinguish between benign and malignant adnexal pathologyUltrasound in Obstetrics & Gynecology, 36
Y. Woo, M. Kyrgiou, A. Bryant, Thomas Everett, H. Dickinson (2012)
Centralisation of services for gynaecological cancers - a Cochrane systematic review.Gynecologic oncology, 126 2
W. Froyman, C. Landolfo, B. Cock, L. Wynants, P. Sladkevicius, A. Testa, C. Holsbeke, E. Domali, R. Fruscio, E. Epstein, Maria Bernardo, D. Franchi, M. Kudla, V. Chiappa, J. Alcázar, F. Leone, F. Buonomo, L. Hochberg, M. Coccia, S. Guerriero, N. Deo, L. Jokubkiene, J. Kaijser, A. Coosemans, I. Vergote, J. Verbakel, T. Bourne, B. calster, L. Valentin, D. Timmerman (2019)
Risk of complications in patients with conservatively managed ovarian tumours (IOTA5): a 2-year interim analysis of a multicentre, prospective, cohort study.The Lancet. Oncology, 20 3
E. Sadowski, V. Paroder, Krupa Patel-Lippmann, J. Robbins, L. Barroilhet, Elizabeth Maddox, T. McMahon, E. Sampene, A. Wasnik, Alexander Blaty, K. Maturen (2018)
Indeterminate Adnexal Cysts at US: Prevalence and Characteristics of Ovarian Cancer.Radiology, 287 3
E. Meys, L. Jeelof, N. Achten, B. Slangen, S. Lambrechts, R. Kruitwagen, T. Gorp (2017)
Estimating risk of malignancy in adnexal masses: external validation of the ADNEX model and comparison with other frequently used ultrasound methodsUltrasound in Obstetrics & Gynecology, 49
E. Meys, J. Kaijser, R. Kruitwagen, B. Slangen, B. Calster, B. Aertgeerts, J. Verbakel, J. Verbakel, D. Timmerman, T. Gorp (2016)
Subjective assessment versus ultrasound models to diagnose ovarian cancer: A systematic review and meta-analysis.European journal of cancer, 58
Els Orvos (2020)
MRI scoring system for adnexal masses
D. Levine, Douglas Brown, R. Andreotti, B. Benacerraf, C. Benson, W. Brewster, B. Coleman, P. DePriest, P. Doubilet, S. Goldstein, U. Hamper, J. Hecht, M. Horrow, H. Hur, M. Marnach, Maitray Patel, L. Platt, E. Puscheck, R. Smith-Bindman (2010)
Management of asymptomatic ovarian and other adnexal cysts imaged at US Society of Radiologists in Ultrasound consensus conference statement.Ultrasound quarterly, 26 3
Scott Fegan (2009)
Sensitivity and specificity of multimodal and ultrasound screening for ovarian cancer, and stage distribution of detected cancers: results of the prevalence screen of the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS)Journal of Family Planning and Reproductive Health Care, 35
(2019)
MR in ovarian cancer
Key Points Question Is the Ovarian-Adnexal IMPORTANCE Approximately one-quarter of adnexal masses detected at ultrasonography are Reporting Data System Magnetic indeterminate for benignity or malignancy, posing a substantial clinical dilemma. Resonance Imaging (O-RADS MRI) score accurate for stratifying the risk of OBJECTIVE To validate the accuracy of a 5-point Ovarian-Adnexal Reporting Data System Magnetic malignancy of sonographically Resonance Imaging (O-RADS MRI) score for risk stratification of adnexal masses. indeterminate adnexal masses? Findings In this multicenter cohort DESIGN, SETTING, AND PARTICIPANTS This multicenter cohort study was conducted between study that included 1340 women, the March 1, 2013, and March 31, 2016. Among patients undergoing expectant management, 2-year O-RADS MRI score had a sensitivity of follow-up data were completed by March 31, 2018. A routine pelvic MRI was performed among 0.93 and a specificity of 0.91. consecutive patients referred to characterize a sonographically indeterminate adnexal mass according to routine diagnostic practice at 15 referral centers. The MRI score was prospectively Meaning Applying this score in clinical applied by 2 onsite readers and by 1 reader masked to clinical and ultrasonographic data. Data practice may allow a tailored, patient- analysis was conducted between April and November 2018. centered approach for adnexal masses that are sonographically indeterminate, MAIN OUTCOMES AND MEASURES The primary end point was the joint analysis of true-negative preventing unnecessary surgery, less and false-negative rates according to the MRI score compared with the reference standard (ie, extensive surgery, or fertility histology or 2-year follow-up). preservation when appropriate, while ensuring preoperative detection of RESULTS A total of 1340 women (mean [range] age, 49 [18-96] years) were enrolled. Of 1194 lesions with a high likelihood of evaluable women, 1130 (94.6%) had a pelvic mass on MRI with a reference standard (surgery, 768 malignancy. [67.9%]; 2-year follow-up, 362 [32.1%]). A total of 203 patients (18.0%) had at least 1 malignant adnexal or nonadnexal pelvic mass. No invasive cancer was assigned a score of 2. Positive likelihood Supplemental content ratios were 0.01 for score 2, 0.27 for score 3, 4.42 for score 4, and 38.81 for score 5. Area under the receiver operating characteristic curve was 0.961 (95% CI, 0.948-0.971) among experienced readers, Author affiliations and article information are listed at the end of this article. with a sensitivity of 0.93 (95% CI, 0.89-0.96; 189 of 203 patients) and a specificity of 0.91 (95% CI, 0.89-0.93; 848 of 927 patients). There was good interrater agreement among both experienced and junior readers (κ = 0.784; 95% CI, 0.743-0824). Of 580 of 1130 women (51.3%) with a mass on MRI and no specific gynecological symptoms, 362 (62.4%) underwent surgery. Of them, 244 (67.4%) had benign lesions and a score of 3 or less. The MRI score correctly reclassified the mass origin as nonadnexal with a sensitivity of 0.99 (95% CI, 0.98-0.99; 1360 of 1372 patients) and a specificity of 0.78 (95% CI, 0.71-0.85; 102 of 130 patients). CONCLUSIONS AND RELEVANCE In this study, the O-RADS MRI score was accurate when stratifying the risk of malignancy in adnexal masses. JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 Open Access. This is an open access article distributed under the terms of the CC-BY License. JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 1/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses Introduction Adnexal masses are common, resulting in a significant clinical workload related to diagnostic imaging, 1,2 surgery, and pathology. Most adnexal masses are benign, and most masses can be accurately 3,4 categorized as benign or malignant on ultrasonography. However, between 18% and 31% of adnexal masses remain indeterminate following ultrasonography using International Ovarian Tumor 3,4 Analysis (IOTA) Simple Rules or other ultrasonography scoring systems. Moreover, this prevalence may be an underestimation, given that many studies only report the cases with available surgical reference standards. There are very limited data on patients who undergo only imaging and clinical follow-up. In a prospective external validation of the IOTA Simple Rules among 666 women, 362 women (54.4%) underwent surgery, 309 of whom (85.4%) had benign masses. The authors reported that, among 304 patients (45.6%) who underwent expectant management, 71 patients (23.4%) experienced disappearance of the mass, and 233 (76.6%) had a persistent mass on imaging follow-up that was considered benign after 1 year of follow-up. Percutaneous biopsy of a suspicious adnexal mass is not advised because of the risk of potentially upstaging a confined early-stage ovarian cancer or because of the risk of sampling error, resulting in a missed cancer diagnosis. As a result, despite the low rate of malignant adnexal masses 5,6 discovered at ultrasonography (ie, 8%-20%), a significant number of women with sonographically indeterminate but benign adnexal masses undergo potentially unnecessary or inappropriately 7,8 extensive surgical interventions. This increases the risk of loss of fertility as well as morbidity, as 7,8 reported in the 2 largest ovarian cancer screening trials. Conversely, some women with an indeterminate adnexal mass undergo initial, limited, noncancer surgery and are found to have ovarian cancer, with a risk of suboptimal initial cytoreductive surgery and significantly poorer outcomes. Thus, preoperative characterization and risk stratification of indeterminate adnexal masses are unmet clinical needs. A validated scoring system that standardizes imaging reports and categorizes the risk of malignant neoplasm in these women would be useful as a triage test to decide whether surgery is appropriate and, if so, the extent of surgery required. This could potentially reduce unnecessary or overextensive surgery. In the literature, various scoring systems have been developed based on clinical, biochemical (eg, cancer antigen 125 [CA 125] or human epididymis 10,11 protein 4 [HE 4] levels), and ultrasonographic criteria. Nevertheless, a significant subgroup of adnexal masses remain indeterminate despite optimal sonographic risk assessment, hampering 12,13 treatment planning. A magnetic resonance imaging (MRI) scoring system was developed in a retrospective single-center study among a cohort of 497 patients with indeterminate adnexal masses at ultrasonography. This MRI-based score consisted of 5 categories according to the positive likelihood ratio for a malignant neoplasm. The score was based on MRI features with high positive and high negative predictive values in distinguishing benign from malignant masses that were considered indeterminate on ultrasonography. However, the score warrants validation from a multicenter study among a large cohort of women. Therefore, the primary objective of this study was to test the score for risk stratification in women referred for an MRI of sonographically indeterminate adnexal masses in a large, prospective, multicenter clinical study. The findings provide the evidence to support the publication of the Ovarian-Adnexal Reporting Data System Magnetic (O-RADS) MRI score version 1. Methods This prospective multicenter cohort study was conducted between March 1, 2013, and March 31, 2018. Participant enrollment took place between March 1, 2013, and March 31, 2016, with 2-year follow-up among 362 of 1340 patients (27.0%) undergoing expectant management, which was completed by March 31, 2018. Recruitment was undertaken in 15 centers, each with a principal investigator from the European Society of Urogenital Radiology Female Pelvic Imaging working JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 2/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses group (eAppendix 1 in the Supplement). According to French regulations at the time of study initiation, the study was approved by the Comité Consultatif sur le Traitement de l'Information en matière de Recherche dans le domaine de la Santé. In addition, the protocol was approved by the ethics committee of each participating site. All participating women provided written informed consent. The study protocol appears in eAppendix 2 of the Supplement. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline. Study Population Consecutive women older than 18 years who were referred to a study center for MRI to characterize a sonographically indeterminate adnexal mass were invited to participate. Exclusion criteria were pregnancy or any contraindication to MRI (eAppendix 3 in the Supplement). MRI Acquisition and Analysis Each patient underwent a routine pelvic MRI (1.5 T or 3 T), including morphological sequences (ie, T2-weighted; T1-weighted, with and without fat suppression; and T1-weighted after gadolinium injection) and functional sequences (ie, perfusion-weighted and diffusion-weighted sequences). If no adnexal mass was present on T2-wieghted and T1-weighted sequences, functional sequences and gadolinium injection were not mandatory (eAppendix 4 in the Supplement). The patients’ medical records were reviewed, and gynecological symptoms and ultrasonographic findings were recorded. The quality of the ultrasonography report was recorded using widely used standardized criteria (eAppendix 3 in the Supplement). Levels of CA 125 were recorded, if available. An experienced radiologist (ie, with >10 years of gynecological MRI expertise) (I.T.-N, E.P., A.J.-C., A.G., L.S.F., S.S., I.M., N.B, V.J., T.M.C., G.M, C.B., C.M., N.F.P., M.B., P.T., and A.G.R.) and a junior radiologist (ie, with 6-12 months of gynecological MRI expertise) read MRI scans prospectively and independently. They were unmasked to clinical and sonographic findings. Another experienced reader (with >10 years of gynecological MRI expertise) (I.T.-N., E.P., A.J.-C., V.J., and A.G.R.), masked to clinical and sonographic findings, read the MRI retrospectively. The readers characterized each mass according to a standardized lexicon and assigned a score. If there was no adnexal mass or if the origin of a pelvic mass was nonadnexal, readers were asked to assign a score of 1 to the adnexa and rate the nonadnexal mass as either suspicious or nonsuspicious for malignancy. The presence of solid tissue and its morphology (eg, enhancing solid papillary projection, thickened irregular septa, or the solid part of a mixed cystic solid or purely solid lesion) were evaluated. The reader then analyzed T2-weighted signal intensity within the solid tissue (ie, low or intermediate compared with the outer myometrium) and diffusion-weighted signal intensity within the solid tissue (ie, high diffusion-weighted signal intensity compared with serous fluid, eg, urine within bladder or cerebrospinal fluid). The reader classified the enhancement of the solid tissue using time intensity curve (TIC) classification. When TIC classification was not feasible, it was rated as TIC type 2. An MRI score of 2 was assigned if the reader diagnosed a purely cystic mass (ie, adnexal unilocular cyst with simple fluid and no solid tissue), a purely endometriotic mass (ie, adnexal unilocular cyst with endometriotic fluid and no internal enhancement), a purely fatty mass (ie, adnexal cyst with unilocular or multilocular fatty content and no solid tissue), if there was no wall enhancement, or if solid tissue was detected with homogeneous hypointense T2-weighted as well as homogeneous hypointense high b value of diffusion-weighted solid tissue (ie, a dark-dark pattern). An MRI score of 3 was assigned if the reader diagnosed an adnexal unilocular cyst with proteinaceous or hemorrhagic fluid that did not comply with endometriotic fluid signal intensity and no solid tissue, an adnexal multilocular cyst and no solid tissue, or an adnexal lesion with solid tissue that enhanced with a TIC type 1 on dynamic-contrast enhanced MRI (excluding dark-dark solid tissue). An MRI score of 4 was assigned if the reader diagnosed an adnexal lesion with solid tissue that enhanced with a TIC type 2 on dynamic-contrast enhanced MRI (excluding dark-dark solid tissue). An MRI score of 5 was assigned if the reader diagnosed an adnexal lesion with solid tissue that enhanced with a TIC type 3 on dynamic-contrast enhanced MRI or if peritoneal or omental thickening or nodules were detected. JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 3/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses Presence of ascites was noted. Up to 3 pelvic masses per patient were analyzed. All MRI readers were masked to the final outcome. Reader Training During study setup, a session of 30 anonymized MRI scans (acquired before the beginning of the study) were downloaded for a training session for all teams participating in the multicenter validation to learn how to apply the score. A standardized lexicon was used for interpretation. Reference Standard Patient management was decided by a multidisciplinary team according to standard clinical practice in each center. The final diagnosis recorded for each patient was based on histology or clinical follow-up; if the lesion did not disappear or decrease at imaging follow-up, a minimum of 24 months of observation was performed (with or without imaging) from the date of the study MRI. Borderline lesions were considered malignant. In cases that underwent clinical follow-up, the origin of the pelvic mass was confirmed if there was agreement by the 2 experienced readers. In cases of disagreement, a final decision was made by a consensus panel of 5 radiologists (with >10 years of gynecological MRI expertise) (I.T.-N., I.M., and P.T.) from 2 sites. Statistical Analysis The study end point was the joint analysis of true-negative and false-negative rates according to the MRI score compared with the reference standard. The sample size was determined based on previous results to ensure that this study would have power of at least 90% to show a difference in diagnostic odds ratio between a score of 2 and 3 and between a score of 4 and 5. A total sample size of 1340 patients would ensure a probability of at least 95% to obtain the required 569, 250, 52, and 51 patients with MRI scores of 2, 3, 4, and 5, respectively, assuming that 6% of patients would have lesions classified as a score of 1 and 10% of patients would be lost to follow-up. For statistical analysis, the MRI score was matched to the reference standard. These analyses used the prospective, experienced reader’s rating. Diagnostic accuracy was evaluated both at the patient level and at the lesion level in terms of positive likelihood ratios (PLRs) and negative likelihood ratios (NLRs) for malignant masses. In addition, sensitivities, specificities, positive predictive values (PPVs), and negative predictive values (NPVs) were computed for dichotomized scores (ie, score of 2 and 3 [benign] vs score of 4 and 5 [malignant], according to predefined cutoff at 3 for the score). To evaluate interobserver agreement, we used receiver operating characteristic curve analysis and compared the area under receiver operating characteristic curves between experienced and 16 17 junior readers. We also computed weighted quadratic κ coefficients. Patients lost to follow-up (130 [9.7%]), patients for whom MRI failed to be completed (9 [0.7%]), and patients who withdrew consent (7 [0.5%]) were excluded from analyses. Among patients who were lost to follow-up, subjective assessment by experienced readers was indeterminate, borderline, or invasive in 12 of 130 patients (9.2%). Estimates are provided with their 95% CIs. A 2-tailed P < .05 was considered statistically significant. Statistical analyses were performed using MedCalc version 9.3.0.0 (MedCalc Software) and R version 3.5.0 (R Project for Statistical Computing). Results Patients and Lesions Overall, 1340 patients were enrolled in the study. The mean (range) age was 49 (18-96) years. The final, evaluable population included 1194 patients (89.1%), after 130 (9.7%) patient withdrawals (Figure 1). Of the included patients, 64 (5.4%) were found not to have a pelvic mass. The remaining 1130 patients (94.6%) had a total of 1502 pelvic masses. JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 4/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses Patient characteristics and clinical symptoms are described in Table 1. Patients were referred for indeterminate adnexal masses based on the results of a pelvic ultrasonograph with an issued report rated as high quality, with scores equal to or greater than 5 to 7 in 950 of 1194 patients (79.6%) (eAppendix 3 in the Supplement). Solid tissue was suspected at ultrasonography in 523 women (43.8%), including 166 malignant lesions (11.1%) and 357 benign lesions (23.8%). Ultrasonographic size of the mass was greater than 6 cm in 337 women (28.2%), including 95 malignant lesions (6.3%) and 242 benign lesions (16.1%). Levels of CA 125 were available for 537 patients (44.9%), 398 (74.1%) of whom had benign and 139 (25.9%) of whom had malignant tumors. An elevated CA 125 level (ie,35 U/mL; to convert to kU/L, multiply by 1.0) indicated malignant neoplasm as follows: sensitivity, 0.68 (95% CI, 0.61-0.76; 95 of 139 patients); specificity, 0.82 (95% CI, 0.78-0.86; 327 of 398 patients); PLR, 3.83 (95% CI, 3.02-4.87); NLR, 0.39 (95% CI, 0.30-0.49); PPV, 0.57 (95 of 166 patients); NPV, 0.88 (327 of 371 patients); and accuracy, 0.78 (422 of 537 patients). A total of 915 women (76.7%) had MRI performed with a 1.5-T MRI scanner and 279 women (23.4%) with a 3-T MRI scanner using 3 different vendors (Siemens, Philips, GE Healthcare). Quality of MRI scans was considered good in 1160 of 1194 women (97.1%), with motion artifacts in 184 of 1194 (15.4%), but all scans remained diagnostic. Of 1130 patients with a pelvic mass, 768 (67.9%) underwent surgery, and 362 (32.1%) underwent standard clinical follow-up (Table 1). Of those undergoing clinical follow-up, imaging was included for 263 women (72.6%). There were 130 nonadnexal masses (8.6%) and 1372 adnexal masses (91.3%). The origins of nonadnexal masses are reported in Table 1. The prevalence of malignant neoplasms in the population of women with pelvic mass on MRI was 18.0% (203 of 1130). Figure 1. Study Population Flow Diagram 1340 Individuals consented 146 Excluded 130 Lost to follow-up 5 Incomplete MRI sequences 4 Failed gadolinium injection 3 Inclusion criteria not met 4 Withdrew consent 1194 MRI scans evaluable 671 Asymptomatic 523 Symptomatic 64 Had no MR pelvic mass, ie, score 1 1130 Included in per-patient analysis of score 802 With 1 pelvis mass 284 With 2 pelvis masses 44 With 3 pelvic masses 1502 Pelvic masses 1031 Adnexal diagnoses 99 Nonadnexal diagnoses 1372 Adnexal masses 130 Nonadnexal masses 838 Benign 89 Benign 1110 Benign 115 Benign 193 Malignant 10 Malignant 262 Malignant 15 Malignant 37 Borderline 45 Borderline MR indicates magnetic resonance; MRI, magnetic 156 Invasive 217 Invasive resonance imaging. JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 5/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses Table 1. Population Characteristics Characteristics No. (%) Personal History (n = 1194) Menopausal 511 (42.8) History of pelvic surgery 364 (30.5) History of adnexal surgery 134 (11.2) History of infertility 94 (7.9) History of breast or ovarian cancer 126 (10.5) Known BRCA1/2 carriers 13 (1.1) Clinical Presentation (n = 1194) Pelvic pain 384 (32.2) Vaginal bleeding 60 (5.0) Palpable mass or increasing abdominal volume 10 (0.8) Urinary symptoms 8 (0.7) Amenorrhea 6 (0.5) Constipation or diarrhea 2 (0.2) Combination of previously mentioned symptoms 53 (4.4) None of these symptoms 671 (56.2) MRI Findings (n = 1194) 0 Lesions 64 (5.4) 1 Lesion 802 (67.2) 2 Lesions 284 (23.8) 3 Lesions 44 (3.7) Management (n = 1130) Primary surgery 719 (63.6) Secondary surgery After initial follow-up 44 (3.9) After primary chemotherapy 5 (0.4) 24 mo of clinical and/or imaging follow-up 362 (32) Imaging follow-up 263 (23.3) Disappearance 96 (8.4) Decrease of the mass 15 (1.3) Stability 152 (13.4) Clinical follow-up, ie, stability 99 (8.8) Origin of Pelvic Mass From Reference Standard Masses, No. 1502 Adnexal Masses, No. 1372 Malignant, No./total, No. (%) Ovary 253/1223 (20.7) Tubo-ovarian 9/125 (7.2) Abbreviation: MRI, magnetic resonance imaging. Mesosalpinx 0/24 The 99 women without imaging follow-up had 122 Nonadnexal lesions with the following clinical diagnoses: 23 Masses, No. 130 nonadnexal masses including 14 leiomyomas, 5 Malignant, No./total, No. (%) peritoneal cysts, 3 hematoma, and 1 Nabothian cyst; Uterus 4/72 (5.5) 26 serous cystadenomas; 24 functional cysts; 23 Peritoneal 3/41 (7.3) endometrioma; 9 cystadenofibroma; 5 ovarian fibroma; 5 hydrosalpinx; 4 mature cystic teratoma; 2 Colorectal 5/5 (100) mucinous cystadenoma; and 1 paraovarian cyst. Lymph node 2/2 (100) This category excludes tubo-ovarian peritoneal Other, eg, schwannoma, arterial aneurysm 0/9 carcinoma and includes other purely peritoneal Urothelial 1/1 (100) diseases, such as pseudoperitoneal cysts. JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 6/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses Analysis of the MRI Score at the Patient Level Validation per Patient Discrimination | The score yielded an area under the receiver operating characteristic curve of 0.961 (95% CI, 0.948-0.971) among experienced readers and 0.942 (95% CI, 0.927-0.955) among junior readers, with a higher performance for the experienced readers (P = .03) (eFigure in the Supplement). Among experienced and junior readers, a score of 4 or 5 suggested a malignant mass with a sensitivity of 0.93 (95% CI, 0.89-0.96; 189 of 203 patients) and 0.92 (95% CI, 0.87-0.95; 186 of 203 patients), respectively, and a specificity 0.91 (95% CI, 0.89-0.93; 848 of 927 patients) and 0.90 (95% CI, 0.88-0.91; 833 of 927 patients), respectively. The prevalence of masses that remained indeterminate on MRI (ie, score of 4) remained low, with 122 (10.8%) among experienced readers and 141 (12.5%) among junior readers. Performance per Patient | Among 91 women assigned a score of 1, 78 women (85.7%) with nonadnexal masses were subjectively rated nonsuspicious, and 13 women (14.3%) were subjectively rated suspicious by the readers (Table 2 and Table 3). All nonsuspicious masses were benign, and suspicious masses were highly indicative of a malignant tumor (PLR, 15.22; 95% CI, 4.23-54.82) (Table 3). Of the 13 women with at least 1 suspicious mass, 10 women (76.9%) had malignant tumors, and 3 (23.1%) had benign leiomyomas with degeneration. Among 571 women assigned a score of 2, 569 (99.6%) had benign lesions (Table 3). Two premenopausal women (0.3%) had serous borderline tumors (false-negative rate, 0.3%) (Table 3). Among 213 women assigned a score of 3, malignant tumors were found in 6 premenopausal women (2.8%) and 6 menopausal women (2.8%) (false-negative rate, 5.6%), including 4 women (33.3%) with masses containing no solid tissue (1 premenopausal woman [25.0%] and 3 menopausal women [75.0%]) (Table 3). Eight of 12 malignant tumors (75.0%) were borderline. All 12 fat-containing lesions were benign. Among 122 women assigned a score of 4, 62 women (50.8%) had benign tumors, and 60 (49.2%) had malignant tumors, with a higher prevalence of invasive than borderline tumors (40 [32.8%] vs 20 [16.4%], respectively) (Table 3). All 14 fat-containing lesions were benign. Among 133 women assigned a score of 5, 9 premenopausal (6.8%) and 5 menopausal women (3.8%) had benign lesions (false-positive rate, 10.5%), including 5 (35.7%) with mature teratomas, 2 (14.3%) with pelvic inflammatory disease, 2 (14.3%) with cystadenofibroma, 1 (7.1%) with Brenner tumors, 1 (7.1%) with serous cystadenoma, 1 (7.1%) with ovarian fibroma, 1 (7.1%) with struma ovarii, Table 2. Diagnostic Performance of the Magnetic Resonance Imaging Score Score (95% CI) Characteristic Experienced Readers Junior Readers Performance, No. True-positive result 189 186 False-negative result 14 17 True-negative result 848 833 False-positive result 79 94 Sensitivity 0.93 (0.89-0.96) 0.92 (0.87-0.95) Specificity 0.91 (0.89-0.93) 0.90 (0.88-0.91) Likelihood ratio Positive 10.90 (8.82-13.50) 9.04 (7.43-11.00) A total of 1130 magnetic resonacing imaging scans Negative 0.08 (0.05-0.13) 0.09 (0.06-0.15) were scored. Sensitivities, specificities, and positive Predictive value and negative predictive values were computed for Positive 0.71 (0.65-0.76) 0.66 (0.61-0.72) dichotomized scores (ie, score of 2 and 3 [benign] vs score of 4 and 5 [malignant] or score 1 [nonadnexal Negative 0.98 (0.97-0.99) 0.98 (0.97-0.99) mass rated suspicious]). A total of 203 of 1130 Accuracy 0.92 (0.90-0.93) 0.90 (0.88-0.92) patients (18.0%) had at least 1 malignant mass of Diagnostic odds ratio 145.00 (80.30-261.00) 97.00 (56.50-166.00) adnexal or nonadnexal origin. JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 7/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses and 1 (7.1%) with a luteal cyst. Of the 6 fat-containing lesions, 5 (83.3%) were benign and 1 (16.7%) was a malignant germ cell (endodermal sinus) tumor (Table 3). The PLR for score 2 was 0.01; for score 3, 0.27; for score 4, 4.42; and for score 5, 38.81. Potential Consequences for Management In the study population, 580 of 1130 women (51.3%) with a mass on MRI and no specific gynecological symptoms underwent surgery (362 [62.4%]) or follow-up (218 [37.6%]). Based on the standard MRI report and management, 244 women (67.4%) (121 premenopausal and 123 menopausal) with benign lesions and a score of 3 or less or a nonadnexal mass rated as nonsuspicious underwent surgery, and 1 woman (0.5%) with an invasive tumor with a score of 4 or 5 underwent initial follow-up. Moreover, 8 women (2.2%) who underwent surgery had a score of 2 (2 [25.0%] with borderline tumors) or 3 (4 [50.0%] with borderline tumors and 2 [25.0%] with invasive tumors). Reproducibility The interrater agreement of the score between experienced and junior readers was substantial (κ = 0.784; 95% CI 0.743-0.824). Interrater agreement between experienced readers was also substantial (κ = 0.804; 95% CI, 0.764-0.844). Analysis of the Criteria Used in the MRI Score at the Lesion Level The overall prevalence of malignancy per lesion at histology was 18.4% (277 of 1502), 11.5% (15 of 130) for nonadnexal masses and 19.1% (262 of 1372) for adnexal masses, including 45 (3.0%) borderline tumors. Detailed analysis of imaging criteria and their diagnostic performances are available in eTable 1 and eTable 2 in the Supplement. The origin of each pelvic mass was correctly categorized as adnexal with a sensitivity of 0.99 (95% CI, 0.98-0.99; 1360 of 1372), a specificity of 0.78 (95% CI, 0.71-0.85; 102 of 130), a PLR of 4.60 (95% CI, 3.31-6.39), an NLR of 0.01 (95% CI, 0.01-0.02), a PPV of 0.98 (95% CI, 0.97-0.99), an NPV of 0.89 (95% CI, 0.82-0.94), and an accuracy of 0.97 (95% CI, 0.96-0.98). The diagnosis of the origin was reproducible with a substantial κ of 0.68 (95% CI, 0.59-0.76) between junior and experienced readers. Table 3. Experienced Readers’ MRI Scores Prospectively Assigned to 1130 Patients With Pelvic Masses Patients, No./Total No. (%) Positive Likelihood of Malignant a c MRI Score Patients, No. Tumor (95% CI) With Borderline Tumors With Invasive Tumors With Malignant Tumors 1 91 0.53 (0.30-1.07) 0 10/91 (10.9) 10/91 (10.9) Nonsuspicious nonadnexal 78 0 (0-0.16) 0 0 0 Suspicious nonadnexal 13 15.22 (4.23.54.82) 0 10/13 (76.9) 10/13 (76.9) 2 571 0.01 (0-0.04) 2/571 (0.3) 0 2 (0.3) 3 213 0.27 (0.16-0.48) 8/213 (3.7) 4/213 (1.9) 12/213 (5.6) No solid tissue 120 0.17 (0.06-0.45) 2/120 (1.7) 2/120 (1.7) 4/120 (3.3) Solid tissue 93 0.46 (0.23-0.93) 6/93 (6.4) 2/93 (2.1) 8/93 (8.6) 4 122 4.42 (3.31-6.09) 20/122 (16.4) 40/122 (32.8) 60/122 (49.2) Fatty content 14 0 0 0 No fatty content 108 20/108 (18.5) 40/108 (37.0) 60/108 (55.5) 5 133 38.81 (22.79-66.11) 7/133 (5.2) 112/133 (84.2) 119/133 (89.5) Fatty content 6 0 1/6 (16.7) 1/6 (16.7) No fatty content 127 7/127 (5.5) 111/127 (87.4) 118/127 (92.9) Abbreviation: MRI, magnetic resonance imaging. Malignant includes borderline and invasive tumors. a 18 Confidence intervals obtained by bootstrapping. A total of 64 evaluable patients did not have a pelvic mass at the time of the MRI scan, and 91 patients had a nonadnexal mass. JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 8/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses Discussion In this multicenter prospective cohort study, we demonstrated that a previously published 5-point MRI score provided robust risk stratification of sonographically indeterminate adnexal masses. The study confirms a strong concordance of the PLR of malignant neoplasms for each category. Therefore, the MRI score may provide potentially crucial information for determining the therapeutic strategy, allowing the risks and benefits of expectant management or surgery to be considered case by case. The study demonstrated the feasibility of the acquisition of the multiparametric MRI in multiple centers. Substantial interrater agreement was found, regardless of reader experience, which 20-22 has been reported to be challenging in some ultrasonographic studies. External validations in 23-25 smaller single-center studies have reported similar findings. The O-RADS MRI score is now proposed as the accepted score for risk assignment of sonographically indeterminate adnexal masses, supported by this strong evidence base. The O-RADS MRI score addresses a significant clinical issue, given that approximately 18% to 3,4,26-29 31% of adnexal lesions detected on ultrasound remain indeterminate. Transvaginal sonography is accurate for detecting and characterizing adnexal lesions of classic appearance. 7,8 However, in the 2 largest ovarian cancer screening trials, a significant number of false-positive cases underwent inappropriate surgery. Nonclassical features, such as avascular solid components, large masses, and less experienced sonographers could all contribute to lower accuracy and 20,31 specificity on ultrasound examination. Several sonographic rules and scoring systems have been 3,10,11,27 advocated, such as IOTA, Risk of Malignancy Index, and Risk of Ovarian Malignancy Algorithm. However, performance in real-life clinical settings has been variable, potentially because of 27,32-34 differences in operator experience and cancer prevalence in the population being studied. Correctly classifying an adnexal mass as benign has positive consequences, including the potential to reduce overtreatment by unnecessary or overextensive surgery, to allow consideration of minimally invasive or fertility-preserving surgery, and to improve patient information regarding the risk of ovarian reserve alteration after surgery. The preponderant contribution of MRI in adnexal mass evaluation is its specificity, allowing confident diagnosis of many benign adnexal lesions. Using the O-RADS MRI score, our study demonstrated that, even in sonographically indeterminate masses, a lesion with a score of 2 has a PLR of malignant tumor of no greater than 0.01, and a lesion with a score of 3 has the PLR of malignant tumor of 0.27 among both experienced and junior readers. Thus, patients with lesions with scores of 2 or 3 can make an informed decision with the support of their physicians to undergo a minimally invasive or conservative surgical approach or expectant management. Such a high-performance clinical scoring system could allow for the development of decision-support tools, with referral of patients for appropriate follow-up vs surgery, and ensure that fertility-preserving treatment options are considered for young patients with early-stage disease. Our study showed that the likelihood of a borderline tumor when a lesion scores 5 was very low (<6%), as in a previous publication. However, as borderline tumors are a rare entity, our population included less than 3% (45 of 1502), and larger specific studies are needed. Optimal management also relies on identifying the site of origin of a pelvic mass (ie, adnexal or nonadnexal). Our study showed that MRI helped to correctly reclassify the origin of the presumed adnexal mass on ultrasonography. In 802 women with only 1 mass described on MRI, 81 lesions (10.0%) were nonadnexal. This is particularly important for malignant nonadnexal tumors, for which initial incorrect management could adversely affect prognosis. In our population, 5.4% (15 of 277) of malignant tumors were nonadnexal lesions of uterine, colorectal, urothelial, nonepithelial peritoneal, or lymph node origin. Limitations This study has limitations. It was observational and without randomization, and the score was not integrated into clinical decision-making. Therefore, the clinical consequences on the number of cases in which surgery can be avoided or tailored can only be imputed. However, the validation of the score JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 9/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses now allows studies to test the consequences of the O-RADS MRI score in treatment planning; 2 such 36,37 studies are currently underway. Furthermore, because patients were managed according to clinical recommendations, when no pelvic mass was found on MRI, no specific follow-up was 4,5 undertaken in clinical care as in previous base studies. Consequently, 64 such cases were excluded from our analysis. This is a low proportion compared with the number of resolving lesions that are typically seen in general outpatient adnexal ultrasonography, given that most physiological ovarian masses are recognized and not referred for MRI. Thus, the O-RADS MRI score estimates the risk of malignancy of an existing pelvic mass detected on MRI. Magnetic resonance imaging is not recommended as a screening tool, and as such, the NPV when no mass is found is not available in the literature. In our study, 284 women had 2 lesions and 44 women had 3 lesions. As each mass was not considered independently, a potential clustering effect should be considered. In addition, 99 of 362 patients were observed during 2 years with only clinical assessment that cannot replace imaging evaluation. Furthermore, we did not include patients who were lost to follow-up in the final analysis. This could have biased the prevalence of the disease in the population, and that is why we calculated the PLR of malignant neoplasms and not the PPV. Of note, more than 90% of these patients were diagnosed with benign lesions on MRI, and this is likely to have played a part in the decision not to undertake further clinical follow-up. Conclusions In conclusion, this prospective multicenter cohort study confirmed the performance of a 5-point scoring system developed in a previous retrospective single-center study. The current study provides strong supporting evidence, and the score is now presented as the O-RADS MRI score. Using this score in clinical practice may allow a tailored, patient-centered approach for masses that are sonographically indeterminate, preventing unnecessary surgery, less extensive surgery, or fertility preservation when appropriate, while ensuring preoperative detection of lesions with a high likelihood of malignancy. ARTICLE INFORMATION Accepted for Publication: December 1, 2019. Published: January 24, 2020. doi:10.1001/jamanetworkopen.2019.19896 Open Access: This is an open access article distributed under the terms of the CC-BY License.©2020 Thomassin-Naggara I et al. JAMA Network Open. Corresponding Author: Isabelle Thomassin-Naggara, MD, PhD, Service de Radiologie, Hôpital Tenon, Assistance Publique–Hôpitaux de Paris, Hôpitaux Univesitaires Est Parisien, 58 avenue Gambetta, 75020 Paris, France (isabelle. thomassin@aphp.fr). Author Affiliations: Service de Radiologie, Hôpital Tenon, Assistance Publique–Hôpitaux de Paris, Sorbonne Université, Paris, France (Thomassin-Naggara, Bazot); Institute for Computing and Data Sciences, Sorbonne Université, Paris, France (Thomassin-Naggara, Bazot); American College of Radiology, Ovarian-Adnexal Reporting and Data System Magnetic Resonance Imaging Committee (Thomassin-Naggara, Sadowski, Reinhold, Rockall); Service d’Imagerie de la Femme, Centre Hospitalier de Valenciennes, Valenciennes, France (Poncelet); Institut Paoli Calmettes, Marseille, France (Jalaguier-Coudray); Hospital da Luz, Lisboa, Portugal (Guerra); Department of Radiology, Hôpital Européen Georges Pompidou, Assistance Publique-Hôpitaux de Paris, Université Paris- Descartes, Paris, France (Fournier); Centre for Radiology, Clinical Centre of Vojvodina, Medical Faculty, University of Novi Sad, Novi Sad, Serbia and Montenegro (Stojanovic); Lapeyronie Hospital, University of Montpellier, Montpellier, France (Millet, Taourel); Department of Radiology, Imperial College Healthcare NHS Trust, London, United Kingdom (Bharwani, Rockall); Hôpital de la Timone, Marseille, France (Juhan); Department of Radiology, Instituto Português de Oncologia de Lisboa Francisco Gentil, Lisboa, Portugal (Cunha); Department of Radiology, Umberto I Hospital, Sapienza University Roma, Rome, Italy (Masselli); Institut Gustave Roussy, Paris, France (Balleyguier); Institut Curie, Paris, France (Malhaire); Centre Pyramides, Paris, France (Perrot); University of Wisconsin, Madison, Wisconsin (Sadowski); Centre of Research in Epidemiology and Statistics Sorbonne Paris Cité, Institute national de la santé et de la recherche médicale, Joint Research Unit 1153, Paris, France (Porcher); Service JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 10/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses de Gynecologie et Obstetrique et Médecine de la Reproduction, Hôpital Tenon, Assistance Publique–Hôpitaux de Paris, Hôpitaux Univesitaires Est Parisien, Paris, France (Darai); Faculté de Médecine Pierre et Marie Curie, Sorbonne Université, Paris, France (Darai); Department of Medical Imaging, McGill University Health Centre, Montreal, Quebec, Canada (Reinhold); Division of Cancer and Surgery, Faculty of Medicine, Imperial College London, United Kingdom (Rockall). Author Contributions: Drs Thomassin-Naggara and Porcher had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Reinhold and Rockall contributed equally. Concept and design: Thomassin-Naggara, Guerra, Porcher, Reinhold, Rockall. Acquisition, analysis, or interpretation of data: All authors. Drafting of the manuscript: Thomassin-Naggara, Guerra, Stojanovic, Sadowski, Darai, Reinhold, Rockall. Critical revision of the manuscript for important intellectual content: Thomassin-Naggara, Poncelet, Jalaguier- Coudray, Guerra, Fournier, Stojanovic, Millet, Bharwani, Juhan, Cunha, Masselli, Balleyguier, Malhaire, Perrot, Bazot, Taourel, Porcher, Reinhold, Rockall. Statistical analysis: Thomassin-Naggara, Porcher, Darai. Obtained funding: Thomassin-Naggara. Administrative, technical, or material support: Thomassin-Naggara, Poncelet, Jalaguier-Coudray, Fournier, Stojanovic, Bharwani, Masselli, Malhaire, Perrot, Bazot, Taourel, Reinhold, Rockall. Supervision: Thomassin-Naggara, Stojanovic, Masselli, Balleyguier, Sadowski, Bazot, Taourel, Reinhold, Rockall. Conflict of Interest Disclosures: Dr Thomassin-Naggara reported receiving personal fees and nonfinancial support from General Electric and personal fees from Siemens, Hologic, Canon, and Guerbet outside the submitted work. Dr Fournier reported receiving grants from Invectys and speaking fees from General Electric, Novartis, Sanofi, and Janssen Pharmaceuticals outside the submitted work. Dr Balleyguier reported receiving personal fees and nonfinancial support from General Electric and personal fees from Siemens, Samsung Group, and the Bracco Group outside the submitted work. Dr Rockall reported receiving an educational speaker fee from Guerbet. No other disclosures were reported. Funding/Support: This work was funded by a grant from the Société d’Imagerie de la Femme. Support is acknowledged from the National Institute of Health Research Imperial Biomedical Research Centre and the Cancer Research UK Imperial Centre. Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Additional Contributions: The European Adnex MR Score Group (EURAD) Collaborators include Asma Bekhouche, MD, Antoine Brault, MD, Louise Gervais, MD, and Edith Kermarrec, MD (Service de Radiologie, Hôpital Tenon, Assistance Publique–Hôpitaux de Paris, Hôpitaux Univesitaires Est Parisien, Paris, France), Alexandre Bellucci, MD (Department of Radiology, Hôpital Européen Georges Pompidou, Assistance Publique-Hôpitaux de Paris, Université Paris-Descartes, Paris, France), Julien Brochet, MD (Service d’Imagerie de la Femme, Centre Hospitalier de Valenciennes, Valenciennes, France), Claudia Campos, MD (Hôpital de la Timone, Marseille, France), Danielle Donat, MD, and Marijana Basta Nikolić, MD (Centre for Radiology, Clinical Centre of Vojvodina, Medical Faculty, University of Novi Sad, Novi Sad, Serbia and Montenegro), Federica Laghi, MD (Department of Radiology, Instituto Português de Oncologia de Lisboa Francisco Gentil, Lisboa, Portugal), Emma Pages-Bouic, MD, and Cecile Verheyden, MD (Lapeyronie Hospital, University of Montpellier, Montpellier, France), Miriam Salib, FRCR, and Elena Serena, MD (Service de Gynecologie et Obstetrique et Médecine de la Reproduction, Hôpital Tenon), and Pascal Siles, MD (Department of Radiology, Imperial College Healthcare NHS Trust, London, United Kingdom). The EURAD Collaborators were not compensated for their time. The Steering Committee of Ovarian-Adnexal Reporting Data System Magnetic Resonance Imaging (O-RADS MRI) includes Caroline Reinhold, MD, MSC (Department of Medical Imaging, McGill University Health Centre, Montreal, Quebec, Canada), Andrea Rockall, MRCP, FRCR (Faculty of Medicine, Imperial College London and Department of Radiology, Imperial College Healthcare NHS Trust, London, United Kingdom), Isabelle Thomassin-Naggara, MD, PhD (Service de Radiologie, Hôpital Tenon, Assistance Publique–Hôpitaux de Paris, Hôpitaux Univesitaires Est Parisien, Paris, France), Evan Siegelman, MD (Perelman School of Medicine, University of Pennsylvania, Philadelphia), Elizabeth Sadowski, MD (University of Wisconsin, Madison, Wisconsin), Kate Maturen, MD, MS (University of Michigan Hospitals, Ann Arbor), Alberto Vargas, MD (Memorial Sloan Kettering Cancer Center, New York, New York), and Rosemary Fostner, MD (University of Salzburg, Salzburg, Austria). The members of Steering Committee of O-RADS MRI were not compensated for their time. We acknowledge the support of the study sponsor Société d’Imagerie de la Femme, support for the EURAD study from European Society for Urogenital Radiology Female Pelvic Imaging working group, and support for the O-RADS steering committee from the American College of Radiology and the European Society of JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 11/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses Radiology. The EURAD collaborators contributed to additional secondary image reads. The O-RADS MRI collaborators participated in collaborative work in American College of Radiology O-RADS MRI committee for discussions concerning the transition of the AdnexMR score to become the O-RADS MRI score to align the position of the score in the imaging community. REFERENCES 1. Curtin JP. Management of the adnexal mass. Gynecol Oncol. 1994;55(3, pt 2):S42-S46. doi:10.1006/gyno. 1994.1340 2. Woo YL, Kyrgiou M, Bryant A, Everett T, Dickinson HO. Centralisation of services for gynaecological cancers: a Cochrane systematic review. Gynecol Oncol. 2012;126(2):286-290. doi:10.1016/j.ygyno.2012.04.012 3. Meys EMJ, Kaijser J, Kruitwagen RFPM, et al. Subjective assessment versus ultrasound models to diagnose ovarian cancer: a systematic review and meta-analysis. Eur J Cancer. 2016;58:17-29. doi:10.1016/j.ejca.2016. 01.007 4. Froyman W, Landolfo C, De Cock B, et al. Risk of complications in patients with conservatively managed ovarian tumours (IOTA5): a 2-year interim analysis of a multicentre, prospective, cohort study. Lancet Oncol. 2019;20(3): 448-458. doi:10.1016/S1470-2045(18)30837-4 5. Alcázar JL, Pascual MA, Graupera B, et al. External validation of IOTA simple descriptors and simple rules for classifying adnexal masses. Ultrasound Obstet Gynecol. 2016;48(3):397-402. doi:10.1002/uog.15854 6. Menon U, Gentry-Maharaj A, Hallett R, et al. Sensitivity and specificity of multimodal and ultrasound screening for ovarian cancer, and stage distribution of detected cancers: results of the prevalence screen of the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS). Lancet Oncol. 2009;10(4):327-340. doi:10.1016/ S1470-2045(09)70026-9 7. Buys SS, Partridge E, Black A, et al; PLCO Project Team. Effect of screening on ovarian cancer mortality: the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening randomized controlled trial. JAMA. 2011;305(22): 2295-2303. doi:10.1001/jama.2011.766 8. Jacobs IJ, Menon U, Ryan A, et al. Ovarian cancer screening and mortality in the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS): a randomised controlled trial. Lancet. 2016;387(10022):945-956. doi:10. 1016/S0140-6736(15)01224-6 9. Borley J, Wilhelm-Benartzi C, Yazbek J, et al. Radiological predictors of cytoreductive outcomes in patients with advanced ovarian cancer. BJOG. 2015;122(6):843-849. doi:10.1111/1471-0528.12992 10. Timmerman D, Van Calster B, Testa A, et al. Predicting the risk of malignancy in adnexal masses based on the Simple Rules from the International Ovarian Tumor Analysis group. Am J Obstet Gynecol. 2016;214(4):424-437. doi:10.1016/j.ajog.2016.01.007 11. Anton C, Carvalho FM, Oliveira EI, Maciel GAR, Baracat EC, Carvalho JP. A comparison of CA125, HE4, risk ovarian malignancy algorithm (ROMA), and risk malignancy index (RMI) for the classification of ovarian masses. Clinics (Sao Paulo). 2012;67(5):437-441. doi:10.6061/clinics/2012(05)06 12. Moszynski R, Szubert S, Szpurek D, Michalak S, Krygowska J, Sajdak S. Usefulness of the HE4 biomarker as a second-line test in the assessment of suspicious ovarian tumors. Arch Gynecol Obstet. 2013;288(6):1377-1383. doi: 10.1007/s00404-013-2901-1 13. Kaijser J, Vandecaveye V, Deroose CM, et al. Imaging techniques for the pre-surgical diagnosis of adnexal tumours. Best Pract Res Clin Obstet Gynaecol. 2014;28(5):683-695. doi:10.1016/j.bpobgyn.2014.03.013 14. Thomassin-Naggara I, Aubert E, Rockall A, et al. Adnexal masses: development and preliminary validation of an MR imaging scoring system. Radiology. 2013;267(2):432-443. doi:10.1148/radiol.13121161 15. Hsieh FY, Bloch DA, Larsen MD. A simple method of sample size calculation for linear and logistic regression. Stat Med. 1998;17(14):1623-1634. doi:10.1002/(SICI)1097-0258(19980730)17:14<1623::AID-SIM871>3.0.CO;2-S 16. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837-845. doi:10.2307/ 17. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1): 159-174. doi:10.2307/2529310 18. Marill KA, Chang Y, Wong KF, Friedman AB. Estimating negative likelihood ratio confidence when test sensitivity is 100%: a bootstrapping approach. Stat Methods Med Res. 2017;26(4):1936-1948. doi:10.1177/ JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 12/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses 19. Anthoulakis C, Nikoloudis N. Pelvic MRI as the “gold standard” in the subsequent evaluation of ultrasound- indeterminate adnexal lesions: a systematic review. Gynecol Oncol. 2014;132(3):661-668. doi:10.1016/j.ygyno. 2013.10.022 20. Yazbek J, Raju SK, Ben-Nagi J, Holland TK, Hillaby K, Jurkovic D. Effect of quality of gynaecological ultrasonography on management of patients with suspected ovarian cancer: a randomised controlled trial. Lancet Oncol. 2008;9(2):124-131. doi:10.1016/S1470-2045(08)70005-6 21. Van Holsbeke C, Van Belle V, Leone FPG, et al. Prospective external validation of the ‘ovarian crescent sign’ as a single ultrasound parameter to distinguish between benign and malignant adnexal pathology. Ultrasound Obstet Gynecol. 2010;36(1):81-87. doi:10.1002/uog.7625 22. Faschingbauer F, Benz M, Häberle L, et al. Subjective assessment of ovarian masses using pattern recognition: the impact of experience on diagnostic performance and interobserver variability. Arch Gynecol Obstet. 2012;285 (6):1663-1669. doi:10.1007/s00404-012-2229-2 23. Ruiz M, Labauge P, Louboutin A, Limot O, Fauconnier A, Huchon C. External validation of the MR imaging scoring system for the management of adnexal masses. Eur J Obstet Gynecol Reprod Biol. 2016;205:115-119. doi:10. 1016/j.ejogrb.2016.07.493 24. Pereira PN, Sarian LO, Yoshida A, et al. Accuracy of the AdnexMR scoring system based on a simplified MRI protocol for the assessment of adnexal masses. Diagn Interv Radiol. 2018;24(2):63-71. doi:10.5152/dir.2018.17378 25. Sasaguri K, Yamaguchi K, Nakazono T, et al. External validation of AdnexMR Scoring system: a single-centre retrospective study. Clin Radiol. 2019;74(2):131-139. doi:10.1016/j.crad.2018.10.014 26. Van Calster B, Timmerman D, Valentin L, et al. Triaging women with ovarian masses for surgery: observational diagnostic study to compare RCOG guidelines with an International Ovarian Tumour Analysis (IOTA) group protocol. BJOG. 2012;119(6):662-671. doi:10.1111/j.1471-0528.2012.03297.x 27. Meys EMJ, Jeelof LS, Achten NMJ, et al. Estimating risk of malignancy in adnexal masses: external validation of the ADNEX model and comparison with other frequently used ultrasound methods. Ultrasound Obstet Gynecol. 2017;49(6):784-792. doi:10.1002/uog.17225 28. Sadowski EA, Paroder V, Patel-Lippmann K, et al. Indeterminate adnexal cysts at US: prevalence and characteristics of ovarian cancer. Radiology. 2018;287(3):1041-1049. doi:10.1148/radiol.2018172271 29. Basha MAA, Refaat R, Ibrahim SA, et al. Gynecology Imaging Reporting and Data System (GI-RADS): diagnostic performance and inter-reviewer agreement. Eur Radiol. 2019;29(11):5981-5990. doi:10.1007/s00330-019- 06181-0 30. Levine D, Brown DL, Andreotti RF, et al; Society of Radiologists in Ultrasound. Management of asymptomatic ovarian and other adnexal cysts imaged at US Society of Radiologists in Ultrasound consensus conference statement. Ultrasound Q. 2010;26(3):121-131. doi:10.1097/RUQ.0b013e3181f09099 31. Yazbek J, Ameye L, Testa AC, et al. Confidence of expert ultrasound operators in making a diagnosis of adnexal tumor: effect on diagnostic accuracy and interobserver agreement. Ultrasound Obstet Gynecol. 2010;35(1):89-93. doi:10.1002/uog.7335 32. Kaijser J, Van Gorp T, Van Hoorde K, et al. A comparison between an ultrasound based prediction model (LR2) and the risk of ovarian malignancy algorithm (ROMA) to assess the risk of malignancy in women with an adnexal mass. Gynecol Oncol. 2013;129(2):377-383. doi:10.1016/j.ygyno.2013.01.018 33. Kaijser J, Sayasneh A, Van Hoorde K, et al. Presurgical diagnosis of adnexal tumours using mathematical models and scoring systems: a systematic review and meta-analysis. Hum Reprod Update. 2014;20(3):449-462. doi:10.1093/humupd/dmt059 34. Nunes N, Ambler G, Foo X, Widschwendter M, Jurkovic D. Prospective evaluation of IOTA logistic regression models LR1 and LR2 in comparison with subjective pattern recognition for diagnosis of ovarian cancer in an outpatient setting. Ultrasound Obstet Gynecol. 2018;51(6):829-835. doi:10.1002/uog.18918 35. Morice P, Uzan C, Fauvet R, Gouy S, Duvillard P, Darai E. Borderline ovarian tumour: pathological diagnostic dilemma and risk factors for invasive or lethal recurrence. Lancet Oncol. 2012;13(3):e103-e115. doi:10.1016/S1470- 2045(11)70288-1 36. ClinicalTrials.gov. ADNEXMR scoring system: impact of an MR scoring system of therapeutic strategy of pelvic adnexal masses (ASCORDIA). https://clinicaltrials.gov/ct2/show/NCT02664597?term=NCT02664597&draw= 2&rank=1. Accessed December 9, 2019. 37. ISRCTN Registry. MR in ovarian cancer. http://www.isrctn.com/ISRCTN51246892. Accessed December 9, 2019. 38. Thomassin-Naggara I, Daraï E, Cuenod CA, Rouzier R, Callard P, Bazot M. Dynamic contrast-enhanced magnetic resonance imaging: a useful tool for characterizing ovarian epithelial tumors. J Magn Reson Imaging. 2008;28(1):111-120. doi:10.1002/jmri.21377 JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 13/14 JAMA Network Open | Imaging O-RADS MRI Score for Sonographically Indeterminate Adnexal Masses SUPPLEMENT. eAppendix 1. Study Centers, Names of Principal Investigators, and Number of Patients Recruited and Included eAppendix 2. Trial Protocol eAppendix 3. Population Selection Criteria eAppendix 4. Methodological Details eTable 1. Lexicon and Supplementary Material with Detailed Analysis of Adnexal Lesions Based on Morphological Characteristics eTable 2. Diagnostic Values of Different MRI Features eFigure. Score Performance Among Senior and Junior Readers JAMA Network Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 (Reprinted) January 24, 2020 14/14 Supplementary Online Content Thomassin-Naggara I, Poncelet E, Jalaguier-Coudray A, et al. Ovarian-Adnexal Reporting Data System Magnetic Resonance Imaging (O-RADS MRI) score for risk stratification of sonographically indeterminate adnexal masses. JAMA Netw Open. 2020;3(1):e1919896. doi:10.1001/jamanetworkopen.2019.19896 eAppendix 1. Study Centers, Names of Principal Investigators, and Number of Patients Recruited and Included eAppendix 2. Trial Protocol eAppendix 3. Population Selection Criteria eAppendix 4. Methodological Details eTable 1. Lexicon and Supplementary Material with Detailed Analysis of Adnexal Lesions Based on Morphological Characteristics eTable 2. Diagnostic Values of Different MRI Features eFigure. Score Performance Among Senior and Junior Readers This supplementary material has been provided by the authors to give readers additional information about their work. © 2020 Thomassin Naggara I et al. JAMA Network Open. eAppendix 1. Study Centers, Names of Principal Investigators, and Number of Patients Recruited and Included Population (blue: inclusion / orange: final population) Clinical trial NCT01738789 TNN: Hopital Tenon; VAL : Hopital de Valenciennes / IPC : Institut Paoli Calmettes, LUZ Hospital da Luz, HEGP Hopital Européen Georges Pompidou, VOJ : Clinical Centre of Vojvodine, IMP: Imperial College Healthcare NHS Trust, LAP : Hopital de Lapeyronie, TIM: Hopital de la Timone, IPOL: Instituto Português de Oncologia de Lisboa Francisco Gentil , UMB Umberto I hospital. Sapienza University (UHS) , IGR : Institut Gustave Roussy, CUR : Institut Curie, PYR : Centre Pyramides, LAR : Hopital Lariboisiere © 2020 Thomassin Naggara I et al. JAMA Network Open. eAppendix 2. Trial Protocol European multicenter validation of an A MR System for DNEX SCORING characterizing adnexal masses: EURAD MR classification Title European multicenter validation of an A MR DNEX SCORING System for characterizing adnexal masses: EURAD MR classification. Short Title Version Date Main investigator CoMain investigator CoMain investigator © 2020 Thomassin Naggara I et al. JAMA Network Open. TABLE © 2020 Thomassin Naggara I et al. JAMA Network Open. Appendix 2 : system © 2020 Thomassin Naggara I et al. JAMA Network Open. Section 1 : Abstract Tit le of t he proj ect abst ract: INTRODUCTION EXPERIMENTAL PLAN Reference standard: Sample size: 1340 patients © 2020 Thomassin Naggara I et al. JAMA Network Open. Tit re du proj et Résumé: INTRODUCTION EXPERIMENTAL PLAN Reference standard: Mille trois cent quarante patientes © 2020 Thomassin Naggara I et al. JAMA Network Open. Section 2 : Specific information CCTIRS I Nom, titres, expériences et fonctions de la personneresponsable de la mise en oeuvre du traitementautomatisé Isabelle Thomassin-Naggara Maitre de Conférences Universitaires- Praticien Hospitalier Investigateur coordonnateur II Catégories de personnes : . qui seront appelées à mettre en oeuvre le traitement automatisé des données : Investigateur principal dans chaque site . qui auront accès aux données : Investigateur coordonnateur Assistant de recherche clinique du centre coordonateur III La recherche fait elle appel aux données du SNIRAM? Si oui, il faut l’indiquer dans le protocole de recherche (en justifiant de leur utilisation). Dans ce cas, l’avis favorable du comité dispense de soumettre à l’Institut des données de santé chargé de contrôler cet accès. Non © 2020 Thomassin Naggara I et al. JAMA Network Open. Section 3: Project description Chapter 1 : Coordinator and particpating teams List of participating teams Add as many lines as necessary Title, name of Name of the Affiliated the team team/laboratory or N° institution manager in the Contact email hospital and city frame of the department project Institut Paoli- 1 Marseille, FR Dr A.Jalaguier aureliejalaguier@yahoo.fr Calmettes (IPC) Centre Oscar 2 Lille, FR Dr S.Taieb S-Taieb@o-lambret.fr Lambret (COL) Institut Gustave Dr 3 Villejuif, FR balleyguier@igr.fr Roussy (IGR) C.Balleyguier Pr Hôpital de la Pitié 4 Paris, FR O.Lucidarme olivier.lucidarme@gmail.com Salpétrière (SAL) Hôpital Européen 5 Georges Pompidou Paris, FR Dr L.Fournier laure.fournier@gmail.com (HEGP) Hopital Montpellier, Dr I.Millet patricetaourel@wanadoo.fr Lapeyronie (LAP) FR Centre imagerie 7 Paris, FR Dr N.Perrot vitenson@wanadoo.fr pyramides (CIP) Hopital 8 Paris, FR Dr S.Bendavid sandrabendavid@free.fr Lariboisière (LAR) Hopital de Valenciennes, 9 Valenciennes Dr E.Poncelet poncelet.edouard@gmail.com FR (VAL) Hopital de la Marseille, FR Dr V.Juhan Valerie.JUHAN@ap-hm.fr Timone (TIM) Institut Curie 11 Paris, FR Dr E.Aubert emi.aubert@yahoo.fr (CUR) Hôpital Tenon Dr (Coordinator) Paris, FR I.Thomassin- isabellethomassin@gmail.com (TNN) Naggara Imperial College Pr A.Rockall 13 Healthcare NHS London, UK a.rockall@imperial.ac.uk Dr N.Bharwani Trust (ICH) Barts Health NHS London, UK Dr Anju Anju.Sahdev@bartshealth.nhs.uk © 2020 Thomassin Naggara I et al. JAMA Network Open. Trust (BAR) Sahdev Steeping Hill StockPort, Dr M.Lewinski Maryna.Lewinski@stockport.nhs.uk hospital UK Addenbrookes Cambridge, Dr S.Freeman hospital (ADD) UK University College Prof M Hall- London UK Margaret.hall-Craggs@uclh.nhs.uk Hospital London Craggs University Bonn, SW Dr R.Kubik Rahel.Kubik@ksb.ch Hodpital Dubrav John Paul Catholic Campobasso, Pr G.Reistano grestaino@rm.unicatt.it 19 University (JPCU) IT Umberto I 20 hospital. Sapienza Roma, IT Dr G.Masseli g.masselli@policlinicoumberto1.it University (UHS) Pr 21 Instituto di R.Manfredi/Pr Verone, IT roberto.pozzimucelli@uniivr.it Radiologia R.Pozzi Mucelli Clinique des Geneve, SW Pr K.Kinkel karen.kinkel-trugli@wanadoo.fr Grangettes University Institute Salzburg, 23 of Radiology Pr R Forstner R.Forstner@salk.at AUS (UIR) Instituto Português 24 de Oncologia de Pr T.M. Cuhna Lisboa, PT tmargarida@gmail.com Lisboa Francisco Dr C.Campos Gentil (IPOLFG) Hospital da Luz Lisboa, PT Dr Gisa Guerra gisaguerra@gmail.com (LUZ) 26 Clinical Center of Novi Sad, Dr tupsons@gmail.com Vojvodine SER S.Stojanovic CHU liège Liege, BE Dr A.Thille alain@thille.be University Zagreb,CR Dr I.Gordana gordana.augustan@gmail.com Hodpital Dubrav © 2020 Thomassin Naggara I et al. JAMA Network Open. Chapter 2 : Coordinator experience in the field Principal significant articles published over the last five years Radiology Eur Radiol Radiology Eur Radiol J Magn Reson Imaging Radiology Magn Reson Imaging Clin N Am Radiology © 2020 Thomassin Naggara I et al. JAMA Network Open. Chapter 3 : Clinical research project 3 1 Synopsis 3 1 1 Project description Justification and interest of the study Main orientations of the project Objectives of the clinical research Primary objective: Secondary objectives: © 2020 Thomassin Naggara I et al. JAMA Network Open. If ADNEXMR SCORING system improves reproducibility of MR report for characterization of adnexal masses If ADNEXMR SCORING system is as accurate if the radiologist is blinded from any clinical and ultrasonographic data Main endpoint Experimental plan Reference standard 3 1 2 Project feasibility Inclusions Number of required Number of required patients: 1340 patients © 2020 Thomassin Naggara I et al. JAMA Network Open. Number of centers 28 centers for patients accrual Feasibility All patients addressed to the 28 study centers that fulfill the selection criteria are included. The average number of available patients per year is 570 patients in France and 320 in other European countries. These numbers are based on the report of pelvic MRI for characterization of indeterminate adnexal masses at the radiology department in the different hospitals 3 2 2 Planned schedule and key steps Estimated schedule and identification of key steps Inclusion: 18 months Follow up after inclusion: 24 months Total: 3.5 years First inclusion: 1 January 2013 End of the inclusions: 30 June 2014 Follow up: from 1 July 2014 to 31 June 2016 Data analysis: July 2016 © 2020 Thomassin Naggara I et al. JAMA Network Open. 3 2 Introduction 3 2 1 Pelvic MR imaging 3 2 2 Development of an MR scoring system 3 2 3 Potential significance of a MR scoring system © 2020 Thomassin Naggara I et al. JAMA Network Open. 3 3 Objectives Primary objective: Secondary objectives and endpoints: The potential impact of applying the score to the therapeutic strategy, in particular to measure the possible reduction of unnecessary surgery in benign cases If ADNEXMR SCORING system improves reproducibility of MR report for characterization of adnexal masses If A MR system is as accurate if the radiologist is blinded from any DNEX SCORING clinical and ultrasonographic data Main endpoint © 2020 Thomassin Naggara I et al. JAMA Network Open. 3 4 Experimental plan 3 4 1 Type of experimental plan 3 4 2 Study workflow Inclusion of patients Pelvic MR imaging Surgery or clinical follow up Inclusion Criteria Patient 18 years old With sonographically indeterminate adnexal mass Informed consent Exclusion Criteria Pregnant women (relative contra indication for gadoliniuminjection) Pacemaker, ferromagnetic materials, or foreign body at risk of mobilization or any other contra indication to MR imaging. Intolerance to gadolinium contrast agents, or severe renal insufficiency (GFR <30 ml/min/1.73m²). 3 4 2 2 Data acquisition © 2020 Thomassin Naggara I et al. JAMA Network Open. 3 4 2 3 Data analysis 3 4 2 3 Surgery or follow up © 2020 Thomassin Naggara I et al. JAMA Network Open. 3 4 3 Expected duration of the research 3 3 4 Calendar First inclusion: 1 January 2013 End of the inclusions: 30 june 2014 Follow up: from 1 July 2014 to 30 june 2016 Data analysis: July 2016 3 4 5 Criteria for cancellation 3 5 Evaluated parameters 3 5 1 Primary evaluation parameter 3 5 2 Secondary evaluation parameter 3 6 Safety evaluation © 2020 Thomassin Naggara I et al. JAMA Network Open. 3 7 Statistics External validation of A MR system 3 7 1 DNEX SCORING It would thus be necessary to have at least 569 patients classified as ADNEXMR SCORE 2, 259 as A MR 3, 52 as A MR 4 and 51 as A MR 5 (18). Given the DNEX SCORE DNEX SCORE DNEX SCORE 3 7 2 Evaluation of agreement between readers 3 8 Ethics and CRF © 2020 Thomassin Naggara I et al. JAMA Network Open. 3 9 Data management 3 9 1 Data management SITE : Abbreviate name of the center (given in table page5) PATIENT NUMBER : From 1 for each center INITIALS of first name and last name INCLUSION DATE Copies of the letter of the mandatory opinion of CPP Successive versions of the protocol (identified by version number anddate) Signed consent forms in sealed storage with a list or register of enrollments CRF completed and validated for each patient All the appendices specified in the study The final study report, statistical analysis and quality control for thestudy Audit certificates from any audits done during the research 3 9 2 Right of access to data and source documents 3 10 Ethical and legal considerations 3 10 1 Request for the opinion of the CCP © 2020 Thomassin Naggara I et al. JAMA Network Open. 3 10 2 Modifications 3 10 3 CCTIRS and Commission of Informatics and Freedom (CNIL)Declaration 3 10 4 Information letter and informed consent © 2020 Thomassin Naggara I et al. JAMA Network Open. 3 10 5 Final research report 3 11 Rules concerning publication 3 12 References Curtin, J.P., Management of the adnexal mass. Gynecol Oncol, 1994. 55(3 Pt 2): p. S42 6. Timmerman, D., et al., Logistic regression model to distinguish between the benign and malignant adnexal mass before surgery: a multicenter study by the International Ovarian Tumor Analysis Group. J Clin Oncol, 2005. 23(34): p. 8794 801. Brown, D.L., et al., Benign and malignant ovarian masses: selection of the most discriminating gray scale and Doppler sonographic features. Radiology, 1998. 208(1): p. 103 Thomassin Naggara I., et al. Development and preliminary validation of an MRI rd Scoring system for Adnexal Masses. Radiology in press, accepted on 2012, October,14 Gynecologic sonography. Report of the ultrasonography task force. Council on Scientific Affairs, American Medical Association. Jama, 1991. 265(21): p.2851 5. Yazbek, J., et al., Effect of quality of gynaecological ultrasonography on management of patients with suspected ovarian cancer: a randomised controlled trial. Lancet Oncol, 2008. 9(2): p. 124 31. © 2020 Thomassin Naggara I et al. JAMA Network Open. Kinkel, K., et al., Indeterminate ovarian mass at US: incremental value of second imaging test for characterization meta analysis and Bayesian analysis. Radiology, 2005. 236(1): p. 85 94. Tsili, A.C., et al., Comparative evaluation of multidetector CT and MR imaging in the differentiation of adnexal masses. Eur Radiol, 2008. 18(5): p.1049 57. Hricak, H., et al., Complex adnexal masses: detection and characterization with MR imaging multivariate analysis. Radiology, 2000. 214(1): p.39 46. Rieber, A., et al., Preoperative diagnosis of ovarian tumors with MR imaging: comparison with transvaginal sonography, positron emission tomography, and histologic findings. AJR Am J Roentgenol, 2001. 177(1): p. 123 9. Sohaib, S.A., et al., Characterization of adnexal mass lesions on MR imaging. AJR Am J Roentgenol, 2003. 180(5): p. 1297 304. Thomassin Naggara, I., et al., Characterization of Complex Adnexal Masses: Value of Adding Perfusion and Diffusion weighted MR Imaging to Conventional MR Imaging. Radiology, 2011. 258(3): p. 793 803. Siegelman, E.S. and E.K. Outwater, Tissue characterization in the female pelvis by means of MR imaging. Radiology, 1999. 212(1): p. 5 18. Thomassin Naggara, I., et al., Contribution of diffusion weighted MR imaging for predicting benignity of complex adnexal masses. Eur Radiol, 2009. 19(6): p. 1544 52. Thomassin Naggara, I., et al., Dynamic Contrast Enhanced Magnetic Resonance Imaging: a Useful Tool for Characterizing Ovarian Epithelial Tumors. J Magn Reson Imaging, 2008. 25(1): p. 111 120. Bernardin L, et al. Effectiveness of semi quantitative multiphase dynamic contrast enhanced MRI as a predictor of malignancy in comple adnexal masses : radiological and pathological correlation. Eur radiol, 2011. Nov 18. Spencer JA et al. ESUR guidelines for MR imaging of the sonographically indeterminate adnexal mass : an algorithmic approach. Eur radiol, 2010 Jan ;20(1) : p. 25 Hsieh FY, Bloch DA, Larsen MD et al. A simple method of sample size calculation for linear and logistic regression. Stat Med, 1998. 17: p. 1623 1634. © 2020 Thomassin Naggara I et al. JAMA Network Open. Appendix 1 : Lexicon* Terms Definitions Purely endometriotic mass Solid papillary projections From Thomassin Naggara et al. Radiology 2012 in press © 2020 Thomassin Naggara I et al. JAMA Network Open. Appendix 2 : A MR syst em (From Thomassin Naggara et al. DNEX SCORING Radiology 2012 in press) PLR 1: No mass 2 : Benign mass <0.01 3: Probably benign mass 0.1 10 4: Indeterminate MR mass >10 5 : Probably malignant mass * Only one feature enough to classify in each category © 2020 Thomassin Naggara I et al. JAMA Network Open. Appendix 3 : Informed form Fiche dinformation et de consentement dans le cadre de létude observationnelle « Evaluation multicentrique dune classification diagnostique en IRM pour la caractérisation dune masse annexielle : EURADMR classification» Vous allez réaliser une IRM pelvienne pour caractériser une masse annexielle complexe détectée en échographie. Nous vous proposons de participer à létude intitulée : « Evaluation multicentrique dune classification diagnostique en IRM pour la caractérisation dune masse annexielle : EURADMR classification » Justification de létude : Objectifs de létude : Durée et suivi de létude : Risques et contraintes : © 2020 Thomassin Naggara I et al. JAMA Network Open. Effets indésirables et bénéfices attendus : Participation volontaire et droit de se retirer de létude : Confidentialité et législation : Dr.THOMASSIN NAGGARA (n° de téléphone : 01 56 01 70 00 (bip 15212)) Linvestigateur : Le patient : Fait à ............................. , le Fait à ............................. , le : Nom, prénom : Nom, prénom Signature Signature © 2020 Thomassin Naggara I et al. JAMA Network Open. Appendix 6: Informed consent: ENGLAND Patient information sheet Title Multicentre validation of ADNEX-MR classification for characterizing sonographically indeterminate adnexal masses. Short Title EurRad Study Imperial College Healthcare NHS Trust Sponsor Version 1.0 Date September 2012 PI Name <<Insert Local PI Name>> Dear Madam, We would like to invite you to take part in a research study. Before you decide whether to take part, it is important for you to understand why the research is being done and what it will involve. Please take time to read the following information carefully. Talk to others about the study if you wish. Ask us if there is anything that is not clear or if you would like more information. Take time to decide whether or not you wish to take part. 1. What is the purpose of the study? Ultrasound of the pelvis sometimes detects cysts or masses in the pelvis.These are known as adnexal masses. Although most are harmless a few can be more serious including a very small number which may have cancerous changes. If the ultrasound findings are not certain then MRI scanning is often performed to get more information about the mass. MRI has been found to be very accurate in the diagnosis of indeterminate adnexal masses and provides detailed information about the nature of adnexal masses. MRI is already used routinely to provide more information concerning indeterminate adnexal masses in order to help direct the doctor to the best treatment for the patient. For example, the doctor may need to decide whether surgery will be necessary or not and if surgery is needed, then the type of surgery that will be best for the individual patient. There have now been many published scientific studies that have reported the ability of MRI to determine the nature of adnexal masses that could not be confidently diagnosed using ultrasound and clinical findings. A classification system for MRI scan results has been designed which gives a numerical score of 1-5 to indicate the likely risk of the mass being cancer. The importance of the classification is to allow a reliable and reproducible scale which predicts the risk of cancerous change within an adnexal mass. This information will allow the best possible treatment strategy to be chosen for each individual patient. The name of the diagnostic classification is the ‘Adnex-MR score’. © 2020 Thomassin Naggara I et al. JAMA Network Open. The purpose of this study is to validate this ADNEX MR score for adnexal masses that were not confidently diagnosed on ultrasound (‘sonographically indeterminate’). The validation of this classification is being undertaken as a multicentre European collaborative trial. If the scoring system proves to be valid then doctors reporting MRI scans will be able to provide a clear and standardised indication of the level of risk that a mass may be cancerous. This will then, in turn, help determine the most appropriate treatment for patients with this condition. 2. Why have I been chosen? You have been chosen because your doctor has referred you for an MRI scan of the pelvis, following your recent ultrasound scan, in order to further evaluate the pelvic cyst or mass (an adnexal mass) that was found. We are inviting all patients who have been referred for MRI of an indeterminate adnexal mass to take part in this study. 3. Do I have to take part? No, it is up to you to decide whether or not to take part. If you choose to take part you will be asked to sign an informed consent form and you will be given a copy to keep, together with this information sheet. If you do not wish to take part in the study you do not have to give a reason. You will not be disadvantaged in any way, and it will not affect the standard of care you receive. This also applies if you initially decide to take part and then change your mind at a later date. 4. What are the alternatives? If you decide not to go into this study, you will be offered the MRI scan that your doctor has requested and your MRI scan will be reported in the normal way. You will follow the standard care pathway appropriate for your individual medical needs whether you take part in the study or not . 5. What will happen to me if I take part? If you decide to take part in the study you will be asked to sign a consent form and your doctor will make sure that you are suitable for entry onto the study. He or she will check that you meet all the requirements for entry and there are no reasons why you should be not take part. If you are pregnant you will not be able to participate in this study as the scoring system uses information from one scan sequence which uses an injection of a liquid dye called Gadolinium. That sequence is not used in these circumstances as Gadolinium is not recommended in pregnant women. The MRI scan that you will then undergo will be better suited to pregnant patients. Once these checks have taken place you will then be enrolled onto the study. Your participation in the study will involve your attending for for a standard MRI scan, as requested by your doctor, which will take approximately 45 minutes. Information about the standard MRI scan will be given to you by the imaging department when your appointment is made. Your MRI scan will be reported and the results will be made available to your doctors. Following discussion with your doctor, © 2020 Thomassin Naggara I et al. JAMA Network Open. you will proceed to treatment according the standard of care for your condition. This is your normal clinical care. If you consent to participate in the study then your scans will be reviewed and scored by the radiologists participating in the research. In addition to the routine clinical report they will use a predefined scoring system to categorise your scan appearances. The score uses a 5 point scale to indicate the likelihood that the cysts or mass in your pelvis might be cancerous. Similar scoring systems have been used for some time in breast imaging. Following the scan we would collect information to see how effective the scoring system was at predicting the correct diagnosis. If you have surgery we will collect information from the surgeon and the pathologists looking at sample in order to know the final diagnosis. If you do not have surgery we will collect information from any follow up appointments, scans or surgery that you have over the next two years. This information will be collected by the researcher at your hospital who will review your clinical notes and imaging studies that you may undergo during that period of time. By consenting to the study, you give us permission to use the proposed scoring system to evaluate your scan and then to collect information about what happens to you after the scan for a period of two years. The research team will compare the Adnex-MR score with the outcome of your surgery or any follow-up appointments 6. What do I have to do? Should you become a participant in the study, then you will be required to attend your scheduled visit for MRI. If you decide to take part in this study, you should not be involved in any other imaging trial at the same time. No other study procedure is required. However, we will follow your progress to record whether you have surgery or not and record the results of any surgery or follow-up for a two year period. 7. Will my taking part in this study be kept confidential? Yes. All information which is collected about you during the course of the research will be kept strictly confidential. If you consent to take part in the research, the people conducting the study will abide by the Data Protection Act 1988, and the rights you have under this Act. If you join the study, some parts of your medical records and the data collected for the study will be looked at by authorised personnel from your treating centre and authorised personnel from <<Name of Hospital taking part in the study>>. It may also be looked at by representatives of regulatory authorities and other authorised personnel from your trust, to check that the study is being carried out correctly. All will have a duty of confidentiality to you as a research participant and nothing that could reveal your identity will be disclosed outside the research site. The information that we will collect for the purposes of this study will be entered on a paper form on which your name will be anonymised, although at each centre, the research team will securely store information that will be able to link you to the anonymisation number. The anonymised data will be entered on an electronic database. MRI scans will be reported on the clinical system normally. An anonymised copy of the scan will be stored on CD for two radiologists to score your scan, in order to test the reproducibility of the score between different radiologists. These radiologists may be © 2020 Thomassin Naggara I et al. JAMA Network Open. other investigators in the UK and in Europe as part of the multicentre collaboration. All data will be stored in a locked and dedicated room at <<Imperial College Healthcare NHS Trust>> which will only be accessible by authorised personnel. 8. Expenses and payments If you choose to participate in this research study, you will not have any additional visits to the hospital other than those required for your routine care. Unfortunately, we are unable to reimburse any travel costs that may be incurred for attending your MRI scan. If this is a problem for you, then please discuss this with your study doctor. 9. What are the other possible disadvantages and risks of taking part? There is no specific risk related to taking part in this study, as the study focuses on testing a diagnostic classification of the standard MRI scan that your doctor has requested. You will not undergo any different or additional scan or test for the purposes of the study. The hospital department which will undertake the MRI scan performs this type of scan routinely. 10. Risks to an Unborn Child If at the time of any imaging procedures you suspect you may be pregnant, then you must inform your doctor, who will advise and help you. An MRI scan may be done in pregnancy but the liquid dye that is used as standard (gadolinium) would not be used and you would not be eligible to take part in the study. 11. What happens if there is a problem? We would not expect you to suffer any harm or injury because of your participation in this study, as the study does not involve any change from your normal care. Regardless of this, if you wish to complain or have any concerns about any aspect of the way you have been approached or treated during the course of this study, the normal National Health Service complaints mechanisms should be available to you. Please contact Patient Advisory Liaison Service (PALS) if you have any concerns regarding the care you have received, or as an initial point of contact if you have a complaint. Please telephone <<Insert PALS Telephone Number>> or email <<Insert PALS Email>> you can also visit PALS by asking at any hospital reception. 12. What if relevant new information becomes available? Sometimes during the course of a research project new information becomes available about the diagnostic test being studied. This is very unlikely to happen in this type of study. However, if this happens, your consultant will tell you about it and discuss whether you want to take part in the study. If you decide to withdraw, your study doctor will make arrangements for your care to continue. 13. What will happen if I don’t want to carry on with the study? You are free to withdraw from the study at any time and do not have to give a reason. Your future treatment will not be affected and your doctor will discuss this with you. We would like your permission to continue to receive information on your progress. If © 2020 Thomassin Naggara I et al. JAMA Network Open. you decide that we may have no further information from you for the study, we will need to use the data collected up to the time of your withdrawal. 14. What are the possible benefits of taking part? You may not benefit directly by taking part in this trial. However, it is hoped that the information from this study will allow doctors in the future to give better information to patients with this kind of problem allowing them to make informed choices regarding their care, 15. What happens when the research study stops? Following completion of your MRI scan, you will have ended your active study participation. We will then monitor your treatment and follow-up either until you have surgery for the adnexal mass or, if you do not have surgery, we will monitor any further imaging tests that you have of the adnexal mass, such as future ultrasound or MRI scans. The reason for this follow-up monitoring is to see whether the scoring classification was correct or whether it was wrong. When the results of the study are available a summary can be provided to you upon request. 16. Who is organising and funding the research and where was it reviewed? This is an investigator-initiated study. {Dr Isabelle Thomassin-Naggara of Hopital is the Chief Investigator}. Imperial College Healthcare NHS Trust is sponsoring the study. The doctors and other members of the clinical research teams are not being paid for participating in this study. 17. Who has reviewed the research? This study has been through a peer review process. A peer review involves the examination of an author’s work by other experts in the same field. These referees each return an evaluation of the work which may include suggestions of improvements if necessary. Your local NHS Trust has been given approval for the study to take place at your hospital. The study has also been reviewed by a Research Ethics Committee. 18. Who can I contact for further information? For questions about the study you may contact your research doctor or nurse, the contact details are below: Dr <<Insert Doctor Name>> Telephone: <<Insert Doctor Telephone Number>> Or; Research Nurse <<Insert Research Nurse Name>> © 2020 Thomassin Naggara I et al. JAMA Network Open. Telephone: <<Insert Research Nurse Telephone Number>> For information about your disease: ??Ovarian cancer helpline? CancerHelp UK provides general information for patients about cancer and its treatment on their website, www.cancerhelp.org.uk. The information can also be available over the phone by contacting a specialist research nurse on the 020 7061 8355 or free-phone 0808 800 4040. We are Macmillan provides support and counselling to help people living with cancer, information can be found on their website, www.macmillan.org.uk, or by free- phone on 0808 808 2020 © 2020 Thomassin Naggara I et al. JAMA Network Open. eAppendix 3. Population Selection Criteria Inclusion Criteria Patient 18 years old Sonographically indeterminate adnexal mass Informed consent Exclusion Criteria Pregnancy (relative contraindication for gadolinium injection) Pacemaker, ferromagnetic materials, or foreign body at risk of mobilization or any other contraindication to MR imaging. Intolerance to gadolinium contrast agents, or severe renal insufficiency (GFR <30 ml/min/1.73m²). Ultrasound findings (Quoted 7 points in all criteria) (950/1194 (79.6%) patients had a score 5/7) 1 Side 2 Size 3 Origin 4 Presence of solid component 5 Ipsilateral ovarian parenchyma 6 Positive Doppler 7 Pelvic Fluid © 2020 Thomassin Naggara I et al. JAMA Network Open. eAppendix 4. Methodological Details 2A : Technical MR acquisition Each patient underwent a routine pelvic MR imaging (1.5T or 3T), including morphological sequences (T2, T1 with and without fat suppression and T1 after gadolinium injection) and functional sequences (perfusion and diffusion weighted sequences). If at the time of inclusion and MRI scan, the adnexal mass has disappeared on T2 and T1 weighted sequences, functional sequences and gadolinium injection were not mandatory. Each centre used functional sequences with the following criteria: DCE MR sequence: 3D isotropic, Delay between the beginning of the sequence and injection: 1 minute, Total duration after injection: 3min, Slice 3mm no gap (1 slice on 2 with the exact same location than T2), Spatial resolution: 3mm, Temporal resolution < 15s, Box size: 15cm DWI sequence : Exactly same slice thickness as the T2, b1000 b1200 (to ensure that urine in bladder is black) Technical requirements for DCE MR sequence were checked at a preliminary period by an MR scientist. 2B Quality Assurance: The first patient from each centre was sent to main investigator to check if the acquisition protocol was correct especially regarding technical requirements for DCE MR sequence and DW sequence (supplementary material 1). Subsequent quality assurance was then undertaken by the site PI to ensure on going adherence to the study imaging protocol. 2C Reader training: During study set up, a session of 30 anonymized DICOM cases (previously acquired before the beginning of the study) were downloaded for a training session to learn how to apply the MR score for all teams participating in the multicentre validation. Readers were trained in using the standardized lexicon (suppl. table 2) for interpretation. © 2020 Thomassin Naggara I et al. JAMA Network Open. eTable 1. Lexicon and Supplementary Material with Detailed Analysis of Adnexal Lesions Based on Morphological Characteristics Terms and Definitions Per lesion study results (n=1502) Adnexal origin Adnexal origin corresponds to all lesion developed on ovary, tube or mesosalpinx. Adnexal origin was described in 91.3% (1372/1502). Twelve adnexal masses were misclassified as non adnexal (8 benign masses classified as score 1 non suspicious and 4 malignant tuboovarian masses rated as score 1 suspicious). Ten (83.3%) of these 12 misclassified adnexal masses were larger than 5cm. Twenty eight non adnexal masses misclassified as adnexal masses. Eighteen non adnexal masses were rated score 2 or 3 and all corresponded to benign masses. Nine non adnexal masses were rated as adnexal masses scored 4 or 5 and corresponded to 5 extra adnexal cancers, 3 uterine leiomyoma and one schwanoma. Purely cystic mass Purely cystic mass was described in 27.4% (411/1502) and Absence of internal enhancement after were all benign apart from one serous carcinoma injection and corresponding to a (however, due to peritoneal carcinomatosis, this patient unilocular cyst or hydrosalpinx, both of was assigned MR score 5). This group corresponds mainly which have low T1 weighted and high T2 to serous cystadenoma (n=148), functional cyst (n=134), weighted MR signal intensity of internal pelvic inflammatory disease or hydrosalpinx (n=48), cyst content peritoneal cyst (n=27), paraovarian or paratubal cyst (n=16), others (n=38). Purely endometriotic mass Purely endometriotic mass was described in 12.4% Lesion displaying high T1 weighted signal (187/1502) and were all benign apart from one borderline intensity to subcutaneous fat, shading serous tumour. This description was mainly used for on T2 weighted MR sequence and no endometrioma (n=154) or endometriotic hematosalpinx internal enhancement (n=6). Other diagnoses were functional cyst (n=15), pelvic inflammatory disease or hydrosalpinx (n=7), others (n=5) Fatty mass without enhanced component Purely fatty mass was described in 6.4% (97/1502) and all Lesion displaying high T1 weighted signal corresponded to benign lesions with mature cystic intensity that disappeared after fat teratoma in 95.9% (93/97). The four other lesions were saturation and potentially displaying non serous cystadenoma, functional cyst and indeterminate enhancing solid component due to adnexal torsion. Wall enhancement No wall enhancement was reported in 22.6% (339/1502) Enhancement of the wall of a cyst and were all benign lesions except one serous borderline tumour (the same false negative described in purely endometriotic group). This feature was described mainly in endometrioma (n=112), mature cystic teratoma (n=55), functional cyst (n=53), serous benign cystadenoma (n=50) and peritoneal cyst (n=22). © 2020 Thomassin Naggara I et al. JAMA Network Open. Bi or multilocularity Bi or multiloculate cyst without solid tissue was A cyst that has two or more septa. A described in 12.7% (191/1502) of adnexal masses, septum is defined as a thin strand of including only five malignant tumours (one tubal cancer, tissue running across the cyst cavity from two borderline mucinous tumours, one invasive mucinous one internal surface to the contra lateral carcinoma, and one peritoneal carcinoma). Thickened side smooth septa were present in three malignant epithelial Grouped septa tumours but also described in 57 benign lesions (PPV = A cyst contains grouped septae if three or 5%). In bi or multiloculate cysts with thin smooth septa more septae are close together in one and without solid tissue (n=131), the PPV for malignancy part of the cyst was 1.5% (2/131). Thickened regular septa A smooth septation with a thickness 3mm within a cystic tissue. Solid tissue No Solid tissue (n=911, 60.6%) was reported in 902 benign A solid tissue enhances after gadolinium lesions and 9 malignant tumours. Solid tissue was present injection. In adnexal tumours, diffuse wall in 591 tumours including 323 benign and 268 malignant thickening, normal ovarian stroma and tumours. Solid papillary projections were described in 170 regular septa are not regarded as solid tumours (86% were epithelial tumours at final diagnosis tissue according to IOTA group (13). Thus, (146/170)) with a PPV of malignancy of 55.8% (95/170) solid tissue is either thickened irregular (including 35 borderline and 60 invasive). Mixed masses septa, and/or vegetation and/or solid were described in 257 tumours with a PPV of malignancy portion (including completely solid mass). of 62.6% (161/257). Purely solid masses were described in Solid papillary projections are 211 tumours with a PPV of malignancy of 32.3% (66/211). defined as any solid projections into the Thickened irregular septa were described in 74 masses cyst from the cyst wall with height 3mm with a PPV of malignancy of 66.2% (49/74). Benign Mixed or purely solid mass is defined as tumours with thickened irregular septa (n=25) were any solid tissue which is not a wall, a mainly epithelial tumours (n=10), mature cystic teratoma septum or a vegetation. This group (n=5), pelvic inflammatory disease or hydrosalpinx (n=5) comprises completely solid masses Thickened irregular septa Focal areas of septal thickening with a thickness 3 mm within a cystic tissue. T2 weighted signal intensity within solid Solid tissue with low T2 and low DW signal intensity was tissue present in only two malignant tumours (one borderline Signal intensity defined in comparison serous tumour and one metastasis) (false negative rate with adjacent external myometrium 1.9% (2/105)). However, when solid tissue was low only on T2W sequence, there were 19 malignant tumours b weighted signal intensity within (false negative rate 20.8% (19/91)) and when solid tissue solid tissue was low only on DW sequence, there were 10 malignant Signal intensity defined in comparison with serous fluid (i.e., urine in bladder or tumours (false negative rate 18.8% (10/53)) . cerebrospinal fluid [CSF]) Time Type 1: A gradual increase in When solid tissue enhanced according a time intensity intensity the signal of the solid tissue, curve type 3, malignant tumours were encountered in curve without a well defined 85.6% (143/167). False positives were 10 uterine within shoulder leiomyomas, five mature cystic teratomas, two Brenner solid Type 2: A moderate initial tumours, two cystadenofibromas, one functional cyst, one tissue rise in the signal of solid schwannoma, one struma ovarii, one benign tissue relative to that of cystadenoma and one ovarian fibroma. Invasive myometrium malignant tumours enhanced according a time intensity Type 3: An initial rise in the curve type 3 in 59.4% (138/232), type 2 in 32.3% (75/232), signal of solid tissue that was type 1 in 2.1% (5/232). In the fourteen remaining invasive steeper than that of adnexal tumours, time intensity curve was not feasible in myometrium eight cases and no solid tissue was detected in six. © 2020 Thomassin Naggara I et al. JAMA Network Open. Terms and Definitions Per patient study results (n=1130) Peritoneal implants Peritoneal implants were described in 72/1130 patients Nodular thickening of the peritoneum that with four false positive cases (two pelvic inflammatory enhances after gadolinium injection disease, one functional cyst and one leiomyoma) (LR+ = 77.63 95% CI =28.65 210.37). Pelvic fluid Pelvic fluid was described in 433/1130 patients (38.3%) Presence of fluid in peritoneal cavity including 287 with benign and 146 with malignant lesions (LR+ = 2.04 95% CI ). When there was =2.04 2.64 abdominopelvic fluid (4.6%, n=52/1130), all lesions were malignant apart from four (two serous cystadenomas, one pelvic tuberculosis and one ovarian fibroma) (LR+= 54.8, 95% CI =19.99 150.25). © 2020 Thomassin Naggara I et al. JAMA Network Open. eTable 2. Diagnostic Values of Different MRI Features LR+ LR- Prevalence Rate of False negatives Kappa (experienced/junior) (n=1502) malignancy (PPV ) Purely cystic 0.01 1.50 27.4% 0.2% (1) 1 serous carcinoma* 0.73 (0-0.08) (1.44- ( 0.68-0.77) mass (n=411) 1.56) Purely 0.02 1.17 12.4% 0.5% (1) 1 borderline serous 0.82 (0-0.17) (1.15- (0.78-0.87) endometriotic tumour^ 0.20) mass (n=187) Purely fatty 0 1.08 0 0% - 0.72 (1.07- (0.65-0.79) mass (n=97) 1.10) Absence of 0.01 1.38 22.6% 0.3% (1) 1 borderline serous 0.43 (0-0.09) (1.33- (0.39-0.47) wall tumour^ 1.43) enhancement (n=339) Bi- or 0.12 1.16 12.7% 2.6% (5) 0.63 (0.05- (1.13- (0.59-0.67) multiloculate 0.29) 1.19) cyst without solid tissue 2 borderline (n=191) mucinous tumour** Thick 4% 5% (3) 1 invasive mucinous 0.23 1.04 septa carcinoma** (0.07- (1.02- (n=60) 0.74) 1.06) 1 tubal cancer 0.07 1.11 8.7% 1 peritoneal Thin septa 1.5% (2) (0.02- (1.09- carcinoma (n=131) 0.28) 1.13) Absence of 0.04 3.67 60.6% 1% (9) 3 borderline tumours 0.81 (0.02- (3.33- (0.78-0.84) solid tissue 3 ovarian carcinoma 0.08) 4.04) (n=911) 2 tubal carcinoma 1 peritoneal carcinoma Solid tissue 0.09 1.08 7% 1.9% (2) 1 borderline serous 0.70 (0.02- (1.06- (0.664-0.735) with lowT2 tumour 0.35) 1.11) and low DW 1 ovarian metastasis signal (n=105) Solid tissue 0.32 1.11 11.9% 6.7% (12) 7 borderline 0.66 (0.18- (1.07- (0.617-0.71) with TIC cystadenoma 0.56) 1.15) type 1 (n=180) 1 ovarian cystadenocarcinoma 1 Granulosa 3 ovarian metastasis Solid tissue 3.9 0.69 14.8% 46.6% (3.08- (0.63- with TIC (104) 4.86) 0.76) type 2 (n=223) Solid tissue 26.3 0.49 11.1% 85.6% (17.4- (0.44- with TIC (143) 39.7) 0.56) type 3 (n=167) * because of peritoneal carcinomatosis, this patient was assigned O-RADS Score 5 **Displays thickened regular septa (3 malignant tumours of the 60 lesions with thickened smooth septa) (95%CI ) ^ these were the same lesions © 2020 Thomassin Naggara I et al. JAMA Network Open. eFigure. Score Performance Among Senior and Junior Readers © 2020 Thomassin Naggara I et al. JAMA Network Open.
JAMA Network Open – American Medical Association
Published: Jan 24, 2020
You can share this free article with as many people as you like with the url below! We hope you enjoy this feature!
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.