Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/american-journal-of-neuroradiology/interrater-agreement-and-detection-accuracy-for-medium-vessel-0J8fH9Ceve
References
References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.
- Publisher
- American Journal of Neuroradiology
- Copyright
- © 2022 by American Journal of Neuroradiology. Indicates open access to non-subscribers at www.ajnr.org
- ISSN
- 0195-6108
- eISSN
- 1936-959X
- DOI
- 10.3174/ajnr.a7361
- Publisher site
- See Article on Publisher Site
Abstract
ORIGINAL RESEARCH INTERVENTIONAL Interrater Agreement and Detection Accuracy for Medium- Vessel Occlusions Using Single-Phase and Multiphase CT Angiography J.M. Ospel, F. Bala, R.V. McDonough, O. Volny, N. Kashani, W. Qiu, B.K. Menon, and M. Goyal ABSTRACT BACKGROUND AND PURPOSE: Accurate and reliable detection of medium-vessel occlusions is important to establish the diagnosis of acute ischemic stroke and initiate appropriate treatment with intravenous thrombolysis or endovascular thrombectomy. However, medium-vessel occlusions are often challenging to detect, especially for unexperienced readers. We aimed to evaluate the accuracy and interrater agreement of the detection of medium-vessel occlusions using single-phase and multiphase CTA. MATERIALS AND METHODS: Single-phase and multiphase CTA of 120 patients with acute ischemic stroke (20 with no occlusion, 44 with large-vessel occlusion, and 56 with medium-vessel occlusion in the anterior and posterior circulation) were assessed by 3 read- ers with varying levels of experience (session 1: single-phase CTA; session 2: multiphase CTA). Interrater agreement for occlusion type (large-vessel occlusion versus medium-vessel occlusion versus no occlusion) and for detailed occlusion sites was calculated using the Fleiss k with 95% confidence intervals. Accuracy for the detection of medium-vessel occlusions was calculated for each reader using classification tables. RESULTS: Interrater agreement for occlusion type was moderate for single-phase CTA (k ¼ 0.58; 95% CI, 0.56–0.62) and almost per- fect for multiphase CTA (k ¼ 0.81; 95% CI, 0.78–0.83). Interrater agreement for detailed occlusion sites was moderate for single- phase CTA (k ¼ 0.55; 95% CI, 0.53–0.56) and substantial for multiphase CTA (k ¼ 0.71; 95% CI, 0.67–0.74). On single-phase CTA, readers 1, 2, and 3 classified 33/56 (59%), 34/56 (61%), and 32/56 (57%) correctly as medium-vessel occlusions. On multiphase CTA, 48/56 (86%), 50/56 (89%), and 50/56 (89%) medium-vessel occlusions were classified correctly. CONCLUSIONS: Interrater agreement for medium-vessel occlusions is moderate when using single-phase CTA and almost perfect with multiphase CTA. Detection accuracy is substantially higher with multiphase CTA compared with single-phase CTA, suggesting that multiphase CTA might be a valuable tool for assessment of medium-vessel occlusion stroke. ABBREVIATIONS: EVT ¼ endovascular treatment; LVO ¼ large-vessel occlusion; mCTA ¼ multiphase CTA; MeVO ¼ medium-vessel occlusion edium-vessel occlusions (MeVOs) are defined as occlu- segments of the posterior cerebral artery. MeVOs account for Msions of the M2 and M3/4 segments of the MCA, A2 and approximately 25%–40% of all acute ischemic strokes, and A3/4 segments of the anterior cerebral artery, and P2 and P3/4 although they have traditionally been thought to cause only minor symptoms, MeVO strokes frequently result in disabling 2,3 deficits. This outcome has led to a paradigm shift in treatment: Received August 10, 2021; accepted after revision September 22. An increasing number of physicians now routinely offers endo- From the Department of Diagnostic Imaging (J.M.O., F.B., R.V.M., N.K., W.Q., B.K.M., vascular treatment (EVT) as a stand-alone therapy instead of or M.G.), and Department of Clinical Neurosciences (N.K., B.K.M., M.G.), University of 4-6 Calgary, Calgary, Alberta, Canada; Department of Radiology (J.M.O.), University in addition to intravenous thrombolysis for MeVO stroke. Hospital of Basel, Basel, Switzerland; Czech National Centre for Evidence-Based Furthermore, 2 randomized trials, ESCAPE-MeVO and DISTAL, Healthcare and Knowledge Translation (O.V.), Masaryk University, Brno, Czech; and Department of Neurology (O.V.), University Hospital Ostrava, Ostrava, Czech. are being planned to generate level 1A evidence for EVT in This study was funded, in part, by the Canadian Institutes of Health Research. MeVO stroke (personal communication: Mayank Goyal, Marios- Please address correspondence to Mayank Goyal, MD, Departments of Diagnostic Nikos Psychogios, oral communication). The prerequisite for Imaging and Clinical Neurosciences, Foothills Medical Centre, 1403 29th St NW, conducting these trials and for appropriate treatment of MeVOs University of Calgary, Calgary, Alberta, Canada, T2N2T9; e-mail: mgoyal@ucalgary.ca; @johanna_ospel; @rosevmcd; @mayank_G0 is accurate and reliable MeVO detection and distinction between Indicates open access to non-subscribers at www.ajnr.org MeVOs and large-vessel occlusions (LVOs). However, MeVOs are missed in up to one-third of cases. Furthermore, distinguish- Indicates article with online supplemental data. http://dx.doi.org/10.3174/ajnr.A7361 ing MeVOs from LVOs is not always straightforward because AJNR Am J Neuroradiol 43:93–97 Jan 2022 www.ajnr.org 93 various definitions for the borders between the M1 and M2 seg- Image Analysis ments exist. The same holds true for the anterior and posterior Three blinded readers (a general radiologist with 4 years of expe- cerebral arteries. Multiphase CTA (mCTA) is a dynamic imaging rience and 2 neuroradiologists with 7 and 2 years of experience) method in which an arch-to-vertex CTA is obtained in a manner interpreted the images in 2 separate reading sessions. In the first identical to single-phase CTA. The same contrast bolus is then session, only single-phase CTA (ie, the first CTA phase with used to obtain 2 additional series during the peak-venous and arch-to-vertex coverage) was available. In the second session, all late-venous phases, covering only the intracranial vasculature 3 mCTA phases were available. Readers were informed that the from the skull base to the vertex (Online Supplemental Data). set of cases included patients with LVO and MeVO in the ante- The delayed washout on the second and third phases in the vas- rior and posterior circulation as well as cases without any occlu- cular territory downstream to the occlusion may help to improve sion, but they did not know the distribution of occlusions and detection accuracy and interrater agreement for MeVOs. occlusion locations. They were blinded to all other baseline imag- In this study, we aimed to compare interrater agreement and ing, follow-up imaging, and clinical information. Occlusion sites detection accuracy for MeVOs using single-phase CTA and were captured as either no occlusion; intracranial internal carotid mCTA. artery occlusion; M1, M2, or M3/4 segment MCA occlusion; A1, A2, or A3/4 segment anterior cerebral artery occlusion; or P1, P2, or P3/4 segment posterior cerebral artery occlusion. Detailed def- MATERIALS AND METHODS initions of vessel segments and borders between them as they Patient Sample were used in this study are shown in the Online Supplemental This study was approved by the local institutional review board at Data. Besides these detailed occlusion sites, occlusions were also the University of Calgary. Data are from the Precise and Rapid grouped into no occlusion versus LVOs (occlusions of the ICA, Assessment of Collaterals Using Multi-Phase CTA in the Triage M1, A1, or P1 segments) versus MeVOs (occlusions of the M2/3/ of Patients with Acute Ischemic Stroke for IV or IA Therapy 4, A2/3/4, or P2/3/4 segments). The reference standard was set by (PRove-IT) study (clinicaltrials.gov identifier: NCT02184936). an independent core lab (M.G., interventional neuroradiologist PRove-IT was a prospective multicenter cohort study that en- with 24 years of neuroimaging experience), with all imaging in- rolled 595 patients who presented with symptoms of acute ische- formation (baseline noncontrast head CT, mCTA, CT perfusion, mic stroke. Patients were included if they presented to the and follow-up imaging) as well as clinical information being emergency department with symptoms consistent with acute is- available. chemic stroke, were older than 18 years of age, and had mCTA and CT perfusion performed within 12 hours of symptom onset Statistical Analysis and before recanalization therapy. Exclusion criteria were intra- Interrater agreement for occlusion type (LVO versus MeVO ver- cranial hemorrhage at baseline NCCT, previous sizeable stroke in sus no occlusion) and for detailed occlusion sites was assessed the ipsilateral hemisphere, mRS. 2 at baseline, estimated creati- using the Fleiss k for multiple raters with respective optimism- nine clearance of ,60 mL/min, contrast material allergy or other corrected bootstrapped 95% confidence intervals. According to contraindications for iodinated contrast, and estimated life common convention, k , 0 was interpreted as poor agreement; expectancy of ,1year. The enrollment period was July 2012 to k ¼ 0–0.2, as slight agreement; k ¼ 0.21–0.40, as fair agreement; October 2016. For this study, a sample of 120 patients (20 with k ¼ 0.41–0.60, as moderate agreement; k ¼ 0.61–0.80, as sub- no occlusion, 44 with LVO, and 56 with MeVO in the anterior stantial agreement; and k ¼ 0.81–1, as almost perfect agree- and posterior circulation) was randomly chosen. The proportions ment. The accuracy of MeVO detection overall and for each of patients with MeVOs in relation to LVOs and scans without MeVO occlusion site was calculated for each reader using classifi- occlusions were chosen so that they approximately reflected the cation tables. All analyses were performed in STATA, Version distribution of occlusions in the PRove-IT study. 15.1 (StataCorp). RESULTS Imaging A total of 120 cases were included in the study, among them 20 We exclusively used baseline CTA imaging in this study. The first with no visible occlusion, 44 LVOs (22 intracranial ICA occlu- CTA phase consisted of an arch-to-vertex coverage (CTA head sions, 2 vertebral artery occlusions, 1 basilar artery occlusion, 18 and neck and conventional single-phase CTA). This first phase M1 occlusions, and 1 P1 occlusion), and 56 MeVOs (21 M2 was used for the single-phase CTA reading session and was fol- occlusions, 12 M3/4 occlusions, 5 A2 occlusions, 4 A3/4 occlu- lowed by skull base-to-vertex coverage for the second (peak ve- sions, and 14 P2 occlusions). nous) and third (late venous) phases of an mCTA acquisition. All 3 phases were available in the mCTA reading session. Detailed mCTA acquisition parameters have been published previously. Interrater Agreement Axial images with 1-mm overlap and multiplanar axial, coronal, Interrater agreement for occlusion type was moderate for single- and sagittal reconstructions with 3-mm thickness, 1-mm inter- phase CTA (k ¼ 0.58; 95% CI, 0.56–0.62) and significantly vals, and 1-mm overlap for the first phase were obtained, along improved to excellent agreement when mCTA was used (k ¼ 0.81; with axial minimum intensity projections for all 3 phases with 95% CI, 0.78–0.83). Interrater agreement for detailed occlusion sites 24-mm thickness and 4-mm intervals, available for the readers. (see the Online Supplemental Data for detailed definitions) was 94 Ospel Jan 2022 www.ajnr.org FIGURE. Proportion of MeVOs that were correctly classified as MeVOs (green), misclassified as LVOs (yellow), and misclassified as no occlusion (red) on single-phase and multiphase CTA, stratified per occlusion site. A, Proportions for M2, M3/4, A2, and A3/4 MeVOs. B, Proportions for P2 MeVOs. sp-CTA indicates single-phase CT angiography. moderate for single-phase CTA (k ¼ 0.55; 95% CI, 0.53–0.56) and occlusion on single-phase and multiphase CTA, stratified per substantial for mCTA (k ¼ 0.71; 95% CI, 0.67–0.74). occlusion site. Accuracy of MeVO Detection DISCUSSION Classification tables for the 3 readers on single-phase CTA and Interrater agreement for occlusion sites in this study was mod- mCTA are shown in the Online Supplemental Data. On single- erate when using single-phase CTA and almost perfect with phase CTA, readers 1, 2, and 3 classified 33/56 (59%), 34/56 mCTA. The accuracy of MeVO detection for all 3 readers ranged (61%), and 32/56 (57%) MeVOs correctly as such. Most of the between 57% and 61% on single-phase CTA and improved to misclassified MeVOs on single-phase CTA (21/23 [91%], 16/22 86%–89% with mCTA. [73%], and 22/24 [92%]) were erroneously classified as “no occlu- In the past, it was thought that MeVO strokes resulted in rela- sion,” while only very few were mistaken for LVOs. On mCTA, tively favorable clinical outcomes due to their more distal occlusion 48/56 (86%), 50/56 (89%), and 50/56 (89%) MeVOs were classi- location and smaller ischemic tissue volume compared with LVOs. fied correctly, and most misclassified MeVOs (6/8 [75%], 4/6 However, data from large, prospective cohort studies and a pooled [67%], and 5/6 [83%]) were misclassified as no occlusion. The meta-analysis from randomized trials showed that only half of the Figure shows the proportion of MeVOs that were correctly classi- fied as MeVOs, misclassified as LVOs, and misclassified as no patients with MeVOs achieve a good functional outcome at AJNR Am J Neuroradiol 43:93–97 Jan 2022 www.ajnr.org 95 12-14 90days with currentbestmedical management. Together with posterior cerebral arteries, in which anatomy is even more the high efficacy and safety of EVT in LVO stroke, this finding has variable and includes variants such as an azygos anterior cere- 4-6 bral artery and fetal origin of the posterior cerebral artery. led many physicians to routinely offer EVT for MeVO stroke. Nevertheless, this variability simply reflects the clinical reality, To offer the appropriate treatment, it is, however, necessary that and some disagreement in distinguishing MeVOs versus LVOs is, MeVOs are quickly and reliably identified on imaging. Missing a therefore, expected. Second, we chose to compare single-phase MeVO on baseline imaging can lead to a delayed diagnosis or mis- CTA with mCTA because they are closely related in their acquisi- diagnosis, which may result in a patient not receiving intravenous tion technique, have identical contrast doses and comparable thrombolysis or not getting transferred to a comprehensive stroke radiation doses, and can be performed without any postprocess- center for EVT. It may also cause misclassification of the patient’s ing software. We did not include other imaging modalities such symptoms as a stroke mimic and, as a consequence, failure to initi- as CT perfusion or MR imaging and are, therefore, not able to ate appropriate stroke work-up and secondary stroke-prevention comment on the value of these imaging modalities for MeVO measures. Unfortunately, recent data suggest that this scenario is assessment. Third, not all MeVO sites were represented in this not an uncommon one: Fasen et al retrospectively reviewed 520 study; for example, we could not include any cases of P3 occlu- single-phase CTA studies of patients with clinically confirmed acute sion simply because they did not occur in our dataset. Fourth, the ischemic stroke and found that M2 occlusions, which are arguably readers in this study relied exclusively on single-phase CTA and themost easily detectableMeVOs, were 5 times more likely to be mCTA to assess occlusion sites and did not have access to any overlooked compared with LVOs and were missed in up to one- other imaging or clinical information, while in a real-world sce- third of cases. Misses were most common when non-neuroradiolo- nario, the reader will almost always have access to clinical infor- mation and baseline noncontrast CT. However, one could argue gists and trainees interpreted the scans. that access to this information may, if anything, have improved We have recently commented on the potential value of mCTA the readers’ performance. Last, distinguishing between MeVOs in improving MeVO detection. Indeed, our study showed an excellent interrater agreement and substantially higher detection and LVOs, which was part of the current study, while being crit- ically important for randomized MeVO trials, may be only of accuracy for MeVOs with mCTA compared with single-phase limited usefulness in clinical practice once the safety and efficacy CTA, suggesting that mCTA could be a valuable tool for MeVO of MeVO EVT has been proven. detection. Innovative mCTA display formats, such as color-coded time-variant mCTA maps or mCTA tissue-level perfusion maps, may be able to facilitate MeVO detection even further, but they are CONCLUSIONS 15,16 not widely available yet. Interrater agreement for MeVOs is moderate when using single- Given the complexity in the anatomic definition of MeVOs phase CTA and almost perfect with mCTA. The accuracy of (for example, there are different ways to define the border 8 MeVO detection is higher with mCTA compared with single- between the M1 and M2 segments of the MCA ), one may sus- phase CTA, suggesting that mCTA might be a valuable tool that pect that some MeVOs will be misclassified as LVOs. This is par- allows reliable diagnosis of MeVO stroke. ticularly problematic because establishing high-level evidence for MeVO EVT in randomized trials requires an accurate, reliable, ACKNOWLEDGMENTS and reproducible MeVO definition as part of the trial inclusion The authors are most grateful to all enrolling sites. criteria. However, in our study, most misclassified MeVOs were not mistaken for LVOs but rather erroneously classified as no Disclosure forms provided by the authors are available with the full text and occlusion, suggesting that the problem of distinguishing MeVOs PDF of this article at www.ajnr.org. from LVOs is only a minor one and confirming that the MeVO definition that was previously suggested by our group and has been used in the current study could potentially be used as a REFERENCES randomized trial inclusion criterion. Furthermore, detection of 1. Goyal M, Ospel JM, Menon BK, et al. MeVO: the next frontier? J occlusion of any kind will lead to the correct diagnosis of acute is- Neurointerv Surg 2020;12:545–47 CrossRef Medline chemic stroke and trigger appropriate treatment with IV throm- 2. Duloquin G, Graber M, Garnier L, et al. Incidence of acute ischemic bolysis and potentially EVT; in other words, mistaking an MeVO stroke with visible arterial occlusion: a population-based study as an LVO is not as fatal as missing it altogether. (Dijon Stroke Registry). Stroke 2020;51:2122–30 CrossRef Medline The added radiation dose of 1.0 mSv for the 2 additional 3. Saver JL, Chapot R, Agid R, et al; Distal Thrombectomy Summit Group. Thrombectomy for distal, medium vessel occlusions: a con- phases that are obtained in multiphase CTA slightly increases the sensus statement on present knowledge and promising directions. mean effective dose for a CT-based acute ischemic stroke imaging Stroke 2020;51:2872–84 CrossRef Medline protocol from 7.0 to 8.0 mSv, and it is generally deemed accepta- 4. Almekhlafi M, Ospel JM, Saposnik G, et al. Endovascular treatment ble, given the severity of the condition. decisions in patients with M2 segment MCA occlusions. AJNR Am JNeuroradiol 2020;41:280–85 CrossRef Medline Limitations 5. Cimflova P, Kappelhof M, Singh N, et al. Factors influencing throm- This study has limitations. First, there is controversy about the bectomy decision making for primary medium vessel occlusion exact anatomic definition of the M1 versus the M2 segment, and stroke. J Neurointerv Surg 2021 May 5 [Epub ahead of print] the situation is arguably even more complex in the anterior and CrossRef Medline 96 Ospel Jan 2022 www.ajnr.org 6. Kappelhof M, Ospel J, Kashani N, et al. Influence of intravenous 12. Lima FO, Furie KL, Silva GS, et al. Prognosis of untreated strokes alteplase on endovascular treatment decision-making in acute is- due to anterior circulation proximal intracranial arterial occlu- chemic stroke due to primary medium-vessel occlusion: a case- sions detected by use of computed tomography angiography. based survey study. J Neurointerv Surg 2021 May 25 [Epub ahead of JAMA Neurol 2014;71:151–57 CrossRef Medline print] CrossRef Medline 13. Menon BK, Hill MD, Davalos A, et al. Efficacy of endovascular 7. Fasen B, Heijboer RJJ, Hulsmans FH, et al. CT angiography in evalu- thrombectomy in patients with M2 segment middle cerebral artery ating large-vessel occlusion in acute anterior circulation ischemic occlusions: meta-analysis of data from the HERMES Collaboration. stroke: factors associated with diagnostic error in clinical practice. J Neurointerv Surg 2019;11:1065–69 CrossRef Medline AJNR Am J Neuroradiol 2020;41:607–11 CrossRef Medline 14. Ospel JM, Menon BK, Demchuk AM, et al. Clinical course of acute is- 8. Goyal M, Menon BK, Krings T, et al. What constitutes the M1 seg- chemic stroke due to medium vessel occlusion with and without intra- ment of the middle cerebral artery? J Neurointerv Surg 2016;8:1273– venous alteplase treatment. Stroke 2020;51:3232–40 CrossRef Medline 77 CrossRef Medline 15. Ospel JM, Qiu W, Goyal M. Missed medium-vessel occlusions on 9. Menon BK, d’Esterre CD, Qazi EM, et al. Multiphase CT angiogra- CT angiography: make it easier ... easily! AJNR Am J Neuroradiol phy: a new tool for the imaging triage of patients with acute ische- 2020;41:E73–74 CrossRef Medline mic stroke. Radiology 2015;275:510–20 CrossRef Medline 16. Ospel JM, Volny O, Qiu W, et al. Displaying multiphase CT angiog- 10. Fleiss JL, Cohen J. The equivalence of weighted kappa and the intra- raphy using a time-variant color map: practical considerations and class correlation coefficient as measures of reliability. Educational potential applications in patients with acute stroke. AJNR Am J and Psychological Measurement 1973;33:613–19 CrossRef Neuroradiol 2020;41:200–05 CrossRef Medline 11. Hartling LH, Milne A. Validity and inter-rater reliability testing of 17. van der Zwan A, Hillen B, Tulleken CA, et al. Variability of the terr- quality assessment instruments. https://www.ncbi.nlm.nih.gov/ ritories of the major cerebral arteries. J Neurosurg 1992;77:927–40 books/NBK92295/table/methods.t2/. Accessed August 9, 2021 CrossRef Medline AJNR Am J Neuroradiol 43:93–97 Jan 2022 www.ajnr.org 97
Journal
American Journal of Neuroradiology – American Journal of Neuroradiology
Published: Jan 1, 2022