2D CNN versus 3D CNN for false-positive reduction in lung cancer screeningYu, Juezhao; Yang, Bohan; Wang, Jing; Leader, Joseph; Wilson, David; Pu, Jiantao
2020 Journal of Medical Imaging
doi: 10.1117/1.JMI.7.5.051202pmid: 33062802
Abstract.Purpose: To clarify whether and to what extent three-dimensional (3D) convolutional neural network (CNN) is superior to 2D CNN when applied to reduce false-positive nodule detections in the scenario of low-dose computed tomography (CT) lung cancer screening.Approach: We established a dataset consisting of 1600 chest CT examinations acquired on different subjects from various sources. There were in total 18,280 candidate nodules in these CT examinations, among which 9185 were nodules and 9095 were not nodules. For each candidate nodule, we extracted a number of cubic subvolumes with a dimension of 72 × 72 × 72 mm3 by rotating the CT examinations randomly for 25 times prior to the extraction of the axis-aligned subvolumes. These subvolumes were split into three groups in a ratio of 8 ∶ 1 ∶ 1 for training, validation, and independent testing purposes. We developed a multiscale CNN architecture and implemented its 2D and 3D versions to classify pulmonary nodules into two categories, namely true positive and false positive. The performance of the 2D/3D-CNN classification schemes was evaluated using the area under the receiver operating characteristic curves (AUC). The p-values and the 95% confidence intervals (CI) were calculated.Results: The AUC for the optimal 2D-CNN model is 0.9307 (95% CI: 0.9285 to 0.9330) with a sensitivity of 92.70% and a specificity of 76.21%. The 3D-CNN model with the best performance had an AUC of 0.9541 (95% CI: 0.9495 to 0.9583) with a sensitivity of 89.98% and a specificity of 87.30%. The developed multiscale CNN architecture had a better performance than the vanilla architecture did.Conclusions: The 3D-CNN model has a better performance in false-positive reduction compared with its 2D counterpart; however, the improvement is relatively limited and demands more computational resources for training purposes.
Eye tracking reveals expertise-related differences in the time-course of medical image inspection and diagnosisBrunyé, Tad T.; Drew, Trafton; Kerr, Kathleen F.; Shucard, Hannah; Weaver, Donald L.; Elmore, Joann G.
2020 Journal of Medical Imaging
doi: 10.1117/1.JMI.7.5.051203pmid: 37476351
Abstract.Purpose: Physicians’ eye movements provide insights into relative reliance on different visual features during medical image review and diagnosis. Current theories posit that increasing expertise is associated with relatively holistic viewing strategies activated early in the image viewing experience. This study examined whether early image viewing behavior is associated with experience level and diagnostic accuracy when pathologists and trainees interpreted breast biopsies.Approach: Ninety-two residents in training and experienced pathologists at nine major U.S. medical centers interpreted digitized whole slide images of breast biopsy cases while eye movements were monitored. The breadth of visual attention and frequency and duration of eye fixations on critical image regions were recorded. We dissociated eye movements occurring early during initial viewing (prior to first zoom) versus later viewing, examining seven viewing behaviors of interest.Results: Residents and faculty pathologists were similarly likely to detect critical image regions during early image viewing, but faculty members showed more and longer duration eye fixations in these regions. Among pathology residents, year of residency predicted increasingly higher odds of fixating on critical image regions during early viewing. No viewing behavior was significantly associated with diagnostic accuracy.Conclusions: Results suggest early detection and recognition of critical image features by experienced pathologists, with relatively directed and efficient search behavior. The results also suggest that the immediate distribution of eye movements over medical images warrants further exploration as a potential metric for the objective monitoring and evaluation of progress during medical training.
Overcoming calcium blooming and improving the quantification accuracy of percent area luminal stenosis by material decomposition of multi-energy computed tomography datasetsLi, Zhoubo; Leng, Shuai; Halaweish, Ahmed F.; Yu, Zhicong; Yu, Lifeng; Ritman, Erik L.; McCollough, Cynthia H.
2020 Journal of Medical Imaging
doi: 10.1117/1.JMI.7.5.053501pmid: 33033732
Abstract.Purpose: Conventional stenosis quantification from single-energy computed tomography (SECT) images relies on segmentation of lumen boundaries, which suffers from partial volume averaging and calcium blooming effects. We present and evaluate a method for quantifying percent area stenosis using multienergy CT (MECT) images.Approach: We utilize material decomposition of MECT images to measure stenosis based on the ratio of iodine mass between vessel locations with and without a stenosis, thereby eliminating the requirement for segmentation of iodinated lumen. The method was first assessed using simulated MECT images created with different spatial resolutions. To experimentally assess this method, four phantoms with different stenosis severity (30% to 51%), vessel diameters (5.5 to 14 mm), and calcification densities (700 to 1100 mgHA / cc) were fabricated. Conventional SECT images were acquired using a commercial CT system and were analyzed with commercial software. MECT images were acquired using a commercial dual-energy CT (DECT) system and also from a research photon-counting detector CT (PCD-CT) system. Three-material-decomposition was performed on MECT data, and iodine density maps were used to quantify stenosis. Clinical radiation doses were used for all data acquisitions.Results: Computer simulation verified that this method reduced partial volume and blooming effects, resulting in consistent stenosis measurements. Phantom experiments showed accurate and reproducible stenosis measurements from MECT images. For DECT and two-threshold PCD-CT images, the estimation errors were 4.0% to 7.0%, 2.0% to 9.0%, 10.0% to 18.0%, and −1.0 % to −5.0 % (ground truth: 51%, 51%, 51%, and 30%). For four-threshold PCD-CT images, the errors were 1.0% to 3.0%, 4.0% to 6.0%, −1.0 % to 9.0%, and 0.0% to 6.0%. Errors using SECT were much larger, ranging from 4.4% to 46%, and were especially worse in the presence of dense calcifications.Conclusions: The proposed approach was shown to be insensitive to acquisition parameters, demonstrating the potential to improve the accuracy and precision of stenosis measurements in clinical practice.
Optimization-based algorithm for solving the discrete x-ray transform with nonlinear partial volume effectChen, Buxin; Liu, Xin; Zhang, Zheng; Xia, Dan; Sidky, Emil Y.; Pan, Xiaochuan
2020 Journal of Medical Imaging
doi: 10.1117/1.JMI.7.5.053502pmid: 33033733
Abstract.Purpose: Inverting the discrete x-ray transform (DXT) with the nonlinear partial volume (NLPV) effect, which we refer to as the NLPV DXT, remains of theoretical and practical interest. We propose an optimization-based algorithm for accurately and directly inverting the NLPV DXT.Methods: Formulating the inversion of the NLPV DXT as a nonconvex optimization program, we propose an iterative algorithm, referred to as the nonconvex primal-dual (NCPD) algorithm, to solve the problem. We obtain the NCPD algorithm by modifying a first-order primal-dual algorithm to address the nonconvex optimization. Subsequently, we perform quantitative studies to verify and characterize the NCPD algorithm.Results: In addition to proposing the NCPD algorithm, we perform numerical studies to verify that the NCPD algorithm can reach the devised numerically necessary convergence conditions and, under the study conditions considered, invert the NLPV DXT by yielding numerically accurate image reconstruction.Conclusion: We have developed and verified with numerical studies the NCPD algorithm for accurate inversion of the NLPV DXT. The study and results may yield insights into the effective compensation for the NLPV artifacts in CT imaging and into the algorithm development for nonconvex optimization programs in CT and other tomographic imaging technologies.
Silicon photon-counting detector for full-field CT using an ASIC with adjustable shaping timeSundberg, Christel; Persson, Mats; Sjölin, Martin; Wikner, J. Jacob; Danielsson, Mats
2020 Journal of Medical Imaging
doi: 10.1117/1.JMI.7.5.053503pmid: 33033734
Abstract.Purpose: Photon-counting silicon strip detectors are attracting interest for use in next-generation CT scanners. For CT detectors in a clinical environment, it is desirable to have a low power consumption. However, decreasing the power consumption leads to higher noise. This is particularly detrimental for silicon detectors, which require a low noise floor to obtain a good dose efficiency. The increase in noise can be mitigated using a longer shaping time in the readout electronics. This also results in longer pulses, which requires an increased deadtime, thereby degrading the count-rate performance. However, as the photon flux varies greatly during a typical CT scan, not all projection lines require a high count-rate capability. We propose adjusting the shaping time to counteract the increased noise that results from decreasing the power consumption.Approach: To show the potential of increasing the shaping time to decrease the noise level, synchrotron measurements were performed using a detector prototype with two shaping time settings. From the measurements, a simulation model was developed and used to predict the performance of a future channel design.Results: Based on the synchrotron measurements, we show that increasing the shaping time from 28.1 to 39.4 ns decreases the noise and increases the signal-to-noise ratio with 6.5% at low count rates. With the developed simulation model, we predict that a 50% decrease in power can be attained in a proposed future detector design by increasing the shaping time with a factor of 1.875.Conclusion: Our results show that the shaping time can be an important tool to adapt the pulse length and noise level to the photon flux and thereby optimize the dose efficiency of photon-counting silicon detectors.
Radio-pathomic mapping model generated using annotations from five pathologists reliably distinguishes high-grade prostate cancerMcGarry, Sean D.; Bukowy, John D.; Iczkowski, Kenneth A.; Lowman, Allison K.; Brehler, Michael; Bobholz, Samuel; Nencka, Andrew; Barrington, Alex; Jacobsohn, Kenneth; Unteriner, Jackson; Duvnjak, Petar; Griffin, Michael; Hohenwalter, Mark; Keuter, Tucker; Huang, Wei; Antic, Tatjana; Paner, Gladell; Palangmonthip, Watchareepohn; Banerjee, Anjishnu; LaViolette, Peter S.
2020 Journal of Medical Imaging
doi: 10.1117/1.JMI.7.5.054501pmid: 32923510
Abstract.Purpose: Our study predictively maps epithelium density in magnetic resonance imaging (MRI) space while varying the ground truth labels provided by five pathologists to quantify the downstream effects of interobserver variability.Approach: Clinical imaging and postsurgical tissue from 48 recruited prospective patients were used in our study. Tissue was sliced to match the MRI orientation and whole-mount slides were stained and digitized. Data from 28 patients (n = 33 slides) were sent to five pathologists to be annotated. Slides from the remaining 20 patients (n = 123 slides) were annotated by one of the five pathologists. Interpathologist variability was measured using Krippendorff’s alpha. Pathologist-specific radiopathomic mapping models were trained using a partial least-squares regression using MRI values to predict epithelium density, a known marker for disease severity. An analysis of variance characterized intermodel means difference in epithelium density. A consensus model was created and evaluated using a receiver operator characteristic classifying high grade versus low grade and benign, and was statistically compared to apparent diffusion coefficient (ADC).Results: Interobserver variability ranged from low to acceptable agreement (0.31 to 0.69). There was a statistically significant difference in mean predicted epithelium density values (p < 0.001) between the five models. The consensus model outperformed ADC (areas under the curve = 0.80 and 0.71, respectively, p < 0.05).Conclusion: We demonstrate that radiopathomic maps of epithelium density are sensitive to the pathologist annotating the dataset; however, it is unclear if these differences are clinically significant. The consensus model produced the best maps, matched the performance of the best individual model, and outperformed ADC.
Algorithm to quantify nuclear features and confidence intervals for classification of oral neoplasia from high-resolution optical imagesYang, Eric C.; Brenes, David R.; Vohra, Imran S.; Schwarz, Richard A.; Williams, Michelle D.; Vigneswaran, Nadarajah; Gillenwater, Ann M.; Richards-Kortum, Rebecca R.
2020 Journal of Medical Imaging
doi: 10.1117/1.JMI.7.5.054502pmid: 32999894
Abstract.Purpose:In vivo optical imaging technologies like high-resolution microendoscopy (HRME) can image nuclei of the oral epithelium. In principle, automated algorithms can then calculate nuclear features to distinguish neoplastic from benign tissue. However, images frequently contain regions without visible nuclei, due to biological and technical factors, decreasing the data available to and accuracy of image analysis algorithms.Approach: We developed the nuclear density-confidence interval (ND-CI) algorithm to determine if an HRME image contains sufficient nuclei for classification, or if a better image is required. The algorithm uses a convolutional neural network to exclude image regions without visible nuclei. Then the remaining regions are used to estimate a confidence interval (CI) for the number of abnormal nuclei per mm2, a feature used by a previously developed algorithm (called the ND algorithm), to classify images as benign or neoplastic. The range of the CI determines whether the ND-CI algorithm can classify an image with confidence, and if so, the predicted category. The ND and ND-CI algorithm were compared by calculating their positive predictive value (PPV) and negative predictive value (NPV) on 82 oral biopsies with histopathologically confirmed diagnoses.Results: After excluding the images that could not be classified with confidence, the ND-CI algorithm had higher PPV (65% versus 59%) and NPV (78% versus 75%) than the ND algorithm.Conclusions: The ND-CI algorithm could improve the real-time classification of HRME images of the oral epithelium by informing the user if an improved image is required for diagnosis.
Whole mammographic mass segmentation using attention mechanism and multiscale pooling adversarial networkWang, Yuehang; Wang, Shengsheng; Chen, Juan; Wu, Chun
2020 Journal of Medical Imaging
doi: 10.1117/1.JMI.7.5.054503pmid: 33102621
Abstract.Purpose: Since breast mass is a clear sign of breast cancer, its precise segmentation is of great significance for the diagnosis of breast cancer. However, the current diagnosis relies mainly on radiologists who spend time extracting features manually, which inevitably reduces the efficiency of diagnosis. Therefore, designing an automatic segmentation method is urgently necessary for the accurate segmentation of breast masses.Approach: We propose an effective attention mechanism and multiscale pooling conditional generative adversarial network (AM-MSP-cGAN), which accurately achieves mass automatic segmentation in whole mammograms. In AM-MSP-cGAN, U-Net is utilized as a generator network by incorporating attention mechanism (AM) into it, which allows U-Net to pay more attention to the target mass regions without additional cost. As a discriminator network, a convolutional neural network with multiscale pooling module is used to learn more meticulous features from the masses with rough and fuzzy boundaries. The proposed model is trained and tested on two public datasets: CBIS-DDSM and INbreast.Results: Compared with other state-of-the-art methods, the AM-MSP-cGAN can achieve better segmentation results in terms of the dice similarity coefficient (Dice) and Hausdorff distance metrics, achieving top scores of 84.49% and 5.01 on CBIS-DDSM, and 83.92% and 5.81 on INbreast, respectively. Therefore, qualitative and quantitative experiments illustrate that the proposed model is effective and robust for the mass segmentation in whole mammograms.Conclusions: The proposed deep learning model is suitable for the automatic segmentation of breast masses, which provides technical assistance for subsequent pathological structure analysis.
Automatic multi-organ segmentation in computed tomography images using hierarchical convolutional neural networkSultana, Sharmin; Robinson, Adam; Song, Daniel Y.; Lee, Junghoon
2020 Journal of Medical Imaging
doi: 10.1117/1.JMI.7.5.055001pmid: 33102622
Abstract.Purpose: Accurate segmentation of treatment planning computed tomography (CT) images is important for radiation therapy (RT) planning. However, low soft tissue contrast in CT makes the segmentation task challenging. We propose a two-step hierarchical convolutional neural network (CNN) segmentation strategy to automatically segment multiple organs from CT.Approach: The first step generates a coarse segmentation from which organ-specific regions of interest (ROIs) are produced. The second step produces detailed segmentation of each organ. The ROIs are generated using UNet, which automatically identifies the area of each organ and improves computational efficiency by eliminating irrelevant background information. For the fine segmentation step, we combined UNet with a generative adversarial network. The generator is designed as a UNet that is trained to segment organ structures and the discriminator is a fully convolutional network, which distinguishes whether the segmentation is real or generator-predicted, thus improving the segmentation accuracy. We validated the proposed method on male pelvic and head and neck (H&N) CTs used for RT planning of prostate and H&N cancer, respectively. For the pelvic structure segmentation, the network was trained to segment the prostate, bladder, and rectum. For H&N, the network was trained to segment the parotid glands (PG) and submandibular glands (SMG).Results: The trained segmentation networks were tested on 15 pelvic and 20 H&N independent datasets. The H&N segmentation network was also tested on a public domain dataset (N = 38) and showed similar performance. The average dice similarity coefficients (mean ± SD) of pelvic structures are 0.91 ± 0.05 (prostate), 0.95 ± 0.06 (bladder), 0.90 ± 0.09 (rectum), and H&N structures are 0.87 ± 0.04 (PG) and 0.86 ± 0.05 (SMG). The segmentation for each CT takes <10 s on average.Conclusions: Experimental results demonstrate that the proposed method can produce fast, accurate, and reproducible segmentation of multiple organs of different sizes and shapes and show its potential to be applicable to different disease sites.