Image Updating for Brain Shift Compensation During Resection

Image Updating for Brain Shift Compensation During Resection Abstract BACKGROUND In open-cranial neurosurgery, preoperative magnetic resonance (pMR) images are typically coregistered for intraoperative guidance. Their accuracy can be significantly degraded by intraoperative brain deformation, especially when resection is involved. OBJECTIVE To produce model updated MR (uMR) images to compensate for brain shift that occurred during resection, and evaluate the performance of the image-updating process in terms of accuracy and computational efficiency. METHODS In 14 resection cases, intraoperative stereovision image pairs were acquired after dural opening and during resection to generate displacement maps of the surgical field. These data were assimilated by a biomechanical model to create uMR volumes of the evolving surgical field. A tracked stylus provided independent measurements of feature locations to quantify target registration errors (TREs) in the original coregistered pMR and uMR as surgery progressed. RESULTS Updated MR TREs were 1.66 ± 0.27 and 1.92 ± 0.49 mm in the 14 cases after dural opening and after partial resection, respectively, compared to 8.48 ± 3.74 and 8.77 ± 4.61 mm for pMR, respectively. The overall computational time for generating uMRs after partial resection was less than 10 min. CONCLUSION We have developed an image-updating system to compensate for brain deformation during resection using a computational model with data assimilation of displacements measured with intraoperative stereovision imaging that maintains TREs less than 2 mm on average. Brain deformation, FEM model, Image-guided neurosurgery, Intraoperative stereovision, Optical flow, Resection, Sparse data ABBREVIATIONS ABBREVIATIONS 3-D 3-dimensional CCD charge-coupled device FEM finite element model iMR intraoperative magnetic resonance iSV intraoperative stereovision iUS intraoperative ultrasound OF optical flow OR operating room pMR preoperative magnetic resonance POC percentage of correction TRE target registration error uMR updated magnetic resonance Preoperative magnetic resonance (pMR) images are typically used for intraoperative image guidance in open cranial surgery. Commercial navigation systems are available to rigidly align pMR with the patient's head in the operating room (OR),1 and provide real-time navigation with an average accuracy of 2 to 5 mm.2 However, coregistration can be significantly degraded by nonrigid brain deformation, especially as surgery progresses. To maintain accuracy, image-updating systems have been developed to generate updated MR (uMR) images using data-driven biomechanical models.3-13 Compared to intraoperative MR (iMR), model-based image updating is cost-effective and minimally disruptive to surgical workflow. As a result, the image-updating process can be repeated multiple times throughout a procedure, unlike iMR, which is typically acquired only at the presumed end of resection, because of the time and disruption involved. In a previous work,14 we produced uMRs in the OR to compensate for brain deformation but only after dural opening. Sun et al15 used laser range scanning to account for postresection brain shift, but tumor volume was not removed to represent extent of resection. In this paper, we report extensions of our image-updating process that enable multiple (second and beyond) uMRs to be generated by incorporating optical flow (OF) motion tracking of successive intraoperative stereovision (iSV) acquisitions during resection in 14 surgeries. We also assess the accuracy of uMRs with independent measurements of feature locations in the surgical field with a tracked stylus, allowing target registration error (TRE) estimates of multiple uMRs to be generated for the first time. We report the overall computational efficiency of the processes involved. Results show that registration accuracy is significantly improved over pMR with computational efficiency that can be maintained within clinically acceptable ranges. TABLE 1. Summary of Patient Information Patient  Gender  Age  Lesion  Location  Craniotomy size (cm × cm)  Registration accuracy (mm)  1  F  39  Arteriovenous Malformation  Right frontal  4.9 × 4.8  1.72  2  M  41  Oligodendroglioma  Right temporal  5.3 × 5.0  2.11  3  F  61  Meningioma  Right parietal  7.3 × 5.3  2.14  4  F  53  Epilepsy  Left temporal  7.6 × 5.6  0.98  5  F  24  Mixed oligoastrocytoma  Right parietal  6.2 × 4.4  1.18  6  F  49  Metastatic melanoma  Right temporal  3.1 × 2.6  2.38  7  M  78  Metastatic neuroendocrine carcinoma  Right occipital  7.8 × 5.8  4.82  8  F  65  Meningioma  Right temporal  5.0 × 3.9  1.84  9  M  60  Glioblastoma  Left temporal  4.6 × 3.6  1.49  10  F  51  Meningioma  Left frontal  7.8 × 5.4  1.94  11  F  59  Anaplastic mixed oligoastrocytoma  Right frontal  4.7 × 3.8  2.01  12  F  46  Meningioma  Right parietal  5.3 × 4.8  1.40  13  M  75  Glioblastoma  Left temporal  7.4 × 4.6  1.40  14  M  40  Meningioma  Left frontal  7.8 × 5.7  1.98  Patient  Gender  Age  Lesion  Location  Craniotomy size (cm × cm)  Registration accuracy (mm)  1  F  39  Arteriovenous Malformation  Right frontal  4.9 × 4.8  1.72  2  M  41  Oligodendroglioma  Right temporal  5.3 × 5.0  2.11  3  F  61  Meningioma  Right parietal  7.3 × 5.3  2.14  4  F  53  Epilepsy  Left temporal  7.6 × 5.6  0.98  5  F  24  Mixed oligoastrocytoma  Right parietal  6.2 × 4.4  1.18  6  F  49  Metastatic melanoma  Right temporal  3.1 × 2.6  2.38  7  M  78  Metastatic neuroendocrine carcinoma  Right occipital  7.8 × 5.8  4.82  8  F  65  Meningioma  Right temporal  5.0 × 3.9  1.84  9  M  60  Glioblastoma  Left temporal  4.6 × 3.6  1.49  10  F  51  Meningioma  Left frontal  7.8 × 5.4  1.94  11  F  59  Anaplastic mixed oligoastrocytoma  Right frontal  4.7 × 3.8  2.01  12  F  46  Meningioma  Right parietal  5.3 × 4.8  1.40  13  M  75  Glioblastoma  Left temporal  7.4 × 4.6  1.40  14  M  40  Meningioma  Left frontal  7.8 × 5.7  1.98  View Large METHODS Surgical Cases and Procedure Image data from 14 patients undergoing open cranial procedures were evaluated, retrospectively. The study was approved by the Institutional Review Board and patient consent was obtained. Surgical procedures included craniotomies for tumor, epilepsy, or arteriovenous malformation. Criteria for data inclusion were (1) T1-weighted contrast-enhanced MRI acquired prior to surgery (scan size = 256 × 256, 96–160 slices, voxel size = 0.9375 mm × 0.9375 mm × 1.5 mm); and (2) common features, such as blood vessels and sulci, visible on both MR scans and the exposed cortical surface. Subject gender, age, type of lesion, and size of craniotomy are reported in Table 1. At the time of surgery, patient registration was performed with pMR on a commercial navigation system (StealthStation® S7®, Medtronic, Louisville, Colorado) to provide standard-of-care intraoperative navigation and tracking. The reported accuracy of patient registration was 1.96 mm on average and is listed in Table 1 for each individual case. A surgical microscope (OPMI Pentero® Carl Zeiss Surgical GmbH, Oberkochen, Germany) was connected to the StealthStation® for intraoperative tracking, and an iSV system (Figure 1) was attached to and draped together with the microscope. The iSV system consisted of 2 charge-coupled device (CCD) cameras (Flea2 FL2G-50S5C, Point Grey Research, Inc, Richmond, British Columbia, Canada; image resolution: 1024 × 768 pixels), and was precalibrated16-18 and coregistered with pMR using tracking data acquired from the StealthStation® through the Medtronic StealthLink® (Medtronic, Louisville, Colorado) communications framework. FIGURE 1. View largeDownload slide The iSV system (red box). Two CCD cameras (yellow arrow) were connected to an adapter that was attached to one of the optical ports on the microscope head. FIGURE 1. View largeDownload slide The iSV system (red box). Two CCD cameras (yellow arrow) were connected to an adapter that was attached to one of the optical ports on the microscope head. MR Image-Updating Procedure A flowchart of the model-based image-updating process is shown in Figure 2. First, uMR was produced to compensate for brain deformation due to dural opening.14 Specifically, the brain was segmented from pMR using an adaptive preterm segmentation algorithm.19 After dural opening, an iSV image pair was acquired and a 3-dimensional (3-D) profile of the exposed cortical surface was reconstructed (iSV1). A tetrahedral mesh (mesh0) was generated with local refinement near the craniotomy, whose location and size were determined from iSV1. Surface displacements (sparse data1) were extracted by registering iSV1 with pMR,20 and employed to drive a biomechanical finite element model (FEM), in which the misfit between model estimates and measured displacements was minimized.21 The FEM model computed a whole-brain deformation field, in the form of the 3-D displacements of each node in the brain mesh. This information was then used to compute the new locations of each voxel in pMR and produce the updated MR (uMR1). FIGURE 2. View largeDownload slide Flowchart of the MR image-updating process. FIGURE 2. View largeDownload slide Flowchart of the MR image-updating process. During resection, a second iSV image pair was acquired to reconstruct a 3-D surface (iSV2) of the current surgical field. OF registration was applied to register iSV1 with iSV2 to measure surface shift (sparse data2).22 The amount of tissue removal was then estimated using an approach previously described in detail.23 Specifically, the resection cavity on iSV2 provided an initial estimate of tissue removal relative to uMR1, a brain mesh (mesh1) with a resection cavity was created, and the FEM model was executed. If the resulting brain deformation caused the resection cavity to deform further, and thus, no longer be aligned with iSV2, a revised resection cavity was created from iSV2 by shifting it in the opposite direction by the average model-estimated cavity shift. A new brain mesh (mesh2) with the revised cavity was created, and the FEM model was executed again and the model-estimated whole-brain deformation was used to deform uMR1 and produce uMR2. Data Analysis To assess accuracy, uMR1 and uMR2 were first visually compared with the corresponding iSV surfaces in both 2-D and 3-D views. Furthermore, we quantified the misfit and TRE for pMR, uMR1, and uMR2, respectively. Specifically, the misfit of uMR1 and uMR2 was computed as the root-mean-squared error between measured displacements (determined by surface registration) and model estimates at the same locations (uMR1-iSV1 and uMR2-iSV2, respectively). The pMR-iSV1 misfit was computed as the average measured displacements between iSV1 and pMR, ie, sparse data1, and the uMR1-iSV2 misfit was computed as the average measured displacements between iSV1 and iSV2, ie, sparse data2. In addition, we analyzed the displacements to study the patterns of shift due to tissue removal by decomposing the 3-D displacements (sparse data2) into their components along and perpendicular to the surface normal, and along the direction of gravity. The percentage of correction (POC) of uMR1 (POC1) was calculated as (1 – uMR1 misfit/pMR misfit) × 100%, and the POC of uMR2 (POC2) was calculated as (1 – uMR2 misfit/uMR1-iSV2 misfit) × 100%. To quantify the TRE of pMR and uMRs, we used features in the surgical field as target points. The technical details have been published previously.14 Specifically, at the time of iSV1 acquisition, the neurosurgeon used a tracked sterile stylus probe to locate features that were visible on both the exposed cortical surface and MR scans, such as vessel junctions, and the coordinates of these locations in MR image space (through patient registration) were stored. The same features were localized on pMR and uMR1, respectively. The same process was repeated at the time of iSV2 acquisition and features were localized on pMR, uMR1, and uMR2. TRE was computed as the distance between the intraoperatively tracked locations of these features (transformed into MR space) and their corresponding positions in the pMR or uMR image volumes. RESULTS Model-Updated MR Accuracy Evaluation Figures 3 to 6 illustrate the qualitative evaluation of accuracy in 4 representative cases. Misalignments between pMR and both iSV1 and iSV2 indicate that pMR was not accurate at either time point. Specifically, iSV surfaces were either above or underneath the brain surface in 2-D views (image A), indicating brain bulging or sagging, and cortical features (eg, white arrows in image B) from iSV1 were not aligned with pMR in 3-D views, denoting lateral shift. After the first model update, uMR1 and iSV1 (images C and D) were well aligned in terms of both geometry (shown in 2-D) and texture (shown in 3-D), indicating favorable accuracy. After partial resection, uMR1 was no longer accurate when compared with iSV2 (image C), while uMR2 (images E and F) aligned well with iSV2 in terms of both geometry (shown in 2-D) and texture (shown in 3-D) after a second model update. Patient 11 had a third iSV acquisition after more tissue was removed, and uMR2 served as the input for repeating the image-updating process and producing uMR3. Figures 6G and 6H show that uMR3 aligned well with iSV3. FIGURE 3. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 1. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 3. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 1. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 4. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 10. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 4. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 10. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 5. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 14. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 5. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 14. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 6. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 11. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1, iSV2 and iSV3 (yellow, red and green lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrows point to features misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1, iSV2 and iSV3 (yellow, red, and green lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrows point to features accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrows point to features accurately aligned. G, Two-dimensional view of uMR3 at the same coronal slice location, overlaid with iSV3 (red line). H, Three-dimensional view of uMR3 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 6. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 11. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1, iSV2 and iSV3 (yellow, red and green lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrows point to features misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1, iSV2 and iSV3 (yellow, red, and green lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrows point to features accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrows point to features accurately aligned. G, Two-dimensional view of uMR3 at the same coronal slice location, overlaid with iSV3 (red line). H, Three-dimensional view of uMR3 overlaid with iSV2, white arrow points to feature accurately aligned. Quantitative accuracy assessments are reported in Tables 2 (misfit) and 3 (TRE). The initial misfit between pMR and iSV1 is reported in column 2, along with its direction along the surface normal (“+” and “–” signs for outward and inward shift, respectively). The results show that the initial misfit was 8.61 ± 3.83 mm across the 14 cases, ranging from 2.72 to 16.91 mm (column 2). After model compensation, the remaining misfit of uMR1 was 1.95 ± 0.45 mm (column 3), and the corresponding POC of the first image update was 74% ± 11% (POC1, column 4). The misfit between uMR1 and iSV2 is reported in column 5, along with its direction along the surface normal. The results show an average shift of 8.55 ± 4.38 mm as a result of tissue removal, ranging from 3.16 to 16.89 mm, and the direction of cortical shift was inward (indicating brain collapse) in all 14 cases. Their components along the surface normal, in the lateral direction, and along gravity are reported in columns 6 to 8, respectively. After a second update, image updating accounted for 71% ± 14% (POC2; column 10) of the brain shift and the remaining misfit of uMR2 was 2.08 ± 0.56 mm (column 9). The time from dural opening to partial resection (ie, the time between iSV acquisitions) is reported in the last column. Similarly, TREs of distinct features on the exposed surgical surface are reported in Table 3. The average TRE after the first image update was 1.66 ± 0.27 mm (uMR1-iSV1; column 3), relative to 8.48 ± 3.74 mm for pMR (pMR-iSV1; column 2). After the second image update, the average TRE was 1.92 ± 0.49 mm (uMR2-iSV2; column 6), relative to 8.77 ± 4.61 mm, if the first, but not second image update occurred (uMR1-iSV2; column 5). The average TRE for pMR (pMR-iSV2; column 4) was 8.57 ± 4.67 mm. Computational Efficiency The computational time of first image update was 7 to 8 min on average as reported previously.14 Here, we present the time involved in generating the second update. Reconstruction of the iSV surface was completed in ∼6 to 7 s with graphics processing unit acceleration. The OF-based iSV–iSV surface registration was finished within 20 s, and voxels within the cavity were removed from uMR1 within 5 s. Meshing was performed within 10 s, and model computation was finished in ∼4 min. The resection cavity was then revised within 5 s, the model computation was repeated in ∼4 min, and uMR1 was warped to produce uMR2 within 15 s. Thus, the total computational time of the second image update was within 10 min. DISCUSSION Preoperative MR images are conventionally used to intraoperatively guide resection. Results from 14 clinical cases show that pMR was inaccurate as soon as the dura was opened, and continued to be inaccurate during resection. Specifically, an error of >5 mm was observed in 12 of 14 surgeries (misfit, Table 2 column 2; and TRE, Table 3 column 2) after dural opening, and in 9 out of the 14 cases after partial resection (TRE, Table 3 column 4), respectively. Updated MR images produced after dural opening (uMR1) proved to be more accurate with an average error of <2 mm (misfit, Table 2 column 3; and TRE, Table 3 column 3) and agree with results from a previous study.14 However, uMR1 became inaccurate after tissue was removed, and further shift >5 mm (since dural opening) was observed in 10 of 14 cases (misfit, Table 2 column 5; and TRE, Table 3 column 5). After the second image update, uMR2 proved to be more accurate than both pMR and uMR1 with average errors ∼2 mm (misfit, Table 2 column 9; and TRE, Table 3 column 6). The visual comparison between pMR/iMR and iSV does have an inherent bias because the images were clearly identifiable as either preoperative or updated, and thus, the comparison cannot be performed in a blinded fashion. The TRE was quantified by touching features on the cortical surface with a tracked stylus and identifying them in the MR texture map, and is, therefore, subject to human and system localization and tracking errors. The quantification of model-data misfit, however, was automatic and performed free of user input, and independently of any visual comparisons. TABLE 2. Summary of Misfit of pMR and uMR in 14 Patient Cases   After dural opening  After partial resection    pMR  uMR1    uMR1  Normal  Lateral  Gravity  uMR2    Time  Patient  (mm)  (mm)  POC1  (mm)  (mm)  (mm)  (mm)  (mm)  POC2  (h:min)  1  6.07 ± 0.91(–)  1.96  68%  3.16 ± 1.02(–)  2.76 ± 1.32  1.20 ± 0.48  2.89 ± 1.09  1.49  53%  1:56  2  11.04 ± 0.85(+)  2.19  80%  11.69 ± 2.58(–)  11.15 ± 2.42  3.30 ± 1.47  9.33 ± 1.95  1.65  86%  2:32  3  7.25 ± 2.24(+)  1.62  78%  5.65 ± 1.48(–)  3.71 ± 1.83  4.06 ± 0.70  4.73 ± 1.42  2.03  64%  0:59  4  4.19 ± 0.69(–)  1.58  62%  8.30 ± 2.05(–)  4.37 ± 1.24  6.67 ± 2.80  6.40 ± 1.67  2.98  64%  2:16  5  13.98 ± 3.58(+)  1.81  87%  3.43 ± 1.19(–)  2.45 ± 0.88  2.20 ± 1.26  2.17 ± 0.83  1.25  64%  2:48  6  5.73 ± 0.81(+)  2.26  61%  12.83 ± 0.96(–)  11.61 ± 1.69  4.74 ± 2.29  12.30 ± 0.91  2.23  83%  1:25  7  16.91 ± 2.45(+)  2.59  85%  10.90 ± 2.92(–)  10.45 ± 3.08  2.80 ± 0.97  9.96 ± 2.13  2.43  78%  1:27  8  2.72 ± 1.02(–)  1.35  50%  14.11 ± 2.21(–)  11.86 ± 2.30  7.37 ± 1.89  13.85 ± 2.21  2.86  80%  2:17  9  6.94 ± 1.36(+)  1.75  75%  6.23 ± 0.97(–)  5.75 ± 0.86  2.29 ± 0.80  5.65 ± 0.75  1.50  76%  1:24  10  6.47 ± 0.87(–)  1.42  78%  11.22 ± 2.10(–)  7.39 ± 1.89  8.19 ± 2.22  9.78 ± 2.31  2.13  81%  2:25  11  10.12 ± 1.34(+)  1.51  85%  10.69 ± 1.39(–)  10.55 ± 1.45  1.49 ± 0.86  8.82 ± 1.31  1.94  82%  0:38    –  –  –  3.55 ± 0.53(–)  1.54 ± 0.22  3.20 ± 0.49  1.56 ± 0.19  1.28  64%  1:04a  12  9.25 ± 1.13(+)  2.69  71%  4.92 ± 1.57(–)  2.65 ± 1.56  3.68 ± 1.91  2.38 ± 1.88  2.80  43%  2:25  13  8.75 ± 2.12(+)  2.52  71%  16.89 ± 3.05(-)  15.66 ± 2.85  6.07 ± 2.15  16.12 ± 3.01  2.43  86%  2:34  14  11.12 ± 2.04(+)  2.06  81%  4.66 ± 0.74(–)  3.63 ± 0.96  2.57 ± 1.26  3.66 ± 0.78  2.14  54%  1:15  Average  8.61 ± 3.83  1.95 ± 0.45  74% ± 11%  8.55 ± 4.38  7.04 ± 4.47  3.99 ± 2.17  7.31 ± 4.56  2.08 ± 0.56  71% ± 14%      After dural opening  After partial resection    pMR  uMR1    uMR1  Normal  Lateral  Gravity  uMR2    Time  Patient  (mm)  (mm)  POC1  (mm)  (mm)  (mm)  (mm)  (mm)  POC2  (h:min)  1  6.07 ± 0.91(–)  1.96  68%  3.16 ± 1.02(–)  2.76 ± 1.32  1.20 ± 0.48  2.89 ± 1.09  1.49  53%  1:56  2  11.04 ± 0.85(+)  2.19  80%  11.69 ± 2.58(–)  11.15 ± 2.42  3.30 ± 1.47  9.33 ± 1.95  1.65  86%  2:32  3  7.25 ± 2.24(+)  1.62  78%  5.65 ± 1.48(–)  3.71 ± 1.83  4.06 ± 0.70  4.73 ± 1.42  2.03  64%  0:59  4  4.19 ± 0.69(–)  1.58  62%  8.30 ± 2.05(–)  4.37 ± 1.24  6.67 ± 2.80  6.40 ± 1.67  2.98  64%  2:16  5  13.98 ± 3.58(+)  1.81  87%  3.43 ± 1.19(–)  2.45 ± 0.88  2.20 ± 1.26  2.17 ± 0.83  1.25  64%  2:48  6  5.73 ± 0.81(+)  2.26  61%  12.83 ± 0.96(–)  11.61 ± 1.69  4.74 ± 2.29  12.30 ± 0.91  2.23  83%  1:25  7  16.91 ± 2.45(+)  2.59  85%  10.90 ± 2.92(–)  10.45 ± 3.08  2.80 ± 0.97  9.96 ± 2.13  2.43  78%  1:27  8  2.72 ± 1.02(–)  1.35  50%  14.11 ± 2.21(–)  11.86 ± 2.30  7.37 ± 1.89  13.85 ± 2.21  2.86  80%  2:17  9  6.94 ± 1.36(+)  1.75  75%  6.23 ± 0.97(–)  5.75 ± 0.86  2.29 ± 0.80  5.65 ± 0.75  1.50  76%  1:24  10  6.47 ± 0.87(–)  1.42  78%  11.22 ± 2.10(–)  7.39 ± 1.89  8.19 ± 2.22  9.78 ± 2.31  2.13  81%  2:25  11  10.12 ± 1.34(+)  1.51  85%  10.69 ± 1.39(–)  10.55 ± 1.45  1.49 ± 0.86  8.82 ± 1.31  1.94  82%  0:38    –  –  –  3.55 ± 0.53(–)  1.54 ± 0.22  3.20 ± 0.49  1.56 ± 0.19  1.28  64%  1:04a  12  9.25 ± 1.13(+)  2.69  71%  4.92 ± 1.57(–)  2.65 ± 1.56  3.68 ± 1.91  2.38 ± 1.88  2.80  43%  2:25  13  8.75 ± 2.12(+)  2.52  71%  16.89 ± 3.05(-)  15.66 ± 2.85  6.07 ± 2.15  16.12 ± 3.01  2.43  86%  2:34  14  11.12 ± 2.04(+)  2.06  81%  4.66 ± 0.74(–)  3.63 ± 0.96  2.57 ± 1.26  3.66 ± 0.78  2.14  54%  1:15  Average  8.61 ± 3.83  1.95 ± 0.45  74% ± 11%  8.55 ± 4.38  7.04 ± 4.47  3.99 ± 2.17  7.31 ± 4.56  2.08 ± 0.56  71% ± 14%    pMR = preoperative magnetic resonance images; uMR = updated magnetic resonance images; POC = percentage of correction. aThe time between the first iSV (after dural opening) and the third iSV (during resection). View Large TABLE 3. Summary of TREs of pMR and uMR in 14 Patient Cases. The Number of Target Points is Reported in Parenthesis   After dural opening  After partial resection  Patient  pMR-iSV1  uMR1-iSV1  pMR-iSV2  uMR1-iSV2  uMR2-iSV2  1  6.80 ± 0.97(5)  2.00 ± 0.45  7.65 ± 0.62(2)  3.58 ± 2.93  1.49 ± 0.35  2  10.10 ± 2.75(2)  1.83 ± 1.20  4.68 ± 1.24(2)  12.23 ± 0.25  1.63 ± 1.43  3  5.84 ± 1.81(4)  1.15 ± 0.35  4.16 ± 0.25(3)  4.85 ± 1.09  1.52 ± 1.66  4  4.80 ± 0.64(5)  1.38 ± 0.84  11.99 ± 1.15(4)  8.42 ± 0.37  2.52 ± 1.15  5  13.06 ± 1.54(2)  1.52 ± 0.27  10.17(1)  2.86  1.30  6  6.95 ± 1.58(4)  2.00 ± 0.28  12.75 ± 3.62(2)  13.85 ± 0.88  2.19 ± 0.43  7  16.74 ± 2.46(4)  1.36 ± 0.72  2.86(1)  9.46  1.43  8  2.92 ± 1.08(4)  1.85 ± 0.07  16.64 ± 1.66(3)  16.54 ± 1.71  2.57 ± 0.67  9  6.15 ± 0.26(2)  1.73 ± 0.16  2.22(1)  6.41  1.61  10  5.31 ± 0.98(5)  1.50 ± 0.81  13.15 ± 2.35(4)  12.24 ± 0.40  1.80 ± 1.58  11  9.77 ± 1.50(5)  1.96 ± 0.19  4.44 ± 0.73(2)  8.62 ± 0.34  1.95 ± 0.76    –  –  3.20(1)  5.39  1.52  12  10.08 ± 1.75(2)  1.76 ± 0.65  13.09 ± 3.25(2)  6.04 ± 1.39  2.92 ± 0.12  13  8.03 ± 1.27(4)  1.81 ± 1.11  12.09 ± 1.65(2)  16.85 ± 2.31  2.25 ± 0.34  14  12.16 ± 1.07(2)  1.45 ± 0.12  9.51(1)  4.26  2.07  Average  8.48 ± 3.74  1.66 ± 0.27  8.57 ± 4.67  8.77 ± 4.61  1.92 ± 0.49    After dural opening  After partial resection  Patient  pMR-iSV1  uMR1-iSV1  pMR-iSV2  uMR1-iSV2  uMR2-iSV2  1  6.80 ± 0.97(5)  2.00 ± 0.45  7.65 ± 0.62(2)  3.58 ± 2.93  1.49 ± 0.35  2  10.10 ± 2.75(2)  1.83 ± 1.20  4.68 ± 1.24(2)  12.23 ± 0.25  1.63 ± 1.43  3  5.84 ± 1.81(4)  1.15 ± 0.35  4.16 ± 0.25(3)  4.85 ± 1.09  1.52 ± 1.66  4  4.80 ± 0.64(5)  1.38 ± 0.84  11.99 ± 1.15(4)  8.42 ± 0.37  2.52 ± 1.15  5  13.06 ± 1.54(2)  1.52 ± 0.27  10.17(1)  2.86  1.30  6  6.95 ± 1.58(4)  2.00 ± 0.28  12.75 ± 3.62(2)  13.85 ± 0.88  2.19 ± 0.43  7  16.74 ± 2.46(4)  1.36 ± 0.72  2.86(1)  9.46  1.43  8  2.92 ± 1.08(4)  1.85 ± 0.07  16.64 ± 1.66(3)  16.54 ± 1.71  2.57 ± 0.67  9  6.15 ± 0.26(2)  1.73 ± 0.16  2.22(1)  6.41  1.61  10  5.31 ± 0.98(5)  1.50 ± 0.81  13.15 ± 2.35(4)  12.24 ± 0.40  1.80 ± 1.58  11  9.77 ± 1.50(5)  1.96 ± 0.19  4.44 ± 0.73(2)  8.62 ± 0.34  1.95 ± 0.76    –  –  3.20(1)  5.39  1.52  12  10.08 ± 1.75(2)  1.76 ± 0.65  13.09 ± 3.25(2)  6.04 ± 1.39  2.92 ± 0.12  13  8.03 ± 1.27(4)  1.81 ± 1.11  12.09 ± 1.65(2)  16.85 ± 2.31  2.25 ± 0.34  14  12.16 ± 1.07(2)  1.45 ± 0.12  9.51(1)  4.26  2.07  Average  8.48 ± 3.74  1.66 ± 0.27  8.57 ± 4.67  8.77 ± 4.61  1.92 ± 0.49  pMR = preoperative magnetic resonance images; uMR = updated magnetic resonance images; iSV = intraoperative stereovision. View Large In addition, results show that the components of shift along the surface normal (Table 2 column 6) were similar to the components along gravity (Table 2 column 8) because the gravity vectors were largely parallel with the surface normals due to patient position. The lateral shift (Table 2 column 7) was smaller than the normal/gravity component, indicating more collapse than lateral shift of the surgical surface. Analyses and comparisons between cases are challenging because lesion types and volumes varied from case to case, and iSV images were acquired with varying extents of resection. Interestingly, errors in pMR continued to degrade/accumulate after partial resection about half the time (in 7 of 14 surgeries). The magnitude of coregistration error demonstrated after dural opening and during later resection is substantial and clinically relevant. Image-guided volumetric resections, where completeness of tumor resection has been demonstrated to be associated with improved survival,24 are dependent on accuracies generally better than a few millimeter and this is only more important later in the resection. The overall update time after partial resection was longer than the update time after dural opening, mainly due to the second model solution required to adjust the initial estimate of tissue removal. The image-updating process was not fully automated and involved personnel in the OR for data acquisition, processing, and model computation. Although user input is likely to be required, eg, for selecting regions of interest for iSV reconstruction and OF registration, the overall efficiency of image updating can be improved by graphical user interfaces, code optimization, and further automation. Nevertheless, image updating is executed on a dedicated workstation, which allows the surgeon to continue to operate during model computation. Compared to iMR, the image-updating approach is more efficient as 20 min or more are needed to acquire iMR, and the acquisition interrupts surgical workflow. Limitations Some limitations do exist in the study. First, the OF surface registration method relies on common features being present at both iSV acquisition time points, and is not appropriate in some cases, eg, recurrent tumor resections in which no cortical features are visible within the craniotomy. In addition, if all features in the previous iSV are lost due to significant shift and/or resection, the OF algorithm cannot be applied. As a result, iSV surfaces need to be acquired within a certain time frame, and feature tracking can be challenging towards the end of resection. In this study, we report only cases in which features sufficient for OF registration were available to illustrate the image-updating method, and the second iSV surfaces were acquired before all features were lost. If surface displacements cannot be extracted, intraoperative ultrasound (iUS) can also be used to measure displacements deeper in the brain to drive the FEM model.25,26 Furthermore, data from iUS can be combined with iSV, as they sample different regions of the brain and are complementary, accordingly.26 In addition, coregistered iUS images can potentially be used as a way to measure ground-truth locations of internal features independently to quantify the accuracy of uMR deeper in the brain. CONCLUSION In summary, we have developed methods to produce uMR images to compensate for brain deformation due to resection by incorporating data acquired from iSV. We assessed the accuracy of uMRs qualitatively by visually comparing them against iSV surfaces, and quantitatively by calculating misfit, percentage of brain shift correction, and TRE, and compared these accuracy assessments to the corresponding values for pMR. The results show that the accuracy of uMR was ∼2 mm for both image updates (after dural opening and after partial resection), whereas pMR was inaccurate at both stages. The total computational time to generate the image updates after partial resection was ∼10 min, and had minimal influence on surgical workflow. These findings suggest that the image-updating process can be applied in the OR to provide improved image guidance accuracy during tissue resection strictly with iSV as long as common image features continue to be available in the surgical field during the procedure. Disclosures Authors at Dartmouth are named inventors on patents and/or patents-pending related to some of the image-updating technology described, the rights to which are currently held by the Trustees of Dartmouth College. This research was supported by National Institute of Health Grant No. R01 CA159324-03. Medtronic Navigation (Medtronic PLC, Louisville, Colorado) and Carl Zeiss (Carl Zeiss Surgical GmbH, Oberkochen, Germany) provided the StealthStation® S7® navigation system and the OPMI Pentero® operating microscope, respectively. Dr Timothy Schaewe and Dr David Simon are employees of Medtronic PLC. REFERENCES 1. Eggers G, Muhling J, Marmulla R. Image-to-patient registration techniques in head surgery. Int J Oral Maxillofac Surg . 2006; 35( 12): 1081- 1095. Google Scholar CrossRef Search ADS PubMed  2. Helm PA, Eckel TS. Accuracy of registration methods in frameless stereotaxis. Comput Aided Surg . 1998; 3( 2): 51- 56. Google Scholar CrossRef Search ADS PubMed  3. Carter TJ, Sermesant M, Cash DM, Barratt DC, Tanner C, Hawkes DJ. Application of soft tissue modelling to image-guided surgery. Med EngPhys . 2005; 27( 10): 893- 909. Google Scholar CrossRef Search ADS   4. Mostayed A, Garlapati RR, Joldes GR et al.   Biomechanical model as a registration tool for image-guided neurosurgery: evaluation against BSpline registration. Ann Biomed Eng . 2013; 41( 11): 2409- 2425. Google Scholar CrossRef Search ADS PubMed  5. Paulsen KD, Miga MI, Kennedy FE, Hoopes PJ, Hartov A, Roberts DW. A computational model for tracking subsurface tissue deformation during stereotactic neurosurgery. IEEE Trans Biomed Eng . 1999; 46( 2): 213- 225. Google Scholar CrossRef Search ADS PubMed  6. Chen I, Coffey AM, Ding S et al.   Intraoperative brain shift compensation: accounting for dural septa. IEEE Trans Biomed Eng . 2011; 58( 3): 499- 508. Google Scholar CrossRef Search ADS PubMed  7. Clatz O, Delingette H, Talos IF et al.   Robust nonrigid registration to capture brain shift from intraoperative MRI. IEEE Trans Med Imaging . 2005; 24( 11): 1417- 1427. Google Scholar CrossRef Search ADS PubMed  8. Ding S, Miga MI, Thompson RC, Dumpuri P, Cao A, Dawant BM. Estimation of intra-operative brain shift using a tracked laser range scanner. Conf Proc IEEE Eng Med Biol Soc.  2007; 2007: 848- 851. Google Scholar PubMed  9. Ferrant M, Nabavi A, Macq B, Jolesz FA, Kikinis R, Warfield SK. Registration of 3-D intraoperative MR images of the brain using a finite-element biomechanical model. IEEE Trans Med Imaging . 2001; 20( 12): 1384- 1397. Google Scholar CrossRef Search ADS PubMed  10. Garlapati RR, Roy A, Joldes GR et al.   More accurate neuronavigation data provided by biomechanical modeling instead of rigid registration. J Neurosurg . 2014; 120( 6): 1477- 1483. Google Scholar CrossRef Search ADS PubMed  11. Ji S, Hartov A, Roberts D, Paulsen K. Data assimilation using a gradient descent method for estimation of intraoperative brain deformation. Med Image Anal . 2009; 13( 5): 744- 756. Google Scholar CrossRef Search ADS PubMed  12. Joldes GR, Wittek A, Couton M, Warfield SK, Miller K. Real-time prediction of brain shift using nonlinear finite element algorithms. Med Image Comput Comput Assist Interv . 2009; 12( pt 2): 300- 307. Google Scholar PubMed  13. Nimsky C, Ganslandt O, Cerny S, Hastreiter P, Greiner G, Fahlbusch R. Quantification of, visualization of, and compensation for brain shift using intraoperative magnetic resonance imaging. Neurosurgery . 2000; 47( 5): 1070- 1079; discussion 1079-1080. Google Scholar CrossRef Search ADS PubMed  14. Fan X, Roberts DW, Schaewe TJ et al.   Intraoperative image updating for brain shift following dural opening. J Neurosurg . 2016: 1- 10. 15. Sun K, Pheiffer TS, Simpson AL, Weis JA, Thompson RC, Miga MI. Near real-time computer assisted surgery for brain shift correction using biomechanical Models. IEEE J Transl Eng Health Med . 2014; 2: 2500113. doi:10.1109/JTEHM.2014.2327628. Google Scholar PubMed  16. Tsai RY. A versatile camera calibration technique for high-accuracy 3d machine vision metrology using off-the-shelf Tv cameras and lenses. IEEE Trans Rob Autom . 1987; 3( 4): 323- 344. Google Scholar CrossRef Search ADS   17. Ji S, Fan X, Roberts DW, Hartov A, Paulsen KD. Flow-based correspondence matching in stereovision. In: Wu G, Zhang D, Shen D, Yan P, Suzuki K, Wang F, eds. Machine Learning in Medical Imaging: 4th International Workshop, MLMI 2013, Held in Conjunction with MICCAI 2013, Nagoya, Japan, September 22, 2013. Proceedings . Cham: Springer International Publishing; 2013: 106- 113. Google Scholar CrossRef Search ADS   18. Ji S, Fan X, Roberts DW, Paulsen KD. Efficient stereo image geometrical reconstruction at arbitrary camera settings from a single calibration. Med Image Comput Comput Assist Interv . 2014; 17( pt 1): 440- 447. Google Scholar PubMed  19. Cardoso MJ, Melbourne A, Kendall GS et al.   AdaPT: An adaptive preterm segmentation algorithm for neonatal brain MRI. Neuroimage . 2013; 65: 97- 108. Google Scholar CrossRef Search ADS PubMed  20. Fan X, Ji S, Hartov A, Roberts DW, Paulsen KD. Stereovision to MR image registration for cortical surface displacement mapping to enhance image-guided neurosurgery. Med Phys . 2014; 41( 10): 102302. Google Scholar CrossRef Search ADS PubMed  21. Ji S, Roberts DW, Hartov A, Paulsen KD. Brain-skull contact boundary conditions in an inverse computational deformation model. Med Image Anal . 2009; 13( 4): 659- 672. Google Scholar CrossRef Search ADS PubMed  22. Ji S, Fan X, Roberts DW, Hartov A, Paulsen KD. Cortical surface shift estimation using stereovision and optical flow motion tracking via projection image registration. Med Image Anal . 2014; 18( 7): 1169- 1183. Google Scholar CrossRef Search ADS PubMed  23. Fan X, Ji S, Fontaine K, Hartov A, Roberts D, Paulsen K. Simulation of brain tumor resection in image-guided neurosurgery. Paper presented at: Proc. SPIE 7964, Medical Imaging 2011: Visualization, Image-Guided Procedures, and Modeling, 2011;7964, 79640U1-11. 24. Lacroix M, Abi-Said D, Fourney DR et al.   A multivariate analysis of 416 patients with glioblastoma multiforme: prognosis, extent of resection, and survival. J Neurosurg . 2001; 95( 2): 190- 198. Google Scholar CrossRef Search ADS PubMed  25. Valdes PA, Fan X, Ji S, Harris BT, Paulsen KD, Roberts DW. Estimation of brain deformation for volumetric image updating in protoporphyrin IX fluorescence-guided resection. Stereotact Funct Neurosurg . 2010; 88( 1): 1- 10. Google Scholar CrossRef Search ADS PubMed  26. Ji S, Fan X, Hartov A, Roberts D, Paulsen K. Estimation of intraoperative brain deformation. In: Payan Y, ed. Soft Tissue Biomechanical Modeling for Computer Assisted Surgery . Vol 11. Springer Berlin Heidelberg; 2012: 97- 133. Google Scholar CrossRef Search ADS   COMMENTS The use of navigation for cranial neurosurgery has become widespread and relies largely on preoperative magnetic resonance imaging. Intraoperatively, however, the reliability of this modality is reduced largely by brain shift and tissue deformation. This often leaves the surgeon with questionable image guidance at the end of the procedure, when its utility is arguably at its greatest, particularly during resection of a mass lesion. Alternative intraoperative imaging modalities, including CT, MRI, and ultrasound, have been used to complement cranial navigation and compensate for the issues with relying on preoperative MR images alone. The authors describe their use of intraoperative stereovision images to modify and update the preoperative MRI-based navigation. They demonstrate promising results, generating updated imaging data fairly quickly and accurately after dural opening and during resection. While their fusion technique is reliant on, and perhaps limited by, the pairing of consistent cortical landmarks, it may provide an alternative method for dealing with intraoperative brain shift and its detrimental effect on navigation. Tyler J. Kenning Albany, New York It is accepted that image-guided surgery based on a preoperative MRI scan can suffer from loss of accuracy during surgery, due to brain shift as well as tumor resection. While intraoperative MRI imaging can compensate for these factors, iMRI is not always available and is associated with a significant financial investment on the part of the institution. Thus, other techniques which can address these issues are welcomed. The authors report their use of image processing techniques which utilize intraoperative stereoscopic imaging data to update the preoperative MRI model and thus provide continuous accurate guidance throughout surgery. Their technique, which is more time efficient than serial intraoperative MRI image acquisition, may be useful when intraoperative MRI is not available or not feasible for other reasons. Alon Y. Mogilner New York, New York Copyright © 2017 by the Congress of Neurological Surgeons http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Operative Neurosurgery Oxford University Press

Loading next page...
 
/lp/ou_press/image-updating-for-brain-shift-compensation-during-resection-RJ7eOEA3TN
Publisher
Congress of Neurological Surgeons
Copyright
Copyright © 2017 by the Congress of Neurological Surgeons
ISSN
2332-4252
eISSN
2332-4260
D.O.I.
10.1093/ons/opx123
Publisher site
See Article on Publisher Site

Abstract

Abstract BACKGROUND In open-cranial neurosurgery, preoperative magnetic resonance (pMR) images are typically coregistered for intraoperative guidance. Their accuracy can be significantly degraded by intraoperative brain deformation, especially when resection is involved. OBJECTIVE To produce model updated MR (uMR) images to compensate for brain shift that occurred during resection, and evaluate the performance of the image-updating process in terms of accuracy and computational efficiency. METHODS In 14 resection cases, intraoperative stereovision image pairs were acquired after dural opening and during resection to generate displacement maps of the surgical field. These data were assimilated by a biomechanical model to create uMR volumes of the evolving surgical field. A tracked stylus provided independent measurements of feature locations to quantify target registration errors (TREs) in the original coregistered pMR and uMR as surgery progressed. RESULTS Updated MR TREs were 1.66 ± 0.27 and 1.92 ± 0.49 mm in the 14 cases after dural opening and after partial resection, respectively, compared to 8.48 ± 3.74 and 8.77 ± 4.61 mm for pMR, respectively. The overall computational time for generating uMRs after partial resection was less than 10 min. CONCLUSION We have developed an image-updating system to compensate for brain deformation during resection using a computational model with data assimilation of displacements measured with intraoperative stereovision imaging that maintains TREs less than 2 mm on average. Brain deformation, FEM model, Image-guided neurosurgery, Intraoperative stereovision, Optical flow, Resection, Sparse data ABBREVIATIONS ABBREVIATIONS 3-D 3-dimensional CCD charge-coupled device FEM finite element model iMR intraoperative magnetic resonance iSV intraoperative stereovision iUS intraoperative ultrasound OF optical flow OR operating room pMR preoperative magnetic resonance POC percentage of correction TRE target registration error uMR updated magnetic resonance Preoperative magnetic resonance (pMR) images are typically used for intraoperative image guidance in open cranial surgery. Commercial navigation systems are available to rigidly align pMR with the patient's head in the operating room (OR),1 and provide real-time navigation with an average accuracy of 2 to 5 mm.2 However, coregistration can be significantly degraded by nonrigid brain deformation, especially as surgery progresses. To maintain accuracy, image-updating systems have been developed to generate updated MR (uMR) images using data-driven biomechanical models.3-13 Compared to intraoperative MR (iMR), model-based image updating is cost-effective and minimally disruptive to surgical workflow. As a result, the image-updating process can be repeated multiple times throughout a procedure, unlike iMR, which is typically acquired only at the presumed end of resection, because of the time and disruption involved. In a previous work,14 we produced uMRs in the OR to compensate for brain deformation but only after dural opening. Sun et al15 used laser range scanning to account for postresection brain shift, but tumor volume was not removed to represent extent of resection. In this paper, we report extensions of our image-updating process that enable multiple (second and beyond) uMRs to be generated by incorporating optical flow (OF) motion tracking of successive intraoperative stereovision (iSV) acquisitions during resection in 14 surgeries. We also assess the accuracy of uMRs with independent measurements of feature locations in the surgical field with a tracked stylus, allowing target registration error (TRE) estimates of multiple uMRs to be generated for the first time. We report the overall computational efficiency of the processes involved. Results show that registration accuracy is significantly improved over pMR with computational efficiency that can be maintained within clinically acceptable ranges. TABLE 1. Summary of Patient Information Patient  Gender  Age  Lesion  Location  Craniotomy size (cm × cm)  Registration accuracy (mm)  1  F  39  Arteriovenous Malformation  Right frontal  4.9 × 4.8  1.72  2  M  41  Oligodendroglioma  Right temporal  5.3 × 5.0  2.11  3  F  61  Meningioma  Right parietal  7.3 × 5.3  2.14  4  F  53  Epilepsy  Left temporal  7.6 × 5.6  0.98  5  F  24  Mixed oligoastrocytoma  Right parietal  6.2 × 4.4  1.18  6  F  49  Metastatic melanoma  Right temporal  3.1 × 2.6  2.38  7  M  78  Metastatic neuroendocrine carcinoma  Right occipital  7.8 × 5.8  4.82  8  F  65  Meningioma  Right temporal  5.0 × 3.9  1.84  9  M  60  Glioblastoma  Left temporal  4.6 × 3.6  1.49  10  F  51  Meningioma  Left frontal  7.8 × 5.4  1.94  11  F  59  Anaplastic mixed oligoastrocytoma  Right frontal  4.7 × 3.8  2.01  12  F  46  Meningioma  Right parietal  5.3 × 4.8  1.40  13  M  75  Glioblastoma  Left temporal  7.4 × 4.6  1.40  14  M  40  Meningioma  Left frontal  7.8 × 5.7  1.98  Patient  Gender  Age  Lesion  Location  Craniotomy size (cm × cm)  Registration accuracy (mm)  1  F  39  Arteriovenous Malformation  Right frontal  4.9 × 4.8  1.72  2  M  41  Oligodendroglioma  Right temporal  5.3 × 5.0  2.11  3  F  61  Meningioma  Right parietal  7.3 × 5.3  2.14  4  F  53  Epilepsy  Left temporal  7.6 × 5.6  0.98  5  F  24  Mixed oligoastrocytoma  Right parietal  6.2 × 4.4  1.18  6  F  49  Metastatic melanoma  Right temporal  3.1 × 2.6  2.38  7  M  78  Metastatic neuroendocrine carcinoma  Right occipital  7.8 × 5.8  4.82  8  F  65  Meningioma  Right temporal  5.0 × 3.9  1.84  9  M  60  Glioblastoma  Left temporal  4.6 × 3.6  1.49  10  F  51  Meningioma  Left frontal  7.8 × 5.4  1.94  11  F  59  Anaplastic mixed oligoastrocytoma  Right frontal  4.7 × 3.8  2.01  12  F  46  Meningioma  Right parietal  5.3 × 4.8  1.40  13  M  75  Glioblastoma  Left temporal  7.4 × 4.6  1.40  14  M  40  Meningioma  Left frontal  7.8 × 5.7  1.98  View Large METHODS Surgical Cases and Procedure Image data from 14 patients undergoing open cranial procedures were evaluated, retrospectively. The study was approved by the Institutional Review Board and patient consent was obtained. Surgical procedures included craniotomies for tumor, epilepsy, or arteriovenous malformation. Criteria for data inclusion were (1) T1-weighted contrast-enhanced MRI acquired prior to surgery (scan size = 256 × 256, 96–160 slices, voxel size = 0.9375 mm × 0.9375 mm × 1.5 mm); and (2) common features, such as blood vessels and sulci, visible on both MR scans and the exposed cortical surface. Subject gender, age, type of lesion, and size of craniotomy are reported in Table 1. At the time of surgery, patient registration was performed with pMR on a commercial navigation system (StealthStation® S7®, Medtronic, Louisville, Colorado) to provide standard-of-care intraoperative navigation and tracking. The reported accuracy of patient registration was 1.96 mm on average and is listed in Table 1 for each individual case. A surgical microscope (OPMI Pentero® Carl Zeiss Surgical GmbH, Oberkochen, Germany) was connected to the StealthStation® for intraoperative tracking, and an iSV system (Figure 1) was attached to and draped together with the microscope. The iSV system consisted of 2 charge-coupled device (CCD) cameras (Flea2 FL2G-50S5C, Point Grey Research, Inc, Richmond, British Columbia, Canada; image resolution: 1024 × 768 pixels), and was precalibrated16-18 and coregistered with pMR using tracking data acquired from the StealthStation® through the Medtronic StealthLink® (Medtronic, Louisville, Colorado) communications framework. FIGURE 1. View largeDownload slide The iSV system (red box). Two CCD cameras (yellow arrow) were connected to an adapter that was attached to one of the optical ports on the microscope head. FIGURE 1. View largeDownload slide The iSV system (red box). Two CCD cameras (yellow arrow) were connected to an adapter that was attached to one of the optical ports on the microscope head. MR Image-Updating Procedure A flowchart of the model-based image-updating process is shown in Figure 2. First, uMR was produced to compensate for brain deformation due to dural opening.14 Specifically, the brain was segmented from pMR using an adaptive preterm segmentation algorithm.19 After dural opening, an iSV image pair was acquired and a 3-dimensional (3-D) profile of the exposed cortical surface was reconstructed (iSV1). A tetrahedral mesh (mesh0) was generated with local refinement near the craniotomy, whose location and size were determined from iSV1. Surface displacements (sparse data1) were extracted by registering iSV1 with pMR,20 and employed to drive a biomechanical finite element model (FEM), in which the misfit between model estimates and measured displacements was minimized.21 The FEM model computed a whole-brain deformation field, in the form of the 3-D displacements of each node in the brain mesh. This information was then used to compute the new locations of each voxel in pMR and produce the updated MR (uMR1). FIGURE 2. View largeDownload slide Flowchart of the MR image-updating process. FIGURE 2. View largeDownload slide Flowchart of the MR image-updating process. During resection, a second iSV image pair was acquired to reconstruct a 3-D surface (iSV2) of the current surgical field. OF registration was applied to register iSV1 with iSV2 to measure surface shift (sparse data2).22 The amount of tissue removal was then estimated using an approach previously described in detail.23 Specifically, the resection cavity on iSV2 provided an initial estimate of tissue removal relative to uMR1, a brain mesh (mesh1) with a resection cavity was created, and the FEM model was executed. If the resulting brain deformation caused the resection cavity to deform further, and thus, no longer be aligned with iSV2, a revised resection cavity was created from iSV2 by shifting it in the opposite direction by the average model-estimated cavity shift. A new brain mesh (mesh2) with the revised cavity was created, and the FEM model was executed again and the model-estimated whole-brain deformation was used to deform uMR1 and produce uMR2. Data Analysis To assess accuracy, uMR1 and uMR2 were first visually compared with the corresponding iSV surfaces in both 2-D and 3-D views. Furthermore, we quantified the misfit and TRE for pMR, uMR1, and uMR2, respectively. Specifically, the misfit of uMR1 and uMR2 was computed as the root-mean-squared error between measured displacements (determined by surface registration) and model estimates at the same locations (uMR1-iSV1 and uMR2-iSV2, respectively). The pMR-iSV1 misfit was computed as the average measured displacements between iSV1 and pMR, ie, sparse data1, and the uMR1-iSV2 misfit was computed as the average measured displacements between iSV1 and iSV2, ie, sparse data2. In addition, we analyzed the displacements to study the patterns of shift due to tissue removal by decomposing the 3-D displacements (sparse data2) into their components along and perpendicular to the surface normal, and along the direction of gravity. The percentage of correction (POC) of uMR1 (POC1) was calculated as (1 – uMR1 misfit/pMR misfit) × 100%, and the POC of uMR2 (POC2) was calculated as (1 – uMR2 misfit/uMR1-iSV2 misfit) × 100%. To quantify the TRE of pMR and uMRs, we used features in the surgical field as target points. The technical details have been published previously.14 Specifically, at the time of iSV1 acquisition, the neurosurgeon used a tracked sterile stylus probe to locate features that were visible on both the exposed cortical surface and MR scans, such as vessel junctions, and the coordinates of these locations in MR image space (through patient registration) were stored. The same features were localized on pMR and uMR1, respectively. The same process was repeated at the time of iSV2 acquisition and features were localized on pMR, uMR1, and uMR2. TRE was computed as the distance between the intraoperatively tracked locations of these features (transformed into MR space) and their corresponding positions in the pMR or uMR image volumes. RESULTS Model-Updated MR Accuracy Evaluation Figures 3 to 6 illustrate the qualitative evaluation of accuracy in 4 representative cases. Misalignments between pMR and both iSV1 and iSV2 indicate that pMR was not accurate at either time point. Specifically, iSV surfaces were either above or underneath the brain surface in 2-D views (image A), indicating brain bulging or sagging, and cortical features (eg, white arrows in image B) from iSV1 were not aligned with pMR in 3-D views, denoting lateral shift. After the first model update, uMR1 and iSV1 (images C and D) were well aligned in terms of both geometry (shown in 2-D) and texture (shown in 3-D), indicating favorable accuracy. After partial resection, uMR1 was no longer accurate when compared with iSV2 (image C), while uMR2 (images E and F) aligned well with iSV2 in terms of both geometry (shown in 2-D) and texture (shown in 3-D) after a second model update. Patient 11 had a third iSV acquisition after more tissue was removed, and uMR2 served as the input for repeating the image-updating process and producing uMR3. Figures 6G and 6H show that uMR3 aligned well with iSV3. FIGURE 3. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 1. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 3. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 1. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 4. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 10. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 4. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 10. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 5. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 14. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 5. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 14. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1 and iSV2 (yellow and red lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrow points to feature misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1 and iSV2 (yellow and red lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrow points to feature accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 6. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 11. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1, iSV2 and iSV3 (yellow, red and green lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrows point to features misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1, iSV2 and iSV3 (yellow, red, and green lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrows point to features accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrows point to features accurately aligned. G, Two-dimensional view of uMR3 at the same coronal slice location, overlaid with iSV3 (red line). H, Three-dimensional view of uMR3 overlaid with iSV2, white arrow points to feature accurately aligned. FIGURE 6. View largeDownload slide Comparison of pMR and uMRs with iSV surfaces from case 11. pMR = preoperative MR images; uMR = updated MR images; iSV = intraoperative stereovision. A, Representative 2-D view of pMR overlaid with iSV1, iSV2 and iSV3 (yellow, red and green lines, respectively). B, Three-dimensional view of pMR overlaid with iSV1, white arrows point to features misaligned. C, Two-dimensional view of uMR1 at the same coronal slice location, overlaid with iSV1, iSV2 and iSV3 (yellow, red, and green lines, respectively). D, Three-dimensional view of uMR1 overlaid with iSV1, white arrows point to features accurately aligned. E, Two-dimensional view of uMR2 at the same coronal slice location, overlaid with iSV2 (red line). F, Three-dimensional view of uMR2 overlaid with iSV2, white arrows point to features accurately aligned. G, Two-dimensional view of uMR3 at the same coronal slice location, overlaid with iSV3 (red line). H, Three-dimensional view of uMR3 overlaid with iSV2, white arrow points to feature accurately aligned. Quantitative accuracy assessments are reported in Tables 2 (misfit) and 3 (TRE). The initial misfit between pMR and iSV1 is reported in column 2, along with its direction along the surface normal (“+” and “–” signs for outward and inward shift, respectively). The results show that the initial misfit was 8.61 ± 3.83 mm across the 14 cases, ranging from 2.72 to 16.91 mm (column 2). After model compensation, the remaining misfit of uMR1 was 1.95 ± 0.45 mm (column 3), and the corresponding POC of the first image update was 74% ± 11% (POC1, column 4). The misfit between uMR1 and iSV2 is reported in column 5, along with its direction along the surface normal. The results show an average shift of 8.55 ± 4.38 mm as a result of tissue removal, ranging from 3.16 to 16.89 mm, and the direction of cortical shift was inward (indicating brain collapse) in all 14 cases. Their components along the surface normal, in the lateral direction, and along gravity are reported in columns 6 to 8, respectively. After a second update, image updating accounted for 71% ± 14% (POC2; column 10) of the brain shift and the remaining misfit of uMR2 was 2.08 ± 0.56 mm (column 9). The time from dural opening to partial resection (ie, the time between iSV acquisitions) is reported in the last column. Similarly, TREs of distinct features on the exposed surgical surface are reported in Table 3. The average TRE after the first image update was 1.66 ± 0.27 mm (uMR1-iSV1; column 3), relative to 8.48 ± 3.74 mm for pMR (pMR-iSV1; column 2). After the second image update, the average TRE was 1.92 ± 0.49 mm (uMR2-iSV2; column 6), relative to 8.77 ± 4.61 mm, if the first, but not second image update occurred (uMR1-iSV2; column 5). The average TRE for pMR (pMR-iSV2; column 4) was 8.57 ± 4.67 mm. Computational Efficiency The computational time of first image update was 7 to 8 min on average as reported previously.14 Here, we present the time involved in generating the second update. Reconstruction of the iSV surface was completed in ∼6 to 7 s with graphics processing unit acceleration. The OF-based iSV–iSV surface registration was finished within 20 s, and voxels within the cavity were removed from uMR1 within 5 s. Meshing was performed within 10 s, and model computation was finished in ∼4 min. The resection cavity was then revised within 5 s, the model computation was repeated in ∼4 min, and uMR1 was warped to produce uMR2 within 15 s. Thus, the total computational time of the second image update was within 10 min. DISCUSSION Preoperative MR images are conventionally used to intraoperatively guide resection. Results from 14 clinical cases show that pMR was inaccurate as soon as the dura was opened, and continued to be inaccurate during resection. Specifically, an error of >5 mm was observed in 12 of 14 surgeries (misfit, Table 2 column 2; and TRE, Table 3 column 2) after dural opening, and in 9 out of the 14 cases after partial resection (TRE, Table 3 column 4), respectively. Updated MR images produced after dural opening (uMR1) proved to be more accurate with an average error of <2 mm (misfit, Table 2 column 3; and TRE, Table 3 column 3) and agree with results from a previous study.14 However, uMR1 became inaccurate after tissue was removed, and further shift >5 mm (since dural opening) was observed in 10 of 14 cases (misfit, Table 2 column 5; and TRE, Table 3 column 5). After the second image update, uMR2 proved to be more accurate than both pMR and uMR1 with average errors ∼2 mm (misfit, Table 2 column 9; and TRE, Table 3 column 6). The visual comparison between pMR/iMR and iSV does have an inherent bias because the images were clearly identifiable as either preoperative or updated, and thus, the comparison cannot be performed in a blinded fashion. The TRE was quantified by touching features on the cortical surface with a tracked stylus and identifying them in the MR texture map, and is, therefore, subject to human and system localization and tracking errors. The quantification of model-data misfit, however, was automatic and performed free of user input, and independently of any visual comparisons. TABLE 2. Summary of Misfit of pMR and uMR in 14 Patient Cases   After dural opening  After partial resection    pMR  uMR1    uMR1  Normal  Lateral  Gravity  uMR2    Time  Patient  (mm)  (mm)  POC1  (mm)  (mm)  (mm)  (mm)  (mm)  POC2  (h:min)  1  6.07 ± 0.91(–)  1.96  68%  3.16 ± 1.02(–)  2.76 ± 1.32  1.20 ± 0.48  2.89 ± 1.09  1.49  53%  1:56  2  11.04 ± 0.85(+)  2.19  80%  11.69 ± 2.58(–)  11.15 ± 2.42  3.30 ± 1.47  9.33 ± 1.95  1.65  86%  2:32  3  7.25 ± 2.24(+)  1.62  78%  5.65 ± 1.48(–)  3.71 ± 1.83  4.06 ± 0.70  4.73 ± 1.42  2.03  64%  0:59  4  4.19 ± 0.69(–)  1.58  62%  8.30 ± 2.05(–)  4.37 ± 1.24  6.67 ± 2.80  6.40 ± 1.67  2.98  64%  2:16  5  13.98 ± 3.58(+)  1.81  87%  3.43 ± 1.19(–)  2.45 ± 0.88  2.20 ± 1.26  2.17 ± 0.83  1.25  64%  2:48  6  5.73 ± 0.81(+)  2.26  61%  12.83 ± 0.96(–)  11.61 ± 1.69  4.74 ± 2.29  12.30 ± 0.91  2.23  83%  1:25  7  16.91 ± 2.45(+)  2.59  85%  10.90 ± 2.92(–)  10.45 ± 3.08  2.80 ± 0.97  9.96 ± 2.13  2.43  78%  1:27  8  2.72 ± 1.02(–)  1.35  50%  14.11 ± 2.21(–)  11.86 ± 2.30  7.37 ± 1.89  13.85 ± 2.21  2.86  80%  2:17  9  6.94 ± 1.36(+)  1.75  75%  6.23 ± 0.97(–)  5.75 ± 0.86  2.29 ± 0.80  5.65 ± 0.75  1.50  76%  1:24  10  6.47 ± 0.87(–)  1.42  78%  11.22 ± 2.10(–)  7.39 ± 1.89  8.19 ± 2.22  9.78 ± 2.31  2.13  81%  2:25  11  10.12 ± 1.34(+)  1.51  85%  10.69 ± 1.39(–)  10.55 ± 1.45  1.49 ± 0.86  8.82 ± 1.31  1.94  82%  0:38    –  –  –  3.55 ± 0.53(–)  1.54 ± 0.22  3.20 ± 0.49  1.56 ± 0.19  1.28  64%  1:04a  12  9.25 ± 1.13(+)  2.69  71%  4.92 ± 1.57(–)  2.65 ± 1.56  3.68 ± 1.91  2.38 ± 1.88  2.80  43%  2:25  13  8.75 ± 2.12(+)  2.52  71%  16.89 ± 3.05(-)  15.66 ± 2.85  6.07 ± 2.15  16.12 ± 3.01  2.43  86%  2:34  14  11.12 ± 2.04(+)  2.06  81%  4.66 ± 0.74(–)  3.63 ± 0.96  2.57 ± 1.26  3.66 ± 0.78  2.14  54%  1:15  Average  8.61 ± 3.83  1.95 ± 0.45  74% ± 11%  8.55 ± 4.38  7.04 ± 4.47  3.99 ± 2.17  7.31 ± 4.56  2.08 ± 0.56  71% ± 14%      After dural opening  After partial resection    pMR  uMR1    uMR1  Normal  Lateral  Gravity  uMR2    Time  Patient  (mm)  (mm)  POC1  (mm)  (mm)  (mm)  (mm)  (mm)  POC2  (h:min)  1  6.07 ± 0.91(–)  1.96  68%  3.16 ± 1.02(–)  2.76 ± 1.32  1.20 ± 0.48  2.89 ± 1.09  1.49  53%  1:56  2  11.04 ± 0.85(+)  2.19  80%  11.69 ± 2.58(–)  11.15 ± 2.42  3.30 ± 1.47  9.33 ± 1.95  1.65  86%  2:32  3  7.25 ± 2.24(+)  1.62  78%  5.65 ± 1.48(–)  3.71 ± 1.83  4.06 ± 0.70  4.73 ± 1.42  2.03  64%  0:59  4  4.19 ± 0.69(–)  1.58  62%  8.30 ± 2.05(–)  4.37 ± 1.24  6.67 ± 2.80  6.40 ± 1.67  2.98  64%  2:16  5  13.98 ± 3.58(+)  1.81  87%  3.43 ± 1.19(–)  2.45 ± 0.88  2.20 ± 1.26  2.17 ± 0.83  1.25  64%  2:48  6  5.73 ± 0.81(+)  2.26  61%  12.83 ± 0.96(–)  11.61 ± 1.69  4.74 ± 2.29  12.30 ± 0.91  2.23  83%  1:25  7  16.91 ± 2.45(+)  2.59  85%  10.90 ± 2.92(–)  10.45 ± 3.08  2.80 ± 0.97  9.96 ± 2.13  2.43  78%  1:27  8  2.72 ± 1.02(–)  1.35  50%  14.11 ± 2.21(–)  11.86 ± 2.30  7.37 ± 1.89  13.85 ± 2.21  2.86  80%  2:17  9  6.94 ± 1.36(+)  1.75  75%  6.23 ± 0.97(–)  5.75 ± 0.86  2.29 ± 0.80  5.65 ± 0.75  1.50  76%  1:24  10  6.47 ± 0.87(–)  1.42  78%  11.22 ± 2.10(–)  7.39 ± 1.89  8.19 ± 2.22  9.78 ± 2.31  2.13  81%  2:25  11  10.12 ± 1.34(+)  1.51  85%  10.69 ± 1.39(–)  10.55 ± 1.45  1.49 ± 0.86  8.82 ± 1.31  1.94  82%  0:38    –  –  –  3.55 ± 0.53(–)  1.54 ± 0.22  3.20 ± 0.49  1.56 ± 0.19  1.28  64%  1:04a  12  9.25 ± 1.13(+)  2.69  71%  4.92 ± 1.57(–)  2.65 ± 1.56  3.68 ± 1.91  2.38 ± 1.88  2.80  43%  2:25  13  8.75 ± 2.12(+)  2.52  71%  16.89 ± 3.05(-)  15.66 ± 2.85  6.07 ± 2.15  16.12 ± 3.01  2.43  86%  2:34  14  11.12 ± 2.04(+)  2.06  81%  4.66 ± 0.74(–)  3.63 ± 0.96  2.57 ± 1.26  3.66 ± 0.78  2.14  54%  1:15  Average  8.61 ± 3.83  1.95 ± 0.45  74% ± 11%  8.55 ± 4.38  7.04 ± 4.47  3.99 ± 2.17  7.31 ± 4.56  2.08 ± 0.56  71% ± 14%    pMR = preoperative magnetic resonance images; uMR = updated magnetic resonance images; POC = percentage of correction. aThe time between the first iSV (after dural opening) and the third iSV (during resection). View Large TABLE 3. Summary of TREs of pMR and uMR in 14 Patient Cases. The Number of Target Points is Reported in Parenthesis   After dural opening  After partial resection  Patient  pMR-iSV1  uMR1-iSV1  pMR-iSV2  uMR1-iSV2  uMR2-iSV2  1  6.80 ± 0.97(5)  2.00 ± 0.45  7.65 ± 0.62(2)  3.58 ± 2.93  1.49 ± 0.35  2  10.10 ± 2.75(2)  1.83 ± 1.20  4.68 ± 1.24(2)  12.23 ± 0.25  1.63 ± 1.43  3  5.84 ± 1.81(4)  1.15 ± 0.35  4.16 ± 0.25(3)  4.85 ± 1.09  1.52 ± 1.66  4  4.80 ± 0.64(5)  1.38 ± 0.84  11.99 ± 1.15(4)  8.42 ± 0.37  2.52 ± 1.15  5  13.06 ± 1.54(2)  1.52 ± 0.27  10.17(1)  2.86  1.30  6  6.95 ± 1.58(4)  2.00 ± 0.28  12.75 ± 3.62(2)  13.85 ± 0.88  2.19 ± 0.43  7  16.74 ± 2.46(4)  1.36 ± 0.72  2.86(1)  9.46  1.43  8  2.92 ± 1.08(4)  1.85 ± 0.07  16.64 ± 1.66(3)  16.54 ± 1.71  2.57 ± 0.67  9  6.15 ± 0.26(2)  1.73 ± 0.16  2.22(1)  6.41  1.61  10  5.31 ± 0.98(5)  1.50 ± 0.81  13.15 ± 2.35(4)  12.24 ± 0.40  1.80 ± 1.58  11  9.77 ± 1.50(5)  1.96 ± 0.19  4.44 ± 0.73(2)  8.62 ± 0.34  1.95 ± 0.76    –  –  3.20(1)  5.39  1.52  12  10.08 ± 1.75(2)  1.76 ± 0.65  13.09 ± 3.25(2)  6.04 ± 1.39  2.92 ± 0.12  13  8.03 ± 1.27(4)  1.81 ± 1.11  12.09 ± 1.65(2)  16.85 ± 2.31  2.25 ± 0.34  14  12.16 ± 1.07(2)  1.45 ± 0.12  9.51(1)  4.26  2.07  Average  8.48 ± 3.74  1.66 ± 0.27  8.57 ± 4.67  8.77 ± 4.61  1.92 ± 0.49    After dural opening  After partial resection  Patient  pMR-iSV1  uMR1-iSV1  pMR-iSV2  uMR1-iSV2  uMR2-iSV2  1  6.80 ± 0.97(5)  2.00 ± 0.45  7.65 ± 0.62(2)  3.58 ± 2.93  1.49 ± 0.35  2  10.10 ± 2.75(2)  1.83 ± 1.20  4.68 ± 1.24(2)  12.23 ± 0.25  1.63 ± 1.43  3  5.84 ± 1.81(4)  1.15 ± 0.35  4.16 ± 0.25(3)  4.85 ± 1.09  1.52 ± 1.66  4  4.80 ± 0.64(5)  1.38 ± 0.84  11.99 ± 1.15(4)  8.42 ± 0.37  2.52 ± 1.15  5  13.06 ± 1.54(2)  1.52 ± 0.27  10.17(1)  2.86  1.30  6  6.95 ± 1.58(4)  2.00 ± 0.28  12.75 ± 3.62(2)  13.85 ± 0.88  2.19 ± 0.43  7  16.74 ± 2.46(4)  1.36 ± 0.72  2.86(1)  9.46  1.43  8  2.92 ± 1.08(4)  1.85 ± 0.07  16.64 ± 1.66(3)  16.54 ± 1.71  2.57 ± 0.67  9  6.15 ± 0.26(2)  1.73 ± 0.16  2.22(1)  6.41  1.61  10  5.31 ± 0.98(5)  1.50 ± 0.81  13.15 ± 2.35(4)  12.24 ± 0.40  1.80 ± 1.58  11  9.77 ± 1.50(5)  1.96 ± 0.19  4.44 ± 0.73(2)  8.62 ± 0.34  1.95 ± 0.76    –  –  3.20(1)  5.39  1.52  12  10.08 ± 1.75(2)  1.76 ± 0.65  13.09 ± 3.25(2)  6.04 ± 1.39  2.92 ± 0.12  13  8.03 ± 1.27(4)  1.81 ± 1.11  12.09 ± 1.65(2)  16.85 ± 2.31  2.25 ± 0.34  14  12.16 ± 1.07(2)  1.45 ± 0.12  9.51(1)  4.26  2.07  Average  8.48 ± 3.74  1.66 ± 0.27  8.57 ± 4.67  8.77 ± 4.61  1.92 ± 0.49  pMR = preoperative magnetic resonance images; uMR = updated magnetic resonance images; iSV = intraoperative stereovision. View Large In addition, results show that the components of shift along the surface normal (Table 2 column 6) were similar to the components along gravity (Table 2 column 8) because the gravity vectors were largely parallel with the surface normals due to patient position. The lateral shift (Table 2 column 7) was smaller than the normal/gravity component, indicating more collapse than lateral shift of the surgical surface. Analyses and comparisons between cases are challenging because lesion types and volumes varied from case to case, and iSV images were acquired with varying extents of resection. Interestingly, errors in pMR continued to degrade/accumulate after partial resection about half the time (in 7 of 14 surgeries). The magnitude of coregistration error demonstrated after dural opening and during later resection is substantial and clinically relevant. Image-guided volumetric resections, where completeness of tumor resection has been demonstrated to be associated with improved survival,24 are dependent on accuracies generally better than a few millimeter and this is only more important later in the resection. The overall update time after partial resection was longer than the update time after dural opening, mainly due to the second model solution required to adjust the initial estimate of tissue removal. The image-updating process was not fully automated and involved personnel in the OR for data acquisition, processing, and model computation. Although user input is likely to be required, eg, for selecting regions of interest for iSV reconstruction and OF registration, the overall efficiency of image updating can be improved by graphical user interfaces, code optimization, and further automation. Nevertheless, image updating is executed on a dedicated workstation, which allows the surgeon to continue to operate during model computation. Compared to iMR, the image-updating approach is more efficient as 20 min or more are needed to acquire iMR, and the acquisition interrupts surgical workflow. Limitations Some limitations do exist in the study. First, the OF surface registration method relies on common features being present at both iSV acquisition time points, and is not appropriate in some cases, eg, recurrent tumor resections in which no cortical features are visible within the craniotomy. In addition, if all features in the previous iSV are lost due to significant shift and/or resection, the OF algorithm cannot be applied. As a result, iSV surfaces need to be acquired within a certain time frame, and feature tracking can be challenging towards the end of resection. In this study, we report only cases in which features sufficient for OF registration were available to illustrate the image-updating method, and the second iSV surfaces were acquired before all features were lost. If surface displacements cannot be extracted, intraoperative ultrasound (iUS) can also be used to measure displacements deeper in the brain to drive the FEM model.25,26 Furthermore, data from iUS can be combined with iSV, as they sample different regions of the brain and are complementary, accordingly.26 In addition, coregistered iUS images can potentially be used as a way to measure ground-truth locations of internal features independently to quantify the accuracy of uMR deeper in the brain. CONCLUSION In summary, we have developed methods to produce uMR images to compensate for brain deformation due to resection by incorporating data acquired from iSV. We assessed the accuracy of uMRs qualitatively by visually comparing them against iSV surfaces, and quantitatively by calculating misfit, percentage of brain shift correction, and TRE, and compared these accuracy assessments to the corresponding values for pMR. The results show that the accuracy of uMR was ∼2 mm for both image updates (after dural opening and after partial resection), whereas pMR was inaccurate at both stages. The total computational time to generate the image updates after partial resection was ∼10 min, and had minimal influence on surgical workflow. These findings suggest that the image-updating process can be applied in the OR to provide improved image guidance accuracy during tissue resection strictly with iSV as long as common image features continue to be available in the surgical field during the procedure. Disclosures Authors at Dartmouth are named inventors on patents and/or patents-pending related to some of the image-updating technology described, the rights to which are currently held by the Trustees of Dartmouth College. This research was supported by National Institute of Health Grant No. R01 CA159324-03. Medtronic Navigation (Medtronic PLC, Louisville, Colorado) and Carl Zeiss (Carl Zeiss Surgical GmbH, Oberkochen, Germany) provided the StealthStation® S7® navigation system and the OPMI Pentero® operating microscope, respectively. Dr Timothy Schaewe and Dr David Simon are employees of Medtronic PLC. REFERENCES 1. Eggers G, Muhling J, Marmulla R. Image-to-patient registration techniques in head surgery. Int J Oral Maxillofac Surg . 2006; 35( 12): 1081- 1095. Google Scholar CrossRef Search ADS PubMed  2. Helm PA, Eckel TS. Accuracy of registration methods in frameless stereotaxis. Comput Aided Surg . 1998; 3( 2): 51- 56. Google Scholar CrossRef Search ADS PubMed  3. Carter TJ, Sermesant M, Cash DM, Barratt DC, Tanner C, Hawkes DJ. Application of soft tissue modelling to image-guided surgery. Med EngPhys . 2005; 27( 10): 893- 909. Google Scholar CrossRef Search ADS   4. Mostayed A, Garlapati RR, Joldes GR et al.   Biomechanical model as a registration tool for image-guided neurosurgery: evaluation against BSpline registration. Ann Biomed Eng . 2013; 41( 11): 2409- 2425. Google Scholar CrossRef Search ADS PubMed  5. Paulsen KD, Miga MI, Kennedy FE, Hoopes PJ, Hartov A, Roberts DW. A computational model for tracking subsurface tissue deformation during stereotactic neurosurgery. IEEE Trans Biomed Eng . 1999; 46( 2): 213- 225. Google Scholar CrossRef Search ADS PubMed  6. Chen I, Coffey AM, Ding S et al.   Intraoperative brain shift compensation: accounting for dural septa. IEEE Trans Biomed Eng . 2011; 58( 3): 499- 508. Google Scholar CrossRef Search ADS PubMed  7. Clatz O, Delingette H, Talos IF et al.   Robust nonrigid registration to capture brain shift from intraoperative MRI. IEEE Trans Med Imaging . 2005; 24( 11): 1417- 1427. Google Scholar CrossRef Search ADS PubMed  8. Ding S, Miga MI, Thompson RC, Dumpuri P, Cao A, Dawant BM. Estimation of intra-operative brain shift using a tracked laser range scanner. Conf Proc IEEE Eng Med Biol Soc.  2007; 2007: 848- 851. Google Scholar PubMed  9. Ferrant M, Nabavi A, Macq B, Jolesz FA, Kikinis R, Warfield SK. Registration of 3-D intraoperative MR images of the brain using a finite-element biomechanical model. IEEE Trans Med Imaging . 2001; 20( 12): 1384- 1397. Google Scholar CrossRef Search ADS PubMed  10. Garlapati RR, Roy A, Joldes GR et al.   More accurate neuronavigation data provided by biomechanical modeling instead of rigid registration. J Neurosurg . 2014; 120( 6): 1477- 1483. Google Scholar CrossRef Search ADS PubMed  11. Ji S, Hartov A, Roberts D, Paulsen K. Data assimilation using a gradient descent method for estimation of intraoperative brain deformation. Med Image Anal . 2009; 13( 5): 744- 756. Google Scholar CrossRef Search ADS PubMed  12. Joldes GR, Wittek A, Couton M, Warfield SK, Miller K. Real-time prediction of brain shift using nonlinear finite element algorithms. Med Image Comput Comput Assist Interv . 2009; 12( pt 2): 300- 307. Google Scholar PubMed  13. Nimsky C, Ganslandt O, Cerny S, Hastreiter P, Greiner G, Fahlbusch R. Quantification of, visualization of, and compensation for brain shift using intraoperative magnetic resonance imaging. Neurosurgery . 2000; 47( 5): 1070- 1079; discussion 1079-1080. Google Scholar CrossRef Search ADS PubMed  14. Fan X, Roberts DW, Schaewe TJ et al.   Intraoperative image updating for brain shift following dural opening. J Neurosurg . 2016: 1- 10. 15. Sun K, Pheiffer TS, Simpson AL, Weis JA, Thompson RC, Miga MI. Near real-time computer assisted surgery for brain shift correction using biomechanical Models. IEEE J Transl Eng Health Med . 2014; 2: 2500113. doi:10.1109/JTEHM.2014.2327628. Google Scholar PubMed  16. Tsai RY. A versatile camera calibration technique for high-accuracy 3d machine vision metrology using off-the-shelf Tv cameras and lenses. IEEE Trans Rob Autom . 1987; 3( 4): 323- 344. Google Scholar CrossRef Search ADS   17. Ji S, Fan X, Roberts DW, Hartov A, Paulsen KD. Flow-based correspondence matching in stereovision. In: Wu G, Zhang D, Shen D, Yan P, Suzuki K, Wang F, eds. Machine Learning in Medical Imaging: 4th International Workshop, MLMI 2013, Held in Conjunction with MICCAI 2013, Nagoya, Japan, September 22, 2013. Proceedings . Cham: Springer International Publishing; 2013: 106- 113. Google Scholar CrossRef Search ADS   18. Ji S, Fan X, Roberts DW, Paulsen KD. Efficient stereo image geometrical reconstruction at arbitrary camera settings from a single calibration. Med Image Comput Comput Assist Interv . 2014; 17( pt 1): 440- 447. Google Scholar PubMed  19. Cardoso MJ, Melbourne A, Kendall GS et al.   AdaPT: An adaptive preterm segmentation algorithm for neonatal brain MRI. Neuroimage . 2013; 65: 97- 108. Google Scholar CrossRef Search ADS PubMed  20. Fan X, Ji S, Hartov A, Roberts DW, Paulsen KD. Stereovision to MR image registration for cortical surface displacement mapping to enhance image-guided neurosurgery. Med Phys . 2014; 41( 10): 102302. Google Scholar CrossRef Search ADS PubMed  21. Ji S, Roberts DW, Hartov A, Paulsen KD. Brain-skull contact boundary conditions in an inverse computational deformation model. Med Image Anal . 2009; 13( 4): 659- 672. Google Scholar CrossRef Search ADS PubMed  22. Ji S, Fan X, Roberts DW, Hartov A, Paulsen KD. Cortical surface shift estimation using stereovision and optical flow motion tracking via projection image registration. Med Image Anal . 2014; 18( 7): 1169- 1183. Google Scholar CrossRef Search ADS PubMed  23. Fan X, Ji S, Fontaine K, Hartov A, Roberts D, Paulsen K. Simulation of brain tumor resection in image-guided neurosurgery. Paper presented at: Proc. SPIE 7964, Medical Imaging 2011: Visualization, Image-Guided Procedures, and Modeling, 2011;7964, 79640U1-11. 24. Lacroix M, Abi-Said D, Fourney DR et al.   A multivariate analysis of 416 patients with glioblastoma multiforme: prognosis, extent of resection, and survival. J Neurosurg . 2001; 95( 2): 190- 198. Google Scholar CrossRef Search ADS PubMed  25. Valdes PA, Fan X, Ji S, Harris BT, Paulsen KD, Roberts DW. Estimation of brain deformation for volumetric image updating in protoporphyrin IX fluorescence-guided resection. Stereotact Funct Neurosurg . 2010; 88( 1): 1- 10. Google Scholar CrossRef Search ADS PubMed  26. Ji S, Fan X, Hartov A, Roberts D, Paulsen K. Estimation of intraoperative brain deformation. In: Payan Y, ed. Soft Tissue Biomechanical Modeling for Computer Assisted Surgery . Vol 11. Springer Berlin Heidelberg; 2012: 97- 133. Google Scholar CrossRef Search ADS   COMMENTS The use of navigation for cranial neurosurgery has become widespread and relies largely on preoperative magnetic resonance imaging. Intraoperatively, however, the reliability of this modality is reduced largely by brain shift and tissue deformation. This often leaves the surgeon with questionable image guidance at the end of the procedure, when its utility is arguably at its greatest, particularly during resection of a mass lesion. Alternative intraoperative imaging modalities, including CT, MRI, and ultrasound, have been used to complement cranial navigation and compensate for the issues with relying on preoperative MR images alone. The authors describe their use of intraoperative stereovision images to modify and update the preoperative MRI-based navigation. They demonstrate promising results, generating updated imaging data fairly quickly and accurately after dural opening and during resection. While their fusion technique is reliant on, and perhaps limited by, the pairing of consistent cortical landmarks, it may provide an alternative method for dealing with intraoperative brain shift and its detrimental effect on navigation. Tyler J. Kenning Albany, New York It is accepted that image-guided surgery based on a preoperative MRI scan can suffer from loss of accuracy during surgery, due to brain shift as well as tumor resection. While intraoperative MRI imaging can compensate for these factors, iMRI is not always available and is associated with a significant financial investment on the part of the institution. Thus, other techniques which can address these issues are welcomed. The authors report their use of image processing techniques which utilize intraoperative stereoscopic imaging data to update the preoperative MRI model and thus provide continuous accurate guidance throughout surgery. Their technique, which is more time efficient than serial intraoperative MRI image acquisition, may be useful when intraoperative MRI is not available or not feasible for other reasons. Alon Y. Mogilner New York, New York Copyright © 2017 by the Congress of Neurological Surgeons

Journal

Operative NeurosurgeryOxford University Press

Published: Apr 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 12 million articles from more than
10,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Unlimited reading

Read as many articles as you need. Full articles with original layout, charts and figures. Read online, from anywhere.

Stay up to date

Keep up with your field with Personalized Recommendations and Follow Journals to get automatic updates.

Organize your research

It’s easy to organize your research with our built-in tools.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

Monthly Plan

  • Read unlimited articles
  • Personalized recommendations
  • No expiration
  • Print 20 pages per month
  • 20% off on PDF purchases
  • Organize your research
  • Get updates on your journals and topic searches

$49/month

Start Free Trial

14-day Free Trial

Best Deal — 39% off

Annual Plan

  • All the features of the Professional Plan, but for 39% off!
  • Billed annually
  • No expiration
  • For the normal price of 10 articles elsewhere, you get one full year of unlimited access to articles.

$588

$360/year

billed annually
Start Free Trial

14-day Free Trial