ObjectivesTo compare the method of photographic screening using the RetCam 120 to the standard method of screening for retinopathy of prematurity (ROP) by ophthalmologic examination.MethodsA total of 100 RetCam 120 photoscreening examinations of the retina were performed on 32 premature infants. These were stored in a separate file from which all identifying information was removed. At this same examination, a detailed ophthalmological evaluation, employing the indirect ophthalmoscope with scleral depression, was performed by an experienced examiner. Masked examiners performed an evaluation of the fundus photographs to identify presence or absence of ROP, the location and extent of the disease, and the presence or absence of plus disease. These data were then compared with the results of the ophthalmological examination to determine the specificity, sensitivity, and the positive and negative predictive value (PPV and NPV) of the method.ResultsRetinopathy of prematurity was detected in 68 of 100 subjects by ophthalmologic examinations and in 58 of 100 subjects' photoscreening examinations. No ROP was detected in 32 of 100 subjects. The sensitivity of the method was 56 (82.4%) of 68 and the specificity was 30 (93.8%) of 32. The PPV was 96.6%; NPV, 76.9%.ConclusionsThe sensitivity of the method was low. The ROP that was missed was peripheral stage 1 or stage 2 disease in peripheral zone 2 or zone 3. This was largely due to the technical limitations of the speculum-camera interface preventing a better view of the periphery. The ROP cases that were missed by the photographic examination regressed spontaneously on follow-up. No disease more posterior to peripheral zone 2 was overlooked. These results detail the accuracy of the method employing the technique of photoscreening as a potential substitute for detailed ophthalmological examination. At present there are clear technical limitations to such a substitution. The study is part of an ongoing project to determine the feasibility of employing neonatal nurses trained to take digitized images of the premature infant's retina and telemeter the results to be read by an experienced ophthalmologist remote from the site.EARLY DETECTION and treatment at threshold of retinopathy of prematurity (ROP) has been shown to significantly decrease the incidence of severe visual loss in premature infants.Current standards recommend that all infants with a birth weight less than or equal to 1500 g or with a gestational age of 28 weeks or less undergo screening for ROP.Indirect ophthalmoscopy by an experienced ophthalmologist in the neonatal intensive care unit (NICU) has been traditionally considered the criterion standard method for detection of ROP.These examinations can be physiologically stressful for the infants, time consuming for the ophthalmologist, and costly for the medical system.An alternative screening test for ROP that would minimize cost and be less invasive is desirable.Any screening test designed to replace the standard of a complete ophthalmolscopic examination must have an acceptable sensitivity in detecting disease when it is present and specificity in not falsely indicating patients as having disease when they do not. We employed the RetCam 120 camerasystem (Massie Laboratories, Pleasantville, Calif) as the method to photographically document the ROP screening examination. The ability of the photographs to accurately serve as a screening examination was statistically compared with simultaneous clinical examinations.SUBJECTS AND METHODSPOPULATION SAMPLEDThis study conformed to the policies and procedures outlined by the Institutional Review Board of the University of Miami School of Medicine, Miami, Fla. Between October 1997 and May 1998, 32 consecutive infants with birth weights less than 1500 g or gestational age less than 32 weeks at the University of Miami Jackson Memorial Medical Center whose parents or guardians signed the informed consent form were evaluated for the presence of ROP. Gestational age was determined by the neonatologist caring for the patient and represented a best estimate based on menstrual history, obstetrical dating (including ultrasound data when available), and neonatal physical assessment. During that interval a total of 118 infants were examined by an ophthalmologist in that age and weight group.CLINICAL EXAMINATIONScreening examinations were performed by an experienced examiner (J.T.F.) in the nursery area with the approval of an attending neonatologist. The infants' pupils were dilated with 0.5% cyclopentolate and 2.5% phenylephrine combination drops. Infants in the NICU received dilating drops 30 to 60 minutes before the scheduled examination. To reduce risk of aspiration, feeding was delayed from 1 hour prior to examination to 3 hours postexamination. All examinations were performed with cardiac monitoring and many with oxygen saturation monitoring under the supervision of a trained NICU nurse. Whenever an infant's heart rate decreased or oxygen saturation was unacceptably low, the examination was interrupted until levels returned to normal.Topical anesthetic was placed in each eye before examination. A lid speculum and a scleral depressor, individually sterilized for each infant's examination, were used to visualize the peripheral retina and ora serrata, the anterior border of retinal vascularization, or retinopathy. Examinations were performed with the binocular indirect ophthalmoscope and 30-diopter lens, first evaluating the posterior pole and then the periphery with scleral depression. The presence or absence of ROP disease, its location and extent, and the presence or absence of plus disease were documented at the time of examination according to the international classification of ROP.The first examination was performed between 32 to 34 weeks' postconceptional age. Subsequent examinations were performed at intervals of 2 weeks until either the retina became vascularized or retinopathy developed. When ROP developed and it was clinically indicated by the severity of the disease, more frequent examinations were performed. When the retinopathy progressed to threshold level, patients were treated with indirect laser therapy within 72 hours of diagnosis. Threshold ROP was defined as 5 or more contiguous or 8 cumulative clock hours of stage 3 ROP in either zone 1 or zone 2 as in the Cryotherapy for ROP Study.RetCam EXAMINATIONThe RetCam 120 was used to photographically document the fundus features at the time of examination (Figure 1). An experienced ophthalmic photographer skilled in the proper use of the camera performed most of the examinations. A neonatal nurse performed the rest of the examinations under supervision by the ophthalmic photographer. A series of 1 to 6 photographs were obtained to adequately capture the posterior pole and as much of the periphery as possible (Figure 2). This study was conducted under standard nursery conditions. As desirable as it might be to obtain a standard set of views on every baby, this is not possible at this time with the existing instrumentation. Each series of photographs was saved in a data file. These images were then transferred to another file devoid of identifying patient information. These latter images were then randomized as to order of presentation and read in a masked fashion by ophthalmologists familiar with the disease.Figure 1.RetCam 120 (Massie Laboratories, Pleasantville, Calif) being employed in the neonatal intensive care unit to obtain digitized images of the premature fundus.Figure 2.A, Wide-angle RetCam image (Massie Laboratories, Pleasantville, Calif) of premature fundus without retinopathy of prematurity. B, Wide-angle RetCam image with retinopathy of prematurity.DATA INTERPRETATIONThe data from the clinical examinations and the photographic sessions were collected and tabulated by separate observers. For each patient, the gestational age, birth weight, and age at examination were tabulated from the clinical examination data. The presence or absence of retinopathy, the stage of disease, the zonal location of disease, the number of clock hours involved by retinopathy, and the presence or absence of plus disease were abstracted from the clinical record.Each of the photographic sessions was identified exclusively by the randomization number. The images were read independently by 2 observers (D.B.R. and J.T.F.). Readings were performed several months after the images were taken. Where discrepancies existed, these were re-reviewed until consensus was reached. For each of the series of photographs of a single eye's examination, the presence or absence of retinopathy, the stage of disease, the zonal location of disease, the number of clock hours involved by retinopathy, and the presence or absence of plus disease was noted. If the presence of laser photocoagulation scars was seen in the photograph, this was noted as well. All initial and follow-up examinations were randomized, and sessions and eyes were intermixed for reading purposes. For each posterior pole photograph containing it, the distance from the optic nerve to the macula was measured, and the border of zone 1 was determined as twice this distance from the optic nerve. The photographs were felt to inadequately capture zone 3 without scleral indentation. Therefore, if retinopathy were appreciated in the photograph, it was assumed to be located in zone 1 or zone 2.STATISTICAL METHODSPhotographs from each examination for each eye were read in a random order assigned by a computer-generated sequence of numbers so that the interpretation of the photographs was independent for each examination of each eye of each infant. Sensitivity and specificity were computed for the RetCam photographic readings using the clinical examination as the gold standard. A statistical computer program (StatXact; Cytel Software Corp, Cambridge, Mass) was used to calculate sensitivity, specificity, and confidence intervals. Positive predictive values (PPVs) and negative predictive values (NPVs)were calculated and used to assess the usefulness of the RetCam photographic readings. The sample prevalence of ROP as determined by clinical examination was used in these calculations.RESULTSOf 32 premature infants examined by 2 methods, the mean birth weight was 905.6 g (range, 495-1375 g); mean gestational age at birth, 27.4 weeks (range, 24-33 weeks); mean gestational age at the first RetCam examination, 37 weeks (32-46 weeks); and mean gestational age at the last RetCam examination, 38 weeks (32-50 weeks). These demographics were in keeping with the demographics of the overall nursery population at the time of this study.A total of 100 simultaneous ophthalmological and RetCam examinations were performed. Fifty-nine eyes of 32 patients were evaluated. Forty eyes contributed only 1 examination. Ophthalmological examination revealed ROP to be present in 68 examinations. The RetCam examination was positive for ROP in 58 examinations (85.3%). Thirty-two clinical examinations were free of ROP, and 42 RetCam examinations were free of ROP (Table 1). Twelve examinations were false-negative and 2 were false-positive. The sensitivity for the RetCam examination was 82.4%, and the specificity was 93.8%. The PPV was 96.6%, and the NPV was 71.5%.Table 1. Clinical and RetCam* Examination FindingsRetCam Examination FindingsClinical Examination FindingsTotalPositiveNegativePositive56258Negative123042Total Examinations6832100*Massie Laboratories, Pleasantville, Calif.Of 56 examinations in which the clinical and RetCam examinations both revealed the presence of ROP, 100% were located in zone 2 or zone 1. Twelve examinations were judged to be stage 2 by clinical and RetCam examination. Eleven examinations were evaluated as stage 3 by RetCam and clinical examination. Two examinations were evaluated as stage 2 by RetCam examination and stage 1 by clinical examination. Four examinations were evaluated as stage 2 by RetCam examination and stage 3 by clinical examination. One examination was evaluated as stage 3 according to the RetCam and stage 2 by clinical examination (Table 2). Four examinations revealed plus disease by both RetCam and clinical examination. Thirty-four examinations revealed the absence of plus disease by RetCam and clinical examination. One examination revealed plus disease by RetCam but not clinical examination. Four examinations revealed plus disease clinically but not by RetCam.Table 2. Comparison of Retinopathy of Prematurity Stage Between Clinical Examination and RetCam 120*Clinical Stage 1Clinical Stage 2Clinical Stage 3RetCam stage 1000RetCam stage 22124RetCam stage 30111*Massie Laboratories, Pleasantville, Calif.Of 12 false-negative results (ROP by clinical examination; no ROP by RetCam), all were in zone 2; 8 were outer zone 2; 5 were stage 1; 3 were stage 2; and none had plus disease. Of 2 false-positive results (no ROP by clinical examination; ROP by RetCam), both were in zone 2, both were stage 2; and none had plus disease. Review of these photographs revealed that ROP that had not been detected on clinical examination was present. For purposes of this study, these were treated as false-positive results although clinical examination later revealed that the photograph interpretation was correct.COMMENTTwo sets of issues present themselves for discussion in this study. The first are concerned with the accuracy of the screening test itself compared with the ophthalmic examination performed by an experienced examiner, the standard. The second are the technical issues surrounding the clearance between the camera lens and the metallic arms of the speculum (Figure 3). These issues are intertwined.Figure 3.Photograph of the incompatibility between the lid speculum arms and the camera head.While the sensitivity and specificity are proximate measures of the accuracy of a screening test, the test's ultimate usefulness in clinical practice is best captured by the positive and negative predictive power respectively. The PPV is the probability that a positive test result will identify a case of the disease in question. Conversely, the NPV is the probability that a negative test result will correctly identify a normal subject free of the disease. These probabilities depend both on the accuracy of the testing device (constrained in our case by the limitations in fully applying the technology of the instrument to the eye) and the prevalence of the disease to be detected. In the standard clinical scenario where prevalence is low, incremental increases in specificity can have a large effect on the PPV. However in our study in which the prevalence of the ROP is high (68%), the RetCam requires a high sensitivity to be useful. At this stage of our study, we are seeking the best way to use this technology to correctly identify ROP in eyes, not examinations. To the extent that the eyes of these infants have multiple examinations, the chance of missing ROP in all examinations of any given infant diminishes.The detection of ROP requires at present an experienced ophthalmologist to visit each premature infant in the NICU and submit the newborn to a detailed ophthalmological examination. The examination is stressful for the infants,and ophthalmologists must coordinate their schedule with that of the NICU. Often, the ophthalmologist may arrive to examine an infant when it would be preferable for the infant to be fed or not be handled outside of the incubator at that moment. It is not unusual for the ophthalmologist to obtain suboptimal views of the retinal details due to a small pupil or the infant's movements. An ideal screening modality would be one that would be less traumatic to the infants, could be used at the convenience of the NICU staff, and would permanently document the details of any disease in the eyes of premature infants being evaluated for ROP. Photoscreening, although not at this time recommended as a substitute for routine ophthalmological examination to screen for ROP, with improvement of the instrumentation might in future substitute for ROP screening by examination.When peripheral views were obtained by the RetCam, this technology was extremely efficient in detecting ROP and photodocumenting the specific extent and location of disease. The RetCam however missed several cases of ROP. While the occurrence of complications from disease in this location is low,they do occur.This adds an additional note of caution to the findings of this study. The diseases missed by imaging were in outer zone 2, either stage 1 or stage 2 disease. None had plus disease, and no cases of posterior disease were missed. As alluded to earlier in this article, the sensitivity particularly, and to a much lesser extent the specificity, of the RetCam in this study are not adequate to justify replacing routine ROP screening by ophthalmic examination with the RetCam. However, the flaws detected in this study are primarily due to mechanical problems preventing fully adequate visualization of the periphery. They are fixable and are being worked on at this time. It is probable that with improvement of the technology to obtain adequate peripheral photographs, both parameters can be improved to fully acceptable levels.One recent studywas similar in some respects to the current study. All of the eyes in that study were at prethreshold or threshold and about to receive therapy. There was 100% accord between the masked examiners and the on-site examiners. This was not the case in the present study. We cannot at this time account for the discrepancy between the 2 studies' accuracy.Telemedicine is becoming more prevalent in the clinical practice of medicine and has been applied to ophthalmology as well.It may become feasible to implement this technology to telemeter the infant's photographs to an experienced ophthalmologist remote from the site. Digitized photoimages are an effective method of transmitting information to physicians at any site for review and evaluation. This may also allow NICU facilities that today have no access to an ophthalmologist experienced in ROP screening to send the digitized images to an ophthalmologist remote from their facility.Cryotherapy for Retinopathy of Prematurity Cooperative GroupMulticenter trial of cryotherapy for retinopathy of prematurity: preliminary results.Arch Ophthalmol.1988;106:471-479.JTFlynnEBancalariBNBachynskiRetinopathy of prematurity: diagnosis, severity, and natural history.Ophthalmology.1987;94:620-629.KEWPTanBPCatsTimely incidence of retinopathy of prematurity and its consequence for the screening strategy.Am J Perinatol.1989;6:337-340.Cryotherapy for Retinopathy of Prematurity Cooperative GroupMulticenter trial of cryotherapy for retinopathy of prematurity: three-month outcome.Arch Ophthalmol.1990;108:195-204.Cryotherapy for Retinopathy of Prematurity Cooperative GroupMulticenter trial of cryotherapy for retinopathy of prematurity: one-year outcome: structure and function.Arch Ophthalmol.1990;108:1408-1416.EAPalmerJTFlynnRJHardyIncidence and early course of retinopathy of prematurity.Ophthalmology.1991;98:1628-1640.DELawsCMortonMWeindlingDClarkSystemic effects of screening for retinopathy of prematurity.Br J Ophthalmol.1996;80:425-428.KWrightMEAndersonEWalkerVLorchShould fewer premature infants be screened for retinopathy of prematurity in the managed care era?Pediatrics.1998;102:31-34.DBRothDMoralesWFeuerDHessRJohnsonJTFlynnScreening for retinopathy of prematurity employing the RetCam 120: sensitivity and specificity [abstract].Invest Ophthalmol Vis Sci.1999;40:2997.The Committee for the Classification of Retinopathy of PrematurityAn international classification for retinopathy of prematurity.Arch Ophthalmol.1984;102:1130-1134.JLFleissStatistical Methods for Rates and Proportions.2nd ed. New York, NY: John Wiley & Sons. 1981:4-8.MSlevinJFMurphyLDalyMO'KeefeRetinopathy of prematurity screening, stress related responses, the role of nesting.Br J Ophthalmol.1997;81:762-764.Cryotherapy for Retinopathy of Prematurity Cooperative GroupThe natural ocular outcome of premature birth and retinopathy.Arch Ophthalmol.1994;112:903-909.SMCardenWVGoodMacular folds and poor vision associated with zone III retinopathy of prematurity.Am J Ophthalmol.1998;126:460-462.SDSchwartzSAHarrisonPJFerroneMTTreseTelemedical evaluation and management of retinopathy of prematurity using a fiberoptic digital camera.Ophthalmology.2000;107:25-28.MLimaDistributed imaging links physicians with imaging services.Radiol Manage.1999;21:13-14.Accepted for publication June 23, 2000.This investigation was supported in part by the Lincoln Health Care Foundation; grant EY 10900 from the Public Health Service, Department of Health and Human Services, National Eye Institute, Bethesda, Md; Research to Prevent Blindness Inc, New York, NY (Dr Flynn); grant CP-113-C4B from Fight for Sight Inc; Research to Prevent Blindness America, Schaumburg, Ill; and the Alex J. Weinstein Foundation, New York.Corresponding author and reprints: John T. Flynn, MD, Harkness Eye Institute, 635 W 165th St, New York, NY 10032 (e-mail: firstname.lastname@example.org).
JAMA Ophthalmology – American Medical Association
Published: Feb 1, 2001