ObjectiveTo measure the effect of a multimodal intervention on well-child care visit (WCV) and immunization rates in an inner-city population.DesignCluster randomized controlled trial.Setting and ParticipantsOne-year cohort of 2843 infants born at a hospital in an integrated inner-city health care system.InterventionsEleven clinics were randomly allocated to 1 of 3 study arms: WCV intervention (n = 3), immunization intervention (n = 4), and controls (n = 4). Interventions to improve immunization and WCV rates included both patient-based and clinic-based activities.Main Outcome MeasuresUp-to-date status with childhood immunizations and WCVs by age 12 months (primary) and health care utilization and charges (secondary).ResultsCompared with the control arm, the WCV and immunization arms had 5% to 6% higher immunization rates and 7% to 8% higher WCV rates. In multivariate analyses that accounted for the clustered nature of the data, the number of immunizations received was greater in the WCV arm than in controls. However, neither the WCV nor the immunization intervention increased WCV or immunization up-to-date rates. The WCV arm had slightly higher health care charges. Neither intervention affected emergency, urgent care or inpatient utilization.ConclusionsThis multimodal intervention produced a small increase in the number of childhood immunizations delivered. However, patient- and clinic-based methods did not lead to significant increases in WCV or immunization up-to-date rates after controlling for other factors. Methods found in some settings to increase immunization up-to-date rates may not be as effective in a population of inner-city socioeconomically disadvantaged children.SINCE THE measles epidemic in the United States in the early 1990s, local, state, and federal research efforts and policy decisions have improved childhood immunization rates. Today, many areas of the country are approaching the immunization goals set forth in Healthy People 2010.Despite these successes, pockets of underimmunization continue in many of the same inner-city areas that were the primary sites of the measles epidemic.Given that immunizations are a marker for receipt of other childhood preventive services,many inner-city children who are underimmunized are probably not receiving other health services, such as lead and anemia screening and preventive counseling.Numerous studies have identified risk factors for childhood underimmunizationand have tested interventions to increase immunization rates at the level of the individual child (eg, reminder/recall)and at the level of the clinic (eg, AFIX [Assessment, Feedback, Incentives, and eXchange of information]).Reminder/recall systems have been effective in a variety of settings and have increased immunization rates 5% to 20%.The AFIX-based interventions have shown an increase in childhood immunization rates of 5% to 6% per year.The Task Force on Community Preventive Services strongly recommends both methods to increase vaccination coverage.Fewer studies have examined risk factors for lack of receipt of well-child care visits (WCVs)or the interrelationship of immunization status and well-child care receipt.Studies that examined outcomes of increased well-child care have almost universally focused on the process of care (eg, how many children were screened for anemia)rather than on the impact of that process on child health (eg, how many cases of anemia were prevented).To examine outcomes associated with improved receipt of immunizations and WCVs, we asked 2 research questions: Can interventions at the level of the clinic and the patient improve delivery of immunizations and WCVs in an economically disadvantaged inner-city population? Do interventions to improve components of primary care for children, such as immunizations and WCVs, have an impact on other health care utilization (eg, hospitalizations and emergency department or urgent care visits) or on health care charges? To address these questions, we conducted a cluster randomized controlled trial, with randomization at the level of the clinic, to increase immunizations and WCVs in an inner-city population of disadvantaged, largely minority children.METHODSSTUDY SETTINGDenver Health (DH) is the largest vertically integrated community health center system funded by the Bureau of Primary Health Care (Section 330) in the United States. This system comprises 11 community health centers, DH Medical Center (the city hospital, with 17 000 admissions per year), the DH emergency department (50 000 visits per year), 12 school-based clinics, and a public health department. Denver Health provides health care to more than 140 000 persons, or more than 20% of the population of the city and county of Denver, per year. Its community health centers, located in areas with the lowest health status and economic indices, have more than 500 000 patient visits per year. Patient medical records are shared among all DH facilities. All records associated with a patient are scanned into an electronic imaging system accessible across the citywide network; inpatient records are filed at the hospital, and outpatient encounters are filed in one unified patient chart. When transferring care from one network community health center to another, the patient's outpatient medical chart is also transferred.STUDY POPULATION AND RANDOMIZATIONAll patients born at DH Medical Center between July 1, 1998, and June 30, 1999, were enrolled and followed for 12 months; a flow diagram of study participants and clinics is shown in Figure 1. Randomization was conducted at the clinic level rather than at the patient level because a substantial component of the intervention focused on changes in the clinic site as a whole, which would result in contamination within each site if the child were used as the unit of randomization. Randomization was achieved using a standard random-number table to assign each of the 11 clinics a number between 1 and 11 (1-4, 5-8, and 9-11 having been previously assigned to a study arm). The identity of individual clinics was concealed until study arms were assigned. Clinics were not matched on baseline characteristics such as racial and ethnic distribution or baseline immunization rates because matching in cluster randomized trials with small numbers of randomization units may reduce statistical power compared with simple randomization.Distribution of the birth cohort in the cluster randomized controlled trial. DH indicates Denver Health; WCV, well-child care visit.All infants born at DH Medical Center were given an appointment at one of the community health centers before hospital discharge; the choice of clinic was determined by the family's site of residence. The clinic at which the infant attended the 2-week WCV determined the intervention group assignment. For the few infants who did not attend a 2-week WCV, research group allocation was determined by the clinic chosen by the family in the newborn nursery logbook. Because research staff acted on behalf of the clinics to carry out enhanced clinic activities such as reminder/recall, we did not obtain informed consent from individual families. This study was approved by the Colorado Multiple Institutional Review Board and by the review board at the Centers for Disease Control and Prevention.INTERVENTIONSThis study aimed to increase WCVs and immunizations using a multimodal intervention targeted at the individual patient and the clinic where infants received preventive health care. Some components of the intervention were specific to the WCV arm, and others were specific to the immunization arm. In addition, some systemwide interventions were occurring simultaneously that were not related to this study. A summary of all the interventions across the DH system during the period of the study is given in Table 1. Overall, the key elements of the intervention were intensive reminder/recall at the level of the patient and the process of AFIX at the level of the clinic. Briefly, AFIX is an intervention promoted by the Centers for Disease and Control and Prevention that was found to improve immunization rates at public health immunization sites.In our study, AFIX was applied to WCVs at clinics assigned to that intervention arm or to immunizations at sites assigned to the immunization intervention. This study, to our knowledge, is the first application of AFIX methods, which were originally developed to boost immunization rates, to increase WCVs.Table 1. Summary of Study Interventions by Study ArmDescription of InterventionIZ ArmWCV ArmControl ArmPatient-Based InterventionsRecall for missed appointments++−Recall if behind on IZs or WCVs++−Reminder postcards and telephone calls for future appointments++−Parents given WCV patient education materials−+−Parents given IZ patient education materials+−−EPSDT referral for home visit if missed appointment++−Assistance with transportation services++−Clinic-Based InterventionsMedical charts marked if behind on IZs or WCVs++−Baseline clinic practice assessment++−Monthly AFIX cyclesDirected at WCVs−+−Directed at IZs+−−Staff education (CQI) for childhood IZs*+−−Staff education (CQI) for missed appointment recall++−Staff education (CQI) for future appointment practices++−Create patient chart before first WCV++−Literature review for clinic staff++−Quarterly incentives for highest IZ/WCV rates++−System-Based InterventionsFlag patients in the WIC program system if behind on IZs+−−Automated appointment reminder (postcard)†+++Quarterly AFIX assessments for IZ rates†+++Computerized IZ registry†+++Newborn nursery hepatitis B virus vaccine entry into computerized IZ registry†+++Abbreviations: AFIX, Assessment, Feedback, Incentives, and eXchange of information; CQI, continuous quality improvement; EPSDT, Early Periodic Screening and Developmental Testing services of Medicaid; IZ, immunization; WCV, well-child care visit; WIC, Women, Infants, and Children.*Immunization education included missed opportunities, documentation of children who had moved or gone elsewhere for health care, IZ registry reliability, and IZ delivery practice emphasis.†Interventions already in place at the start of the study (not part of the intervention).Research staff performed weekly assessments of infants in the birth cohort to maintain a "real-time" overview of up-to-date rates for WCVs and immunizations. Before the monthly clinic AFIX meetings, research staff compiled detailed lists of children with delayed preventive care. In the WCV intervention clinics, monthly recall lists were generated for children needing a WCV who did not have an appointment for a future visit. Similarly, in the immunization intervention clinics, children in need of immunizations were listed for recall efforts. Research staff supplemented basic clinic recall efforts (see Table 1) with telephone calls, postcards, letters, and referrals to Early Periodic Screening and Developmental Testing, a federally financed resource that provides individual case management, home visitation, and transportation to WCVs for children enrolled in Medicaid.DATA SOURCESThis study used data from 3 DH sources: (1) an electronic immunization registry, (2) medical chart review, and (3) administrative and billing records. In addition, a second Denver hospital, which provided pediatric care to a subset of the birth cohort in this study, provided utilization (but not charge) data on those children.Immunization RegistryThe DH electronic immunization registry serves as the systemwide repository for any pediatric immunization given in the system; it meets all Centers for Disease Control and Prevention criteria for electronic registries.This registry has been shown to be more accurate than the conventional medical record; by January 1, 2001, it contained 551 622 immunizations for 64 329 children.Medical Chart ReviewFive trained abstractors reviewed all infant medical records to determine the reliability of electronic data sources, search for missing data, and categorize encounter visit types (inpatient, outpatient, and well-child care) for all DH encounters. In total, 97.3% (29 748/ 30 575) of the billed encounters received a medical chart review. Ongoing review of a sample of medical charts by other project staff assessed reliability and ensured consistent reviewer interpretation. Maternal antenatal and postnatal forms were also reviewed to abstract maternal covariates (eg, prenatal care use, maternal risk behaviors, and postnatal breastfeeding status) that might affect study outcomes.Administrative DataAll billing and diagnosis data for the study cohort were downloaded from the DH computer system. The DH registration and administrative data provided information on infant date of birth, race and ethnicity, percentage above the federal poverty level, family size, insurance status, International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM)diagnoses,and hospital service codes and charges. After matching on name, sex, and date of birth, administrative data for infants in the study cohort were obtained from a second hospital used by a subset of DH patients. Using hospital service codes and ICD-9-CMdiagnosis codes, visits were then classified into 3 categories: inpatient, emergency department or urgent care, and primary care.MEASURESPrimary Outcome MeasuresThe primary outcome measures for this study included measures of WCV and immunization receipt. Well-child care visits were defined as visits in which a systemwide age-appropriate WCV form was used, corresponding to American Academy of Pediatrics WCV guidelines, which recommend 5 WCVs by age 12 months (at 2 weeks and 2, 4, 6, and 9 months). Use of the WCV form to define a WCV was found to be more accurate than use of the ICD-9-CMcode for WCVs (V20.2) from administrative claims. The WCV outcomes were defined as (1) attendance at 4 or more WCVs and (2) number of WCVs attended (range, 0-5).Immunization data from medical chart review and the electronic registry were merged into the final database. Only valid doses of immunizations in accordance with standard minimum ages and intervals set by the Centers for Disease Control and Preventionwere counted. The immunization outcomes were (1) percentage of children up-to-date with the 3:2:2:2 series (3 diphtheria-tetanus-pertussis, 2 poliovirus, 2 Haemophilus influenzaetype B, and 2 hepatitis B vaccines) at age 12 months, (2) number of vaccines received by age 12 months, and (3) mean number of days underimmunized.Secondary Outcome MeasuresThe secondary outcome measures were health care utilization and charges. Utilization outcomes were dichotomous variables for (1) emergency or urgent care service utilization and (2) inpatient utilization. Charges incurred by children in this birth cohort at a second hospital in Denver and its outpatient clinics were imputed using DH charges for similar services.CovariatesThe following covariates were abstracted from maternal antepartum records: maternal education, language, parity, gravity, estimated date of delivery, birth country, total number of prenatal visits, date of first prenatal visit, psychiatric diagnoses, history of domestic violence, and use of tobacco, alcohol, or illicit drugs. The maternal postnatal visit provided information about breastfeeding. Administrative data, supplemented and validated by medical chart review, provided an income surrogate (percentage of federal poverty guidelines), family size, prenatal care visits, maternal risk factors, infant race and ethnicity, insurance status, date of birth, and birth weight. Pediatric chronic health conditions were defined using the rubric of the National Association of Children's Hospitals and Related Institutions,not including asthma diagnosis codes.DATA ANALYSISAll analyses were conducted using an intent-to-treat model, based on the clinic at which each infant planned to obtain primary care after discharge from the hospital at birth. Comparisons of the intervention and control groups were assessed using nonlinear mixed model methods adjusted for the clustered design of the study, with the 11 clinics entered as random effects, but unadjusted for other covariates.We used nonlinear mixed models to assess the impact of the interventions on each of the outcome variables (7 models in total). In all cases, the clinic was considered a random effect, as it was the unit of randomization in the study design. The exact type of model was determined by the distribution of the outcome variable (dichotomous assumed a binomial distribution, continuous assumed a normal distribution, and discrete counts assumed a Poisson distribution). Covariates were entered into the full regression models if they differed between study intervention arms and were related to the outcome of interest (P<.20 for both). A log transformation was applied to health care charges.For the only normally distributed continuous outcome variable (log charges for health care), the intraclass correlation coefficient was zero. We used statistical software to conduct all analyses (SAS version 8.01; SAS Institute Inc, Cary, NC).RESULTSDenver Health had 2843 live births between July 1, 1998, and June 30, 1999. Of these, 2665 (94%) were included in 1 of the 3 study groups (Figure 1). Thirty-eight infants were excluded because their parent(s) indicated that the infant would receive care outside the DH system, 97 because the infant received no care at DH after newborn discharge (this includes 7 neonatal deaths), and 43 because the infant attended a separately funded clinic that did not use DH's administrative billing system.At baseline there were differences in infant and maternal characteristics, but none were statistically significant after controlling for clinic clustering (Table 2). The immunization study arm consisted of 4 clinics (mean number of study cohort children per clinic, 258; range, 65-697), the WCV arm had 3 clinics (mean number of study cohort children per clinic, 158; range, 45-273), and the control arm had 4 clinics (mean number of study cohort children per clinic, 290; range, 72-718). Each study arm had 1 clinic with resident continuity practices, except for the immunization arm, which had 2 such clinics. Baseline immunization up-to-date status was higher in the immunization (59%) and control (57%) clinics than in the WCV clinics (49%). These up-to-date estimates were calculated in 1997, based on a sample of patients from each clinic, and assessed 1-year immunization status at age 2 years. Thus, the percentages are not directly comparable to the numbers in this study but are indicative of the previous immunization delivery of the clinics.Table 2. Infant and Maternal Characteristics by Study ArmCharacteristicImmunization Arm (4 Clinics and 1030 Infants)Well-Childcare Visit Arm (3 Clinics and 475 Infants)Control Arm (4 Clinics and 1160 Infants)PValue*InfantsFemale sex, %484947.58Gestational age at delivery, %<38 wk181719.8038 to <42 wk798279.68≥42 wk222.56Birth weight, mean (SD), kg3.2 (0.5)3.2 (0.5)3.2 (0.6).91Race/ethnicity, %Latino887083.15African American5236.25Non-Latino white559.52Other212.89Chronic health condition, %467.13Principal health insurance, %Medicaid/Medicare889088.19Medically indigent432.16Private212.46Self-pay547.11All primary care at 1 clinic, %868478.22Use of health care not in system, %101811.45MaternalEnglish is primary language, %395235.06Age at delivery, mean (SD), y24 (6)25 (6)26 (6).35Born in the United States, %394833.06Education, grade completed (SD), No.9 (3)10 (3)9 (3).20Prenatal visits, mean (SD), No.11 (5)12 (5)10 (5).10Onset of prenatal care, %First trimester455346.20Second trimester373433.54Third trimester191421.09Primigravida, %413738.47Live births, mean (SD), No.1.1 (1.2)1.3 (1.5)1.2 (1.4).22Married, %383339.32Occupation, %Homemaker666170.46Outside work242525>.99Student9145.07<185% Poverty level, %989898.64Breastfeeding noted on medical chart, %696566.51Maternal risk factor, %Smoking182219.37Alcohol10146.13Domestic violence999.91Psychiatric issues111010.81Illicit drug use9137.06*P<.20 values are shown in boldface and indicate eligibility for inclusion in multivariate analyses. All comparisons are among 3 study arms, using hierarchical models and adjusting for the clinic as the unit of randomization (see the "Methods" section).Research staff completed more than 10 000 patient contacts for the 2 intervention arms. A total of 150 clinic-level AFIX meetings took place during the 2-year intervention period. Patient-level interventions included more than 500 Early Periodic Screening and Developmental Testing referrals, 2000 appointment confirmations by telephone, 1000 other telephone messages left, and 8000 postcards to remind people of scheduled appointments and to recall patients for missed appointments. Nearly 2000 unsuccessful attempts were made to contact patients, including calls to disconnected telephone numbers, returned postcards, and unanswered telephone calls. The average number of successful contacts per child in the 2 intervention arms was 7 (10 542 contacts per 1505 children); 2757 of these were telephone contacts (average, 2 per child). Ninety-nine percent of the children in the 2 intervention arms (1485 of 1505) received at least 1 contact by either telephone or postcard.Compared with control infants, 5% to 6% more infants at the intervention sites were up-to-date with immunizations and 7% to 8% more attended at least 4 of 5 recommended WCVs by age 1 year (Table 3). Children in the 2 intervention arms were underimmunized a median of 57 and 62 days, compared with 67 days in the control arm. After adjusting for clinic clustering, however, none of these differences were statistically significant.Table 3. Preventive Health Services Measures by Study GroupHealth Service MeasureImmunization ArmWCV ArmControl ArmImmunization ServicesUp-to-date at 12 mo, %*767771Vaccines by 12 mo, mean (SD), No.9.7 (3.0)10.1 (3.1)9.4 (3.4)Days under-immunized at 12 mo (maximum, 365 d), mean (SD); median105 (105); 57109 (104); 62119 (111); 67Well-Child Care ServicesAttended ≥4 WCVs (maximum, 5), %575850WCVs attended by age 12 mo, mean (SD), No.3.4 (1.4)3.5 (1.3)3.2 (1.5)Abbreviation: WCV, well-child care visit.*Three diphtheria-tetanus-acellular pertussis, 2 polio (either oral or inactivated), 2 Haemophilis influenzaeb, and 2 hepatitis B inoculations.In the immunization, WCV, and control arms of the study, children averaged 6.0, 6.8, and 5.3 primary care visits, and 2.1, 2.0, and 2.2 emergency department and urgent care visits, respectively (Table 4). The median total health care charges for the children in the 3 study arms were $1136; $1236; and $1079, respectively. None of the health care utilization and charges outcomes were statistically significant, although the intervention arms did show a trend toward more primary care visits (P= .10).Table 4. Health Service Utilization and Charges by Study ArmVariableImmunization ArmWCV ArmControl ArmPValue*Visits, mean No.All primary care6.046.755.31.10Specialty care0.320.450.39.32Inpatient admissions0.150.160.15.81ED and urgent care2.091.972.19.96Other care†0.360.350.36.89≥1 Visit, %WCVs989997.17Primary care9910098.23Specialty care121413.59Inpatient admissions121212.96ED and urgent care666567.90Per-patient cost, mean/median, $Primary care885/7411002/772828/676.98Specialty care57/063/056/0.75Inpatient920/01091/01001/0.56ED and urgent care299/120269/75316/114.78Other care†80/097/062/0.40Total outpatient charges1339/10981468/11911280/1033.99Total health care charges2258/11362559/12362281/1079.10Abbreviations: ED, emergency department; WCV, well-child care visit.*All comparisons are among 3 study arms, using hierarchical models and adjusting for the clinic as the unit of randomization.†Other care is other health services, including laboratory visits.After adjusting for clinics as random effects and significant covariates, the immunization intervention did not demonstrate a statistically significant increase in immunization or WCV receipt or with health care utilization or charges (Table 5). This intervention was associated with a trend (P<.10) toward higher immunization receipt, a greater number of immunizations and WCVs, and higher total charges for health care. The WCV intervention was associated with an increased number of immunizations (risk ratio, 1.06; 95% confidence interval [CI], 1.02-1.10) and greater total charges for health care (risk ratio for the log of total charges, 1.02; 95% CI, 1.01-1.04), as well as trends toward a greater number of WCVs (risk ratio, 1.07; 95% CI, 0.98-1.16) and increased immunization up-to-date rates (rate ratio, 1.14; 95% CI, 0.99-1.29). There was no difference in the use of urgent or emergency health care or inpatient health services.Table 5. Multivariable Analysis of Intervention Effects on WCV and Immunization Receipt and on Health Care Utilization and Charges*OutcomeRatio (95% CI)†Immunization ArmWCV ArmImmunizations up-to-date1.09 (0.97-1.20)‡1.14 (0.99-1.29)‡WCVs up-to-date1.02 (0.57-1.47)‡1.31 (0.70-1.93)‡No. of immunizations1.02 (0.99-1.05)§1.06 (1.02-1.10)§No. of WCVs1.04 (0.97-1.12)§1.07 (0.98-1.16)§Used urgent or emergency care1.00 (0.68-1.31)‡1.01 (0.66-1.36)‡Used inpatient services1.07 (0.80-1.34)‡1.00 (0.67-1.32)‡Log total charges for health care1.01 (1.00-1.02)§1.02 (1.01-1.04)§Abbreviations: CI, confidence interval; WCV, well-child care visit.*Random clinic effects included in mixed models and adjusted for the following covariates: race or ethnicity, presence of a chronic medical condition, insurance status, primary language, maternal country of birth, number of prenatal visits, and trimester at which prenatal care began. Covariates were entered into the regression models only if they were not comparable between interventions and were related to the outcome of interest (P<.20 for both).†Compared with the control arm. Boldfaced numbers indicate a significant association.‡Rate ratio.§Risk ratio.COMMENTWe assessed the impact of patient-based (intensive reminder/recall) and clinic-based (AFIX) interventions on immunization and well-child care receipt in a clinic-based cluster randomized controlled trial with a prospectively enrolled birth cohort of disadvantaged inner-city children. The interventions resulted in a modest improvement in the number of immunizations delivered and a small but not statistically significant improvement in WCV and immunization up-to-date rates. The interventions had no effect on health care utilization and charges, except for a modest increase in the log of total health care charges in the WCV arm.In other studies,assessment and feedback at the level of the clinic and reminder/recall systems at the level of the patient have improved childhood immunization rates. In this trial, we used both strategies, which were associated with 5% to 6% higher receipt of adequate immunizations and 7% to 8% higher receipt of WCVs in the intervention arms. This finding is consistent with previous effects for AFIX-based interventionsand is at the lower end of published effects for reminder/recall-based interventions.What were the reasons for the modest effect seen in this trial? We tracked each child in the intervention arms of the study until age 12 months. Although 1 year may not be adequate time to see the full impact of clinic-based AFIX interventions,it should be sufficient to observe a positive effect of reminder/recall interventions.In addition, we exposed every clinic and 99% of the families in the intervention groups to at least 1 intervention (see the "Results" section).First, this was a real-world trial. Our interventions took place against a backdrop of rising immunization rates in the DH system. The percentage of 2-year-olds up-to-date with all infant vaccinations rose from 38% in 1995 to 67% in the summer of 1999, the midpoint of our study. In addition, there were parallel nonstudy interventions occurring concomitantly at sites randomized to receive no intervention through our study. For example, an independent, nonstudy, clinic-based AFIX immunization intervention was active in the largest of the control clinics during the study, impacting 60% (601/1003) of the infants in the control arm.Second, there may be a ceiling effect on immunization rates in this study. The challenge of boosting up-to-date immunization status from 70% to 90% is different from boosting it from 50% to 70%, as the population that is most amenable to intervention has already been reached. We used a strict intent-to-treat analysis and did not exclude children from the denominator in the intervention arms who were identified as having moved or gone elsewhere for care by study personnel, as this would result in selective removal of children and bias favoring the intervention. However, in the immunization arm, study personnel found that children who could be defined as having moved or gone elsewhere for care by standard definitionsaccounted for approximately 20% of the cohort (data not shown). In addition, clinics in the immunization intervention arm had the highest baseline immunization rates and thus may have had less margin for improvement.In addition to children who have moved or gone elsewhere for care, in a low socioeconomic inner-city population a proportion of children are likely to be in highly mobile or homeless families, to have no telephone service, or to have parents with substantial physical or mental health burdens of their own. Such children will be difficult to reach even with the intensive interventions described herein. Disadvantaged inner-city populations may be particularly difficult to reach with such interventions,despite intensive case management strategies.Finally, by evaluating immunization coverage for the 6-month vaccines at age 12 months, we may have minimized chances to detect significant differences between intervention and control arms because of the window of catch-up time available to vaccinate underimmunized children. These differences may well have been more pronounced after age 1 year, once the 1-year vaccinations are due and the "toddler gap" in immunization coverage becomes apparent.This study has several limitations. First, statistical significance is difficult to achieve in a cluster randomized controlled trial with a relatively small number of clinics. This study design was believed to be essential to carry out interventions at the level of the clinic and the individual family to avoid contamination within each clinic. Second, because the WCV arm of the intervention had lower baseline immunization rates than the control and immunization arms, we might have missed a positive impact of the WCV intervention on immunization rates. Third, because of the size of the association of the intervention with WCV (7%-8% increase) and immunization receipt (5%-6% increase), we could have missed an impact of improved childhood preventive care on health care utilization and charges. For example, a recent studyin the DH system demonstrates that children enrolled in the Child Health Insurance Plan are more likely to have a WCV and less likely to have a visit to the emergency department compared with uninsured children. However, we detected an increase in the number of immunizations delivered and in the log of total charges for health care. Finally, we analyzed a cohort of children born at DH based on data from the DH system and one other hospital in Denver. Approximately 20% of our cohort was lost to follow-up. We did not capture health care utilization data for children who had moved away from central Denver. However, more than 70% of this birth cohort still actively used the DH system at age 1 year, and 10% of the cohort had additional health care utilization captured at a second Denver hospital. For immunization rates, true population-based data will require citywide or statewide registries.In conclusion, we found that a combination of patient- and< clinic-based interventions targeted at increasing preventive health visits could increase the number of immunizations delivered to a cohort of children. However, this study also suggests that methods found in some settings to increase immunization up-to-date rates may not be as effective in a population of inner-city socioeconomically disadvantaged children. The challenge remains to find ways to provide full pediatric preventive health services to children in the greatest pockets of need.What This Study AddsDespite rising childhood immunization rates in the United States, underimmunization remains a problem in some inner-city populations. Patient- and clinic-based interventions have been shown to increase childhood immunization rates in a variety of settings. This study showed that patient- and clinic-based interventions targeted at increasing well-child care can be used to increase immunization delivery in an inner-city population. However, patient- and clinic-based interventions used to increase childhood immunizations in other settings may not be as effective in a population of inner-city socioeconomically disadvantaged children.US Department of Health and Human ServicesHealthy People 2010, Volume I: Understanding and Improving Health and Objectives for Improving Health.2nd ed. Washington, DC: US Dept of Health and Human Services; November 2000.Centers for Disease Control and PreventionNational, state, and urban area vaccination coverage levels among children aged 19-35 months—United States, 2001.MMWR Morb Mortal Wkly Rep.2002;51:664-666.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=12197212&dopt=AbstractJAbramsonLPickeringUS immunization policy.JAMA.2002;287:505-509.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11798374&dopt=AbstractNational Vaccine Advisory CommitteeStrategies to sustain success in childhood immunizations.JAMA.1999;282:363-370.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=10432034&dopt=AbstractJGindlerWAtkinsonLMarkowitzSHutchinsEpidemiology of measles in the United States in 1989 and 1990.Pediatr Infect Dis J.1992;11:841-846.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=1408483&dopt=AbstractTAKenyonMAMatuckGStrohPersistent low immunization coverage among inner-city preschool children despite access to free vaccine.Pediatrics.1998;101:612-616.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9521942&dopt=AbstractCenters for Disease Control and PreventionVaccination coverage by race/ethnicity and poverty level among children aged 19-35 months—United States, 1996.MMWR Morb Mortal Wkly Rep.1997;46:963-969.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9347908&dopt=AbstractRKlevensETLumanUS children living in and near poverty: risk of vaccine-preventable diseases.Am J Prev Med.2001;20(suppl):41-46.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11331131&dopt=AbstractDDanielsRJilesRKlevensGHerreraUndervaccinated African-American preschoolers: a case of missed opportunities.Am J Prev Med.2001;20(suppl):61-68.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11331134&dopt=AbstractJSantoliSSetiaLRodewaldImmunization pockets of need: science and practice.Am J Prev Med.2000;19(suppl):89-98.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11024333&dopt=AbstractGFairbrotherSFriedmanKADuMontKSLobachMarkers for primary care: missed opportunities to immunize and screen for lead and tuberculosis by private physicians serving large numbers of inner-city Medicaid-eligible children.Pediatrics.1996;97:785-790.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=8657515&dopt=AbstractWCBordleyPAMargolisCMLannonThe delivery of immunizations and other preventive services in private practices.Pediatrics.1996;97:467-473.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=8632930&dopt=AbstractLRodewaldPSzilagyiTShiuhIs underimmunization a marker for insufficient utilization of preventive and primary care?Arch Pediatr Adolesc Med.1995;149:393-397.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=7704167&dopt=AbstractGFreedSClarkDPathmanRSchectmanInfluences on the receipt of well-child visits in the first two years of life.Pediatrics.1999;103:864-869.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=10103323&dopt=AbstractASheferJFritchleyJStevensonLinking WIC and immunization services to improve preventive health care among low-income children in WIC.J Public Health Manag Pract.2002;8:56-65.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11889853&dopt=AbstractRBrennerBSimons-MortonBBhaskarPrevalence and predictors of immunization among inner-city infants: a birth cohort study.Pediatrics.2001;108:661-670.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11533333&dopt=AbstractASBatesFDWolinskyPersonal, financial and structural barriers to immunization in socioeconomically disadvantaged urban children.Pediatrics.1998;101:591-596.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9521939&dopt=AbstractDWoodCDonald-SherbourneNHalfonFactors related to immunization status among inner-city Latino and African-American preschoolers.Pediatrics.1995;96:295-301.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=7630688&dopt=AbstractLAMillerREHoffmanAEBaronRisk factors for delayed immunization against measles, mumps and rubella in Colorado two-year-olds.Pediatrics.1994;94:213-219.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=8036076&dopt=AbstractJKBoboJLGalePBThapaSGFWassilakRisk factors for delayed immunization in a random sample of 1163 children from Oregon and Washington.Pediatrics.1993;91:308-314.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=8424004&dopt=AbstractPGSzilagyiCBordleyJVannEffect of patient reminder/recall interventions on immunization rates: a review.JAMA.2000;284:1820-1827.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11025835&dopt=AbstractPGSzilagyiSSchafferLShoneReducing geographic, racial, and ethnic disparities in childhood immunization rates by using reminder/recall interventions in urban primary care practices.Pediatrics.2002;110:e58.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=12415064&dopt=AbstractCWLeBaronMChaneyALBaughmanImpact of measurement and feedback on vaccination coverage in public clinics, 1988-1994.JAMA.1997;277:631-635.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9039880&dopt=AbstractCBordleyAChelminskiPMargolisThe effect of audit and feedback on immunization delivery: a systematic review.Am J Prev Med.2000;18:343-350.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=10788739&dopt=AbstractCWLeBaronTMercerMSMassoudiChanges in clinic vaccination coverage after institution of measurement and feedback in 4 states and 2 cities.Arch Pediatr Adolesc Med.1999;153:879-886.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=10437765&dopt=AbstractNot AvailableVaccine-preventable diseases: improving vaccination coverage in children, adolescents, and adults: a report on recommendations from the Task Force on Community Preventive Services.MMWR Recomm Rep.1999;48:1-15.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=10428099&dopt=AbstractNHughartPVivierARossAre immunizations an incentive for well-child visits?Arch Pediatr Adolesc Med.1997;151:690-695.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9232043&dopt=AbstractHDMustinVLHoltFAConnellAdequacy of well-child care and immunizations in US infants born in 1988.JAMA.1994;272:1111-1115.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=7933323&dopt=AbstractPDiehrDMartinTKoepsellACheadleBreaking the matches in a paired t-test for community interventions when the number of pairs is small.Stat Med.1995;14:1491-1504.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=7481187&dopt=AbstractEFDiniMChaneyRLMoolenaarCWLeBaronInformation as intervention: how Georgia used vaccination coverage data to double public sector vaccination coverage in seven years.J Public Health Manag Pract.1996;2:45-49.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=10186655&dopt=AbstractTSchlenkerSSukhanCSwensonImproving vaccination coverage through accelerated measurement and feedback.JAMA.1998;280:1482-1483.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9809725&dopt=AbstractCenters for Disease Control and PreventionProgress in development of immunization registries—United States, 1999.MMWR Morb Mortal Wkly Rep.2000;49:274-278.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=10778791&dopt=AbstractADavidsonPMelinkovichBRenfrewImproved immunization registry accuracy with progressive clinical application.Am J Prev Med.2003;24:276-280.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=12657348&dopt=AbstractNational Center for Health StatisticsInternational Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM).Washington, DC: National Center for Health Statistics; 1991. DHHS publication (PHS) 91-1260.Centers for Disease Control and PreventionNotice to readers: recommended childhood immunization schedule—United States, 1999.MMWR Morb Mortal Wkly Rep.1999;48:8-16.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9933122&dopt=AbstractJGayJMuldoonJNeffLWingProfiling the health service needs of populations: description and uses of the NACHRI classification of congenital and chronic health conditions.Pediatr Ann.1997;26:655-663.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9397444&dopt=AbstractMPMHRutten-Van MolkenEKAVDoorslaerRCJAVVlietStatistical analysis of cost outcomes in a randomized controlled clinical trial.Health Econ.1994;3:333-345.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=7827649&dopt=AbstractCenters for Disease Control and PreventionGuidelines for Assessing Vaccination Records in Private and Public Settings.Atlanta, Ga: National Immunization Program, Centers for Disease Control and Prevention; 1997.GFairbrotherGFreedJThompsonMeasuring immunization coverage.Am J Prev Med.2000;19(suppl):78-88.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11024332&dopt=AbstractAKempeELoweryKAPearsonImmunization recall: effectiveness and barriers to success in an urban teaching clinic.J Pediatr.2001;139:630-635.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11713438&dopt=AbstractDWoodNHalfonCDonald-SherbourneIncreasing immunization rates among inner-city, African American children: a randomized trial of case management.JAMA.1998;279:29-34.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9424040&dopt=AbstractMASchusterDLWoodNDuanRMMazelCDSherbourneNHalfonUtilization of well-child care services for African-American infants in a low-income community: results of a randomized, controlled case management/home visitation intervention.Pediatrics.1998;101:999-1005.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9606226&dopt=AbstractSEisertPGabowEffect of child health insurance plan enrollment on the utilization of health care services by children using a public safety net system.Pediatrics.2002;110:940-945.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=12415034&dopt=AbstractCorresponding author: Simon J. Hambidge, MD, PhD, Denver Health Medical Center, 777 Bannock St, Mail Code 0132, Denver, CO 80204 (e-mail: firstname.lastname@example.org).Accepted for publication July 18, 2003.This study was funded by cooperative agreement TS 252-13/15 from the Association of Teachers of Preventive Medicine and the Centers for Disease Control and Prevention.We are indebted to Luis Espinosa Organista, Kristen Miller, BA, and Susan Macaskill-Deloach, BA, for their hard work and creativity in implementing the interventions and in data collection.
JAMA Pediatrics – American Medical Association
Published: Feb 1, 2004