Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

The Status of Immunization Measurement and Feedback in the United States

The Status of Immunization Measurement and Feedback in the United States BackgroundA large body of scientific and programmatic data has demonstrated that provider measurement and feedback raises immunization coverage. Starting in 1995, Congress required that all states measure childhood immunization coverage in all public clinics, and federal grant guidelines encourage private practice measurements.ObjectivesTo determine state immunization measurement rates and examine risk factors for high rates.MethodsReview of 1997 state reports, with correlation of measurement rates to birth cohort and provider numbers, public/private proportions, and vaccine distribution systems.ResultsOf the 9505 public clinics, 48% were measured; 4 states measured all clinics; 29 measured a majority. Measurement rates were highest for Health Department clinics (67%), lower for community/migrant health centers (39%), and lowest for other clinics (22%). Rates were highly correlated among categories of clinics (r>+0.308, P<.03), and the fewer the clinics, the higher the measurement rates (r = −0.351, P= .01), but other factors were not significant. Of the 41,378 private practices, 6% were measured; no state measured all its practices; 1 measured a majority. Private practice measurement rates were not correlated to public clinic measurement rates or other factors examined. Of the 50,883 total providers, 14% were measured; no state measured all providers; 2 measured a majority. A trend toward higher measurement rates was found in states with fewer providers (r = −0.266, P= .06).ConclusionsThree years after the congressional mandate, only a minority of public clinics and very few private practices had their immunization coverage measured. Greater efforts will be needed to assure implementation of the intervention.IMMUNIZATION coverage rose from 53% to 89% over 7 years in Georgia public clinics following implementation of a measurement and feedback intervention involving low direct costs ($80,000 per year), which was characterized as Assessment, Feedback, Incentives, and eXchange of information (AFIX).Other states and cities adapted the Georgia AFIX model to local conditions, achieving comparable results at comparable costs,and successful efforts have been made to export the approach to private practices.Based on these data and a substantial body of scientific literature,annual provider-based measurement and feedback received strong formal endorsement from a wide range of public and private organizations. It was made one of the Standards for Pediatric Immunization Practices by the American Academy of Pediatrics, Elk Grove Village, Ill, and the American Medical Association, Chicago, Ill.It was the subject of a special recommendation by the Advisory Committee on Immunization Practices.It was strongly recommended by the Task Force on Community Preventive Services.It was one of the key strategies identified by the National Vaccine Advisory Committee, Washington, DC.Starting in 1995, Congress directed the Centers for Disease Control and Prevention (CDC) to "ensure that all states receiving IAP [federal immunization] funds conduct annual provider site assessments in all public clinics, using CDC-approved methodology."Federal immunization grant application guidelines (§8a) additionally "encourage periodic private provider assessments." In part because provider denominators were not readily available, the extent to which measurements actually took place was not systematically monitored. Recently, we used state and other data to obtain provider denominators for 1997,and we now report the first evaluation of the implementation of provider-based immunization measurements in the United States.METHODSDATA SOURCESAll data were for 1997 and aggregated by state (District of Columbia treated as a state). Provider denominators were estimated using Vaccines for Children (VFC) and National Immunization Survey data, by methods previously described.Briefly, VFC supplies health care providers with federally purchased vaccines for administration to children who are uninsured, Medicaid eligible, Native American, or Alaska native. Vaccines for Children instructions define a provider site as a health care facility at which routine vaccinations are administered to children and where medical records are kept. From each state's annual VFC report, we abstracted counts of sites that were enrolled in VFC. To estimate the total number of provider sites in each state, VFC counts were adjusted upward based on state-specific estimates of the proportion of providers who were enrolled in VFC according to the National Immunization Survey.Public/Private ProportionsThe National Immunization Survey also furnished information on the proportion of infants vaccinated in the public/private sector. We assigned children with mixed sector vaccination histories (25%) to the private sector as their likely medical home.Measurement RatesFrom each state's annual report to CDC, we abstracted the number of provider sites whose immunization coverage had been measured by CDC-approved methods, sent the data via fax to each state for review and/or correction, followed with a telephone call, then resent the final data via fax for confirmation. We restricted our study to measurements performed by state immunization programs and did not attempt to examine the extent to which private bodies, such as managed care organizations, measured immunization rates among providers.DATA ANALYSISAll analyses were performed using the computer program SAS 6.12 (SAS Institute, Cary, NC). The unit of analysis was the state, and the outcome was the rate of measurement. We used the Spearman rank correlation test to examine the correlation of measurement rates to the number of providers, size of the birth cohort, and public/private sector proportions. We used the Wilcoxon rank sum test to examine the association of measurement rates with different state vaccine distribution systems: (1) Universal (state distribution of all Advisory Committee on Immunization Practices–approved routine childhood vaccines to all providers), (2) VFC full (VFC available to both public and private providers), and (3) VFC public (VFC available only to public providers).RESULTSPUBLIC CLINIC MEASUREMENTSOf the 9505 total clinics, 48% were measured; 4 states (8%) measured all clinics; 29 (57%), a majority (Table 1). The rate of measurement was highest (67%) for Health Department clinics (22 states measured all; 38, a majority), lower (39%) for community/migrant health centers (14 states measured all; 24, a majority), and lowest (22%) for other clinics (5 states measured all; 12, a majority). Measurement rates were highly correlated among categories of clinics (r>+0.308, P<.03). The fewer clinics, the higher the rate of measurement (r = −0.351, P= .01), but measurement rates were not associated with size of the birth cohort (P= .69), proportion of children vaccinated in the public sector (P= .42), or vaccine distribution system (P= .50).Table 1. Public Clinic Immunization Measurement Rates, 1997State*Birth Cohort†Public Clinic Immunization Measurement RatesHealth DepartmentCommunity/Migrant Health CentersOtherTotalTotalPublic%ClinicsMeasured%ClinicsMeasured%ClinicsMeasured%ClinicsMeasured%1 Mo74,74717,9392411311310017171000001301301002 Ore43,895790118494910088100292910086861003 NH14,443101073310066100454510054541004 RI12,38814871200020201003310023231005 Mont10,851271325464610032675252100101100996 Ind83,44720,02724949096441006868100166162987 Iowa36,933997227999910033100191684121118988 Utah43,88511,41026666610013129232678280989 Neb23,32741991811111002210075729688859710 Hawaii17,381104361313100101010040027238511 Ga118,34737,871322302301006614210002962448212 Okla48,11015,87633919110055100338241291048113 Vt66672674121210030000015128014 Nev27,039865232221986856326186956427515 Ala61,03816,480278867767050710001581177416 SC51,90415,57130747410017171003413125927417 Idaho18,5944649254443981616100241484607118 Kan37,44611,9833211010595811339001571066819 Fla192,55636,5861916716710091898002661756620 Ill180,89825,3261417216495123975448893492216321 Tex333,88970,1172140423257291966202145726353966222 Ariz75,753833311373710035195433515105615823 Va91,99017,4781911872613516461300166885324 Wyo642412211930301001100171658315325 Md70,6275621846286180000054285226 La66,01221,78433108108100561254362181125127 NJ113,1414526410567641212100721724189965128 WVa20,75239431954541006427424200160815129 NM26,874483718515098221005012103535130 Wis66,60210,6561610874691154557916176885031 Colo56,53910,74219633251531528422457158714532 NC107,01323,5432210510095126312570003011314433 Mich133,62129,39722434310055471218110484434 Calif526,03352,603104381964518416489236008583604235 Me13,67068453310027271001032423133544136 Ohio152,26527,4081814295676009200240954037 Pa144,23512,98191706840883034000258983838 Ky53,22818,098343091264140513183173671343739 Wash79,024790210515110026207712900206713440 Alaska97052523262626100200490077263441 Conn42,977171947811141716941083936203663342 Minn64,525258148327334222524749172533143 Del10,24351251010100300190032103144 Tenn74,57723,11931140634551002700218632945 Ark36,72020,196551126760670012100300672246 NY263,33923,701911657492831156071704671047 DC7905870110001100102202121048 Mass82,448824510112006128462786245134849 ND835627573346241001300602350 SD10,2081429145400670071141281151 Miss42,74723,9385611000200010013100Total3,894,968694,4261847763191671731675392998655229505452148*States listed in descending order of proportion of total clinics measured and, within identical proportions, in descending order of number of clinics measured.†Total indicates all live births according to state natality data; public, infants vaccinated entirely in the public sector according to the National Immunization Survey; and %, public/total.PRIVATE PRACTICE MEASUREMENTSOf the 41,378 private practices, 2436 (6%) were measured; no state measured all its practices; 1 (2%) measured a majority (Table 2). Rates of private practice measurement were not associated with rates of public clinic measurement (P= .19), number of practices (P= .58), or the other factors examined (P>.18).Table 2. Private Practice Immunization Measurement Rates, 1997State*Birth Cohort†Private Practice Immunization Measurement RatesTotalPrivate%PracticesMeasured%1 Mont10,8518138759992932 Ala61,03844,55773300137463 Me13,67012,9869532899304 Wyo642452038114933225 Nev27,03918,3866813327206 WVa20,75216,8098124446197 Utah43,88532,4747414823168 Minn64,52561,9449646772159 Conn42,97741,25796450641410 Ill180,898155,5728618082171211 Idaho18,59413,94575103121212 Calif526,033473,4299047595221113 Iowa36,93326,96173311341114 Pa144,235131,2539124552531015 Wis66,60255,94584351361016 Mo74,74756,8077652035717 NM26,87422,0368216911718 NY263,339239,638916182302519 Mich133,621104,22478165478520 NC107,01383,4707877335521 NH14,43313,4229325912522 Ga118,34780,4756875631423 Ariz75,75367,4208958726424 Colo56,53945,7968144717425 Hawaii17,38116,3389435313426 Va91,99074,5118157720327 Kan37,44625,463682308328 Del10,2439730951554329 Tex333,889263,77279161160230 Wash79,02471,1219091021231 Ky53,22835,130662455232 Fla192,556155,97081310040133 Mass82,44874,20390135020134 Md70,26764,6459288911135 Okla48,11032,233674977136 La66,01244,228675183137 DC79057035891362138 Neb23,32719,128822272139 Tenn74,57751,458694552040 NJ113,141108,6159616642041 Alaska97057181742491042 Ind83,44763,419767151043 Ohio152,265124,8578211390044 SC51,90436,332704320045 Vt66676400963880046 Ore43,89535,993823380047 RI12,38810,901882000048 ND83565598671610049 SD10,2088778861580050 Ark36,72016,524451170051 Miss42,74718,8084411200Total3,894,9683,200,5438241,37824366*States listed in descending order of proportion of practices measured and, within identical proportions, in descending order of number of practices measured and total number of practices.†Total indicates all live births according to state natality data; private, infants vaccinated entirely or partly in private sector according to the National Immunization Survey; and %, private/total.TOTAL (PUBLIC AND PRIVATE) PROVIDER MEASUREMENTSOf the 50,883 total providers, 6957 (14%) were measured; no state measured all, 2 (4%) measured a majority. A trend toward higher measurement rates was found in states with fewer providers (r = −0.266, P= .06), but rates were not associated with other factors examined (P>.12).COMMENTThree years after Congress mandated measurement of immunization coverage in all public clinics, 48% of public clinics were measured, and 4 states reported complete compliance. Since measurement of a majority of clinics may be sufficient to achieve coverage impact, as many as 29 states may have exposed their public sector children to some potential benefit from the intervention.Higher clinic measurement rates correlated with fewer public clinics but not with birth cohort size or public sector proportion, suggesting that low burden of effort may have been more important to implementation than impact potential or access to federal resources (size of the cohort is a major determinant of federal immunization grant funding). Reinforcing this notion, measurement rates were highest (67%) for Health Department clinics, probably because of easier access of immunization program staff to sites under direct government control. Nevertheless, the high correlation of measurement rates among different clinic categories suggests that once an organizational commitment was made to measure clinic coverage, the intervention tended to be carried out across the public sector.The potential impact of public clinic measurements should not be underestimated: 43% of all children were vaccinated entirely (18%) or in part (25%) in the public sector in 1997, and numerous studies have suggested that children vaccinated in the public sector have lower immunization rates than those vaccinated in the private sector.Demonstrated success in implementing measurement and feedback in the public sector—where sites are fewer and easily enumerated, where large numbers of undervaccinated children are more likely to be found, and where a congressional mandate exists—would seem to be the necessary prelude to expansion of the intervention to the more complicated environment of the private sector.Apparently, most state programs agreed, since only 6% of the nation's private practices were measured, and just 1 state measured a majority of its private practices, despite the fact that 82% of the birth cohort likely had its medical home in the private sector, and private practice measurements were encouraged by federal grant guidelines. Actual exposure of private sector children to the intervention may have been higher in certain states; for example, in 1996 one state measured all its private providers, found that about 30% of practices administered about 80% of vaccinations, and in 1997 concentrated on these sites.However, data do not suggest that many states followed this targeted approach.Our study demonstrates that the weight of a large body of scientific evidence, repeated demonstrations of "real-world" effectiveness, widespread support from medical bodies and advisory panels, and even a congressional mandate may not be sufficient to assure swift generalization of a low-cost intervention, particularly in the absence of monitoring. Based on these findings and advice from states and private provider groups, CDC has been considering an initiative to improve measurement rates across the nation with features that include intervention-specific funding, use of VFC program data to help focus efforts on providers who vaccinate large numbers of children, simplification of measurement methods, and timely monitoring and feedback of measurement rates. The last factor may be most important, given the intervention's publicized premise of "What gets measured gets done."Our study focused on implementation of measurement and feedback during 1 year and thus could not examine the impact of the intervention on raising immunization rates over time, though this is clearly the ultimate objective of any intervention monitoring system. Furthermore, though we were able to calculate provider site measurement rates, we could not determine the proportion of children exposed to the intervention, a more fundamental index of success. Finally, we relied on state self-reporting, which seems more likely to overestimate than underestimate measurement rates in a context of required compliance with a fiscal mandate.Previous studies have suggested that measurement and feedback can initially produce dramatic apparent rises in individual provider-measured coverage (up to 20-35 percentage points) generated by improvements in individual provider record keeping but that the intervention's effect on population-based immunization rates is more likely to be a steady and incremental increase of 5 percentage points annually.Hence, it may be some years before the impact of even a fully implemented program is apparent in a state's immunization coverage. Impact on US immunization rates would require widespread implementation across states, as was required by Congress and encouraged by federal immunization grant guidelines but not actually carried out.Measurement and feedback involving less than 15% of providers is unlikely to have any considerable effect nationally. A public program based on "What gets measured gets done" has had to learn for itself that lack of monitoring of an intervention can be associated with lack of implementation.CWLeBaronMChaneyALBaughmanImpact of measurement and feedback on vaccination coverage in public clinics, 1988-1994.JAMA.1997;277:631-635.CWLeBaronJTMercerMSMassoudiChanges in clinic vaccination coverage after institution of measurement and feedback in 4 states and 2 cities.Arch Pediatr Adolesc Med.1999;153:879-886.TSchlenkerSSukhanCSwensonImproving vaccination coverage through accelerated measurement and feedback [letter].JAMA.1998;280:1482-1483MSMassoudiJWalshSStokleyAssessing immunization performance of private practitioners in Maine: impact of the assessment, feedback, incentives, and exchange strategy.Pediatrics.1999;103:1218-1223.JBuffingtonKMBellFMLaForceand the Genesee Hospital Medical StaffA target-based model for increasing influenza immunizations in private practice.J Gen Intern Med.1991;6:204-209.RWKouidesBLewisNMBennettA performance-based incentive program for influenza immunization in the elderly.Am J Prev Med.1993;9:250-254.RWMorrowADGoodingCClarkImproving physicians' preventive health care behavior through peer review and financial incentives.Arch Fam Med.1995;4:165-169.RSThompsonSHTaplinTAMcAfeeMTMandelsonAESmithPrimary and secondary prevention services in clinical practice: twenty years' experience in development, implementation, and evaluation.JAMA.1995;273:1130-1135.Ad Hoc Working Group for the Development of Standards of Pediatric Immunization PracticesStandards for pediatric immunization practices.JAMA.1993;269:1817-1822.Not AvailableRecommendations of the Advisory Committee on Immunization Practices: programmatic strategies to increase vaccination rates: assessment and feedback of provider-based vaccination coverage information.MMWR Morb Mortal Wkly Rep.1996;45:219-220.Centers for Disease Control and PreventionVaccine-preventable diseases: improving vaccination coverage in children, adolescents, and adults: a report on recommendations of the Task Force on Community Preventive Services.MMWR Morb Mortal Wkly Rep.1999;48(RR-8):1-15.The National Vaccine Advisory CommitteeStrategies to sustain success in childhood immunizations.JAMA.1999;282:363-370.Not AvailableDepartments of Labor, Health and Human Services, and Education and Related Agencies Appropriation Bill: 1995.103rd Cong, 2nd Sess (1994). Report 103-318:57.CWLeBaronBLyonsMMassoudiJStevensonThe childhood vaccination infrastructure of the United States.In: Programs and abstracts of the 2000 Pediatric Academic Societies and American Academy of Pediatrics joint meeting; May 15, 2000; Boston, Mass. Abstract 1206.JBoboJGalePTharpaSWassilakRisk factors for delayed immunization in a random sample of 1163 children from Oregon and Washington.Pediatrics.1993;91:308-314.AMorrowJRosenthalHLakkisA population-based study of access to immunization among urban Virginia children served by public, private, and military health care systems.Pediatrics [serial online].1998;101:e5. Available at: http://www.pediatrics.org/cgi/content/full/101/2/e5?. Accessed June 9, 2000.MMassoudiCWLeBaronSStokleyEDiniLBelmontRSchultzVaccination levels and access to care during a pertussis outbreak in a rural population.In: Program and abstracts of the Ambulatory Pediatric Association Meeting; May 3, 1998; New Orleans, La. Abstract 109.EMaesLERodewaldVCoronadoMBattagliaDIzraelTEzzati-RiceImmunization providers for impoverished preschool children: results from the 1997 National Immunization Survey.In: Program and abstracts of the Ambulatory Pediatric Association Meeting; May 5, 1998; New Orleans, La. Abstract 365.Accepted for publication February 28, 2000.We wish to thank all the state and local immunization program staff who contributed their time and effort to providing the data for this report and without whose tireless efforts, implementation of measurement and feedback would never take place.Corresponding author: Charles W. LeBaron, MD, Mail: MS E-61, NIP, CDC, Atlanta, GA 30333; Federal Express: Room 5314, 12 Corporate Square Blvd, Atlanta, GA 30329 (e-mail: cel3@cdc.gov). http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png JAMA Pediatrics American Medical Association

The Status of Immunization Measurement and Feedback in the United States

Loading next page...
 
/lp/american-medical-association/the-status-of-immunization-measurement-and-feedback-in-the-united-zp49jwSQ6X

References (17)

Publisher
American Medical Association
Copyright
Copyright 2000 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
ISSN
2168-6203
eISSN
2168-6211
DOI
10.1001/archpedi.154.8.832
Publisher site
See Article on Publisher Site

Abstract

BackgroundA large body of scientific and programmatic data has demonstrated that provider measurement and feedback raises immunization coverage. Starting in 1995, Congress required that all states measure childhood immunization coverage in all public clinics, and federal grant guidelines encourage private practice measurements.ObjectivesTo determine state immunization measurement rates and examine risk factors for high rates.MethodsReview of 1997 state reports, with correlation of measurement rates to birth cohort and provider numbers, public/private proportions, and vaccine distribution systems.ResultsOf the 9505 public clinics, 48% were measured; 4 states measured all clinics; 29 measured a majority. Measurement rates were highest for Health Department clinics (67%), lower for community/migrant health centers (39%), and lowest for other clinics (22%). Rates were highly correlated among categories of clinics (r>+0.308, P<.03), and the fewer the clinics, the higher the measurement rates (r = −0.351, P= .01), but other factors were not significant. Of the 41,378 private practices, 6% were measured; no state measured all its practices; 1 measured a majority. Private practice measurement rates were not correlated to public clinic measurement rates or other factors examined. Of the 50,883 total providers, 14% were measured; no state measured all providers; 2 measured a majority. A trend toward higher measurement rates was found in states with fewer providers (r = −0.266, P= .06).ConclusionsThree years after the congressional mandate, only a minority of public clinics and very few private practices had their immunization coverage measured. Greater efforts will be needed to assure implementation of the intervention.IMMUNIZATION coverage rose from 53% to 89% over 7 years in Georgia public clinics following implementation of a measurement and feedback intervention involving low direct costs ($80,000 per year), which was characterized as Assessment, Feedback, Incentives, and eXchange of information (AFIX).Other states and cities adapted the Georgia AFIX model to local conditions, achieving comparable results at comparable costs,and successful efforts have been made to export the approach to private practices.Based on these data and a substantial body of scientific literature,annual provider-based measurement and feedback received strong formal endorsement from a wide range of public and private organizations. It was made one of the Standards for Pediatric Immunization Practices by the American Academy of Pediatrics, Elk Grove Village, Ill, and the American Medical Association, Chicago, Ill.It was the subject of a special recommendation by the Advisory Committee on Immunization Practices.It was strongly recommended by the Task Force on Community Preventive Services.It was one of the key strategies identified by the National Vaccine Advisory Committee, Washington, DC.Starting in 1995, Congress directed the Centers for Disease Control and Prevention (CDC) to "ensure that all states receiving IAP [federal immunization] funds conduct annual provider site assessments in all public clinics, using CDC-approved methodology."Federal immunization grant application guidelines (§8a) additionally "encourage periodic private provider assessments." In part because provider denominators were not readily available, the extent to which measurements actually took place was not systematically monitored. Recently, we used state and other data to obtain provider denominators for 1997,and we now report the first evaluation of the implementation of provider-based immunization measurements in the United States.METHODSDATA SOURCESAll data were for 1997 and aggregated by state (District of Columbia treated as a state). Provider denominators were estimated using Vaccines for Children (VFC) and National Immunization Survey data, by methods previously described.Briefly, VFC supplies health care providers with federally purchased vaccines for administration to children who are uninsured, Medicaid eligible, Native American, or Alaska native. Vaccines for Children instructions define a provider site as a health care facility at which routine vaccinations are administered to children and where medical records are kept. From each state's annual VFC report, we abstracted counts of sites that were enrolled in VFC. To estimate the total number of provider sites in each state, VFC counts were adjusted upward based on state-specific estimates of the proportion of providers who were enrolled in VFC according to the National Immunization Survey.Public/Private ProportionsThe National Immunization Survey also furnished information on the proportion of infants vaccinated in the public/private sector. We assigned children with mixed sector vaccination histories (25%) to the private sector as their likely medical home.Measurement RatesFrom each state's annual report to CDC, we abstracted the number of provider sites whose immunization coverage had been measured by CDC-approved methods, sent the data via fax to each state for review and/or correction, followed with a telephone call, then resent the final data via fax for confirmation. We restricted our study to measurements performed by state immunization programs and did not attempt to examine the extent to which private bodies, such as managed care organizations, measured immunization rates among providers.DATA ANALYSISAll analyses were performed using the computer program SAS 6.12 (SAS Institute, Cary, NC). The unit of analysis was the state, and the outcome was the rate of measurement. We used the Spearman rank correlation test to examine the correlation of measurement rates to the number of providers, size of the birth cohort, and public/private sector proportions. We used the Wilcoxon rank sum test to examine the association of measurement rates with different state vaccine distribution systems: (1) Universal (state distribution of all Advisory Committee on Immunization Practices–approved routine childhood vaccines to all providers), (2) VFC full (VFC available to both public and private providers), and (3) VFC public (VFC available only to public providers).RESULTSPUBLIC CLINIC MEASUREMENTSOf the 9505 total clinics, 48% were measured; 4 states (8%) measured all clinics; 29 (57%), a majority (Table 1). The rate of measurement was highest (67%) for Health Department clinics (22 states measured all; 38, a majority), lower (39%) for community/migrant health centers (14 states measured all; 24, a majority), and lowest (22%) for other clinics (5 states measured all; 12, a majority). Measurement rates were highly correlated among categories of clinics (r>+0.308, P<.03). The fewer clinics, the higher the rate of measurement (r = −0.351, P= .01), but measurement rates were not associated with size of the birth cohort (P= .69), proportion of children vaccinated in the public sector (P= .42), or vaccine distribution system (P= .50).Table 1. Public Clinic Immunization Measurement Rates, 1997State*Birth Cohort†Public Clinic Immunization Measurement RatesHealth DepartmentCommunity/Migrant Health CentersOtherTotalTotalPublic%ClinicsMeasured%ClinicsMeasured%ClinicsMeasured%ClinicsMeasured%1 Mo74,74717,9392411311310017171000001301301002 Ore43,895790118494910088100292910086861003 NH14,443101073310066100454510054541004 RI12,38814871200020201003310023231005 Mont10,851271325464610032675252100101100996 Ind83,44720,02724949096441006868100166162987 Iowa36,933997227999910033100191684121118988 Utah43,88511,41026666610013129232678280989 Neb23,32741991811111002210075729688859710 Hawaii17,381104361313100101010040027238511 Ga118,34737,871322302301006614210002962448212 Okla48,11015,87633919110055100338241291048113 Vt66672674121210030000015128014 Nev27,039865232221986856326186956427515 Ala61,03816,480278867767050710001581177416 SC51,90415,57130747410017171003413125927417 Idaho18,5944649254443981616100241484607118 Kan37,44611,9833211010595811339001571066819 Fla192,55636,5861916716710091898002661756620 Ill180,89825,3261417216495123975448893492216321 Tex333,88970,1172140423257291966202145726353966222 Ariz75,753833311373710035195433515105615823 Va91,99017,4781911872613516461300166885324 Wyo642412211930301001100171658315325 Md70,6275621846286180000054285226 La66,01221,78433108108100561254362181125127 NJ113,1414526410567641212100721724189965128 WVa20,75239431954541006427424200160815129 NM26,874483718515098221005012103535130 Wis66,60210,6561610874691154557916176885031 Colo56,53910,74219633251531528422457158714532 NC107,01323,5432210510095126312570003011314433 Mich133,62129,39722434310055471218110484434 Calif526,03352,603104381964518416489236008583604235 Me13,67068453310027271001032423133544136 Ohio152,26527,4081814295676009200240954037 Pa144,23512,98191706840883034000258983838 Ky53,22818,098343091264140513183173671343739 Wash79,024790210515110026207712900206713440 Alaska97052523262626100200490077263441 Conn42,977171947811141716941083936203663342 Minn64,525258148327334222524749172533143 Del10,24351251010100300190032103144 Tenn74,57723,11931140634551002700218632945 Ark36,72020,196551126760670012100300672246 NY263,33923,701911657492831156071704671047 DC7905870110001100102202121048 Mass82,448824510112006128462786245134849 ND835627573346241001300602350 SD10,2081429145400670071141281151 Miss42,74723,9385611000200010013100Total3,894,968694,4261847763191671731675392998655229505452148*States listed in descending order of proportion of total clinics measured and, within identical proportions, in descending order of number of clinics measured.†Total indicates all live births according to state natality data; public, infants vaccinated entirely in the public sector according to the National Immunization Survey; and %, public/total.PRIVATE PRACTICE MEASUREMENTSOf the 41,378 private practices, 2436 (6%) were measured; no state measured all its practices; 1 (2%) measured a majority (Table 2). Rates of private practice measurement were not associated with rates of public clinic measurement (P= .19), number of practices (P= .58), or the other factors examined (P>.18).Table 2. Private Practice Immunization Measurement Rates, 1997State*Birth Cohort†Private Practice Immunization Measurement RatesTotalPrivate%PracticesMeasured%1 Mont10,8518138759992932 Ala61,03844,55773300137463 Me13,67012,9869532899304 Wyo642452038114933225 Nev27,03918,3866813327206 WVa20,75216,8098124446197 Utah43,88532,4747414823168 Minn64,52561,9449646772159 Conn42,97741,25796450641410 Ill180,898155,5728618082171211 Idaho18,59413,94575103121212 Calif526,033473,4299047595221113 Iowa36,93326,96173311341114 Pa144,235131,2539124552531015 Wis66,60255,94584351361016 Mo74,74756,8077652035717 NM26,87422,0368216911718 NY263,339239,638916182302519 Mich133,621104,22478165478520 NC107,01383,4707877335521 NH14,43313,4229325912522 Ga118,34780,4756875631423 Ariz75,75367,4208958726424 Colo56,53945,7968144717425 Hawaii17,38116,3389435313426 Va91,99074,5118157720327 Kan37,44625,463682308328 Del10,2439730951554329 Tex333,889263,77279161160230 Wash79,02471,1219091021231 Ky53,22835,130662455232 Fla192,556155,97081310040133 Mass82,44874,20390135020134 Md70,26764,6459288911135 Okla48,11032,233674977136 La66,01244,228675183137 DC79057035891362138 Neb23,32719,128822272139 Tenn74,57751,458694552040 NJ113,141108,6159616642041 Alaska97057181742491042 Ind83,44763,419767151043 Ohio152,265124,8578211390044 SC51,90436,332704320045 Vt66676400963880046 Ore43,89535,993823380047 RI12,38810,901882000048 ND83565598671610049 SD10,2088778861580050 Ark36,72016,524451170051 Miss42,74718,8084411200Total3,894,9683,200,5438241,37824366*States listed in descending order of proportion of practices measured and, within identical proportions, in descending order of number of practices measured and total number of practices.†Total indicates all live births according to state natality data; private, infants vaccinated entirely or partly in private sector according to the National Immunization Survey; and %, private/total.TOTAL (PUBLIC AND PRIVATE) PROVIDER MEASUREMENTSOf the 50,883 total providers, 6957 (14%) were measured; no state measured all, 2 (4%) measured a majority. A trend toward higher measurement rates was found in states with fewer providers (r = −0.266, P= .06), but rates were not associated with other factors examined (P>.12).COMMENTThree years after Congress mandated measurement of immunization coverage in all public clinics, 48% of public clinics were measured, and 4 states reported complete compliance. Since measurement of a majority of clinics may be sufficient to achieve coverage impact, as many as 29 states may have exposed their public sector children to some potential benefit from the intervention.Higher clinic measurement rates correlated with fewer public clinics but not with birth cohort size or public sector proportion, suggesting that low burden of effort may have been more important to implementation than impact potential or access to federal resources (size of the cohort is a major determinant of federal immunization grant funding). Reinforcing this notion, measurement rates were highest (67%) for Health Department clinics, probably because of easier access of immunization program staff to sites under direct government control. Nevertheless, the high correlation of measurement rates among different clinic categories suggests that once an organizational commitment was made to measure clinic coverage, the intervention tended to be carried out across the public sector.The potential impact of public clinic measurements should not be underestimated: 43% of all children were vaccinated entirely (18%) or in part (25%) in the public sector in 1997, and numerous studies have suggested that children vaccinated in the public sector have lower immunization rates than those vaccinated in the private sector.Demonstrated success in implementing measurement and feedback in the public sector—where sites are fewer and easily enumerated, where large numbers of undervaccinated children are more likely to be found, and where a congressional mandate exists—would seem to be the necessary prelude to expansion of the intervention to the more complicated environment of the private sector.Apparently, most state programs agreed, since only 6% of the nation's private practices were measured, and just 1 state measured a majority of its private practices, despite the fact that 82% of the birth cohort likely had its medical home in the private sector, and private practice measurements were encouraged by federal grant guidelines. Actual exposure of private sector children to the intervention may have been higher in certain states; for example, in 1996 one state measured all its private providers, found that about 30% of practices administered about 80% of vaccinations, and in 1997 concentrated on these sites.However, data do not suggest that many states followed this targeted approach.Our study demonstrates that the weight of a large body of scientific evidence, repeated demonstrations of "real-world" effectiveness, widespread support from medical bodies and advisory panels, and even a congressional mandate may not be sufficient to assure swift generalization of a low-cost intervention, particularly in the absence of monitoring. Based on these findings and advice from states and private provider groups, CDC has been considering an initiative to improve measurement rates across the nation with features that include intervention-specific funding, use of VFC program data to help focus efforts on providers who vaccinate large numbers of children, simplification of measurement methods, and timely monitoring and feedback of measurement rates. The last factor may be most important, given the intervention's publicized premise of "What gets measured gets done."Our study focused on implementation of measurement and feedback during 1 year and thus could not examine the impact of the intervention on raising immunization rates over time, though this is clearly the ultimate objective of any intervention monitoring system. Furthermore, though we were able to calculate provider site measurement rates, we could not determine the proportion of children exposed to the intervention, a more fundamental index of success. Finally, we relied on state self-reporting, which seems more likely to overestimate than underestimate measurement rates in a context of required compliance with a fiscal mandate.Previous studies have suggested that measurement and feedback can initially produce dramatic apparent rises in individual provider-measured coverage (up to 20-35 percentage points) generated by improvements in individual provider record keeping but that the intervention's effect on population-based immunization rates is more likely to be a steady and incremental increase of 5 percentage points annually.Hence, it may be some years before the impact of even a fully implemented program is apparent in a state's immunization coverage. Impact on US immunization rates would require widespread implementation across states, as was required by Congress and encouraged by federal immunization grant guidelines but not actually carried out.Measurement and feedback involving less than 15% of providers is unlikely to have any considerable effect nationally. A public program based on "What gets measured gets done" has had to learn for itself that lack of monitoring of an intervention can be associated with lack of implementation.CWLeBaronMChaneyALBaughmanImpact of measurement and feedback on vaccination coverage in public clinics, 1988-1994.JAMA.1997;277:631-635.CWLeBaronJTMercerMSMassoudiChanges in clinic vaccination coverage after institution of measurement and feedback in 4 states and 2 cities.Arch Pediatr Adolesc Med.1999;153:879-886.TSchlenkerSSukhanCSwensonImproving vaccination coverage through accelerated measurement and feedback [letter].JAMA.1998;280:1482-1483MSMassoudiJWalshSStokleyAssessing immunization performance of private practitioners in Maine: impact of the assessment, feedback, incentives, and exchange strategy.Pediatrics.1999;103:1218-1223.JBuffingtonKMBellFMLaForceand the Genesee Hospital Medical StaffA target-based model for increasing influenza immunizations in private practice.J Gen Intern Med.1991;6:204-209.RWKouidesBLewisNMBennettA performance-based incentive program for influenza immunization in the elderly.Am J Prev Med.1993;9:250-254.RWMorrowADGoodingCClarkImproving physicians' preventive health care behavior through peer review and financial incentives.Arch Fam Med.1995;4:165-169.RSThompsonSHTaplinTAMcAfeeMTMandelsonAESmithPrimary and secondary prevention services in clinical practice: twenty years' experience in development, implementation, and evaluation.JAMA.1995;273:1130-1135.Ad Hoc Working Group for the Development of Standards of Pediatric Immunization PracticesStandards for pediatric immunization practices.JAMA.1993;269:1817-1822.Not AvailableRecommendations of the Advisory Committee on Immunization Practices: programmatic strategies to increase vaccination rates: assessment and feedback of provider-based vaccination coverage information.MMWR Morb Mortal Wkly Rep.1996;45:219-220.Centers for Disease Control and PreventionVaccine-preventable diseases: improving vaccination coverage in children, adolescents, and adults: a report on recommendations of the Task Force on Community Preventive Services.MMWR Morb Mortal Wkly Rep.1999;48(RR-8):1-15.The National Vaccine Advisory CommitteeStrategies to sustain success in childhood immunizations.JAMA.1999;282:363-370.Not AvailableDepartments of Labor, Health and Human Services, and Education and Related Agencies Appropriation Bill: 1995.103rd Cong, 2nd Sess (1994). Report 103-318:57.CWLeBaronBLyonsMMassoudiJStevensonThe childhood vaccination infrastructure of the United States.In: Programs and abstracts of the 2000 Pediatric Academic Societies and American Academy of Pediatrics joint meeting; May 15, 2000; Boston, Mass. Abstract 1206.JBoboJGalePTharpaSWassilakRisk factors for delayed immunization in a random sample of 1163 children from Oregon and Washington.Pediatrics.1993;91:308-314.AMorrowJRosenthalHLakkisA population-based study of access to immunization among urban Virginia children served by public, private, and military health care systems.Pediatrics [serial online].1998;101:e5. Available at: http://www.pediatrics.org/cgi/content/full/101/2/e5?. Accessed June 9, 2000.MMassoudiCWLeBaronSStokleyEDiniLBelmontRSchultzVaccination levels and access to care during a pertussis outbreak in a rural population.In: Program and abstracts of the Ambulatory Pediatric Association Meeting; May 3, 1998; New Orleans, La. Abstract 109.EMaesLERodewaldVCoronadoMBattagliaDIzraelTEzzati-RiceImmunization providers for impoverished preschool children: results from the 1997 National Immunization Survey.In: Program and abstracts of the Ambulatory Pediatric Association Meeting; May 5, 1998; New Orleans, La. Abstract 365.Accepted for publication February 28, 2000.We wish to thank all the state and local immunization program staff who contributed their time and effort to providing the data for this report and without whose tireless efforts, implementation of measurement and feedback would never take place.Corresponding author: Charles W. LeBaron, MD, Mail: MS E-61, NIP, CDC, Atlanta, GA 30333; Federal Express: Room 5314, 12 Corporate Square Blvd, Atlanta, GA 30329 (e-mail: cel3@cdc.gov).

Journal

JAMA PediatricsAmerican Medical Association

Published: Aug 1, 2000

There are no references for this article.