Evaluation of the Impact of a Rotavirus Vaccine Program on Pediatric Acute Gastroenteritis Hospitalizations: Estimating the Overall Effect Attributable to the Program as a Whole and as a Per-Unit Change in Rotavirus Vaccine Coverage

Evaluation of the Impact of a Rotavirus Vaccine Program on Pediatric Acute Gastroenteritis... Abstract Estimation of the overall effect of a vaccine program is essential, but the effect is typically estimated for a whole program. We estimated the overall effect of the Quebec rotavirus vaccine program, launched in November 2011, and the effect for each 10% increase in rotavirus vaccine coverage on pediatric hospitalizations for all-cause acute gastroenteritis. We implemented negative binomial regressions adjusted for seasonality, long-term trends, and infection dynamics, to estimate the effect of the vaccine program as: 1) a dichotomous variable, representing program presence/absence, and linear term to account for changes in trend in the period after the program began; and 2) a continuous variable, representing rotavirus vaccine coverage. Using exposure 1, the vaccine program was associated with a 51.2% (95% confidence interval (CI): 28.5, 66.7) relative decline in adjusted weekly hospitalization rates for all-cause acute gastroenteritis as of December 28, 2014. Using exposure 2, a 10% increase in rotavirus ≥1-dose coverage was associated with a 7.1% (95% CI: 3.5, 10.5) relative decline in adjusted weekly rates, with maximum coverage of 87.0% associated with a 47.2% (95% CI: 26.9, 61.9) relative decline. Estimation of the overall effect attributable to a change in vaccine coverage might be a useful addition to standard measurement of the overall effect. overall effect, program evaluation, rotavirus, vaccine program Not all vaccine programs are created equal. With the adoption of a vaccine, policy makers must make numerous allocation decisions, which can influence the overall effect, or impact, of a vaccine program on disease burden. For example, decisions regarding the choice of vaccine(s), the population(s) targeted, and the schedule and dosage can influence the effect of a vaccine program on a disease outcome from a population perspective (1). Prior to program implementation, mathematical models are useful for estimating potential effects of different program scenarios (2–4), but model estimates are often uncertain (3, 5, 6). Evaluation of the overall effect using real-world data after implementation is therefore warranted (3–5), and comparison of results across similar jurisdictions with different vaccine programs might provide insights into the consequences of program decisions on population health outcomes. Yet, estimates of the overall effect from jurisdictions with differing levels of vaccine coverage are not easily compared. The comparison is not straightforward because the overall effect represents the average of effects in vaccinated and unvaccinated individuals, weighted by the proportion of each group in the population (i.e., as determined by vaccine coverage) (4, 7–9). One strategy for making measures of overall effect comparable is to characterize the relative reduction in a disease outcome that is attributable to a per-unit change in vaccine coverage. Doing so separates the overall effect of a program from the level of vaccine coverage achieved in the setting of the program and should make estimates of overall effect easier to compare across jurisdictions. We have provided a real-world example of this strategy, estimating, as a function of vaccine coverage, the overall effect of a rotavirus vaccine program on pediatric hospitalizations for all-cause acute gastroenteritis (AGE) in Quebec, Canada. Before the vaccine was available, rotavirus produced annual epidemics with peak disease through the winter and spring in Quebec (10, 11), causing approximately 72% of AGE hospitalizations among children less than 5 years of age during the 6-month epidemic period (11). We estimated the overall effect of a vaccination program as a rate ratio attributable to a 10% change in rotavirus vaccine coverage. We also characterized the overall effect of the whole program, in accordance with standard practice, and discuss the utility of our nonstandard approach. METHODS To estimate the overall effect of the Quebec rotavirus vaccine program, we conducted a time-series analysis of weekly all-cause AGE hospitalizations between June 1, 1998, and December 28, 2014, among a random sample of children aged 8–35 months, residing in greater Montreal, Quebec. Study setting The Province of Quebec implemented a publicly funded rotavirus vaccine program in November 2011. Under the program, monovalent rotavirus vaccine (Rotarix; GlaxoSmithKline, Brentford, United Kingdom) was available at no cost to Quebec residents, with routine vaccination recommended at 2 and 4 months of age, and series initiation and completion by 20 weeks and 8 months of age, respectively (12). Prior to the program, pentavalent (RotaTeq; Merck & Co., Inc., Kenilworth, New Jersey) and monovalent rotavirus vaccines were licensed in Canada in August 2006 (13) and October 2007 (14) and recommended for use by the National Advisory Committee on Immunization in January 2008 (15) and July 2010 (16), respectively; however recipients were required to pay for vaccination. Greater Montreal is the largest metropolitan area in Quebec and the second largest in Canada, with a population of approximately 4 million residents (17). All Quebec residents are covered by provincial health insurance, except for some temporary residents (18). Study population We included greater Montreal residents aged 8–35 months who were members of the Population Health Record (PopHR) cohort in our analyses. As described elsewhere (19), the PopHR cohort was created for research on health indicators, and represents an open, population-based, deidentified random sample of approximately 25% of greater Montreal residents with provincial health insurance. Health-care utilization data are tracked longitudinally for each cohort member from entry until death, emigration, or the cohort end date. Each year, cohort losses are replaced with a random sample of births and immigrants to maintain the population sampling fraction. The initial PopHR cohort was selected in 1998, with annual updates through 2014 at the time of our analyses. Data sources The PopHR cohort was created via linkages of provincial health insurance administrative databases maintained by the government organization: Régie de l’assurance maladie du Québec. Year and month for each cohort member’s dates of birth, cohort entry, and cohort exit are provided, and member health-care utilization data is tracked via an anonymized, unique identifier. Hospitalization data for each cohort member are obtained from a software system that captures administrative, clinical, and demographic information on hospital discharges (Maintenance et exploitation des données pour l’étude de la clientèle hospitalière (20)), used by all Quebec hospitals. In this system, administrative codes are recorded for each hospital stay using the International Classification of Diseases (ICD), Ninth Revision, adapted for Quebec (ICD-9-QC), until 2006, and International Classification of Diseases, Tenth Revision, adapted for Canada (ICD-10-CA), afterward. Exposure ascertainment In separate analyses, we characterized the Quebec rotavirus vaccine program as either: 1) the presence or absence of the program; or 2) rotavirus vaccine coverage in the population. Vaccine program We divided the study period into periods before and after implementing the program, with the period before the program defined as weeks before November 14, 2011, and the period after the program began as weeks including or after this date. We designated November 14, 2011, as the effective program start date in consideration of the program’s implementation date and a lag of 2 weeks to account for the biological delay in the immune response to vaccination, a duration typically used in rotavirus direct vaccine-effectiveness studies (21–24). Vaccine coverage We estimated weekly ≥1- and ≥2-dose rotavirus vaccine coverage among Montreal residents aged 8–35 months using coverage data from control participants enrolled in an external test-negative case-control study of rotavirus vaccine effectiveness in Quebec (23). For reasons described previously (25), we assumed that rotavirus vaccine coverage among these participants was a reasonable proxy to estimate population rotavirus vaccine coverage, given that their vaccination status was representative of the case source population and their inclusion was unlikely to be associated with vaccine uptake. We used these coverage data because an immunization registry did not exist in Quebec before June 2014 (26), and provincial coverage survey estimates were not available as a time series (27). Coverage estimates derived using this method were previously validated via comparison with provincial coverage surveys, and simulations suggest minimal bias when highly accurate diagnostic tests are used to ascertain controls (25). In our main analyses, we used coverage data from eligible controls recruited at hospitals between February 2011 and December 2014 in Montreal and Sherbrooke, Quebec, because coverage did not vary by geographic location (25). Only controls aged 8 months or older at the time of participation in the external study were eligible for inclusion in our coverage estimates. This age cutoff was chosen because a control’s rotavirus vaccine history was assessed as of their participation date and was not expected to change after 8 months of age (28). Using these eligibility criteria, we formed a retrospective cohort to assess weekly rotavirus vaccine coverage, with controls entering and exiting the cohort at 8 and 36 months of age, respectively. We assessed ≥1- and ≥2-dose rotavirus coverage each week to form a weekly time series, because we expected population vaccine coverage to increase rapidly following program implementation. Due to concerns that the age distribution of cohort members might differ from the PopHR cohort over time, weekly coverage estimates were age-standardized via direct standardization using the weekly distribution of children aged ≥8 and <12 months, 1 year, and 2 years of age in the PopHR cohort, via the R “epitools” package (R Foundation for Statistical Computing, Vienna, Austria) (29). Using these methods, the earliest date that we could derive age-standardized coverage estimates was April 11, 2011, or approximately 29 weeks before the implementation of the Quebec rotavirus vaccine program. Prior to that date, we assumed rotavirus vaccine coverage was 0%, but we modified this assumption in sensitivity analyses. Outcome ascertainment We examined weekly counts of AGE hospitalizations as our outcome. We aggregated outcomes by week because AGE hospitalizations occurred infrequently in our population, so daily counts included many zeros, and aggregation to monthly rates might have inadvertently obscured rotavirus epidemic patterns. We used AGE in consideration of potential changes in laboratory testing practices for rotavirus and/or ICD coding directives for AGE over time (30). To identify relevant hospitalizations, we examined the primary diagnosis ICD code recorded for each hospital stay, and we counted hospitalizations with a matching primary code for AGE, as previously defined by Quebec researchers Bernard et al. (31). Statistical analyses We used negative-binomial generalized linear models with a log-link function to estimate the overall effect of the Quebec vaccine program on weekly counts of AGE hospitalizations among children 8–35 months of age. A continuous, linear term indicating the week number of the time series, subsequently referred to as week time, and a weekly offset term to account for the denominator of children aged 8–35 months in the PopHR cohort were included in all models. In separate models, we characterized the exposure as: 1) the presence or absence of the rotavirus vaccine program; or 2) continuous vaccine coverage in the population. For exposure 1, we created a dichotomous variable to represent the rotavirus vaccination program, coded as 0 in weeks before the program and as 1 in weeks after the program began. Because we expected the program effect to change over time due to increasing coverage, we considered adjustment for a change in trend in the period after the program began (3, 32) in models adjusting for seasonality, long-term trends, and infection dynamics. This change was captured via inclusion of a product term between the binary program variable and a linear variable accounting for week or season number in the period after the program began. We coded the final study week or season as 0 in the latter term to facilitate derivation of the full program effect and 95% confidence intervals of the vaccine program achieved at study completion (i.e., December 28, 2014) (33). Therefore, in models with a linear trend, the exponentiated model coefficient for our dichotomous variable represented the rate ratio comparing fitted weekly rates of AGE hospitalizations in the final week or season of the study period versus the mean rate in the period before the program. We also report the percent relative decrease in weekly rates estimated as (1 – rate ratio) × 100, to facilitate interpretability. For exposure 2, we used weekly age-standardized proportions of either ≥1- or ≥2-dose rotavirus vaccine coverage multiplied by 100 (in separate models), with weeks before April 11, 2011, coded as 0. The exponentiated model coefficient for exposure 2 represented the rate ratio of weekly AGE hospitalizations corresponding to a 1% increase in vaccine coverage. To facilitate interpretability, we also report rate ratios and the percent relative decrease in our weekly outcome associated with a 10% increase in vaccine coverage and with an increase to the maximum coverage level observed during the study period. Because this representation assumes a log-linear relationship between our exposure and outcome, we examined the validity of this characterization in separate analyses (Web Appendix 1, available at https://academic.oup.com/aje). We considered several different approaches to adjust for rotavirus seasonality, and we implemented a series of models with season included as either: 1) time strata; 2) a Fourier transformation of week time (t) using combinations of annual (2πt52) and biennial (2πt104) sine and cosine terms; 3) a cubic B-spline transformation of calendar week time (i.e., using a January start date for week number; with the number of annual knots varying between 3 and 10); or 4) a combination of these approaches (34). To account for possible shifts in rotavirus seasonality after the vaccination program, we also evaluated inclusion of separate Fourier terms for periods before and after the program began (35). We considered adjustment for long-term trends by inclusion of a linear annual term for rotavirus season with June as a starting month, as used previously in Quebec (36). Due to the annual dynamic between the susceptible-infected-recovered populations within a rotavirus season, we also considered adjustment for the “population recovered and immune to rotavirus,” defined as the cumulative number of participants hospitalized for AGE per annual rotavirus season (37, 38), with separate terms for before and after the program began. This approach assumes that: 1) hospitalizations for AGE are a surrogate for rotavirus hospitalizations, and 2) people who contract rotavirus within each season are immune for the remainder of the season after recovery. While these assumptions represent an oversimplification, they are expected to generally hold because rotavirus was the predominant cause of Canadian pediatric AGE in the period before the vaccination program (10, 11, 31, 36, 39–41), and approximately one-half of our cohort was replaced each rotavirus season by an incoming birth cohort, which presumably represented children susceptible to rotavirus. We selected the best-fitting model for each exposure by minimizing the Akaike information criterion (AIC), which includes a penalty to discourage overfitting (42, 43), and we compared model fit among final models selected for each exposure by calculating AIC differences. We used Burnham and Anderson’s (44) general criteria to denote AIC differences of 1–2, 4–7, and >10 as models with similar, moderately better, or conclusively better fit, respectively. In situations where the best candidate models had similar fit, we selected the most parsimonious model as the preferred model. After selection of the best-fitting model for each exposure, we considered inclusion of one of the following autoregressive terms lagged by 1 week to address residual autocorrelation: 1) model residuals, 2) our outcome, 3) the logarithm of our outcome + 1, or 4) the logarithm of our outcome + 0.5. This approach has been suggested by several authors to reduce strong residual autocorrelation in infectious-disease time series induced by the contagious nature of the outcome (37, 45, 46). We chose a lag of 1 week in consideration of the short incubation period of rotavirus (47). For terms 3 and 4, the addition of 1 or 0.5 to outcome counts was implemented to prevent taking the logarithm of zero. We selected the best-fitting autoregressive term in consideration of its effect on residual autocorrelation, examined via autocorrelation and partial autocorrelation functions (48), and AIC differences. For additional model selection details, see Web Appendix 2. For each exposure, we examined the fit of final selected models via plots of the residuals over time and plots of fitted model values versus observed values (48). We also inspected the appropriateness of the negative binomial distribution by examining the relationship between the theoretical and actual residual variance for fitted values using the method developed by Ver Hoef and Boveng (49), as implemented by Imai et al. (37). We conducted multiple sensitivity analyses to examine the robustness of our findings (Web Appendix 3). All analyses were performed in RStudio, version 0.98.1103 (Boston, Massachusetts), with R, version 3.2.5 (R Foundation for Statistical Computing) (50). Ethics approval The McGill University Institutional Review Board approved our study and the formation of the PopHR cohort. The external vaccine-effectiveness study was approved by institutional review boards at each hospital. RESULTS An average of 24,058 children aged 8–35 months were under surveillance in the PopHR cohort each week (range, 21,396–27,061). The total number of AGE hospitalizations over the study period was 2,505, with the number of weekly hospitalizations ranging from 0 to 18. At the effective start date of the rotavirus vaccine program, age-adjusted rotavirus ≥1- and ≥2-dose vaccine coverage among children aged 8–35 months was 6.9% (95% confidence interval (CI): 2.6, 17.0), and 6.6% (95% CI: 2.5, 16.8), respectively. Between the start date of our coverage estimates and the study end date, age-adjusted ≥1-dose vaccine coverage ranged from 3.9% to 87.0%, while ≥2-dose vaccine coverage ranged from 3.9% to 72.5%. Time series plots of weekly AGE hospitalization rates are presented in conjunction with the dichotomous and continuous representations of our exposure in Figure 1. Figure 1. View largeDownload slide Weekly rates of all-cause acute gastroenteritis (AGE) hospitalizations among children aged 8–35 months, Montreal, Quebec, Canada, 1998–2014. A) The periods before and after the vaccine program, used for the dichotomous characterization of our exposure. B) The estimated percentage of rotavirus ≥1- and ≥2-dose vaccine coverage, used for the continuous representation of the exposure. Figure 1. View largeDownload slide Weekly rates of all-cause acute gastroenteritis (AGE) hospitalizations among children aged 8–35 months, Montreal, Quebec, Canada, 1998–2014. A) The periods before and after the vaccine program, used for the dichotomous characterization of our exposure. B) The estimated percentage of rotavirus ≥1- and ≥2-dose vaccine coverage, used for the continuous representation of the exposure. Model selection For both exposures, the final selected models included the following terms to adjust for seasonality, long-term trends, and infection dynamics: a linear term for week time, cubic B-spline transformation of calendar week time with 6 annual knots, separate annual and biennial sine terms for the periods before and after the program began, a linear term for annual rotavirus season, and separate linear terms for the “population recovered and immune” for the periods before and after the program began. For our binary exposure, we additionally included a product term to account for a weekly change in trend in the program period after the program began. To account for residual autocorrelation in the final models, we selected an autoregressive term of 1-week lagged model residuals, with little evidence of residual autocorrelation in the models adjusting for seasonality, long-term trends, and infection dynamics after its implementation. Candidate and final models are presented in Web Tables 1–7. Analyses of nonlinearity between vaccine coverage and the log outcome in adjusted models demonstrated that a linear characterization of our exposure was appropriate (Web Appendix 1, Web Tables 8–11, Web Figures 1–2). The fit of the model with ≥1-dose rotavirus coverage was similar to that with ≥2-dose coverage (AIC difference = 0.2). Models with either ≥1- or ≥2-dose coverage had similar to moderately better fit than the dichotomous exposure model (AIC differences = 2.6 and 2.8, respectively). Plots of the fitted counts from each exposure model versus observed counts of AGE hospitalizations are presented in Figure 2. A negative binomial distribution was deemed appropriate to model all exposures. Figure 2. View largeDownload slide Observed counts versus fitted model estimates of weekly all-cause acute gastroenteritis (AGE) hospitalizations among children aged 8–35 months, Montreal, Quebec, Canada, 1998–2014. Plots of fitted values are derived from the final selected models with the exposure characterized as: A) a dichotomous variable representing presence or absence of the program and linear trend term for the period after the program began; B) a continuous variable representing ≥1-dose rotavirus vaccine coverage; C) a continuous variable representing ≥2-dose rotavirus vaccine coverage. Figure 2. View largeDownload slide Observed counts versus fitted model estimates of weekly all-cause acute gastroenteritis (AGE) hospitalizations among children aged 8–35 months, Montreal, Quebec, Canada, 1998–2014. Plots of fitted values are derived from the final selected models with the exposure characterized as: A) a dichotomous variable representing presence or absence of the program and linear trend term for the period after the program began; B) a continuous variable representing ≥1-dose rotavirus vaccine coverage; C) a continuous variable representing ≥2-dose rotavirus vaccine coverage. Impact of the vaccination program On average, weekly rates of AGE hospitalizations in the periods before and after the program began were 1.341 (95% CI: 1.238, 1.443) and 0.696 (95% CI: 0.606, 0.786) per 10,000 children, respectively, accounting for a relative decrease of 48.1% (95% CI: 37.6, 56.8) in crude weekly rates in the period after the program began. After adjustment, the Quebec vaccine program was associated with a 51.2% (95% CI: 28.5, 66.7) relative decline in weekly rates of AGE hospitalizations among children 8–35 months of age at study completion. Prior to the uptake of any vaccine, the mean weekly rate of AGE hospitalizations was 1.369 (95% CI: 1.263, 1.474) per 10,000 children aged 8–35 months. We estimated that a 10% increase in ≥1-dose rotavirus coverage was associated with an 11.9% (95% CI: 8.8, 14.9) relative decrease in crude weekly rates of AGE hospitalizations, or a 7.1% (95% CI: 3.5, 10.5) relative decrease in adjusted rates. Where the continuous exposure was characterized as a 10% increase in ≥2-dose coverage, we estimated a crude relative weekly rate reduction of 13.8% (95% CI: 10.3, 17.3), or an adjusted relative decline of 8.3% (95% CI: 4.2, 12.3). When we generalized our adjusted estimates to the maximum coverage achieved during the study period, we estimated a relative decline in weekly AGE hospitalization rates of 47.2% (95% CI: 26.9, 61.9) for an increase in ≥1-dose coverage to 87.0%, or 46.8% (95% CI: 26.7, 61.3) for an increase in ≥2-dose coverage to 72.5%. Adjusted model coefficients, rate ratios, and 95% confidence intervals are presented in Table 1. Results of sensitivity analyses were consistent with primary analyses (Web Figure 3). Table 1. Adjusted Model Coefficients and Rate Ratios for the Overall Effect of the November 2011 Rotavirus Vaccine Programa on Weekly All-Cause Acute Gastroenteritis Hospitalizations Among Children 8–35 Months of Age, Montreal, Quebec, Canada, 1998–2014 Exposure Model Coefficient 95% CI RR 95% CI Unit Dichotomous (with linear trend) −0.71699 −1.09831, −0.33568 0.488 0.333, 0.715 Full program effect estimated as of December 28, 2014 (3 years after program implementation) Continuous  ≥1-dose coverage −0.00735 −0.01110, −0.00360 0.993 0.989, 0.996 per 1% increase 0.929 0.895, 0.965 per 10% increase 0.528 0.381, 0.731 per increase to max coverage (87.0%)  ≥2-dose coverage −0.00870 −0.01310, −0.00429 0.991 0.987, 0.996 per 1% increase 0.917 0.877, 0.958 per 10% increase 0.532 0.387, 0.733 per increase to max coverage (72.5%) Exposure Model Coefficient 95% CI RR 95% CI Unit Dichotomous (with linear trend) −0.71699 −1.09831, −0.33568 0.488 0.333, 0.715 Full program effect estimated as of December 28, 2014 (3 years after program implementation) Continuous  ≥1-dose coverage −0.00735 −0.01110, −0.00360 0.993 0.989, 0.996 per 1% increase 0.929 0.895, 0.965 per 10% increase 0.528 0.381, 0.731 per increase to max coverage (87.0%)  ≥2-dose coverage −0.00870 −0.01310, −0.00429 0.991 0.987, 0.996 per 1% increase 0.917 0.877, 0.958 per 10% increase 0.532 0.387, 0.733 per increase to max coverage (72.5%) Abbreviations: CI, confidence interval; RR, rate ratio. a Model coefficients represent the log(RR) for each exposure derived from fully adjusting negative-binomial generalized linear models implemented with a log-link function. In separate models, the Quebec rotavirus vaccine program was characterized as either as a dichotomous variable with linear trend in the period after the program or as a continuous variable, representing the percentage of ≥1- or ≥2-dose rotavirus vaccine coverage in the population. Table 1. Adjusted Model Coefficients and Rate Ratios for the Overall Effect of the November 2011 Rotavirus Vaccine Programa on Weekly All-Cause Acute Gastroenteritis Hospitalizations Among Children 8–35 Months of Age, Montreal, Quebec, Canada, 1998–2014 Exposure Model Coefficient 95% CI RR 95% CI Unit Dichotomous (with linear trend) −0.71699 −1.09831, −0.33568 0.488 0.333, 0.715 Full program effect estimated as of December 28, 2014 (3 years after program implementation) Continuous  ≥1-dose coverage −0.00735 −0.01110, −0.00360 0.993 0.989, 0.996 per 1% increase 0.929 0.895, 0.965 per 10% increase 0.528 0.381, 0.731 per increase to max coverage (87.0%)  ≥2-dose coverage −0.00870 −0.01310, −0.00429 0.991 0.987, 0.996 per 1% increase 0.917 0.877, 0.958 per 10% increase 0.532 0.387, 0.733 per increase to max coverage (72.5%) Exposure Model Coefficient 95% CI RR 95% CI Unit Dichotomous (with linear trend) −0.71699 −1.09831, −0.33568 0.488 0.333, 0.715 Full program effect estimated as of December 28, 2014 (3 years after program implementation) Continuous  ≥1-dose coverage −0.00735 −0.01110, −0.00360 0.993 0.989, 0.996 per 1% increase 0.929 0.895, 0.965 per 10% increase 0.528 0.381, 0.731 per increase to max coverage (87.0%)  ≥2-dose coverage −0.00870 −0.01310, −0.00429 0.991 0.987, 0.996 per 1% increase 0.917 0.877, 0.958 per 10% increase 0.532 0.387, 0.733 per increase to max coverage (72.5%) Abbreviations: CI, confidence interval; RR, rate ratio. a Model coefficients represent the log(RR) for each exposure derived from fully adjusting negative-binomial generalized linear models implemented with a log-link function. In separate models, the Quebec rotavirus vaccine program was characterized as either as a dichotomous variable with linear trend in the period after the program or as a continuous variable, representing the percentage of ≥1- or ≥2-dose rotavirus vaccine coverage in the population. DISCUSSION After adjustment for seasonality and other factors, we estimated that a 10% increase in ≥1-dose rotavirus vaccine coverage was associated with a 7% relative decrease in the weekly rate of AGE hospitalizations among children aged 8–35 months. For an increase to 87% rotavirus ≥1-dose coverage, or the highest coverage level observed during the study period, we estimated that adjusted weekly rates of AGE hospitalization declined by 47%. To our knowledge, only Bar-Zeev et al. (51) have also estimated the overall effect of a 10% increase in rotavirus vaccine coverage (in Malawi); unfortunately, their data are not directly comparable to ours due to differences between study outcomes, populations, and rotavirus disease dynamics. In comparison with our nonstandard estimate, we found that weekly rates of AGE hospitalizations were lower in the period after the program began—by 51% at study completion, 3 years after the implementation of the Quebec vaccine program. While these analyses are valid to ascertain the overall effect of the whole program up to that point, and similar to results found for maximum coverage levels, they are not easily applied to future periods, particularly if there is a change in population vaccine coverage. Direct comparison of the overall program effect with otherwise similar jurisdictions with differing vaccine coverage is also not straightforward, because interpretation of program-related reductions must be made in the context of population vaccine coverage. While most systematic reviews or meta-analyses that compare overall effects between vaccine programs report jurisdictional vaccine coverage in their narrative or a results table (52–56), it is difficult for readers to synthesize this information. Other systematic reviews or meta-analyses account for differences in vaccine coverage by stratifying overall effect estimates by coverage (57, 58); while this is a better method for comparison, coverage strata in these systematic reviews or meta-analyses were overly broad (i.e., coverage <50% vs. ≥50%, or low/moderate/high coverage), which might be insufficient to adjust for differences in program effects due to vaccine coverage. While it is also possible to directly adjust for vaccine coverage in meta-analyses using statistical models (59), the overall effect estimates from primary studies might not reflect a period after a program's implementation that has uniform vaccine coverage. This might be particularly true for the evaluation of new vaccine programs, such as rotavirus, where program estimates represent a period during which population coverage is rapidly changing (60–64); thus, direct adjustment for vaccine coverage in pooled analyses might not be possible or accurate. In each of these scenarios, estimates of the overall effect attributable to a per-unit change in vaccine coverage might be particularly useful to extrapolate estimates to other coverage levels, directly compare estimates between otherwise similar jurisdictions, or to incorporate into statistical models pooling program effect estimates from differing jurisdictions. For these reasons, we suggest that the overall effect attributable to a per-unit change in vaccine coverage might be a useful measure to supplement traditional methods to estimate the overall effect of a vaccine program, where vaccine coverage data are available. Limitations Our analyses have some limitations. First, we caution that our results are intended to estimate the overall effect of vaccination at the time of our evaluation, and thus, are generalizable only to settings with comparable rotavirus disease dynamics and to vaccine-coverage levels observed within the range examined in the present study. Further, although we found that a log-linear relationship was appropriate to model the effect of increasing vaccine coverage, this assumption should be examined by researchers in future analyses and might not be accurate for rotavirus vaccine-coverage levels beyond those observed in this study. Second, our approach ignores the potential implications of vaccination in other population subgroups on indirect effects; however, this is not an issue in the present study because our study population represents the entire population eligible for vaccination, and thus, all indirect effects included in our effect estimates necessarily arise from vaccination within the study population. Where analyses do not include the entire vaccine-eligible population, we recommend that researchers account for vaccine coverage among subpopulations not included in the analyses. Third, we did not include sampling error for coverage estimates in our analyses and, given our method of ascertaining coverage, we might slightly underestimate coverage (25); nonetheless, we conducted multiple sensitivity analyses that explored the effect of differing coverage estimates, and we did not find meaningful differences in our results. Last, we were unable to ascertain the effect of the program on rotavirus hospitalizations directly; however, broadening our definition to AGE hospitalizations should only attenuate our effect estimates because only a portion of these represent rotavirus and, thus, would be vaccine-preventable. Conclusion In the evaluation of a vaccine program, researchers should consider estimating the overall effect attributable to a per-unit increase in vaccine coverage in the population, in addition to estimating the overall effect of the whole program. In our examination of the effect of the Quebec rotavirus vaccine program, we estimated that each 10% increase in ≥1-dose rotavirus vaccine coverage was associated with a 7% relative decrease in the rate of weekly AGE hospitalizations among children 8–35 months of age. Using this approach of estimating the incremental effect of increasing coverage should promote comparisons of overall effect estimates across settings with differing coverage. ACKNOWLEDGMENTS Author affiliations: Department of Epidemiology and Biostatistics, McGill University, Montreal, Quebec, Canada (Margaret K. Doll, Caroline Quach, David L. Buckeridge); Department of Microbiology, Infectious Diseases and Immunology, University of Montreal, Montreal, Quebec, Canada (Caroline Quach); and Infection Control and Prevention Unit, Division of Pediatric Infectious Diseases and Medical Microbiology, Centre Hospitalier Universitaire Sainte-Justine, University of Montreal, Montreal, Quebec, Canada (Caroline Quach). Development and maintenance of the Population Health Record cohort was financed by a grant from the Canadian Foundation of Innovation (project 15649). The external vaccine-effectiveness study was also supported by a grant from GlaxoSmithKline. No funding was received for other aspects of this study. We thank Dr. Aman Verma from the McGill University Department of Epidemiology, Biostatistics and Occupational Health for his administrative help and for sharing his insights regarding the Population Health Record data set. C.Q. has received funding from GlaxoSmithKline, Pfizer, Sage, and AbbVie (all for research grant or support). The other authors report no conflicts. Abbreviations AGE acute gastroenteritis AIC Akaike information criterion CI confidence interval ICD International Classification of Diseases PopHR Population Health Record REFERENCES 1 World Health Organization . Principles and considerations for adding a vaccine to a national immunization programme: from decision to implementation and monitoring. World Health Organization ; 2014 . http://www.who.int/immunization/programmes_systems/policies_strategies/vaccine_intro_resources/nvi_guidelines/en/. Accessed October 18, 2017. 2 Ultsch B , Damm O , Beutels P , et al. . Methods for health economic evaluation of vaccines and immunization decision frameworks: a consensus framework from a European vaccine economics community . Pharmacoeconomics . 2016 ; 34 ( 3 ): 227 – 244 . Google Scholar Crossref Search ADS PubMed 3 Mealing N , Hayen A , Newall AT Assessing the impact of vaccination programmes on burden of disease: underlying complexities and statistical methods . Vaccine . 2016 ; 34 ( 27 ): 3022 – 3029 . Google Scholar Crossref Search ADS PubMed 4 Halloran ME , Longini IM , Struchiner CJ , et al. . Design and Analysis of Vaccine Studies . New York, NY : Springer ; 2010 . 5 Newall AT , Reyes JF , Wood JG , et al. . Economic evaluations of implemented vaccination programmes: key methodological challenges in retrospective analyses . Vaccine . 2014 ; 32 ( 7 ): 759 – 765 . Google Scholar Crossref Search ADS PubMed 6 Bilcke J , Beutels P , Brisson M , et al. . Accounting for methodological, structural, and parameter uncertainty in decision-analytic models: a practical guide . Med Decis Making . 2011 ; 31 ( 4 ): 675 – 692 . Google Scholar Crossref Search ADS PubMed 7 Hanquet G , Valenciano M , Simondon F , et al. . Vaccine effects and impact of vaccination programmes in post-licensure studies . Vaccine . 2013 ; 31 ( 48 ): 5634 – 5642 . Google Scholar Crossref Search ADS PubMed 8 Halloran ME . Overview of vaccine field studies: types of effects and designs . J Biopharm Stat . 2006 ; 16 ( 4 ): 415 – 427 . Google Scholar Crossref Search ADS PubMed 9 Halloran ME , Struchiner CJ . Study designs for dependent happenings . Epidemiology . 1991 ; 2 ( 5 ): 331 – 338 . Google Scholar Crossref Search ADS PubMed 10 Ford-Jones EL , Wang E , Petric M , et al. . Hospitalization for community-acquired, rotavirus-associated diarrhea: a prospective, longitudinal, population-based study during the seasonal outbreak. The Greater Toronto Area/Peel Region PRESI Study Group. Pediatric Rotavirus Epidemiology Study for Immunization . Arch Pediatr Adolesc Med . 2000 ; 154 ( 6 ): 578 – 585 . Google Scholar Crossref Search ADS PubMed 11 Rivest P , Proulx M , Lonergan G , et al. . Hospitalisations for gastroenteritis: the role of rotavirus . Vaccine . 2004 ; 22 ( 15–16 ): 2013 – 2017 . Google Scholar Crossref Search ADS PubMed 12 Québec Ministère de la Santé et des Services Sociaux . Rotavirus Vaccine . Quebec City, Canada : Québec Ministère de la Santé et des Services Sociaux ; 2016 . 13 Health Canada . Summary Basis Decision—RotaTeq. 2007 . https://hpr-rps.hres.ca/reg-content/summary-basis-decision-detailOne.php?lang=en&linkID=SBD00130. Accessed April 11, 2018. 14 Health Canada . Summary Basis of Decision—Rotarix. 2008 . https://hpr-rps.hres.ca/reg-content/summary-basis-decision-detailOne.php?lang=en&linkID=SBD00129. Accessed April 11, 2018. 15 National Advisory Committee on Immunization . Statement on the recommended use of pentavalent human-bovine reassortant rotavirus vaccine. An Advisory Committee Statement (ACS) . Can Commun Dis Rep . 2008 ; 34 ( ACS-1 ): 1 – 33 . 16 National Advisory Committee on Immunization . Updated statement on the use of rotavirus vaccines . Can Commun Dis Rep . 2010 ; 36 ( ACS-4 ): 1 – 37 . Crossref Search ADS 17 Statistics Canada . Census profile, 2016 Census. 2017 . http://www12.statcan.gc.ca/census-recensement/2016/dp-pd/prof/index.cfm?Lang=E. Accessed March 17, 2017. 18 Régie de l’Assurance Maladie Québec . Health insurance: eligibility for the Québec Insurance Plan. http://www.ramq.gouv.qc.ca/en/citizens/health-insurance/registration/Pages/eligibility.aspx. Accessed February 1, 2017. 19 Shaban-Nejad A , Lavigne M , Okhmatovskaia A , et al. . PopHR: a knowledge-based platform to support integration, analysis, and visualization of population health data . Ann N Y Acad Sci . 2017 ; 1387 ( 1 ): 44 – 53 . Google Scholar Crossref Search ADS PubMed 20 Québec Ministère de la Santé et des Services Sociaux . Sources de données et métadonnées: MED-ECHO. 2016 . http://www.msss.gouv.qc.ca/professionnels/documentation-sources-de-donnees-et-indicateurs/sources-de-donnees-et-metadonnees/med-echo/. Accessed June 20, 2018. 21 Braeckman T , Van Herck K , Meyer N , et al. . Effectiveness of rotavirus vaccination in prevention of hospital admissions for rotavirus gastroenteritis among young children in Belgium: case-control study . BMJ . 2012 ; 345 : e4752 . Google Scholar Crossref Search ADS PubMed 22 Cortese MM , Immergluck LC , Held M , et al. . Effectiveness of monovalent and pentavalent rotavirus vaccine . Pediatrics . 2013 ; 132 ( 1 ): e25 – e33 . Google Scholar Crossref Search ADS PubMed 23 Doll MK , Buckeridge DL , Morrison KT , et al. . Effectiveness of monovalent rotavirus vaccine in a high-income, predominant-use setting . Vaccine . 2015 ; 33 ( 51 ): 7307 – 7314 . Google Scholar Crossref Search ADS PubMed 24 Payne DC , Boom JA , Staat MA , et al. . Effectiveness of pentavalent and monovalent rotavirus vaccines in concurrent use among US children <5 years of age, 2009–2011 . Clin Infect Dis . 2013 ; 57 ( 1 ): 13 – 20 . Google Scholar Crossref Search ADS PubMed 25 Doll MK , Morrison KT , Buckeridge DL , et al. . Two birds with one stone: estimating population vaccination coverage from a test-negative vaccine effectiveness case-control study . Clin Infect Dis . 2016 ; 63 ( 8 ): 1080 – 1086 . Google Scholar Crossref Search ADS PubMed 26 Québec Ministère de la Santé et des Services Sociaux . Québec Vaccination Registry. 2017 . http://sante.gouv.qc.ca/en/programmes-et-mesures-daide/registre-de-vaccination-du-quebec/description/. Accessed March 14, 2017. 27 Boulianne N , Audet D , Ouakki M . Enquête sur la Couverture Vaccinale des Enfants de 1 An et 2 Ans au Québec en 2014 . Quebec City, Canada: Institut national de santé publique du Québec; 2015 . http://www.inspq.qc.ca/pdf/publications/1973_Enquete_Couverture_Vaccinale_Enfants.pdf. Accessed June 28, 2016. 28 Ministère de la Santé et des Services Sociaux Québec . Feuilles d’Information Pour les Personnes à Vacciner (Version Anglaise) . 2016 . http://publications.msss.gouv.qc.ca/msss/fichiers/piq/feuillets_complets_en.pdf. Accessed February 16, 2017. 29 Aragon TJ . epitools: Epidemiology Tools. 2012 . https://CRAN.R-project.org/package=epitools. Accessed October 18, 2017. 30 Wilson SE , Deeks SL , Rosella LC . Importance of ICD-10 coding directive change for acute gastroenteritis (unspecified) for rotavirus vaccine impact studies: illustration from a population-based cohort study from Ontario, Canada . BMC Res Notes . 2015 ; 8 : 439 . Google Scholar Crossref Search ADS PubMed 31 Bernard S , Valiquette L , De Wals P , et al. . Burden of rotavirus disease: a population-based study in Eastern Townships, Quebec . Can J Infect Dis Med Microbiol . 2013 ; 24 ( 3 ): 138 – 142 . Google Scholar PubMed 32 Wagner AK , Soumerai SB , Zhang F , et al. . Segmented regression analysis of interrupted time series studies in medication use research . J Clin Pharm Ther . 2002 ; 27 ( 4 ): 299 – 309 . Google Scholar Crossref Search ADS PubMed 33 Figueiras A , Domenech-Massons JM , Cadarso C . Regression models: calculating the confidence interval of effects in the presence of interactions . Stat Med . 1998 ; 17 ( 18 ): 2099 – 2105 . Google Scholar Crossref Search ADS PubMed 34 Bhaskaran K , Gasparrini A , Hajat S , et al. . Time series regression studies in environmental epidemiology . Int J Epidemiol . 2013 ; 42 ( 4 ): 1187 – 1195 . Google Scholar Crossref Search ADS PubMed 35 Platts-Mills JA , Amour C , Gratz J , et al. . Impact of rotavirus vaccine introduction and post-introduction etiology of diarrhea requiring hospital admission in Haydom, Tanzania, a rural African setting . Clin Infect Dis . 2017 ; 65 ( 7 ): 1144 – 1151 . Google Scholar Crossref Search ADS PubMed 36 Buigues RP , Duval B , Rochette L , et al. . Hospitalizations for diarrhea in Quebec children from 1985 to 1998: estimates of rotavirus-associated diarrhea . Can J Infect Dis . 2002 ; 13 ( 4 ): 239 – 244 . Google Scholar Crossref Search ADS PubMed 37 Imai C , Armstrong B , Chalabi Z , et al. . Time series regression model for infectious disease and weather . Environ Res . 2015 ; 142 : 319 – 327 . Google Scholar Crossref Search ADS PubMed 38 Lopman B , Armstrong B , Atchison C , et al. . Host, weather and virological factors drive norovirus epidemiology: time-series analysis of laboratory surveillance data in England and Wales . PLoS One . 2009 ; 4 ( 8 ): e6671 . Google Scholar Crossref Search ADS PubMed 39 Ford-Jones EL , Wang E , Petric M , et al. . Rotavirus-associated diarrhea in outpatient settings and child care centers. The Greater Toronto Area/Peel Region PRESI Study Group. Pediatric Rotavirus Epidemiology Study for Immunization . Arch Pediatr Adolesc Med . 2000 ; 154 ( 6 ): 586 – 593 . Google Scholar Crossref Search ADS PubMed 40 Sénécal M , Brisson M , Lebel MH , et al. . Measuring the Impact of Rotavirus Acute Gastroenteritis Episodes (MIRAGE): a prospective community-based study . Can J Infect Dis Med Microbiol . 2008 ; 19 ( 6 ): 397 – 404 . Google Scholar PubMed 41 Waters V , Ford-Jones EL , Petric M , et al. . Etiology of community-acquired pediatric viral diarrhea: a prospective longitudinal study in hospitals, emergency departments, pediatric practices and child care centers during the winter rotavirus outbreak, 1997 to 1998. The Pediatric Rotavirus Epidemiology Study for Immunization Study Group . Pediatr Infect Dis J . 2000 ; 19 ( 9 ): 843 – 848 . Google Scholar Crossref Search ADS PubMed 42 Peng RD , Dominici F , Louis TA . Model choice in time series studies of air pollution and mortality . J R Stat Soc Ser A Stat Soc . 2006 ; 169 ( 2 ): 179 – 203 . Google Scholar Crossref Search ADS 43 Cameron AC , Trivedi PK . Essentials of count data regression. In: Baltagi B , ed. A Companion to Theoretical Econometrics . Malden, MA : Blackwell Publishing Ltd ; 2003 : 331 – 348 . 44 Burnham KP , Anderson DR . Summary. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach . New York, NY : Springer ; 2002 . 45 Brumback BA , Ryan LM , Schwartz JD , et al. . Transitional regression models, with application to environmental time series . J Am Stat Assoc . 2000 ; 95 ( 449 ): 16 – 27 . Google Scholar Crossref Search ADS 46 Cameron AC , Trivedi PK . Regression Analysis of Count Data . New York, NY : Cambridge University Press ; 2013 . 47 Hamborsky J , Kroger A . Rotavirus. Epidemiology and Prevention of Vaccine-Preventable Diseases, E-Book: The Pink Book . Washington DC : Public Health Foundation ; 2015 . 48 Schwartz J , Spix C , Touloumi G , et al. . Methodological issues in studies of air pollution and daily counts of deaths or hospital admissions . J Epidemiol Community Health . 1996 ; 50 ( suppl 1 ): S3 – S11 . Google Scholar Crossref Search ADS PubMed 49 Ver Hoef JM , Boveng PL . Quasi-Poisson vs. negative binomial regression: how should we model overdispersed count data? Ecology . 2007 ; 88 ( 11 ): 2766 – 2772 . Google Scholar Crossref Search ADS PubMed 50 R Core Team . R: A Language and Environment for Statistical Computing. 2015 . http://www.R-project.org/. Accessed October 18, 2017. 51 Bar-Zeev N , Kapanda L , Tate JE , et al. . Effectiveness of a monovalent rotavirus vaccine in infants in Malawi after programmatic roll-out: an observational and case-control study . Lancet Infect Dis . 2015 ; 15 ( 4 ): 422 – 428 . Google Scholar Crossref Search ADS PubMed 52 Giaquinto C , Dominiak-Felden G , Van Damme P , et al. . Summary of effectiveness and impact of rotavirus vaccination with the oral pentavalent rotavirus vaccine: a systematic review of the experience in industrialized countries . Hum Vaccin . 2011 ; 7 ( 7 ): 734 – 748 . Google Scholar Crossref Search ADS PubMed 53 Mariani L , Vici P , Suligoi B , et al. . Early direct and indirect impact of quadrivalent HPV (4HPV) vaccine on genital warts: a systematic review . Adv Ther . 2015 ; 32 ( 1 ): 10 – 30 . Google Scholar Crossref Search ADS PubMed 54 Mongua-Rodriguez N , Díaz-Ortega JL , García-García L , et al. . A systematic review of rubella vaccination strategies implemented in the Americas: impact on the incidence and seroprevalence rates of rubella and congenital rubella syndrome . Vaccine . 2013 ; 31 ( 17 ): 2145 – 2151 . Google Scholar Crossref Search ADS PubMed 55 Myint TT , Madhava H , Balmer P , et al. . The impact of 7-valent pneumococcal conjugate vaccine on invasive pneumococcal disease: a literature review . Adv Ther . 2013 ; 30 ( 2 ): 127 – 151 . Google Scholar Crossref Search ADS PubMed 56 Stuurman AL , Marano C , Bunge EM , et al. . Impact of universal mass vaccination with monovalent inactivated hepatitis A vaccines—a systematic review . Hum Vaccin Immunother . 2017 ; 13 ( 3 ): 724 – 736 . Google Scholar Crossref Search ADS PubMed 57 Drolet M , Bénard É , Boily MC , et al. . Population-level impact and herd effects following human papillomavirus vaccination programmes: a systematic review and meta-analysis . Lancet Infect Dis . 2015 ; 15 ( 5 ): 565 – 580 . Google Scholar Crossref Search ADS PubMed 58 Garland SM , Kjaer SK , Muñoz N , et al. . Impact and effectiveness of the quadrivalent human papillomavirus vaccine: a systematic review of 10 years of real-world experience . Clin Infect Dis . 2016 ; 63 ( 4 ): 519 – 527 . Google Scholar Crossref Search ADS PubMed 59 Shiri T , Datta S , Madan J , et al. . Indirect effects of childhood pneumococcal conjugate vaccination on invasive pneumococcal disease: a systematic review and meta-analysis . Lancet Glob Health . 2017 ; 5 ( 1 ): e51 – e59 . Google Scholar Crossref Search ADS PubMed 60 Atchison CJ , Stowe J , Andrews N , et al. . Rapid declines in age group-specific rotavirus infection and acute gastroenteritis among vaccinated and unvaccinated individuals within 1 year of rotavirus vaccine introduction in England and Wales . J Infect Dis . 2016 ; 213 ( 2 ): 243 – 249 . Google Scholar Crossref Search ADS PubMed 61 Davey HM , Muscatello DJ , Wood JG , et al. . Impact of high coverage of monovalent human rotavirus vaccine on emergency department presentations for rotavirus gastroenteritis . Vaccine . 2015 ; 33 ( 14 ): 1726 – 1730 . Google Scholar Crossref Search ADS PubMed 62 Ngabo F , Tate JE , Gatera M , et al. . Effect of pentavalent rotavirus vaccine introduction on hospital admissions for diarrhoea and rotavirus in children in Rwanda: a time-series analysis . Lancet Glob Health . 2016 ; 4 ( 2 ): e129 – e136 . Google Scholar Crossref Search ADS PubMed 63 Thomas SL , Walker JL , Fenty J , et al. . Impact of the national rotavirus vaccination programme on acute gastroenteritis in England and associated costs averted . Vaccine . 2017 ; 35 ( 4 ): 680 – 686 . Google Scholar Crossref Search ADS PubMed 64 Wilson SE , Rosella LC , Wang J , et al. . Population-level impact of Ontario’s infant rotavirus immunization program: evidence of direct and indirect effects . PLoS One . 2016 ; 11 ( 5 ): e0154340 . Google Scholar Crossref Search ADS PubMed © The Author(s) 2018. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png American Journal of Epidemiology Oxford University Press

Evaluation of the Impact of a Rotavirus Vaccine Program on Pediatric Acute Gastroenteritis Hospitalizations: Estimating the Overall Effect Attributable to the Program as a Whole and as a Per-Unit Change in Rotavirus Vaccine Coverage

Loading next page...
 
/lp/ou_press/evaluation-of-the-impact-of-a-rotavirus-vaccine-program-on-pediatric-AqlVqWAK0A
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
ISSN
0002-9262
eISSN
1476-6256
D.O.I.
10.1093/aje/kwy097
Publisher site
See Article on Publisher Site

Abstract

Abstract Estimation of the overall effect of a vaccine program is essential, but the effect is typically estimated for a whole program. We estimated the overall effect of the Quebec rotavirus vaccine program, launched in November 2011, and the effect for each 10% increase in rotavirus vaccine coverage on pediatric hospitalizations for all-cause acute gastroenteritis. We implemented negative binomial regressions adjusted for seasonality, long-term trends, and infection dynamics, to estimate the effect of the vaccine program as: 1) a dichotomous variable, representing program presence/absence, and linear term to account for changes in trend in the period after the program began; and 2) a continuous variable, representing rotavirus vaccine coverage. Using exposure 1, the vaccine program was associated with a 51.2% (95% confidence interval (CI): 28.5, 66.7) relative decline in adjusted weekly hospitalization rates for all-cause acute gastroenteritis as of December 28, 2014. Using exposure 2, a 10% increase in rotavirus ≥1-dose coverage was associated with a 7.1% (95% CI: 3.5, 10.5) relative decline in adjusted weekly rates, with maximum coverage of 87.0% associated with a 47.2% (95% CI: 26.9, 61.9) relative decline. Estimation of the overall effect attributable to a change in vaccine coverage might be a useful addition to standard measurement of the overall effect. overall effect, program evaluation, rotavirus, vaccine program Not all vaccine programs are created equal. With the adoption of a vaccine, policy makers must make numerous allocation decisions, which can influence the overall effect, or impact, of a vaccine program on disease burden. For example, decisions regarding the choice of vaccine(s), the population(s) targeted, and the schedule and dosage can influence the effect of a vaccine program on a disease outcome from a population perspective (1). Prior to program implementation, mathematical models are useful for estimating potential effects of different program scenarios (2–4), but model estimates are often uncertain (3, 5, 6). Evaluation of the overall effect using real-world data after implementation is therefore warranted (3–5), and comparison of results across similar jurisdictions with different vaccine programs might provide insights into the consequences of program decisions on population health outcomes. Yet, estimates of the overall effect from jurisdictions with differing levels of vaccine coverage are not easily compared. The comparison is not straightforward because the overall effect represents the average of effects in vaccinated and unvaccinated individuals, weighted by the proportion of each group in the population (i.e., as determined by vaccine coverage) (4, 7–9). One strategy for making measures of overall effect comparable is to characterize the relative reduction in a disease outcome that is attributable to a per-unit change in vaccine coverage. Doing so separates the overall effect of a program from the level of vaccine coverage achieved in the setting of the program and should make estimates of overall effect easier to compare across jurisdictions. We have provided a real-world example of this strategy, estimating, as a function of vaccine coverage, the overall effect of a rotavirus vaccine program on pediatric hospitalizations for all-cause acute gastroenteritis (AGE) in Quebec, Canada. Before the vaccine was available, rotavirus produced annual epidemics with peak disease through the winter and spring in Quebec (10, 11), causing approximately 72% of AGE hospitalizations among children less than 5 years of age during the 6-month epidemic period (11). We estimated the overall effect of a vaccination program as a rate ratio attributable to a 10% change in rotavirus vaccine coverage. We also characterized the overall effect of the whole program, in accordance with standard practice, and discuss the utility of our nonstandard approach. METHODS To estimate the overall effect of the Quebec rotavirus vaccine program, we conducted a time-series analysis of weekly all-cause AGE hospitalizations between June 1, 1998, and December 28, 2014, among a random sample of children aged 8–35 months, residing in greater Montreal, Quebec. Study setting The Province of Quebec implemented a publicly funded rotavirus vaccine program in November 2011. Under the program, monovalent rotavirus vaccine (Rotarix; GlaxoSmithKline, Brentford, United Kingdom) was available at no cost to Quebec residents, with routine vaccination recommended at 2 and 4 months of age, and series initiation and completion by 20 weeks and 8 months of age, respectively (12). Prior to the program, pentavalent (RotaTeq; Merck & Co., Inc., Kenilworth, New Jersey) and monovalent rotavirus vaccines were licensed in Canada in August 2006 (13) and October 2007 (14) and recommended for use by the National Advisory Committee on Immunization in January 2008 (15) and July 2010 (16), respectively; however recipients were required to pay for vaccination. Greater Montreal is the largest metropolitan area in Quebec and the second largest in Canada, with a population of approximately 4 million residents (17). All Quebec residents are covered by provincial health insurance, except for some temporary residents (18). Study population We included greater Montreal residents aged 8–35 months who were members of the Population Health Record (PopHR) cohort in our analyses. As described elsewhere (19), the PopHR cohort was created for research on health indicators, and represents an open, population-based, deidentified random sample of approximately 25% of greater Montreal residents with provincial health insurance. Health-care utilization data are tracked longitudinally for each cohort member from entry until death, emigration, or the cohort end date. Each year, cohort losses are replaced with a random sample of births and immigrants to maintain the population sampling fraction. The initial PopHR cohort was selected in 1998, with annual updates through 2014 at the time of our analyses. Data sources The PopHR cohort was created via linkages of provincial health insurance administrative databases maintained by the government organization: Régie de l’assurance maladie du Québec. Year and month for each cohort member’s dates of birth, cohort entry, and cohort exit are provided, and member health-care utilization data is tracked via an anonymized, unique identifier. Hospitalization data for each cohort member are obtained from a software system that captures administrative, clinical, and demographic information on hospital discharges (Maintenance et exploitation des données pour l’étude de la clientèle hospitalière (20)), used by all Quebec hospitals. In this system, administrative codes are recorded for each hospital stay using the International Classification of Diseases (ICD), Ninth Revision, adapted for Quebec (ICD-9-QC), until 2006, and International Classification of Diseases, Tenth Revision, adapted for Canada (ICD-10-CA), afterward. Exposure ascertainment In separate analyses, we characterized the Quebec rotavirus vaccine program as either: 1) the presence or absence of the program; or 2) rotavirus vaccine coverage in the population. Vaccine program We divided the study period into periods before and after implementing the program, with the period before the program defined as weeks before November 14, 2011, and the period after the program began as weeks including or after this date. We designated November 14, 2011, as the effective program start date in consideration of the program’s implementation date and a lag of 2 weeks to account for the biological delay in the immune response to vaccination, a duration typically used in rotavirus direct vaccine-effectiveness studies (21–24). Vaccine coverage We estimated weekly ≥1- and ≥2-dose rotavirus vaccine coverage among Montreal residents aged 8–35 months using coverage data from control participants enrolled in an external test-negative case-control study of rotavirus vaccine effectiveness in Quebec (23). For reasons described previously (25), we assumed that rotavirus vaccine coverage among these participants was a reasonable proxy to estimate population rotavirus vaccine coverage, given that their vaccination status was representative of the case source population and their inclusion was unlikely to be associated with vaccine uptake. We used these coverage data because an immunization registry did not exist in Quebec before June 2014 (26), and provincial coverage survey estimates were not available as a time series (27). Coverage estimates derived using this method were previously validated via comparison with provincial coverage surveys, and simulations suggest minimal bias when highly accurate diagnostic tests are used to ascertain controls (25). In our main analyses, we used coverage data from eligible controls recruited at hospitals between February 2011 and December 2014 in Montreal and Sherbrooke, Quebec, because coverage did not vary by geographic location (25). Only controls aged 8 months or older at the time of participation in the external study were eligible for inclusion in our coverage estimates. This age cutoff was chosen because a control’s rotavirus vaccine history was assessed as of their participation date and was not expected to change after 8 months of age (28). Using these eligibility criteria, we formed a retrospective cohort to assess weekly rotavirus vaccine coverage, with controls entering and exiting the cohort at 8 and 36 months of age, respectively. We assessed ≥1- and ≥2-dose rotavirus coverage each week to form a weekly time series, because we expected population vaccine coverage to increase rapidly following program implementation. Due to concerns that the age distribution of cohort members might differ from the PopHR cohort over time, weekly coverage estimates were age-standardized via direct standardization using the weekly distribution of children aged ≥8 and <12 months, 1 year, and 2 years of age in the PopHR cohort, via the R “epitools” package (R Foundation for Statistical Computing, Vienna, Austria) (29). Using these methods, the earliest date that we could derive age-standardized coverage estimates was April 11, 2011, or approximately 29 weeks before the implementation of the Quebec rotavirus vaccine program. Prior to that date, we assumed rotavirus vaccine coverage was 0%, but we modified this assumption in sensitivity analyses. Outcome ascertainment We examined weekly counts of AGE hospitalizations as our outcome. We aggregated outcomes by week because AGE hospitalizations occurred infrequently in our population, so daily counts included many zeros, and aggregation to monthly rates might have inadvertently obscured rotavirus epidemic patterns. We used AGE in consideration of potential changes in laboratory testing practices for rotavirus and/or ICD coding directives for AGE over time (30). To identify relevant hospitalizations, we examined the primary diagnosis ICD code recorded for each hospital stay, and we counted hospitalizations with a matching primary code for AGE, as previously defined by Quebec researchers Bernard et al. (31). Statistical analyses We used negative-binomial generalized linear models with a log-link function to estimate the overall effect of the Quebec vaccine program on weekly counts of AGE hospitalizations among children 8–35 months of age. A continuous, linear term indicating the week number of the time series, subsequently referred to as week time, and a weekly offset term to account for the denominator of children aged 8–35 months in the PopHR cohort were included in all models. In separate models, we characterized the exposure as: 1) the presence or absence of the rotavirus vaccine program; or 2) continuous vaccine coverage in the population. For exposure 1, we created a dichotomous variable to represent the rotavirus vaccination program, coded as 0 in weeks before the program and as 1 in weeks after the program began. Because we expected the program effect to change over time due to increasing coverage, we considered adjustment for a change in trend in the period after the program began (3, 32) in models adjusting for seasonality, long-term trends, and infection dynamics. This change was captured via inclusion of a product term between the binary program variable and a linear variable accounting for week or season number in the period after the program began. We coded the final study week or season as 0 in the latter term to facilitate derivation of the full program effect and 95% confidence intervals of the vaccine program achieved at study completion (i.e., December 28, 2014) (33). Therefore, in models with a linear trend, the exponentiated model coefficient for our dichotomous variable represented the rate ratio comparing fitted weekly rates of AGE hospitalizations in the final week or season of the study period versus the mean rate in the period before the program. We also report the percent relative decrease in weekly rates estimated as (1 – rate ratio) × 100, to facilitate interpretability. For exposure 2, we used weekly age-standardized proportions of either ≥1- or ≥2-dose rotavirus vaccine coverage multiplied by 100 (in separate models), with weeks before April 11, 2011, coded as 0. The exponentiated model coefficient for exposure 2 represented the rate ratio of weekly AGE hospitalizations corresponding to a 1% increase in vaccine coverage. To facilitate interpretability, we also report rate ratios and the percent relative decrease in our weekly outcome associated with a 10% increase in vaccine coverage and with an increase to the maximum coverage level observed during the study period. Because this representation assumes a log-linear relationship between our exposure and outcome, we examined the validity of this characterization in separate analyses (Web Appendix 1, available at https://academic.oup.com/aje). We considered several different approaches to adjust for rotavirus seasonality, and we implemented a series of models with season included as either: 1) time strata; 2) a Fourier transformation of week time (t) using combinations of annual (2πt52) and biennial (2πt104) sine and cosine terms; 3) a cubic B-spline transformation of calendar week time (i.e., using a January start date for week number; with the number of annual knots varying between 3 and 10); or 4) a combination of these approaches (34). To account for possible shifts in rotavirus seasonality after the vaccination program, we also evaluated inclusion of separate Fourier terms for periods before and after the program began (35). We considered adjustment for long-term trends by inclusion of a linear annual term for rotavirus season with June as a starting month, as used previously in Quebec (36). Due to the annual dynamic between the susceptible-infected-recovered populations within a rotavirus season, we also considered adjustment for the “population recovered and immune to rotavirus,” defined as the cumulative number of participants hospitalized for AGE per annual rotavirus season (37, 38), with separate terms for before and after the program began. This approach assumes that: 1) hospitalizations for AGE are a surrogate for rotavirus hospitalizations, and 2) people who contract rotavirus within each season are immune for the remainder of the season after recovery. While these assumptions represent an oversimplification, they are expected to generally hold because rotavirus was the predominant cause of Canadian pediatric AGE in the period before the vaccination program (10, 11, 31, 36, 39–41), and approximately one-half of our cohort was replaced each rotavirus season by an incoming birth cohort, which presumably represented children susceptible to rotavirus. We selected the best-fitting model for each exposure by minimizing the Akaike information criterion (AIC), which includes a penalty to discourage overfitting (42, 43), and we compared model fit among final models selected for each exposure by calculating AIC differences. We used Burnham and Anderson’s (44) general criteria to denote AIC differences of 1–2, 4–7, and >10 as models with similar, moderately better, or conclusively better fit, respectively. In situations where the best candidate models had similar fit, we selected the most parsimonious model as the preferred model. After selection of the best-fitting model for each exposure, we considered inclusion of one of the following autoregressive terms lagged by 1 week to address residual autocorrelation: 1) model residuals, 2) our outcome, 3) the logarithm of our outcome + 1, or 4) the logarithm of our outcome + 0.5. This approach has been suggested by several authors to reduce strong residual autocorrelation in infectious-disease time series induced by the contagious nature of the outcome (37, 45, 46). We chose a lag of 1 week in consideration of the short incubation period of rotavirus (47). For terms 3 and 4, the addition of 1 or 0.5 to outcome counts was implemented to prevent taking the logarithm of zero. We selected the best-fitting autoregressive term in consideration of its effect on residual autocorrelation, examined via autocorrelation and partial autocorrelation functions (48), and AIC differences. For additional model selection details, see Web Appendix 2. For each exposure, we examined the fit of final selected models via plots of the residuals over time and plots of fitted model values versus observed values (48). We also inspected the appropriateness of the negative binomial distribution by examining the relationship between the theoretical and actual residual variance for fitted values using the method developed by Ver Hoef and Boveng (49), as implemented by Imai et al. (37). We conducted multiple sensitivity analyses to examine the robustness of our findings (Web Appendix 3). All analyses were performed in RStudio, version 0.98.1103 (Boston, Massachusetts), with R, version 3.2.5 (R Foundation for Statistical Computing) (50). Ethics approval The McGill University Institutional Review Board approved our study and the formation of the PopHR cohort. The external vaccine-effectiveness study was approved by institutional review boards at each hospital. RESULTS An average of 24,058 children aged 8–35 months were under surveillance in the PopHR cohort each week (range, 21,396–27,061). The total number of AGE hospitalizations over the study period was 2,505, with the number of weekly hospitalizations ranging from 0 to 18. At the effective start date of the rotavirus vaccine program, age-adjusted rotavirus ≥1- and ≥2-dose vaccine coverage among children aged 8–35 months was 6.9% (95% confidence interval (CI): 2.6, 17.0), and 6.6% (95% CI: 2.5, 16.8), respectively. Between the start date of our coverage estimates and the study end date, age-adjusted ≥1-dose vaccine coverage ranged from 3.9% to 87.0%, while ≥2-dose vaccine coverage ranged from 3.9% to 72.5%. Time series plots of weekly AGE hospitalization rates are presented in conjunction with the dichotomous and continuous representations of our exposure in Figure 1. Figure 1. View largeDownload slide Weekly rates of all-cause acute gastroenteritis (AGE) hospitalizations among children aged 8–35 months, Montreal, Quebec, Canada, 1998–2014. A) The periods before and after the vaccine program, used for the dichotomous characterization of our exposure. B) The estimated percentage of rotavirus ≥1- and ≥2-dose vaccine coverage, used for the continuous representation of the exposure. Figure 1. View largeDownload slide Weekly rates of all-cause acute gastroenteritis (AGE) hospitalizations among children aged 8–35 months, Montreal, Quebec, Canada, 1998–2014. A) The periods before and after the vaccine program, used for the dichotomous characterization of our exposure. B) The estimated percentage of rotavirus ≥1- and ≥2-dose vaccine coverage, used for the continuous representation of the exposure. Model selection For both exposures, the final selected models included the following terms to adjust for seasonality, long-term trends, and infection dynamics: a linear term for week time, cubic B-spline transformation of calendar week time with 6 annual knots, separate annual and biennial sine terms for the periods before and after the program began, a linear term for annual rotavirus season, and separate linear terms for the “population recovered and immune” for the periods before and after the program began. For our binary exposure, we additionally included a product term to account for a weekly change in trend in the program period after the program began. To account for residual autocorrelation in the final models, we selected an autoregressive term of 1-week lagged model residuals, with little evidence of residual autocorrelation in the models adjusting for seasonality, long-term trends, and infection dynamics after its implementation. Candidate and final models are presented in Web Tables 1–7. Analyses of nonlinearity between vaccine coverage and the log outcome in adjusted models demonstrated that a linear characterization of our exposure was appropriate (Web Appendix 1, Web Tables 8–11, Web Figures 1–2). The fit of the model with ≥1-dose rotavirus coverage was similar to that with ≥2-dose coverage (AIC difference = 0.2). Models with either ≥1- or ≥2-dose coverage had similar to moderately better fit than the dichotomous exposure model (AIC differences = 2.6 and 2.8, respectively). Plots of the fitted counts from each exposure model versus observed counts of AGE hospitalizations are presented in Figure 2. A negative binomial distribution was deemed appropriate to model all exposures. Figure 2. View largeDownload slide Observed counts versus fitted model estimates of weekly all-cause acute gastroenteritis (AGE) hospitalizations among children aged 8–35 months, Montreal, Quebec, Canada, 1998–2014. Plots of fitted values are derived from the final selected models with the exposure characterized as: A) a dichotomous variable representing presence or absence of the program and linear trend term for the period after the program began; B) a continuous variable representing ≥1-dose rotavirus vaccine coverage; C) a continuous variable representing ≥2-dose rotavirus vaccine coverage. Figure 2. View largeDownload slide Observed counts versus fitted model estimates of weekly all-cause acute gastroenteritis (AGE) hospitalizations among children aged 8–35 months, Montreal, Quebec, Canada, 1998–2014. Plots of fitted values are derived from the final selected models with the exposure characterized as: A) a dichotomous variable representing presence or absence of the program and linear trend term for the period after the program began; B) a continuous variable representing ≥1-dose rotavirus vaccine coverage; C) a continuous variable representing ≥2-dose rotavirus vaccine coverage. Impact of the vaccination program On average, weekly rates of AGE hospitalizations in the periods before and after the program began were 1.341 (95% CI: 1.238, 1.443) and 0.696 (95% CI: 0.606, 0.786) per 10,000 children, respectively, accounting for a relative decrease of 48.1% (95% CI: 37.6, 56.8) in crude weekly rates in the period after the program began. After adjustment, the Quebec vaccine program was associated with a 51.2% (95% CI: 28.5, 66.7) relative decline in weekly rates of AGE hospitalizations among children 8–35 months of age at study completion. Prior to the uptake of any vaccine, the mean weekly rate of AGE hospitalizations was 1.369 (95% CI: 1.263, 1.474) per 10,000 children aged 8–35 months. We estimated that a 10% increase in ≥1-dose rotavirus coverage was associated with an 11.9% (95% CI: 8.8, 14.9) relative decrease in crude weekly rates of AGE hospitalizations, or a 7.1% (95% CI: 3.5, 10.5) relative decrease in adjusted rates. Where the continuous exposure was characterized as a 10% increase in ≥2-dose coverage, we estimated a crude relative weekly rate reduction of 13.8% (95% CI: 10.3, 17.3), or an adjusted relative decline of 8.3% (95% CI: 4.2, 12.3). When we generalized our adjusted estimates to the maximum coverage achieved during the study period, we estimated a relative decline in weekly AGE hospitalization rates of 47.2% (95% CI: 26.9, 61.9) for an increase in ≥1-dose coverage to 87.0%, or 46.8% (95% CI: 26.7, 61.3) for an increase in ≥2-dose coverage to 72.5%. Adjusted model coefficients, rate ratios, and 95% confidence intervals are presented in Table 1. Results of sensitivity analyses were consistent with primary analyses (Web Figure 3). Table 1. Adjusted Model Coefficients and Rate Ratios for the Overall Effect of the November 2011 Rotavirus Vaccine Programa on Weekly All-Cause Acute Gastroenteritis Hospitalizations Among Children 8–35 Months of Age, Montreal, Quebec, Canada, 1998–2014 Exposure Model Coefficient 95% CI RR 95% CI Unit Dichotomous (with linear trend) −0.71699 −1.09831, −0.33568 0.488 0.333, 0.715 Full program effect estimated as of December 28, 2014 (3 years after program implementation) Continuous  ≥1-dose coverage −0.00735 −0.01110, −0.00360 0.993 0.989, 0.996 per 1% increase 0.929 0.895, 0.965 per 10% increase 0.528 0.381, 0.731 per increase to max coverage (87.0%)  ≥2-dose coverage −0.00870 −0.01310, −0.00429 0.991 0.987, 0.996 per 1% increase 0.917 0.877, 0.958 per 10% increase 0.532 0.387, 0.733 per increase to max coverage (72.5%) Exposure Model Coefficient 95% CI RR 95% CI Unit Dichotomous (with linear trend) −0.71699 −1.09831, −0.33568 0.488 0.333, 0.715 Full program effect estimated as of December 28, 2014 (3 years after program implementation) Continuous  ≥1-dose coverage −0.00735 −0.01110, −0.00360 0.993 0.989, 0.996 per 1% increase 0.929 0.895, 0.965 per 10% increase 0.528 0.381, 0.731 per increase to max coverage (87.0%)  ≥2-dose coverage −0.00870 −0.01310, −0.00429 0.991 0.987, 0.996 per 1% increase 0.917 0.877, 0.958 per 10% increase 0.532 0.387, 0.733 per increase to max coverage (72.5%) Abbreviations: CI, confidence interval; RR, rate ratio. a Model coefficients represent the log(RR) for each exposure derived from fully adjusting negative-binomial generalized linear models implemented with a log-link function. In separate models, the Quebec rotavirus vaccine program was characterized as either as a dichotomous variable with linear trend in the period after the program or as a continuous variable, representing the percentage of ≥1- or ≥2-dose rotavirus vaccine coverage in the population. Table 1. Adjusted Model Coefficients and Rate Ratios for the Overall Effect of the November 2011 Rotavirus Vaccine Programa on Weekly All-Cause Acute Gastroenteritis Hospitalizations Among Children 8–35 Months of Age, Montreal, Quebec, Canada, 1998–2014 Exposure Model Coefficient 95% CI RR 95% CI Unit Dichotomous (with linear trend) −0.71699 −1.09831, −0.33568 0.488 0.333, 0.715 Full program effect estimated as of December 28, 2014 (3 years after program implementation) Continuous  ≥1-dose coverage −0.00735 −0.01110, −0.00360 0.993 0.989, 0.996 per 1% increase 0.929 0.895, 0.965 per 10% increase 0.528 0.381, 0.731 per increase to max coverage (87.0%)  ≥2-dose coverage −0.00870 −0.01310, −0.00429 0.991 0.987, 0.996 per 1% increase 0.917 0.877, 0.958 per 10% increase 0.532 0.387, 0.733 per increase to max coverage (72.5%) Exposure Model Coefficient 95% CI RR 95% CI Unit Dichotomous (with linear trend) −0.71699 −1.09831, −0.33568 0.488 0.333, 0.715 Full program effect estimated as of December 28, 2014 (3 years after program implementation) Continuous  ≥1-dose coverage −0.00735 −0.01110, −0.00360 0.993 0.989, 0.996 per 1% increase 0.929 0.895, 0.965 per 10% increase 0.528 0.381, 0.731 per increase to max coverage (87.0%)  ≥2-dose coverage −0.00870 −0.01310, −0.00429 0.991 0.987, 0.996 per 1% increase 0.917 0.877, 0.958 per 10% increase 0.532 0.387, 0.733 per increase to max coverage (72.5%) Abbreviations: CI, confidence interval; RR, rate ratio. a Model coefficients represent the log(RR) for each exposure derived from fully adjusting negative-binomial generalized linear models implemented with a log-link function. In separate models, the Quebec rotavirus vaccine program was characterized as either as a dichotomous variable with linear trend in the period after the program or as a continuous variable, representing the percentage of ≥1- or ≥2-dose rotavirus vaccine coverage in the population. DISCUSSION After adjustment for seasonality and other factors, we estimated that a 10% increase in ≥1-dose rotavirus vaccine coverage was associated with a 7% relative decrease in the weekly rate of AGE hospitalizations among children aged 8–35 months. For an increase to 87% rotavirus ≥1-dose coverage, or the highest coverage level observed during the study period, we estimated that adjusted weekly rates of AGE hospitalization declined by 47%. To our knowledge, only Bar-Zeev et al. (51) have also estimated the overall effect of a 10% increase in rotavirus vaccine coverage (in Malawi); unfortunately, their data are not directly comparable to ours due to differences between study outcomes, populations, and rotavirus disease dynamics. In comparison with our nonstandard estimate, we found that weekly rates of AGE hospitalizations were lower in the period after the program began—by 51% at study completion, 3 years after the implementation of the Quebec vaccine program. While these analyses are valid to ascertain the overall effect of the whole program up to that point, and similar to results found for maximum coverage levels, they are not easily applied to future periods, particularly if there is a change in population vaccine coverage. Direct comparison of the overall program effect with otherwise similar jurisdictions with differing vaccine coverage is also not straightforward, because interpretation of program-related reductions must be made in the context of population vaccine coverage. While most systematic reviews or meta-analyses that compare overall effects between vaccine programs report jurisdictional vaccine coverage in their narrative or a results table (52–56), it is difficult for readers to synthesize this information. Other systematic reviews or meta-analyses account for differences in vaccine coverage by stratifying overall effect estimates by coverage (57, 58); while this is a better method for comparison, coverage strata in these systematic reviews or meta-analyses were overly broad (i.e., coverage <50% vs. ≥50%, or low/moderate/high coverage), which might be insufficient to adjust for differences in program effects due to vaccine coverage. While it is also possible to directly adjust for vaccine coverage in meta-analyses using statistical models (59), the overall effect estimates from primary studies might not reflect a period after a program's implementation that has uniform vaccine coverage. This might be particularly true for the evaluation of new vaccine programs, such as rotavirus, where program estimates represent a period during which population coverage is rapidly changing (60–64); thus, direct adjustment for vaccine coverage in pooled analyses might not be possible or accurate. In each of these scenarios, estimates of the overall effect attributable to a per-unit change in vaccine coverage might be particularly useful to extrapolate estimates to other coverage levels, directly compare estimates between otherwise similar jurisdictions, or to incorporate into statistical models pooling program effect estimates from differing jurisdictions. For these reasons, we suggest that the overall effect attributable to a per-unit change in vaccine coverage might be a useful measure to supplement traditional methods to estimate the overall effect of a vaccine program, where vaccine coverage data are available. Limitations Our analyses have some limitations. First, we caution that our results are intended to estimate the overall effect of vaccination at the time of our evaluation, and thus, are generalizable only to settings with comparable rotavirus disease dynamics and to vaccine-coverage levels observed within the range examined in the present study. Further, although we found that a log-linear relationship was appropriate to model the effect of increasing vaccine coverage, this assumption should be examined by researchers in future analyses and might not be accurate for rotavirus vaccine-coverage levels beyond those observed in this study. Second, our approach ignores the potential implications of vaccination in other population subgroups on indirect effects; however, this is not an issue in the present study because our study population represents the entire population eligible for vaccination, and thus, all indirect effects included in our effect estimates necessarily arise from vaccination within the study population. Where analyses do not include the entire vaccine-eligible population, we recommend that researchers account for vaccine coverage among subpopulations not included in the analyses. Third, we did not include sampling error for coverage estimates in our analyses and, given our method of ascertaining coverage, we might slightly underestimate coverage (25); nonetheless, we conducted multiple sensitivity analyses that explored the effect of differing coverage estimates, and we did not find meaningful differences in our results. Last, we were unable to ascertain the effect of the program on rotavirus hospitalizations directly; however, broadening our definition to AGE hospitalizations should only attenuate our effect estimates because only a portion of these represent rotavirus and, thus, would be vaccine-preventable. Conclusion In the evaluation of a vaccine program, researchers should consider estimating the overall effect attributable to a per-unit increase in vaccine coverage in the population, in addition to estimating the overall effect of the whole program. In our examination of the effect of the Quebec rotavirus vaccine program, we estimated that each 10% increase in ≥1-dose rotavirus vaccine coverage was associated with a 7% relative decrease in the rate of weekly AGE hospitalizations among children 8–35 months of age. Using this approach of estimating the incremental effect of increasing coverage should promote comparisons of overall effect estimates across settings with differing coverage. ACKNOWLEDGMENTS Author affiliations: Department of Epidemiology and Biostatistics, McGill University, Montreal, Quebec, Canada (Margaret K. Doll, Caroline Quach, David L. Buckeridge); Department of Microbiology, Infectious Diseases and Immunology, University of Montreal, Montreal, Quebec, Canada (Caroline Quach); and Infection Control and Prevention Unit, Division of Pediatric Infectious Diseases and Medical Microbiology, Centre Hospitalier Universitaire Sainte-Justine, University of Montreal, Montreal, Quebec, Canada (Caroline Quach). Development and maintenance of the Population Health Record cohort was financed by a grant from the Canadian Foundation of Innovation (project 15649). The external vaccine-effectiveness study was also supported by a grant from GlaxoSmithKline. No funding was received for other aspects of this study. We thank Dr. Aman Verma from the McGill University Department of Epidemiology, Biostatistics and Occupational Health for his administrative help and for sharing his insights regarding the Population Health Record data set. C.Q. has received funding from GlaxoSmithKline, Pfizer, Sage, and AbbVie (all for research grant or support). The other authors report no conflicts. Abbreviations AGE acute gastroenteritis AIC Akaike information criterion CI confidence interval ICD International Classification of Diseases PopHR Population Health Record REFERENCES 1 World Health Organization . Principles and considerations for adding a vaccine to a national immunization programme: from decision to implementation and monitoring. World Health Organization ; 2014 . http://www.who.int/immunization/programmes_systems/policies_strategies/vaccine_intro_resources/nvi_guidelines/en/. Accessed October 18, 2017. 2 Ultsch B , Damm O , Beutels P , et al. . Methods for health economic evaluation of vaccines and immunization decision frameworks: a consensus framework from a European vaccine economics community . Pharmacoeconomics . 2016 ; 34 ( 3 ): 227 – 244 . Google Scholar Crossref Search ADS PubMed 3 Mealing N , Hayen A , Newall AT Assessing the impact of vaccination programmes on burden of disease: underlying complexities and statistical methods . Vaccine . 2016 ; 34 ( 27 ): 3022 – 3029 . Google Scholar Crossref Search ADS PubMed 4 Halloran ME , Longini IM , Struchiner CJ , et al. . Design and Analysis of Vaccine Studies . New York, NY : Springer ; 2010 . 5 Newall AT , Reyes JF , Wood JG , et al. . Economic evaluations of implemented vaccination programmes: key methodological challenges in retrospective analyses . Vaccine . 2014 ; 32 ( 7 ): 759 – 765 . Google Scholar Crossref Search ADS PubMed 6 Bilcke J , Beutels P , Brisson M , et al. . Accounting for methodological, structural, and parameter uncertainty in decision-analytic models: a practical guide . Med Decis Making . 2011 ; 31 ( 4 ): 675 – 692 . Google Scholar Crossref Search ADS PubMed 7 Hanquet G , Valenciano M , Simondon F , et al. . Vaccine effects and impact of vaccination programmes in post-licensure studies . Vaccine . 2013 ; 31 ( 48 ): 5634 – 5642 . Google Scholar Crossref Search ADS PubMed 8 Halloran ME . Overview of vaccine field studies: types of effects and designs . J Biopharm Stat . 2006 ; 16 ( 4 ): 415 – 427 . Google Scholar Crossref Search ADS PubMed 9 Halloran ME , Struchiner CJ . Study designs for dependent happenings . Epidemiology . 1991 ; 2 ( 5 ): 331 – 338 . Google Scholar Crossref Search ADS PubMed 10 Ford-Jones EL , Wang E , Petric M , et al. . Hospitalization for community-acquired, rotavirus-associated diarrhea: a prospective, longitudinal, population-based study during the seasonal outbreak. The Greater Toronto Area/Peel Region PRESI Study Group. Pediatric Rotavirus Epidemiology Study for Immunization . Arch Pediatr Adolesc Med . 2000 ; 154 ( 6 ): 578 – 585 . Google Scholar Crossref Search ADS PubMed 11 Rivest P , Proulx M , Lonergan G , et al. . Hospitalisations for gastroenteritis: the role of rotavirus . Vaccine . 2004 ; 22 ( 15–16 ): 2013 – 2017 . Google Scholar Crossref Search ADS PubMed 12 Québec Ministère de la Santé et des Services Sociaux . Rotavirus Vaccine . Quebec City, Canada : Québec Ministère de la Santé et des Services Sociaux ; 2016 . 13 Health Canada . Summary Basis Decision—RotaTeq. 2007 . https://hpr-rps.hres.ca/reg-content/summary-basis-decision-detailOne.php?lang=en&linkID=SBD00130. Accessed April 11, 2018. 14 Health Canada . Summary Basis of Decision—Rotarix. 2008 . https://hpr-rps.hres.ca/reg-content/summary-basis-decision-detailOne.php?lang=en&linkID=SBD00129. Accessed April 11, 2018. 15 National Advisory Committee on Immunization . Statement on the recommended use of pentavalent human-bovine reassortant rotavirus vaccine. An Advisory Committee Statement (ACS) . Can Commun Dis Rep . 2008 ; 34 ( ACS-1 ): 1 – 33 . 16 National Advisory Committee on Immunization . Updated statement on the use of rotavirus vaccines . Can Commun Dis Rep . 2010 ; 36 ( ACS-4 ): 1 – 37 . Crossref Search ADS 17 Statistics Canada . Census profile, 2016 Census. 2017 . http://www12.statcan.gc.ca/census-recensement/2016/dp-pd/prof/index.cfm?Lang=E. Accessed March 17, 2017. 18 Régie de l’Assurance Maladie Québec . Health insurance: eligibility for the Québec Insurance Plan. http://www.ramq.gouv.qc.ca/en/citizens/health-insurance/registration/Pages/eligibility.aspx. Accessed February 1, 2017. 19 Shaban-Nejad A , Lavigne M , Okhmatovskaia A , et al. . PopHR: a knowledge-based platform to support integration, analysis, and visualization of population health data . Ann N Y Acad Sci . 2017 ; 1387 ( 1 ): 44 – 53 . Google Scholar Crossref Search ADS PubMed 20 Québec Ministère de la Santé et des Services Sociaux . Sources de données et métadonnées: MED-ECHO. 2016 . http://www.msss.gouv.qc.ca/professionnels/documentation-sources-de-donnees-et-indicateurs/sources-de-donnees-et-metadonnees/med-echo/. Accessed June 20, 2018. 21 Braeckman T , Van Herck K , Meyer N , et al. . Effectiveness of rotavirus vaccination in prevention of hospital admissions for rotavirus gastroenteritis among young children in Belgium: case-control study . BMJ . 2012 ; 345 : e4752 . Google Scholar Crossref Search ADS PubMed 22 Cortese MM , Immergluck LC , Held M , et al. . Effectiveness of monovalent and pentavalent rotavirus vaccine . Pediatrics . 2013 ; 132 ( 1 ): e25 – e33 . Google Scholar Crossref Search ADS PubMed 23 Doll MK , Buckeridge DL , Morrison KT , et al. . Effectiveness of monovalent rotavirus vaccine in a high-income, predominant-use setting . Vaccine . 2015 ; 33 ( 51 ): 7307 – 7314 . Google Scholar Crossref Search ADS PubMed 24 Payne DC , Boom JA , Staat MA , et al. . Effectiveness of pentavalent and monovalent rotavirus vaccines in concurrent use among US children <5 years of age, 2009–2011 . Clin Infect Dis . 2013 ; 57 ( 1 ): 13 – 20 . Google Scholar Crossref Search ADS PubMed 25 Doll MK , Morrison KT , Buckeridge DL , et al. . Two birds with one stone: estimating population vaccination coverage from a test-negative vaccine effectiveness case-control study . Clin Infect Dis . 2016 ; 63 ( 8 ): 1080 – 1086 . Google Scholar Crossref Search ADS PubMed 26 Québec Ministère de la Santé et des Services Sociaux . Québec Vaccination Registry. 2017 . http://sante.gouv.qc.ca/en/programmes-et-mesures-daide/registre-de-vaccination-du-quebec/description/. Accessed March 14, 2017. 27 Boulianne N , Audet D , Ouakki M . Enquête sur la Couverture Vaccinale des Enfants de 1 An et 2 Ans au Québec en 2014 . Quebec City, Canada: Institut national de santé publique du Québec; 2015 . http://www.inspq.qc.ca/pdf/publications/1973_Enquete_Couverture_Vaccinale_Enfants.pdf. Accessed June 28, 2016. 28 Ministère de la Santé et des Services Sociaux Québec . Feuilles d’Information Pour les Personnes à Vacciner (Version Anglaise) . 2016 . http://publications.msss.gouv.qc.ca/msss/fichiers/piq/feuillets_complets_en.pdf. Accessed February 16, 2017. 29 Aragon TJ . epitools: Epidemiology Tools. 2012 . https://CRAN.R-project.org/package=epitools. Accessed October 18, 2017. 30 Wilson SE , Deeks SL , Rosella LC . Importance of ICD-10 coding directive change for acute gastroenteritis (unspecified) for rotavirus vaccine impact studies: illustration from a population-based cohort study from Ontario, Canada . BMC Res Notes . 2015 ; 8 : 439 . Google Scholar Crossref Search ADS PubMed 31 Bernard S , Valiquette L , De Wals P , et al. . Burden of rotavirus disease: a population-based study in Eastern Townships, Quebec . Can J Infect Dis Med Microbiol . 2013 ; 24 ( 3 ): 138 – 142 . Google Scholar PubMed 32 Wagner AK , Soumerai SB , Zhang F , et al. . Segmented regression analysis of interrupted time series studies in medication use research . J Clin Pharm Ther . 2002 ; 27 ( 4 ): 299 – 309 . Google Scholar Crossref Search ADS PubMed 33 Figueiras A , Domenech-Massons JM , Cadarso C . Regression models: calculating the confidence interval of effects in the presence of interactions . Stat Med . 1998 ; 17 ( 18 ): 2099 – 2105 . Google Scholar Crossref Search ADS PubMed 34 Bhaskaran K , Gasparrini A , Hajat S , et al. . Time series regression studies in environmental epidemiology . Int J Epidemiol . 2013 ; 42 ( 4 ): 1187 – 1195 . Google Scholar Crossref Search ADS PubMed 35 Platts-Mills JA , Amour C , Gratz J , et al. . Impact of rotavirus vaccine introduction and post-introduction etiology of diarrhea requiring hospital admission in Haydom, Tanzania, a rural African setting . Clin Infect Dis . 2017 ; 65 ( 7 ): 1144 – 1151 . Google Scholar Crossref Search ADS PubMed 36 Buigues RP , Duval B , Rochette L , et al. . Hospitalizations for diarrhea in Quebec children from 1985 to 1998: estimates of rotavirus-associated diarrhea . Can J Infect Dis . 2002 ; 13 ( 4 ): 239 – 244 . Google Scholar Crossref Search ADS PubMed 37 Imai C , Armstrong B , Chalabi Z , et al. . Time series regression model for infectious disease and weather . Environ Res . 2015 ; 142 : 319 – 327 . Google Scholar Crossref Search ADS PubMed 38 Lopman B , Armstrong B , Atchison C , et al. . Host, weather and virological factors drive norovirus epidemiology: time-series analysis of laboratory surveillance data in England and Wales . PLoS One . 2009 ; 4 ( 8 ): e6671 . Google Scholar Crossref Search ADS PubMed 39 Ford-Jones EL , Wang E , Petric M , et al. . Rotavirus-associated diarrhea in outpatient settings and child care centers. The Greater Toronto Area/Peel Region PRESI Study Group. Pediatric Rotavirus Epidemiology Study for Immunization . Arch Pediatr Adolesc Med . 2000 ; 154 ( 6 ): 586 – 593 . Google Scholar Crossref Search ADS PubMed 40 Sénécal M , Brisson M , Lebel MH , et al. . Measuring the Impact of Rotavirus Acute Gastroenteritis Episodes (MIRAGE): a prospective community-based study . Can J Infect Dis Med Microbiol . 2008 ; 19 ( 6 ): 397 – 404 . Google Scholar PubMed 41 Waters V , Ford-Jones EL , Petric M , et al. . Etiology of community-acquired pediatric viral diarrhea: a prospective longitudinal study in hospitals, emergency departments, pediatric practices and child care centers during the winter rotavirus outbreak, 1997 to 1998. The Pediatric Rotavirus Epidemiology Study for Immunization Study Group . Pediatr Infect Dis J . 2000 ; 19 ( 9 ): 843 – 848 . Google Scholar Crossref Search ADS PubMed 42 Peng RD , Dominici F , Louis TA . Model choice in time series studies of air pollution and mortality . J R Stat Soc Ser A Stat Soc . 2006 ; 169 ( 2 ): 179 – 203 . Google Scholar Crossref Search ADS 43 Cameron AC , Trivedi PK . Essentials of count data regression. In: Baltagi B , ed. A Companion to Theoretical Econometrics . Malden, MA : Blackwell Publishing Ltd ; 2003 : 331 – 348 . 44 Burnham KP , Anderson DR . Summary. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach . New York, NY : Springer ; 2002 . 45 Brumback BA , Ryan LM , Schwartz JD , et al. . Transitional regression models, with application to environmental time series . J Am Stat Assoc . 2000 ; 95 ( 449 ): 16 – 27 . Google Scholar Crossref Search ADS 46 Cameron AC , Trivedi PK . Regression Analysis of Count Data . New York, NY : Cambridge University Press ; 2013 . 47 Hamborsky J , Kroger A . Rotavirus. Epidemiology and Prevention of Vaccine-Preventable Diseases, E-Book: The Pink Book . Washington DC : Public Health Foundation ; 2015 . 48 Schwartz J , Spix C , Touloumi G , et al. . Methodological issues in studies of air pollution and daily counts of deaths or hospital admissions . J Epidemiol Community Health . 1996 ; 50 ( suppl 1 ): S3 – S11 . Google Scholar Crossref Search ADS PubMed 49 Ver Hoef JM , Boveng PL . Quasi-Poisson vs. negative binomial regression: how should we model overdispersed count data? Ecology . 2007 ; 88 ( 11 ): 2766 – 2772 . Google Scholar Crossref Search ADS PubMed 50 R Core Team . R: A Language and Environment for Statistical Computing. 2015 . http://www.R-project.org/. Accessed October 18, 2017. 51 Bar-Zeev N , Kapanda L , Tate JE , et al. . Effectiveness of a monovalent rotavirus vaccine in infants in Malawi after programmatic roll-out: an observational and case-control study . Lancet Infect Dis . 2015 ; 15 ( 4 ): 422 – 428 . Google Scholar Crossref Search ADS PubMed 52 Giaquinto C , Dominiak-Felden G , Van Damme P , et al. . Summary of effectiveness and impact of rotavirus vaccination with the oral pentavalent rotavirus vaccine: a systematic review of the experience in industrialized countries . Hum Vaccin . 2011 ; 7 ( 7 ): 734 – 748 . Google Scholar Crossref Search ADS PubMed 53 Mariani L , Vici P , Suligoi B , et al. . Early direct and indirect impact of quadrivalent HPV (4HPV) vaccine on genital warts: a systematic review . Adv Ther . 2015 ; 32 ( 1 ): 10 – 30 . Google Scholar Crossref Search ADS PubMed 54 Mongua-Rodriguez N , Díaz-Ortega JL , García-García L , et al. . A systematic review of rubella vaccination strategies implemented in the Americas: impact on the incidence and seroprevalence rates of rubella and congenital rubella syndrome . Vaccine . 2013 ; 31 ( 17 ): 2145 – 2151 . Google Scholar Crossref Search ADS PubMed 55 Myint TT , Madhava H , Balmer P , et al. . The impact of 7-valent pneumococcal conjugate vaccine on invasive pneumococcal disease: a literature review . Adv Ther . 2013 ; 30 ( 2 ): 127 – 151 . Google Scholar Crossref Search ADS PubMed 56 Stuurman AL , Marano C , Bunge EM , et al. . Impact of universal mass vaccination with monovalent inactivated hepatitis A vaccines—a systematic review . Hum Vaccin Immunother . 2017 ; 13 ( 3 ): 724 – 736 . Google Scholar Crossref Search ADS PubMed 57 Drolet M , Bénard É , Boily MC , et al. . Population-level impact and herd effects following human papillomavirus vaccination programmes: a systematic review and meta-analysis . Lancet Infect Dis . 2015 ; 15 ( 5 ): 565 – 580 . Google Scholar Crossref Search ADS PubMed 58 Garland SM , Kjaer SK , Muñoz N , et al. . Impact and effectiveness of the quadrivalent human papillomavirus vaccine: a systematic review of 10 years of real-world experience . Clin Infect Dis . 2016 ; 63 ( 4 ): 519 – 527 . Google Scholar Crossref Search ADS PubMed 59 Shiri T , Datta S , Madan J , et al. . Indirect effects of childhood pneumococcal conjugate vaccination on invasive pneumococcal disease: a systematic review and meta-analysis . Lancet Glob Health . 2017 ; 5 ( 1 ): e51 – e59 . Google Scholar Crossref Search ADS PubMed 60 Atchison CJ , Stowe J , Andrews N , et al. . Rapid declines in age group-specific rotavirus infection and acute gastroenteritis among vaccinated and unvaccinated individuals within 1 year of rotavirus vaccine introduction in England and Wales . J Infect Dis . 2016 ; 213 ( 2 ): 243 – 249 . Google Scholar Crossref Search ADS PubMed 61 Davey HM , Muscatello DJ , Wood JG , et al. . Impact of high coverage of monovalent human rotavirus vaccine on emergency department presentations for rotavirus gastroenteritis . Vaccine . 2015 ; 33 ( 14 ): 1726 – 1730 . Google Scholar Crossref Search ADS PubMed 62 Ngabo F , Tate JE , Gatera M , et al. . Effect of pentavalent rotavirus vaccine introduction on hospital admissions for diarrhoea and rotavirus in children in Rwanda: a time-series analysis . Lancet Glob Health . 2016 ; 4 ( 2 ): e129 – e136 . Google Scholar Crossref Search ADS PubMed 63 Thomas SL , Walker JL , Fenty J , et al. . Impact of the national rotavirus vaccination programme on acute gastroenteritis in England and associated costs averted . Vaccine . 2017 ; 35 ( 4 ): 680 – 686 . Google Scholar Crossref Search ADS PubMed 64 Wilson SE , Rosella LC , Wang J , et al. . Population-level impact of Ontario’s infant rotavirus immunization program: evidence of direct and indirect effects . PLoS One . 2016 ; 11 ( 5 ): e0154340 . Google Scholar Crossref Search ADS PubMed © The Author(s) 2018. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Journal

American Journal of EpidemiologyOxford University Press

Published: Sep 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off