Evaluation of results-based financing in the Republic of the Congo: a comparison group pre–post study

Evaluation of results-based financing in the Republic of the Congo: a comparison group pre–post... Abstract Results-based financing (RBF) has been advocated and increasingly scaled up in low- and middle-income countries to increase utilization and quality of key primary care services, thereby reducing maternal and child mortality rates. This pilot RBF study in the Republic of the Congo from 2012 to 2014 used a quasi-experimental research design. The authors conducted pre- and post-household surveys and gathered health facility services data from both intervention and comparison groups. Using a difference-in-differences approach, the study evaluated the impact of RBF on maternal and child health services. The household survey found statistically significant improvements in quality of services regarding the availability of medicines, perceived quality of care, hygiene of health facilities and being respected at the reception desk. The health facility survey showed no adverse effects and significantly favourable impacts on: curative visits, patient referral, children receiving vitamin A, HIV testing of pregnant women and assisted deliveries. These improvements, in relative terms, ranged from 42% (assisted deliveries) to 155% (children receiving vitamin A). However, the household survey found no statistically significant impacts on the five indicators measuring the use of maternal health services, including the percentage of pregnant women using prenatal care, 3+ prenatal care, postnatal care, assisted delivery, and family planning. Surprisingly, RBF was found to be associated with a reduction of coverage of the third diphtheria, pertussis, and tetanus immunization among children in the household survey. From the health facility survey, no association was found between RBF and full immunization among children. Overall, the study shows a favourable impact of an RBF programme on most, but not all, targeted maternal and child health services. Several aspects of programme implementation, such as timely disbursement of incentives, monitoring health facility performance, and transparency of using funds could be further strengthened to maximize RBF’s impact. Results-based financing, pay for performance, impact evaluation, household survey, the Republic of the Congo Key Messages There is positive impact of RBF programme on some, not all, maternal and child health services that the programme targeted. In spite of general favourable impact of PBF, programme implementation, such as timely disbursement of incentives, monitoring the performance of health facilities, and transparency of using funds, could be further strengthened to maximize the impact. Introduction In spite of substantial achievements during the millennium development goals (MDGs) era, many low- and middle-income countries (LMICs) still face challenges in establishing well-functioning health systems. Reasons include scarcity of supplies, limited infrastructure, shortage of qualified human resources (World Health Organization 2010), absence of incentives, dual practice, brain drain, and the lack of an enabling working environment (Hongoro and Normand 2006). In response to these challenges, health systems in many LMICs over the last two decades have launched provider incentive programmes. With support from the governments of Norway and the UK, the World Bank is managing the Health Results Innovation Trust Fund to pilot results-based financing (RBF) programmes in more than 30 countries (Africa Health Forum 2013). Most RBF programmes aim to improve maternal and child health (MCH) services by offering incentive payments to participating health facilities (including their staff) based on their quantity and quality of health services. In 2012, the Republic of the Congo’s (RoC) rapidly growing economy achieved a gross national income per capita of USD$3, 450 adjusted for purchasing power parity (The World Bank 2015). In spite of being a lower-middle income country, RoC continued to face challenges in addressing MCH problems, suggesting potential concerns of inefficiencies within the health system. The country’s 2013 under-5 mortality rate of 68 per 1000 was almost double the 2015 target of 35 per 1000 (World Bank 2013). One factor was the low utilization and poor quality of key MCH services (UNICEF 2008), despite MCH services having generally favourable cost-effectiveness ratios (Darmstadt et al. 2005; UNICEF 2008; Frost et al. 2014; Stenberg et al. 2014). According to Demographic and Health Survey (DHS) 2011–12 in RoC, 21.2 and 22.7% of pregnant women did not have 4+ prenatal visits and postnatal care, respectively, (Centre Nationale de la Statistique et des Études Économiques (CNSEE) & ICF International 2013) although the coverage of assisted delivery was relatively high (Centre Nationale de la Statistique et des Études Économiques (CNSEE) & ICF International 2013). Additionally, the coverage of prenatal and postnatal visits was much lower in rural areas. (Centre Nationale de la Statistique et des Études Économiques (CNSEE) & ICF International 2013). To respond to the need for improved MCH care, particularly in rural areas, the RoC’s RBF programme targeted to MCH began in January 2012 in three mostly rural departments (Niari, Plateaux and Pool). Supported by the World Bank and managed by the RoC’s Ministry of Health and Population, the RBF programme incentivizes providers by rewarding verified quantity and quality of key MCH services. According to the RBF framework and toolkit developed by the World Bank (Fritsche et al. 2014; RBFHealth 2015), RBF programmes have multiple elements beyond contracting health facilities on indicators, including providing technical assistance on developing business plans for health facilities, trainings on basic financial management, granting health facilities autonomy to use resources gained from incentives and verifying data for the incentive payment. These interventions may result in increased autonomy, strengthened data reporting, and enhanced capacity to manage health facilities. Consequently, RBF has the potential to bring about management and behaviour changes of staff (e.g. improved motivation) to boost the coverage and quality of incentivized health services. RBF programmes have demonstrated beneficial impacts in some LMICs. In Rwanda, the RBF programme led to a significant increase in coverage of institutional deliveries and preventive care visits for children (Basinga et al. 2011). Similar results were found in Burundi, Haiti, Nigeria and Cambodia. (Ashir et al. 2013; Zeng et al. 2013; Bonfrer et al. 2014; Van de Poel et al. 2016). However, a recent review of the impact of global RBF programmes calls for more investigation, partially due to the lack of rigorous research design (Fretheim et al. 2012; Turcotte-Tremblay et al. 2016). This study adds more evidence to the current literature on the impact of RBF by using data from two rounds of household and health facility surveys to quantify the impact of RBF on the utilization of key MCH services in RoC. Methods RBF schemes in the RoC In an effort to reduce maternal and child mortality to meet MDGs in RoC, a pilot RBF scheme, with financial support from the World Bank, began in January 2012. The RBF scheme started in three departments (Niari, Plateaux and Pool), equivalent to regions in many African countries, among 73 public health centres and 7 hospitals. These departments contained 1.26 million of ROC’s 4.41 million residents in 2013 (Ministry of Health and Population of the Republic of the Congo et al. 2014). Under the RBF scheme, Cordaid, the purchasing agency selected by the Ministry of Health and Population, obtained monthly reports of the specified services, verified quantities against facility registers and visited sampled households, assessed quality and transferred payments quarterly into each facility’s bank account. Table 1 lists the incentivized services and the amount of payment per unit for each service. These incentive payments were made in addition to the routine budget allocated by the Ministry of Health and Population. Cordaid also paid financial incentives to technical teams in each district that were responsible for supervising and helping health centres identify service provision issues and suggesting solutions. The RBF programme later expanded to include more health centres, with a total of 97 health centres in the three departments. Due to lengthy administrative and contractual procedures, the programme was interrupted from June 2012 through September 2012 and restarted in October 2012. Cordaid continued monitoring these services and paying incentives through February 2014, when this phase of the RBF scheme ended. Table 1 Health services included in the RBF package and their payments per unit Service number  Service description  Payment per unit (USD)  General population services   1  Curative visits  0.40   2  Curative visits for the poor  1.00   3  Hospitalizations  0.80   4  Small surgeries  3.00   5  Hospital referrals  10.00   6  Tuberculosis and leprosy cases detected  30.00   7  Tuberculosis and leprosy cases cured  60.00   8  Toilets built in the catchment area of the health center  6.00   9  Insecticide-treated bed nets distributed  0.50  HIV/AIDS services   10  Cases with opportunistic infections treated  0.40   11  Clients having voluntary counselling and testing  2.00   12  HIV-positive cases referred to the hospital  10.00   13  Pregnant women tested for HIV  2.00   14  HIV+ pregnant women receiving AZT+NVP  20.00   15  Newborns from HIV+ women receiving AZT+NVP  20.00   16  Patients under ARV followed for 6 months  1.00   17  Condom distribution points  10.00  Maternal and reproductive health services   18  Assisted deliveries  8.00   19  Pregnant women having 3 or more standard prenatal visits  6.00   20  Women who received tetanus vaccination during prenatal care (VAT2+)  2.00   21  Women who received a second dose of malaria prophylaxis during prenatal care  2.00   22  Pregnant women fully immunized  4.00   23  Women of productive age who use family planning method  5.00  Child health services   24  Children under 1 fully immunized  6.00   25  Children under 5 taking vitamin A  2.00  Service number  Service description  Payment per unit (USD)  General population services   1  Curative visits  0.40   2  Curative visits for the poor  1.00   3  Hospitalizations  0.80   4  Small surgeries  3.00   5  Hospital referrals  10.00   6  Tuberculosis and leprosy cases detected  30.00   7  Tuberculosis and leprosy cases cured  60.00   8  Toilets built in the catchment area of the health center  6.00   9  Insecticide-treated bed nets distributed  0.50  HIV/AIDS services   10  Cases with opportunistic infections treated  0.40   11  Clients having voluntary counselling and testing  2.00   12  HIV-positive cases referred to the hospital  10.00   13  Pregnant women tested for HIV  2.00   14  HIV+ pregnant women receiving AZT+NVP  20.00   15  Newborns from HIV+ women receiving AZT+NVP  20.00   16  Patients under ARV followed for 6 months  1.00   17  Condom distribution points  10.00  Maternal and reproductive health services   18  Assisted deliveries  8.00   19  Pregnant women having 3 or more standard prenatal visits  6.00   20  Women who received tetanus vaccination during prenatal care (VAT2+)  2.00   21  Women who received a second dose of malaria prophylaxis during prenatal care  2.00   22  Pregnant women fully immunized  4.00   23  Women of productive age who use family planning method  5.00  Child health services   24  Children under 1 fully immunized  6.00   25  Children under 5 taking vitamin A  2.00  Notes: HIV/AIDS, human immunodeficiency virus/acquired immune deficiency syndrome; AZT, zidovudine, also known as azidothymidine; NVP, niverapine; VAT, tetanus vaccination. Research design This evaluation used a quasi-experimental research design, where the three departments implementing RBF (Niari, Plateaux and Pool) constituted the ‘intervention group’, and two departments, Bouenza and Cuvette, served as the ‘comparison group’. Bouenza and Cuvette, were selected as comparisons because of their geographic proximity to the three intervention departments. Table 2 shows the comparison of some indicators at the department level. Overall, the five departments were quite similar on the indicators except the share of rural and semi-urban population where Plateaux and Pool had much higher share than Bouenza and Cuvette had. To evaluate the impact of the RBF programme, before-and-after rounds of household and health facility surveys were conducted: one of each in March 2012, before the RBF implementation, and one of each in March 2014, after the RBF implementation. Table 2 Comparison of department characteristics   Intervention departments   Comparison departments   Niari  Plateaux  Pool  Bouenza  Cuvette  Gini coefficient (%)  42.1  46.5  34.0  37.0  48.5  Female literacy (secondary or higher) (%)  59.0  47.6  51.9  46.7  67.0  Media exposure (male): Reading newspaper at least once a week (%)  9.6  9.7  9.2  7.5  14.9  Female population currently working (%)  72.6  81.9  82.6  75.2  73.8  Share of rural and semi-urban population (%)  48.2  70.7  84.4  47.0  47.7    Intervention departments   Comparison departments   Niari  Plateaux  Pool  Bouenza  Cuvette  Gini coefficient (%)  42.1  46.5  34.0  37.0  48.5  Female literacy (secondary or higher) (%)  59.0  47.6  51.9  46.7  67.0  Media exposure (male): Reading newspaper at least once a week (%)  9.6  9.7  9.2  7.5  14.9  Female population currently working (%)  72.6  81.9  82.6  75.2  73.8  Share of rural and semi-urban population (%)  48.2  70.7  84.4  47.0  47.7  Source: Demographic and Health Survey 2011–12, Republic of the Congo. In the first round of the household survey, households were selected using a multi-stage sampling approach. The RoC’s Department of Planning has defined ZDs (enumeration zones) as the basis for conducting the national census. The first stage of the survey in this sampling process was to draw random samples of 100 ZDs from the 557 ZDs in the three intervention departments (Niari, Plateaux and Pool) and from the 591 ZDs in the two comparison departments (Bouenza and Cuvette). The sampled ZDs were chosen with probability proportional to the population size (PPS) of each ZD. In the second stage of the sampling process, one village from each ZD was randomly selected. As the Department of Planning did not release information on the population size of villages, we could not conduct PPS sampling directly in this stage. Instead, each village within the ZD was given an equal probability to be selected. In cases where small villages did not have the minimum number of households required (seven households for each village according to our calculation of the sample size), we included households of an adjacent village to complement the one we originally selected. This adjustment brought the second stage closer to PPS. In the third stage, we worked with the village chief or community workers who provided a list of households in selected villages. We aimed to select households with at least one child under 2 years of age. We first randomly selected seven households regardless of whether they met this child criterion or not. If the household had at least one child under 2, the evaluation team then conducted an interview to collect the required information from that household. If the household did not meet this criterion, the team sequentially checked whether the next neighbouring household was eligible, until the evaluators found an eligible household. It then requested the interview. A total of 1349 households were selected for the baseline household survey, with 1344 mothers and 1841 children in the sample. For the second round of household surveys, we visited the same villages as the first round, and used the same sampling approach in the third stage, interviewing 1325 households. The sample sizes for mothers and children were 1307 and 1859, respectively. Since we conducted the two household surveys in March of 2012 and 2014 respectively with a 2-year gap, and considering the requirement of selecting households with at least one child under 2-years old in each round, we suspected that only a few of the households selected in the second round had also been in the first round. For the endline survey, we did not intend to survey the same households included in the baseline to construct panel data, because the requirement of having at least one child might eliminate many households to be used in the endline survey, and the country was unable to provide the research team with a household tracking system to identify the potential participatory households. Each of the household surveys were conducted over the course of about one month, and two research teams, each consisting of a supervisor and two to three surveyors, conducted the surveys. Mothers of children under 2-years old responded to relevant questions on using maternal health services and child services for their children. For health facility surveys, data from 73 health centres in the intervention group and 25 health centres in the comparison group were collected retrospectively in October and November 2012 for the services provided from January to December 2011, to serve as baseline. Data on service provision from the same health centres from January 2012 to February 2014 were collected retrospectively in March 2015. In order to rule out potential bias due to reporting behaviour changes from RBF, data from health facilities were collected from the registry book in both intervention and comparison facilities. A research team of two researchers visited health facilities to collect post-RBF health facility data on a rolling basis quarterly, while the health facility data prior to the RBF programme were collected in the first quarter after RBF was implemented. As we planned to analyse these data by year, the data from January and February 2014 were not included in the analysis. To examine the impact of RBF, only data from 2013 were used as the endline. For both household and health-facility surveys, most outcome variables used in the study focussed on the incentivized indicators, such as assisted delivery, prenatal care, postnatal care, HIV/AIDS testing and vaccinations. Additionally, we also included indictors measuring the quality of services in the household survey, such as availability of drugs, perceived quality of care, waiting time, and hygiene of health facilities. It should be noted that although there were 25 incentivized indicators for health centres and hospitals, not all indicators were measured or were measurable in the household surveys (e.g. hospitalizations, small surgeries, condom distribution points and tuberculosis testing) and thus many indicators were excluded from the analysis. In the end, nine incentivized indicators were included in the analysis. For the health-facility surveys, not all incentivized indicators were reported during the period prior to the RBF programme, and thus we only included the seven common indicators that were reported at the health-centre level in both periods before and after the RBF programme was implemented. Both household and health facility surveys were designed and carried out by external surveyors who were independent from the operation agency of the RBF programme in the country. We estimated the impact of RBF on the key outcome indicators using the difference-in-differences (DIDs) approach with a multivariate regression model for both the household and health-facility surveys. The model specification for the household surveys is provided below:   Service= b0+b1RBF+b2Post+b3RBF*Post+b4Urban+b5Age+b6Educ+b7HHsize+b8With partner+ b9Regular job+b10House ownership+b11Distance+ε, where RBF represents whether the household was in the RBF group or not, Post whether the measures were taken after the implementation of RBF or not, RBF*Post the interaction term between RBF and Post, and ε the random noise. The coefficient for RBF*Post (b3) represents DIDs, measuring the impact of RBF on service utilization. The models also controlled for a series of individual and household characteristics that might affect the utilization of health services. These characteristics measured financial and physical accessibility of households and respondents’ awareness of and education on health care, which included the location of households (Urban), house ownership (House ownership), household size (HHsize), mother’s age (Age), education (Educ), status of living with a partner (With partner), status of having a regular job (Regular job) and distance of households from health facilities (Distance). We conducted two regression models, one with and the other without village fixed effects. Both models adjusted for clustering at the village level. Given the parsimony of the model without village fixed effects, the interpretation of the results was focussed on this model. For the health facility survey, we conducted both random- and fixed-effects models to examine the impact of the RBF programme, while controlling for seasonal fluctuation of service delivery. The full model specification for the health facility surveys is:   Serviceit= b0+b1RBFit+b2Postit+b3RBFit*Postit+b4Quartersit+αi+εit, where i and t represent the ith facility at the time t; RBF, Post and RBF*Post have the same meaning as the model for the household surveys; Quarters are three dummy variables indicating which quarter the measure was taken in; bs are coefficients or coefficient matrix for corresponding variable(s); αi is the facility individual effects (either fixed or random effects); and εit is the random noise. The results from the two types of the models (the fixed and random effects models) were compared using the Housman test and we found no statistical differences for all seven indicators. Thus, we focussed on results from random-effects models given its advantage of the efficiency of estimates, and also reported results from the facility fixed effects model to provide additional information. We conducted all statistical analyses with STATA 12 (Stata Corp LP, College Station, TX). Results At baseline, the average age of mothers who had had one pregnancy in the two years prior to the survey was 27.7 years (27.7 ± 9.5). Respondents in the intervention departments were slightly younger (27.2 ± 8.3 years) than those in the comparison departments (28.2 ± 10.6 years). The difference was statistically significant (P < 0.05). There were no statistically significant differences between the comparison and intervention departments in mothers’ education level or proportion of them living with partners (Table 2). Most baseline differences between comparison and intervention departments on utilization of maternal health services were not statistically significant, except for the use of any prenatal care, four or more prenatal visits, and postnatal visits (Table 3). Higher percentages of pregnant women received prenatal and postnatal visits in the RBF departments than the comparison departments. The coverage of some services was high. For example, the coverage of institutional delivery was 84 and 87% in the comparison and intervention departments, respectively; the coverage of four or more prenatal care visits was 79 and 88% in the comparison and intervention departments, respectively. In contrast, the use of postnatal care and family planning methods remained low among the mothers, with an average of slightly >50% for both intervention and comparison departments combined. Table 3 Characteristics of households and mothers, and utilization of maternal health and HIV/AIDS services at baseline (from household survey) Variable  Intervention departments  Comparison departments  Total mean ± SD  Difference (intervention vs comparison)  mean ± SD  mean ± SD    Household characteristics   Household size  5.28 ± 2.04  5.58 ± 2.17  5.44 ± 2.11  −0.30**   Number of children <5 years of age  1.51 ± 0.67  1.50 ± 0.61  1.50 ± 0.64  0.02   Daily spending (FCFA)  2110 ± 1044  2258 ± 1518  2185 ± 1310  −148*   Own land (Yes = 1, No = 0)  73.1 ± 44.4%  76.8 ± 42.2%  75.0 ± 43.3%  −3.8%   Own house (Yes = 1, No = 0)  63.8 ± 48.1%  68.8 ± 46.4%  66.4 ± 47.3%  −5.0%   Distance from health facility (minutes)  57.53 ± 60.30  62.57 ± 79.27  60.12 ± 70.69  −5.04  Mother's characteristics   Age (years)  27.20 ± 8.25  28.22 ± 10.59  27.72 ± 9.54  −1.03*   Education (% above middle school)  55.3 ± 49.8%  52.4 ± 50.0%  53.8 ± 49.9%  2.9%   Living with partners (%)  82.6 ± 38.0%  78.9 ± 40.8%  80.7 ± 39.5%  3.6%  Maternal health services   Live birth ratio in last 5 years  93.4 ± 16.9%  94.1 ± 16.6%  93.7 ± 16.7%  −0.7%   Institutional delivery (%)  87.5 ± 33.1%  83.7 ± 36.9%  85.6 ± 35.2%  3.7%   Cesarean section for last delivery (%)  15.2 ± 33.9%  12.4 ± 33.0%  13.8 ± 34.5%  2.8%   Any prenatal care visit (%)  95.3 ± 21.3%  85.7 ± 35.0%  90.4 ± 29.5%  9.5%**   Four or more prenatal care visits (%)  87.5 ± 33.1%  79.4 ± 40.5%  83.3 ± 37.3%  8.1%**   Used postnatal care (%)  58.3 ± 49.3%  50.6 ± 50.0%  54.4 ± 49.8%  7.7%**   Ever used family planning methods (%)  52.9 ± 50.0%  51.2 ± 50.0%  52.0 ± 50.0%  1.7%  HIV/AIDS services   Offered HIV test (%)  43.4 ± 49.6%  48.3 ± 50.0%  45.7 ± 49.8%  −4.9%   Received HIV test among those who were offered (%)  87.0 ± 33.7%  88.0 ± 32.6%  87.5 ± 33.1%  −1.0%   Obtained HIV test results for those who received test (%)  89.4 ± 30.9%  77.7 ± 41.7%  83.6 ± 37.0%  11.7%***  Variable  Intervention departments  Comparison departments  Total mean ± SD  Difference (intervention vs comparison)  mean ± SD  mean ± SD    Household characteristics   Household size  5.28 ± 2.04  5.58 ± 2.17  5.44 ± 2.11  −0.30**   Number of children <5 years of age  1.51 ± 0.67  1.50 ± 0.61  1.50 ± 0.64  0.02   Daily spending (FCFA)  2110 ± 1044  2258 ± 1518  2185 ± 1310  −148*   Own land (Yes = 1, No = 0)  73.1 ± 44.4%  76.8 ± 42.2%  75.0 ± 43.3%  −3.8%   Own house (Yes = 1, No = 0)  63.8 ± 48.1%  68.8 ± 46.4%  66.4 ± 47.3%  −5.0%   Distance from health facility (minutes)  57.53 ± 60.30  62.57 ± 79.27  60.12 ± 70.69  −5.04  Mother's characteristics   Age (years)  27.20 ± 8.25  28.22 ± 10.59  27.72 ± 9.54  −1.03*   Education (% above middle school)  55.3 ± 49.8%  52.4 ± 50.0%  53.8 ± 49.9%  2.9%   Living with partners (%)  82.6 ± 38.0%  78.9 ± 40.8%  80.7 ± 39.5%  3.6%  Maternal health services   Live birth ratio in last 5 years  93.4 ± 16.9%  94.1 ± 16.6%  93.7 ± 16.7%  −0.7%   Institutional delivery (%)  87.5 ± 33.1%  83.7 ± 36.9%  85.6 ± 35.2%  3.7%   Cesarean section for last delivery (%)  15.2 ± 33.9%  12.4 ± 33.0%  13.8 ± 34.5%  2.8%   Any prenatal care visit (%)  95.3 ± 21.3%  85.7 ± 35.0%  90.4 ± 29.5%  9.5%**   Four or more prenatal care visits (%)  87.5 ± 33.1%  79.4 ± 40.5%  83.3 ± 37.3%  8.1%**   Used postnatal care (%)  58.3 ± 49.3%  50.6 ± 50.0%  54.4 ± 49.8%  7.7%**   Ever used family planning methods (%)  52.9 ± 50.0%  51.2 ± 50.0%  52.0 ± 50.0%  1.7%  HIV/AIDS services   Offered HIV test (%)  43.4 ± 49.6%  48.3 ± 50.0%  45.7 ± 49.8%  −4.9%   Received HIV test among those who were offered (%)  87.0 ± 33.7%  88.0 ± 32.6%  87.5 ± 33.1%  −1.0%   Obtained HIV test results for those who received test (%)  89.4 ± 30.9%  77.7 ± 41.7%  83.6 ± 37.0%  11.7%***  Notes: HIV/AIDS, human immunodeficiency virus/acquired immune deficiency syndrome; SD, standard deviation; FCFA, Franc Communauté Financière Africaine. * P < 0.05; ** P < 0.01; *** P < 0.001. Health providers played an important role in influencing mothers to receive testing for HIV/AIDS. As shown in Table 3, health providers asked fewer than half of the mothers to take an HIV/AIDS test during their prenatal visits in both comparison and intervention departments. Most of the mothers invited for testing for HIV/AIDS received the test. However, the percentages of women who were tested and obtained HIV test results differed significantly between the intervention and comparison departments (89 vs 78%, P < 0.001). For children under 5 at baseline, there were no differences in age and gender distribution (Table 4). The percentage of children under 5 who had malnutrition consultation was < 10% in both comparison and intervention departments. Most households (79%) had bed nets at home and most children under 5 (>90%) used a bed net if there was one at their home. About 10% and 20% of children under 5 who had illness did not seek care in the comparison and intervention departments at the baseline, respectively, and the difference was statistically significant (P < 0.001), which indicates there is opportunity for the intervention departments to make improvement. The coverage of bacille Calmette–Guerin (BCG) vaccine for tuberculosis was high (>90%). However, among children aged 1–5, coverage of the third diphtheria, pertussis and tetanus vaccine (DPT) was low with ∼40%. Table 4 Child health services and care at baseline (from household survey) Indicator  Intervention departments (mean ± SD)  Comparison departments (mean ± SD)  Total (mean ± SD)  Difference (intervention vs comparison)  Children’s characteristics   Age (years)  1.57 ± 1.30  1.59 ± 1.27  1.58 ± 1.28  −0.02   Male (%)  53.8 ± 49.9%  53.2 ± 49.9%  53.6 ± 49.9%  0.6%  Use of child care   Had malnutrition consultation (%)  9.3 ± 29.1%  8.2 ± 27.5%  8.7 ± 28.2%  1.1%   Had bed net for child (%)  80.5 ± 39.7%  76.9 ± 42.2%  78.6 ± 41.0%  3.6%   Used bed net in households with bed net (%)  97.4 ± 16.1%  94.0 ± 23.7%  95.7 ± 20.3%  3.3%**   Sought care for children with illness (%)  80.3 ± 39.8%  89.1 ± 31.2%  85.0 ± 35.7%  −8.8%***  Vaccinations   All children aged 12–24 months received BCG vaccination (%)  98.3 ± 12.9%  95.4 ± 21.1%  96.8 ± 17.5%  3.0%***   All children aged 1–5 years received three DPT vaccinations (%)  39.2 ± 48.9%  40.3 ± 49.1%  39.8 ± 49.0%  −1.1%  Indicator  Intervention departments (mean ± SD)  Comparison departments (mean ± SD)  Total (mean ± SD)  Difference (intervention vs comparison)  Children’s characteristics   Age (years)  1.57 ± 1.30  1.59 ± 1.27  1.58 ± 1.28  −0.02   Male (%)  53.8 ± 49.9%  53.2 ± 49.9%  53.6 ± 49.9%  0.6%  Use of child care   Had malnutrition consultation (%)  9.3 ± 29.1%  8.2 ± 27.5%  8.7 ± 28.2%  1.1%   Had bed net for child (%)  80.5 ± 39.7%  76.9 ± 42.2%  78.6 ± 41.0%  3.6%   Used bed net in households with bed net (%)  97.4 ± 16.1%  94.0 ± 23.7%  95.7 ± 20.3%  3.3%**   Sought care for children with illness (%)  80.3 ± 39.8%  89.1 ± 31.2%  85.0 ± 35.7%  −8.8%***  Vaccinations   All children aged 12–24 months received BCG vaccination (%)  98.3 ± 12.9%  95.4 ± 21.1%  96.8 ± 17.5%  3.0%***   All children aged 1–5 years received three DPT vaccinations (%)  39.2 ± 48.9%  40.3 ± 49.1%  39.8 ± 49.0%  −1.1%  Notes: SD, standard deviation; BCG, bacille Calmette–Guerin vaccine for tuberculosis; DPT, diphtheria, pertussis and tetanus vaccine. ** P < 0.01; *** P < 0.001. In general, the perceived quality of health services was good (Table 5). More mothers in the intervention departments felt they were well received by health facilities than those in the comparison departments (92 vs 87%, respectively), and most (89%) felt that health centres had good quality, whether they were in the comparison or intervention departments. However, only about 70% of mothers received medications from health centres during their last medical visit. The rate of receiving medication was lower in intervention departments compared with the comparison departments (65 vs 76%), and this difference was highly statistically significant (P < 0.001). Again, this also indicated potential for the intervention departments to improve medication availability. Another noteworthy indicator was the waiting time of mothers at their last outpatient visit, where mothers in the comparison department spent about 9 min longer than those in the intervention Group (47 vs 38 min), and the difference was statistically significant (P < 0.01). Table 5 Quality of health services at baseline (from household survey) Quality of services  Intervention departments (mean ± SD)  Comparison departments (mean ± SD)  Total (mean ± SD)  Difference (intervention vs comparison)  Waiting time for last visit (minute)  37.8 ± 42.6  47.0 ± 60.0  42.5 ± 52.5  −9.2**  Received medication at visit (received = 1, otherwise = 0)  65.1 ± 47.7%  76.1 ± 42.7%  70.8 ± 45.5%  −11.0%***  Reception (% good reception)  91.7 ± 27.6%  87.5 ± 33.1%  89.5 ± 30.6%  4.3%*  Hygiene (% good hygiene)  81.5 ± 38.8%  80.7 ± 39.5%  81.1 ± 39.2%  0.8%  Quality (% good quality)  89.2 ± 31.1%  88.9 ± 31.5%  89.0 ± 31.3%  0.3%  Quality of services  Intervention departments (mean ± SD)  Comparison departments (mean ± SD)  Total (mean ± SD)  Difference (intervention vs comparison)  Waiting time for last visit (minute)  37.8 ± 42.6  47.0 ± 60.0  42.5 ± 52.5  −9.2**  Received medication at visit (received = 1, otherwise = 0)  65.1 ± 47.7%  76.1 ± 42.7%  70.8 ± 45.5%  −11.0%***  Reception (% good reception)  91.7 ± 27.6%  87.5 ± 33.1%  89.5 ± 30.6%  4.3%*  Hygiene (% good hygiene)  81.5 ± 38.8%  80.7 ± 39.5%  81.1 ± 39.2%  0.8%  Quality (% good quality)  89.2 ± 31.1%  88.9 ± 31.5%  89.0 ± 31.3%  0.3%  Notes: SD, standard deviation; min, minutes. * P < 0.05; ** P < 0.01; *** P < 0.001. Using the DIDs approach without village fixed effects, we found that RBF was associated with improved quality of care, and increased use of curative care (Table 6). More specifically, RBF was associated with a 14.5 percentage point increase in receiving medication at the last visit (P < 0.05), representing a 20.5% increase relative to the baseline average of 70.8%. RBF was associated with a 17.6 percentage point increase in seeking care if a child was sick (P < 0.05), indicating a 39.2% relative increase over the baseline average of 45%. Table 6 Impact of RBF on services (from household surveys) Outcome  n  Without village fixed effects   With village fixed effects       Absolute DIDs  Relative %D  Absolute DIDs  Relative %D  Maternal health services             Institutional delivery (%)  2665  –1.8%  –2.1%  0.4%  0.5%   Prenatal care (%)  2652  –4.4%  –4.9%  –3.2%  –3.5%   3+ prenatal care (%)  2565  1.5%  1.8%  1.6%  1.9%   Postnatal care (%)  2452  4.2%  7.7%  11.2%  20.6%   Used family planning methods (%)  2552  3.5%  6.7%  7.2%  13.8%  HIV/AIDS services             Patient offered HIV test (% of pregnant women)  2406  9.9%  21.6%  11.0%  24.0%   Patient received HIV test when offered (%)  2675  5.6%  15.5%  8.5%  23.6%  Quality of services             Woman received medication at last visit (%)  2644  14.5%*  20.5%  13.3%*  18.8%   Reception (% poor reception)  2502  11.7%*  14.4%  8.4%*  10.4%   Hygiene (% poor hygiene)  2619  12.5%*  15.4%  8.9%  10.4%   Quality (% poor quality)  2577  7.4%*  8.3%  7.1%  9.4%  Child care             Had bed net (% children <5)  3681  –7.3%  –9.3%  –8.8%  –11.2%   Sought curative care (% children in last month)  1886  17.6%*  39.2%  25.3%*  56.3%   Received BCG, children aged 0–23 months (%)  2319  0.7%  0.7%  1.2%  1.2%   Received DPT3, children aged 6–23 months (%)  1282  –19.7%*  –38.2%  –13.3%*  –25.8%  Outcome  n  Without village fixed effects   With village fixed effects       Absolute DIDs  Relative %D  Absolute DIDs  Relative %D  Maternal health services             Institutional delivery (%)  2665  –1.8%  –2.1%  0.4%  0.5%   Prenatal care (%)  2652  –4.4%  –4.9%  –3.2%  –3.5%   3+ prenatal care (%)  2565  1.5%  1.8%  1.6%  1.9%   Postnatal care (%)  2452  4.2%  7.7%  11.2%  20.6%   Used family planning methods (%)  2552  3.5%  6.7%  7.2%  13.8%  HIV/AIDS services             Patient offered HIV test (% of pregnant women)  2406  9.9%  21.6%  11.0%  24.0%   Patient received HIV test when offered (%)  2675  5.6%  15.5%  8.5%  23.6%  Quality of services             Woman received medication at last visit (%)  2644  14.5%*  20.5%  13.3%*  18.8%   Reception (% poor reception)  2502  11.7%*  14.4%  8.4%*  10.4%   Hygiene (% poor hygiene)  2619  12.5%*  15.4%  8.9%  10.4%   Quality (% poor quality)  2577  7.4%*  8.3%  7.1%  9.4%  Child care             Had bed net (% children <5)  3681  –7.3%  –9.3%  –8.8%  –11.2%   Sought curative care (% children in last month)  1886  17.6%*  39.2%  25.3%*  56.3%   Received BCG, children aged 0–23 months (%)  2319  0.7%  0.7%  1.2%  1.2%   Received DPT3, children aged 6–23 months (%)  1282  –19.7%*  –38.2%  –13.3%*  –25.8%  Notes: n denotes the number of observations, DIDs is the results from the difference-in-differences analysis. %D shows the relative change to the average of baseline; HIV/AIDS, human immunodeficiency virus/acquired immune deficiency syndrome; BCG, bacille Calmette–Guerin vaccine for tuberculosis; DPT, diphtheria, pertussis and tetanus vaccine; Coef, coefficient; stat., statistically. * P < 0.05 after Bonferroni correction. In addition, RBF was associated with changes in all the quality indicators used in the household survey. It was estimated that 11.7, 12.5 and 7.4 percentage points more pregnant women felt that they received ‘good’ reception at their last visit and the hygiene and quality of services were better, representing relative increases of 14.4, 15.4 and 8.3%, respectively, compared with the baseline averages. Surprisingly, the RBF was associated with a reduction of 19.7 percentage points in the coverage of DPT3 (P < 0.05). Although RBF showed favourable impacts on the use of more than three prenatal care visits, postnatal visits, use of family planning and BCG vaccination, these impacts were not statistically significant. Among 15 indicators, RBF showed favourable directions on 11 indicators and unfavourable on the remaining 4. This preponderance of favourable results is very unlikely to be due to chance (P < 0.001). Overall, the results from the model with village fixed effects were similar to the model without village fixed effects, although the magnitude of the effects was different for some indicators (e.g. postnatal care). The results from the health facility surveys estimated from the random effects model showed a more favourable impact of RBF on some selected indicators. The RBF scheme was associated with relative increases in curative care (83%), patient referrals (472%), vitamin A distribution (155%), assisted delivery (42%) and HIV/AIDS testing among pregnant women (147%). However, RBF did not improve full immunization among children and anti-tetanus vaccination (VAT2+) among pregnant women (Table 7). Generally, the impacts derived from the health facility survey were much larger than those observed in the household survey. For example, the facility survey showed the number of curative visits increased by 83%, while the household survey showed a 38.9% increase for children receiving curative visits. When compared with the baseline, the services of referrals to hospitals, children receiving vitamin A and HIV/AIDS testing among pregnant women more than doubled. The results from the model with facility fixed effects were quite similar to those from the random effects model. Table 7 Impact of RBF on services (from health facility surveys) Health services  n  Baseline monthly visit  Facility random effects   Facility fixed effects   Absolute DIDs  %D  Absolute DIDs  %D  Curative visits  769  76.2  63.30*  83%  63.55*  83%  Patients referred to hospital  757  1.1  4.97*  472%  4.97*  472%  Children receiving full immunization  633  30.2  −5.42  −18%  −5.37  −18%  Children who received vitamin A  648  22.8  35.26*  155%  33.55*  147%  Pregnant women tested HIV/AIDS  768  3.4  5.04*  147%  5.10*  149%  Assisted births  759  6.8  2.81*  42%  2.80*  41%  Pregnant women vaccinated with VAT2+  678  17.8  2.09  12%  2.21  12%  Median  757  17.8  4.97  83%  4.97  83%  Health services  n  Baseline monthly visit  Facility random effects   Facility fixed effects   Absolute DIDs  %D  Absolute DIDs  %D  Curative visits  769  76.2  63.30*  83%  63.55*  83%  Patients referred to hospital  757  1.1  4.97*  472%  4.97*  472%  Children receiving full immunization  633  30.2  −5.42  −18%  −5.37  −18%  Children who received vitamin A  648  22.8  35.26*  155%  33.55*  147%  Pregnant women tested HIV/AIDS  768  3.4  5.04*  147%  5.10*  149%  Assisted births  759  6.8  2.81*  42%  2.80*  41%  Pregnant women vaccinated with VAT2+  678  17.8  2.09  12%  2.21  12%  Median  757  17.8  4.97  83%  4.97  83%  Notes: n denotes facility quarters, with a maximum of 776; DIDs are the results from difference-in-differences analysis using random-effects models; %D shows the relative change to the baseline; HIV/AIDS, human immunodeficiency virus/acquired immune deficiency syndrome; VAT, tetanus vaccination. * P < 0.05 after Bonferroni correction. Site visits and provider interviews showed frequent delays in incentive payments to health facilities and, at some facilities, a lack of transparency in how incentive payments were allocated within the facility. These delays were primarily due to administrative issues, such as funding rupture, delays in verification and inadequate management knowledge and skills. At the health-facility level, some managers did not capture the essence of RBF and did not inform staff about the RBF programme and how it worked in health facilities. Staff in some intervention health facilities did not know about the implementation of the RBF programme, while other health facilities lacked the capacity to calculate bonuses and link them to the performance of each staff member. Discussion In general, this study shows the potential of this pilot RBF programme to improve MCH services. Both the household and health facility surveys revealed that the implementation of RBF was associated with improvement in some, but not all, incentivized MCH services included in the analysis. The household survey suggested that the major impacts of the RBF programme were on the quality of care, curative care and HIV/AIDS testing among pregnant women. However, the household survey did not convey a significant impact of RBF on prenatal care, postnatal care and family planning. The health facility survey, however, displayed more salient impacts favourable for all indicators except for full immunization among children. Partially due to the favourable results from this pilot study, RoC scaled up the RBF programme to other departments, where the government and the World Bank are co-financing the programme. From the household survey, we found that the perceived quality of services improved consistently using multiple quality indicators. This result is consistent with findings from evaluations of RBF in Rwanda (Basinga et al. 2011; Janssen et al. 2015). In the RBF programme in RoC, quality of care was measured primarily on structural indicators, namely the availability of essential infrastructure (e.g. laboratory equipment, qualification of staff, hygiene, availability of medication etc.). The incentive payment formula weighed quality heavily, with about half of a facility’s payment based on quality. This substantial payment for quality of care provided a strong motivation for health facilities to upgrade health infrastructure. Responding to these quality incentives, health facilities tended to use additional resources from RBF incentives to improve those indicators that could be enhanced most quickly, such as availability of drugs and hygiene. In general, facilities distributed a portion of the proceeds of RBF as bonuses to staff at a facility. Typically, all staff received some share of the proceeds, including the least skilled temporary and non-medical personnel, which may strengthen cohesion and engagement within the health facilities. Another indicator showing improvement in the household survey was curative care for children under 5 years of age. This result parallels findings from a study in Haiti (Zeng et al. 2013). The payment per unit of this service in RoC is $0.40 per visit. When compared with other child services, such as full immunization at $6 per child case (see Table 1), the curative care payment is relatively small. However, full immunization may be restricted by (1) availability of vaccines, (2) limited population, as children under age 2 are the key population for the indicator of full immunization and (3) saturation of the immunization coverage. In RoC, certain vaccines, such as BCG, have high coverage already (Centre Nationale de la Statistique et des Études Économiques (CNSEE) & ICF International 2013), leaving limited room for improvement. The payment rate for assisted delivery, $8 per delivery, is in the top 40% of incentivized services. If health providers respond to incentives, it is expected that the coverage of assisted delivery would increase. Surprisingly, we do not observe a statistically significant increase in assisted delivery from the household survey. Several reasons may explain why there is no improvement in this indicator in spite of high incentives. First, RoC has high baseline coverage of assisted delivery (87%), leaving comparatively less room for improvement. Those women who had not used assisted delivery before may have lived in remote areas that remain hard to reach. Second, scaling up assisted delivery may require resources outside the control of an individual health facility (e.g. training and recruiting new personnel), and require more time to pay off than was available in this 2-year study. Third, the 4-month interruption in RBF payments and its expiration in 2014 may have discouraged long-term investments. In addition to assisted delivery, the coverage of any prenatal care visit, bed nets and receiving DPT3 among children aged 6–23 months had a negative association with the RBF programme, which is unexpected. The same reasons for assisted delivery are applicable to prenatal care and bed nets (e.g. high coverage at the baseline and interruption of RBF implementation) for explaining the negative association. Furthermore, bed nets are generally distributed to pregnant women who visit health facilities regularly for prenatal care in RoC (Koukouikila-Koussounda and Ntoumi 2016), which may explain the unexpected consistent impact of RBF on the coverage of prenatal care, assisted delivery and bed nets. However, the impact of RBF on DPT3 could be more complicated given that the supply of vaccine plays an important role in determining the coverage. DTP3 is often used as a proxy of full immunization. We found that the result from the household survey was consistent with that from the health facility survey, showing the negative impact of RBF on immunization. Further investigation reveals a drop in coverage of immunization in both intervention and comparison groups, with a sharper drop in the comparison group. This may suggest a supply issue of vaccines in RoC in the post-RBF period. The results from the health-facility survey are more favourable than are those from the household survey, which is expected as the household survey uses a whole market analysis approach while the health-facility survey focuses on the public sector. After 2 years of implementation of RBF, the health-facility survey showed that curative visits, hospital referral, vitamin A distribution among children, HIV/AIDS testing among pregnant women and assisted delivery improved substantially. The use of curative care and vaccination is consistent with findings from the household survey. However, there are discrepancies between the health facility and household surveys on assisted delivery, where there is a significant improvement recorded from health-facility survey, but no significant increase recorded in the household survey. Three factors may explain the differences: (1) the RBF programme was implemented in public facilities only, and did not cover private facilities. It is likely that the improved quality at public facilities attracted pregnant women who otherwise would have delivered in private health facilities; (2) some pregnant women in both the comparison and intervention groups could not distinguish licenced health facilities from informal health facilities. If this occurred disproportionally more among pregnant women in the comparison group, there was a risk that the coverage of assisted delivery in the comparison group was inflated much more than that in the intervention group, resulting in no differences in assisted delivery between the two groups. As community engagement is a part of the RBF programme design, it is less likely that community workers refer pregnant women to informal health facilities. Our rate of assisted delivery from the household survey was similar to those from DHSs conducted around the same time in the country (Centre Nationale de la Statistique et des Études Économiques (CNSEE) & ICF International 2013); this consistency helped validate the household survey; and (3) it is also possible that health facilities over-reported their service provision. However, all the health facilities’ data were verified by qualified staff, decreasing the chance of over-reporting. Therefore, the discrepancy in the rates of assisted delivery between the health facility and household surveys is most likely due to the switch in preference from private to public facilities. Nevertheless, the result on assisted delivery needs to be interpreted with caution. According to the RBF programme design, supervisors were supposed to visit and monitor health providers in the RBF programme regularly to reinforce and examine their knowledge of RBF and its impact on the process of care. Site visits revealed initial shortcomings. For example, during regular monitoring and evaluation visits, frontline health facility staff expressed limited understanding of RBF, particularly on how it relates to staff behaviour at facility and community levels, and the overall engagement with patients. This knowledge appeared to improve over time. The relationship between RBF at the organizational level and implementation at staff levels could be improved. Nevertheless, in household surveys about 90% of consumers rated the quality of care as good. Several limitations are worth noting. First, this study uses a quasi-experimental research design with DIDs. Although DIDs are a solid approach to estimate the impact of RBF, neither the intervention nor the comparison departments were selected randomly. Thus, there is a potential selection bias that may affect the evaluation results. Second, in this study we used intention-to-treat analysis. In the intervention group, RBF was first implemented among 73 health facilities that met minimal qualifications, leaving out those whose quality was too poor to have the capacity to implement RBF, which also results in a selection bias. Fortunately, the DIDs approach, to some degree, mitigated such biases. Third, the quality measures in the household survey are more related to patient experience, and less about direct and objective indicators of health process and quality outcomes. Current discussions on development of measures for quality of care in LMICs suggest that incentive payments should be based more on process and outcome quality indicators. Fourth, the evaluation was based on only five departments, so some managerial change at the departmental level during the RBF programme could have affected the results. Although we were not aware of any such change, an unobserved change could still have confounded the findings (Kahneman and Tversky 1979). Last, this study could not link households to health facilities, and thus the household survey potentially captured the impact of both the RBF programme implemented at public health facilities, and any initiatives, if there were, in private and non-governmental organization facilities. However, we are not aware of any large scale health initiatives addressing MCH services in the private and non-governmental organization sector during the RBF implementation period. Although the results were favourable for the RBF programme, they were not consistent across all the key MCH services. RBF implementation was not limited to providing payment incentives only. Rather, it was regarded as a vehicle to implement managerial improvements at administrative and health-centre levels. Successful implementation also requires strong monitoring and evaluation, improved transparency, capacity building and increased autonomy. Future versions of RBF could strengthen several elements to yield more favourable outcomes. For instance, in the current study, RBF was implemented without a component of capacity building related to service delivery. Some staff at health facilities were not aware an RBF programme was ongoing, showing that awareness of this programme among providers was not high. Mobilization of providers from the bottom level (community) is considered essential to guarantee the programme’s success (Falisse et al. 2012; Manongi et al. 2014). There is a need for capacity building at all levels to develop a well-coordinated, integrated network of RBF implementation. In addition, prompt payment or incentives and transparency in their allocation within the facility are critical to the effectiveness of the RBF programme. It is also important to consider the demand-side financing alignment with RBF to generate synergy, simultaneously improving the supply side and stimulating demand in the community for life-saving MCH services (Carrin et al. 2005; Witter et al. 2013). Last, the RBF programme was based entirely on positive incentives with no sanctions for substandard performance. ‘Prospect theory’ has found that averting a loss can be a more powerful motivator than seeking a gain (Kahneman and Tversky 1979). Additionally, while positive incentives require additional public funding, sanctions would save the government money, thereby making RBF more financially sustainable. Disclaimer Views expressed in this article are those of the authors and do not necessarily reflect the views of the authors’ institutions and funder of this study. None of the authors have any financial interest in any product discussed in this article. Acknowledgements We thank Alphonse Indouyi-Ibikoue for assistance with data collection and Clare L. Hurley for editorial assistance. Funding This study was funded by the World Bank. Conflict of interest statement. None declared. References Africa Health Forum. 2013. Results-Based Financing for Health . Washington, DC: The World Bank. Ashir GM, Doctor HV, Afenyadu GY. 2013. Performance based financing and uptake of maternal and child health services in Yobe State, northern Nigeria. Global Journal of Health Science  5: 34– 41. Google Scholar CrossRef Search ADS PubMed  Basinga P, Gertler PJ, Binagwaho A. 2011. Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: an impact evaluation. Lancet  377: 1421– 8. Google Scholar CrossRef Search ADS PubMed  Bonfrer I, Soeters R, Van de Poel E et al.   2014. Introduction of performance-based financing in Burundi was associated with improvements in care and quality. Health Affairs (Millwood)  33: 2179– 87. Google Scholar CrossRef Search ADS   Carrin G, Waelkens MP, Criel B. 2005. Community-based health insurance in developing countries: a study of its contribution to the performance of health financing systems. Tropical Medicine and International Health  10: 799– 811. Google Scholar CrossRef Search ADS PubMed  Centre Nationale de la Statistique et des Études Économiques (CNSEE), ICF International. 2013. Enquête Démographique Et De Santé Du Congo (EDSC-II) 2011-2012 . Calverton, MD: CNSEE et ICF International. Darmstadt GL, Bhutta ZA, Cousens S et al.   2005. Evidence-based, cost-effective interventions: how many newborn babies can we save? Lancet  365: 977– 88. Google Scholar CrossRef Search ADS PubMed  Falisse JB, Meessen B, Ndayishimiye J, Bossuyt M. 2012. Community participation and voice mechanisms under performance-based financing schemes in Burundi. Tropical Medicine and International Health  17: 674– 82. Google Scholar CrossRef Search ADS PubMed  Fretheim A, Witter S, Lindahl AK, Olsen IT. 2012. Performance-based financing in low- and middle-income countries: still more questions than answers. Bulletin of the World Health Organization  90: 559– 559A. Google Scholar CrossRef Search ADS PubMed  Fritsche G,B, Soeters R, Meessen B. 2014. Performance-Based Fianncing Toolkit . Washington, DC: The World Bank. Google Scholar CrossRef Search ADS   Frost JJ, Sonfield A, Zolna MR, Finer LB. 2014. Return on investment: a fuller assessment of the benefits and cost savings of the US publicly funded family planning program. Milbank Quraterly  92: 696– 749. Google Scholar CrossRef Search ADS   Hongoro C, Normand C, 2006. Health workers: Building and motivating the workforce. In: Jamisan D, Breman J, Measham A et al.   (eds). Disease Control Priorities in Developing Countries , 2nd edn. Washington, DC: World Bank. Janssen W, Ngirabega J. d D, Matungwa M, Van Bastelaere S. 2015. Improving quality through performance-based financing in district hospitals in Rwanda between 2006 and 2010: a 5-year experience. Tropical Doctor  45: 27– 35. Google Scholar CrossRef Search ADS PubMed  Kahneman D, Tversky A. 1979. Prospect theory: An analysis of decision under risk. Econometrica  47: 263– 91. Google Scholar CrossRef Search ADS   Koukouikila-Koussounda F, Ntoumi F. 2016. Malaria epidemiological research in the Republic of Congo. Malaria Journal  15: 598. Google Scholar CrossRef Search ADS PubMed  Manongi R, Mushi D, Kessy J, Salome S, Njau B. 2014. Does training on performance based financing make a difference in performance and quality of health care delivery? Health care provider’s perspective in Rungwe Tanzania. BMC Health Services Research  14: 154. Google Scholar CrossRef Search ADS PubMed  Ministry of Health and Population of the Republic of the Congo, World Bank, Cordaid, Memisa. 2014. Final activity report of the second phase of the project on performance based financing. Brazzaville, Republic of the Congo: Ministry of Health and Population of the Republic of the Congo, World Bank, Cordaid, and Memisa. RBFHealth. 2015. Performance-based financing conceptual framework. https://www.rbfhealth.org/resource/performance-based-financing-conceptual-framework, accessed 11 May 2017. Stenberg K, Axelson H, Sheehan P et al.   2014. Advancing social and economic development by investing in women’s and children’s health: a new Global Investment Framework. Lancet  383: 1333– 54. Google Scholar CrossRef Search ADS PubMed  The World Bank. 2015. GDP per Capita . Washington, DC: The World Bank. http://data.worldbank.org/indicator/NY.GDP.PCAP.CD, accessed 30 September 2015. Turcotte-Tremblay AM, Spagnolo J, De Allegri M, Ridde V. 2016. Does performance-based financing increase value for money in low- and middle- income countries? A systematic review. Health Economics Review  6: 30. Google Scholar CrossRef Search ADS PubMed  UNICEF. 2008. UNICEF humanitarian action: Republic of the Congo (Brazzaville) in 2008. https://www.unicef.org/har08/files/har08_Congo_countrychapter.pdf, accessed 30 September 2015. Van de Poel E, Flores G, Ir P, O’Donnell O. 2016. Impact of performance-based Financing in a low-resource setting: a decade of experience in Cambodia. Health Economics  25: 688– 705. Google Scholar CrossRef Search ADS PubMed  Witter S, Toonen J, Meessen B et al.   2013. Performance-based financing as a health system reform: mapping the key dimensions for monitoring and evaluation. BMC Health Services Research  13: 367. Google Scholar CrossRef Search ADS PubMed  World Bank. 2013. Health System Strennthening Project II . Washington, DC: World Bank. World Health Organization. 2010. Health Systems Financing: The Path to Universal Coverage . Geneva, Switzland: World Health Organization. PubMed PubMed  Zeng W, Cros M, Wright KD, Shepard DS. 2013. Impact of performance-based financing on primary health care services in Haiti. Health Policy and Planning  28: 596– 605. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press in association with The London School of Hygiene and Tropical Medicine. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Health Policy and Planning Oxford University Press

Evaluation of results-based financing in the Republic of the Congo: a comparison group pre–post study

Loading next page...
 
/lp/ou_press/evaluation-of-results-based-financing-in-the-republic-of-the-congo-a-91MjjlPfCR
Publisher
Oxford University Press
ISSN
0268-1080
eISSN
1460-2237
D.O.I.
10.1093/heapol/czx195
Publisher site
See Article on Publisher Site

Abstract

Abstract Results-based financing (RBF) has been advocated and increasingly scaled up in low- and middle-income countries to increase utilization and quality of key primary care services, thereby reducing maternal and child mortality rates. This pilot RBF study in the Republic of the Congo from 2012 to 2014 used a quasi-experimental research design. The authors conducted pre- and post-household surveys and gathered health facility services data from both intervention and comparison groups. Using a difference-in-differences approach, the study evaluated the impact of RBF on maternal and child health services. The household survey found statistically significant improvements in quality of services regarding the availability of medicines, perceived quality of care, hygiene of health facilities and being respected at the reception desk. The health facility survey showed no adverse effects and significantly favourable impacts on: curative visits, patient referral, children receiving vitamin A, HIV testing of pregnant women and assisted deliveries. These improvements, in relative terms, ranged from 42% (assisted deliveries) to 155% (children receiving vitamin A). However, the household survey found no statistically significant impacts on the five indicators measuring the use of maternal health services, including the percentage of pregnant women using prenatal care, 3+ prenatal care, postnatal care, assisted delivery, and family planning. Surprisingly, RBF was found to be associated with a reduction of coverage of the third diphtheria, pertussis, and tetanus immunization among children in the household survey. From the health facility survey, no association was found between RBF and full immunization among children. Overall, the study shows a favourable impact of an RBF programme on most, but not all, targeted maternal and child health services. Several aspects of programme implementation, such as timely disbursement of incentives, monitoring health facility performance, and transparency of using funds could be further strengthened to maximize RBF’s impact. Results-based financing, pay for performance, impact evaluation, household survey, the Republic of the Congo Key Messages There is positive impact of RBF programme on some, not all, maternal and child health services that the programme targeted. In spite of general favourable impact of PBF, programme implementation, such as timely disbursement of incentives, monitoring the performance of health facilities, and transparency of using funds, could be further strengthened to maximize the impact. Introduction In spite of substantial achievements during the millennium development goals (MDGs) era, many low- and middle-income countries (LMICs) still face challenges in establishing well-functioning health systems. Reasons include scarcity of supplies, limited infrastructure, shortage of qualified human resources (World Health Organization 2010), absence of incentives, dual practice, brain drain, and the lack of an enabling working environment (Hongoro and Normand 2006). In response to these challenges, health systems in many LMICs over the last two decades have launched provider incentive programmes. With support from the governments of Norway and the UK, the World Bank is managing the Health Results Innovation Trust Fund to pilot results-based financing (RBF) programmes in more than 30 countries (Africa Health Forum 2013). Most RBF programmes aim to improve maternal and child health (MCH) services by offering incentive payments to participating health facilities (including their staff) based on their quantity and quality of health services. In 2012, the Republic of the Congo’s (RoC) rapidly growing economy achieved a gross national income per capita of USD$3, 450 adjusted for purchasing power parity (The World Bank 2015). In spite of being a lower-middle income country, RoC continued to face challenges in addressing MCH problems, suggesting potential concerns of inefficiencies within the health system. The country’s 2013 under-5 mortality rate of 68 per 1000 was almost double the 2015 target of 35 per 1000 (World Bank 2013). One factor was the low utilization and poor quality of key MCH services (UNICEF 2008), despite MCH services having generally favourable cost-effectiveness ratios (Darmstadt et al. 2005; UNICEF 2008; Frost et al. 2014; Stenberg et al. 2014). According to Demographic and Health Survey (DHS) 2011–12 in RoC, 21.2 and 22.7% of pregnant women did not have 4+ prenatal visits and postnatal care, respectively, (Centre Nationale de la Statistique et des Études Économiques (CNSEE) & ICF International 2013) although the coverage of assisted delivery was relatively high (Centre Nationale de la Statistique et des Études Économiques (CNSEE) & ICF International 2013). Additionally, the coverage of prenatal and postnatal visits was much lower in rural areas. (Centre Nationale de la Statistique et des Études Économiques (CNSEE) & ICF International 2013). To respond to the need for improved MCH care, particularly in rural areas, the RoC’s RBF programme targeted to MCH began in January 2012 in three mostly rural departments (Niari, Plateaux and Pool). Supported by the World Bank and managed by the RoC’s Ministry of Health and Population, the RBF programme incentivizes providers by rewarding verified quantity and quality of key MCH services. According to the RBF framework and toolkit developed by the World Bank (Fritsche et al. 2014; RBFHealth 2015), RBF programmes have multiple elements beyond contracting health facilities on indicators, including providing technical assistance on developing business plans for health facilities, trainings on basic financial management, granting health facilities autonomy to use resources gained from incentives and verifying data for the incentive payment. These interventions may result in increased autonomy, strengthened data reporting, and enhanced capacity to manage health facilities. Consequently, RBF has the potential to bring about management and behaviour changes of staff (e.g. improved motivation) to boost the coverage and quality of incentivized health services. RBF programmes have demonstrated beneficial impacts in some LMICs. In Rwanda, the RBF programme led to a significant increase in coverage of institutional deliveries and preventive care visits for children (Basinga et al. 2011). Similar results were found in Burundi, Haiti, Nigeria and Cambodia. (Ashir et al. 2013; Zeng et al. 2013; Bonfrer et al. 2014; Van de Poel et al. 2016). However, a recent review of the impact of global RBF programmes calls for more investigation, partially due to the lack of rigorous research design (Fretheim et al. 2012; Turcotte-Tremblay et al. 2016). This study adds more evidence to the current literature on the impact of RBF by using data from two rounds of household and health facility surveys to quantify the impact of RBF on the utilization of key MCH services in RoC. Methods RBF schemes in the RoC In an effort to reduce maternal and child mortality to meet MDGs in RoC, a pilot RBF scheme, with financial support from the World Bank, began in January 2012. The RBF scheme started in three departments (Niari, Plateaux and Pool), equivalent to regions in many African countries, among 73 public health centres and 7 hospitals. These departments contained 1.26 million of ROC’s 4.41 million residents in 2013 (Ministry of Health and Population of the Republic of the Congo et al. 2014). Under the RBF scheme, Cordaid, the purchasing agency selected by the Ministry of Health and Population, obtained monthly reports of the specified services, verified quantities against facility registers and visited sampled households, assessed quality and transferred payments quarterly into each facility’s bank account. Table 1 lists the incentivized services and the amount of payment per unit for each service. These incentive payments were made in addition to the routine budget allocated by the Ministry of Health and Population. Cordaid also paid financial incentives to technical teams in each district that were responsible for supervising and helping health centres identify service provision issues and suggesting solutions. The RBF programme later expanded to include more health centres, with a total of 97 health centres in the three departments. Due to lengthy administrative and contractual procedures, the programme was interrupted from June 2012 through September 2012 and restarted in October 2012. Cordaid continued monitoring these services and paying incentives through February 2014, when this phase of the RBF scheme ended. Table 1 Health services included in the RBF package and their payments per unit Service number  Service description  Payment per unit (USD)  General population services   1  Curative visits  0.40   2  Curative visits for the poor  1.00   3  Hospitalizations  0.80   4  Small surgeries  3.00   5  Hospital referrals  10.00   6  Tuberculosis and leprosy cases detected  30.00   7  Tuberculosis and leprosy cases cured  60.00   8  Toilets built in the catchment area of the health center  6.00   9  Insecticide-treated bed nets distributed  0.50  HIV/AIDS services   10  Cases with opportunistic infections treated  0.40   11  Clients having voluntary counselling and testing  2.00   12  HIV-positive cases referred to the hospital  10.00   13  Pregnant women tested for HIV  2.00   14  HIV+ pregnant women receiving AZT+NVP  20.00   15  Newborns from HIV+ women receiving AZT+NVP  20.00   16  Patients under ARV followed for 6 months  1.00   17  Condom distribution points  10.00  Maternal and reproductive health services   18  Assisted deliveries  8.00   19  Pregnant women having 3 or more standard prenatal visits  6.00   20  Women who received tetanus vaccination during prenatal care (VAT2+)  2.00   21  Women who received a second dose of malaria prophylaxis during prenatal care  2.00   22  Pregnant women fully immunized  4.00   23  Women of productive age who use family planning method  5.00  Child health services   24  Children under 1 fully immunized  6.00   25  Children under 5 taking vitamin A  2.00  Service number  Service description  Payment per unit (USD)  General population services   1  Curative visits  0.40   2  Curative visits for the poor  1.00   3  Hospitalizations  0.80   4  Small surgeries  3.00   5  Hospital referrals  10.00   6  Tuberculosis and leprosy cases detected  30.00   7  Tuberculosis and leprosy cases cured  60.00   8  Toilets built in the catchment area of the health center  6.00   9  Insecticide-treated bed nets distributed  0.50  HIV/AIDS services   10  Cases with opportunistic infections treated  0.40   11  Clients having voluntary counselling and testing  2.00   12  HIV-positive cases referred to the hospital  10.00   13  Pregnant women tested for HIV  2.00   14  HIV+ pregnant women receiving AZT+NVP  20.00   15  Newborns from HIV+ women receiving AZT+NVP  20.00   16  Patients under ARV followed for 6 months  1.00   17  Condom distribution points  10.00  Maternal and reproductive health services   18  Assisted deliveries  8.00   19  Pregnant women having 3 or more standard prenatal visits  6.00   20  Women who received tetanus vaccination during prenatal care (VAT2+)  2.00   21  Women who received a second dose of malaria prophylaxis during prenatal care  2.00   22  Pregnant women fully immunized  4.00   23  Women of productive age who use family planning method  5.00  Child health services   24  Children under 1 fully immunized  6.00   25  Children under 5 taking vitamin A  2.00  Notes: HIV/AIDS, human immunodeficiency virus/acquired immune deficiency syndrome; AZT, zidovudine, also known as azidothymidine; NVP, niverapine; VAT, tetanus vaccination. Research design This evaluation used a quasi-experimental research design, where the three departments implementing RBF (Niari, Plateaux and Pool) constituted the ‘intervention group’, and two departments, Bouenza and Cuvette, served as the ‘comparison group’. Bouenza and Cuvette, were selected as comparisons because of their geographic proximity to the three intervention departments. Table 2 shows the comparison of some indicators at the department level. Overall, the five departments were quite similar on the indicators except the share of rural and semi-urban population where Plateaux and Pool had much higher share than Bouenza and Cuvette had. To evaluate the impact of the RBF programme, before-and-after rounds of household and health facility surveys were conducted: one of each in March 2012, before the RBF implementation, and one of each in March 2014, after the RBF implementation. Table 2 Comparison of department characteristics   Intervention departments   Comparison departments   Niari  Plateaux  Pool  Bouenza  Cuvette  Gini coefficient (%)  42.1  46.5  34.0  37.0  48.5  Female literacy (secondary or higher) (%)  59.0  47.6  51.9  46.7  67.0  Media exposure (male): Reading newspaper at least once a week (%)  9.6  9.7  9.2  7.5  14.9  Female population currently working (%)  72.6  81.9  82.6  75.2  73.8  Share of rural and semi-urban population (%)  48.2  70.7  84.4  47.0  47.7    Intervention departments   Comparison departments   Niari  Plateaux  Pool  Bouenza  Cuvette  Gini coefficient (%)  42.1  46.5  34.0  37.0  48.5  Female literacy (secondary or higher) (%)  59.0  47.6  51.9  46.7  67.0  Media exposure (male): Reading newspaper at least once a week (%)  9.6  9.7  9.2  7.5  14.9  Female population currently working (%)  72.6  81.9  82.6  75.2  73.8  Share of rural and semi-urban population (%)  48.2  70.7  84.4  47.0  47.7  Source: Demographic and Health Survey 2011–12, Republic of the Congo. In the first round of the household survey, households were selected using a multi-stage sampling approach. The RoC’s Department of Planning has defined ZDs (enumeration zones) as the basis for conducting the national census. The first stage of the survey in this sampling process was to draw random samples of 100 ZDs from the 557 ZDs in the three intervention departments (Niari, Plateaux and Pool) and from the 591 ZDs in the two comparison departments (Bouenza and Cuvette). The sampled ZDs were chosen with probability proportional to the population size (PPS) of each ZD. In the second stage of the sampling process, one village from each ZD was randomly selected. As the Department of Planning did not release information on the population size of villages, we could not conduct PPS sampling directly in this stage. Instead, each village within the ZD was given an equal probability to be selected. In cases where small villages did not have the minimum number of households required (seven households for each village according to our calculation of the sample size), we included households of an adjacent village to complement the one we originally selected. This adjustment brought the second stage closer to PPS. In the third stage, we worked with the village chief or community workers who provided a list of households in selected villages. We aimed to select households with at least one child under 2 years of age. We first randomly selected seven households regardless of whether they met this child criterion or not. If the household had at least one child under 2, the evaluation team then conducted an interview to collect the required information from that household. If the household did not meet this criterion, the team sequentially checked whether the next neighbouring household was eligible, until the evaluators found an eligible household. It then requested the interview. A total of 1349 households were selected for the baseline household survey, with 1344 mothers and 1841 children in the sample. For the second round of household surveys, we visited the same villages as the first round, and used the same sampling approach in the third stage, interviewing 1325 households. The sample sizes for mothers and children were 1307 and 1859, respectively. Since we conducted the two household surveys in March of 2012 and 2014 respectively with a 2-year gap, and considering the requirement of selecting households with at least one child under 2-years old in each round, we suspected that only a few of the households selected in the second round had also been in the first round. For the endline survey, we did not intend to survey the same households included in the baseline to construct panel data, because the requirement of having at least one child might eliminate many households to be used in the endline survey, and the country was unable to provide the research team with a household tracking system to identify the potential participatory households. Each of the household surveys were conducted over the course of about one month, and two research teams, each consisting of a supervisor and two to three surveyors, conducted the surveys. Mothers of children under 2-years old responded to relevant questions on using maternal health services and child services for their children. For health facility surveys, data from 73 health centres in the intervention group and 25 health centres in the comparison group were collected retrospectively in October and November 2012 for the services provided from January to December 2011, to serve as baseline. Data on service provision from the same health centres from January 2012 to February 2014 were collected retrospectively in March 2015. In order to rule out potential bias due to reporting behaviour changes from RBF, data from health facilities were collected from the registry book in both intervention and comparison facilities. A research team of two researchers visited health facilities to collect post-RBF health facility data on a rolling basis quarterly, while the health facility data prior to the RBF programme were collected in the first quarter after RBF was implemented. As we planned to analyse these data by year, the data from January and February 2014 were not included in the analysis. To examine the impact of RBF, only data from 2013 were used as the endline. For both household and health-facility surveys, most outcome variables used in the study focussed on the incentivized indicators, such as assisted delivery, prenatal care, postnatal care, HIV/AIDS testing and vaccinations. Additionally, we also included indictors measuring the quality of services in the household survey, such as availability of drugs, perceived quality of care, waiting time, and hygiene of health facilities. It should be noted that although there were 25 incentivized indicators for health centres and hospitals, not all indicators were measured or were measurable in the household surveys (e.g. hospitalizations, small surgeries, condom distribution points and tuberculosis testing) and thus many indicators were excluded from the analysis. In the end, nine incentivized indicators were included in the analysis. For the health-facility surveys, not all incentivized indicators were reported during the period prior to the RBF programme, and thus we only included the seven common indicators that were reported at the health-centre level in both periods before and after the RBF programme was implemented. Both household and health facility surveys were designed and carried out by external surveyors who were independent from the operation agency of the RBF programme in the country. We estimated the impact of RBF on the key outcome indicators using the difference-in-differences (DIDs) approach with a multivariate regression model for both the household and health-facility surveys. The model specification for the household surveys is provided below:   Service= b0+b1RBF+b2Post+b3RBF*Post+b4Urban+b5Age+b6Educ+b7HHsize+b8With partner+ b9Regular job+b10House ownership+b11Distance+ε, where RBF represents whether the household was in the RBF group or not, Post whether the measures were taken after the implementation of RBF or not, RBF*Post the interaction term between RBF and Post, and ε the random noise. The coefficient for RBF*Post (b3) represents DIDs, measuring the impact of RBF on service utilization. The models also controlled for a series of individual and household characteristics that might affect the utilization of health services. These characteristics measured financial and physical accessibility of households and respondents’ awareness of and education on health care, which included the location of households (Urban), house ownership (House ownership), household size (HHsize), mother’s age (Age), education (Educ), status of living with a partner (With partner), status of having a regular job (Regular job) and distance of households from health facilities (Distance). We conducted two regression models, one with and the other without village fixed effects. Both models adjusted for clustering at the village level. Given the parsimony of the model without village fixed effects, the interpretation of the results was focussed on this model. For the health facility survey, we conducted both random- and fixed-effects models to examine the impact of the RBF programme, while controlling for seasonal fluctuation of service delivery. The full model specification for the health facility surveys is:   Serviceit= b0+b1RBFit+b2Postit+b3RBFit*Postit+b4Quartersit+αi+εit, where i and t represent the ith facility at the time t; RBF, Post and RBF*Post have the same meaning as the model for the household surveys; Quarters are three dummy variables indicating which quarter the measure was taken in; bs are coefficients or coefficient matrix for corresponding variable(s); αi is the facility individual effects (either fixed or random effects); and εit is the random noise. The results from the two types of the models (the fixed and random effects models) were compared using the Housman test and we found no statistical differences for all seven indicators. Thus, we focussed on results from random-effects models given its advantage of the efficiency of estimates, and also reported results from the facility fixed effects model to provide additional information. We conducted all statistical analyses with STATA 12 (Stata Corp LP, College Station, TX). Results At baseline, the average age of mothers who had had one pregnancy in the two years prior to the survey was 27.7 years (27.7 ± 9.5). Respondents in the intervention departments were slightly younger (27.2 ± 8.3 years) than those in the comparison departments (28.2 ± 10.6 years). The difference was statistically significant (P < 0.05). There were no statistically significant differences between the comparison and intervention departments in mothers’ education level or proportion of them living with partners (Table 2). Most baseline differences between comparison and intervention departments on utilization of maternal health services were not statistically significant, except for the use of any prenatal care, four or more prenatal visits, and postnatal visits (Table 3). Higher percentages of pregnant women received prenatal and postnatal visits in the RBF departments than the comparison departments. The coverage of some services was high. For example, the coverage of institutional delivery was 84 and 87% in the comparison and intervention departments, respectively; the coverage of four or more prenatal care visits was 79 and 88% in the comparison and intervention departments, respectively. In contrast, the use of postnatal care and family planning methods remained low among the mothers, with an average of slightly >50% for both intervention and comparison departments combined. Table 3 Characteristics of households and mothers, and utilization of maternal health and HIV/AIDS services at baseline (from household survey) Variable  Intervention departments  Comparison departments  Total mean ± SD  Difference (intervention vs comparison)  mean ± SD  mean ± SD    Household characteristics   Household size  5.28 ± 2.04  5.58 ± 2.17  5.44 ± 2.11  −0.30**   Number of children <5 years of age  1.51 ± 0.67  1.50 ± 0.61  1.50 ± 0.64  0.02   Daily spending (FCFA)  2110 ± 1044  2258 ± 1518  2185 ± 1310  −148*   Own land (Yes = 1, No = 0)  73.1 ± 44.4%  76.8 ± 42.2%  75.0 ± 43.3%  −3.8%   Own house (Yes = 1, No = 0)  63.8 ± 48.1%  68.8 ± 46.4%  66.4 ± 47.3%  −5.0%   Distance from health facility (minutes)  57.53 ± 60.30  62.57 ± 79.27  60.12 ± 70.69  −5.04  Mother's characteristics   Age (years)  27.20 ± 8.25  28.22 ± 10.59  27.72 ± 9.54  −1.03*   Education (% above middle school)  55.3 ± 49.8%  52.4 ± 50.0%  53.8 ± 49.9%  2.9%   Living with partners (%)  82.6 ± 38.0%  78.9 ± 40.8%  80.7 ± 39.5%  3.6%  Maternal health services   Live birth ratio in last 5 years  93.4 ± 16.9%  94.1 ± 16.6%  93.7 ± 16.7%  −0.7%   Institutional delivery (%)  87.5 ± 33.1%  83.7 ± 36.9%  85.6 ± 35.2%  3.7%   Cesarean section for last delivery (%)  15.2 ± 33.9%  12.4 ± 33.0%  13.8 ± 34.5%  2.8%   Any prenatal care visit (%)  95.3 ± 21.3%  85.7 ± 35.0%  90.4 ± 29.5%  9.5%**   Four or more prenatal care visits (%)  87.5 ± 33.1%  79.4 ± 40.5%  83.3 ± 37.3%  8.1%**   Used postnatal care (%)  58.3 ± 49.3%  50.6 ± 50.0%  54.4 ± 49.8%  7.7%**   Ever used family planning methods (%)  52.9 ± 50.0%  51.2 ± 50.0%  52.0 ± 50.0%  1.7%  HIV/AIDS services   Offered HIV test (%)  43.4 ± 49.6%  48.3 ± 50.0%  45.7 ± 49.8%  −4.9%   Received HIV test among those who were offered (%)  87.0 ± 33.7%  88.0 ± 32.6%  87.5 ± 33.1%  −1.0%   Obtained HIV test results for those who received test (%)  89.4 ± 30.9%  77.7 ± 41.7%  83.6 ± 37.0%  11.7%***  Variable  Intervention departments  Comparison departments  Total mean ± SD  Difference (intervention vs comparison)  mean ± SD  mean ± SD    Household characteristics   Household size  5.28 ± 2.04  5.58 ± 2.17  5.44 ± 2.11  −0.30**   Number of children <5 years of age  1.51 ± 0.67  1.50 ± 0.61  1.50 ± 0.64  0.02   Daily spending (FCFA)  2110 ± 1044  2258 ± 1518  2185 ± 1310  −148*   Own land (Yes = 1, No = 0)  73.1 ± 44.4%  76.8 ± 42.2%  75.0 ± 43.3%  −3.8%   Own house (Yes = 1, No = 0)  63.8 ± 48.1%  68.8 ± 46.4%  66.4 ± 47.3%  −5.0%   Distance from health facility (minutes)  57.53 ± 60.30  62.57 ± 79.27  60.12 ± 70.69  −5.04  Mother's characteristics   Age (years)  27.20 ± 8.25  28.22 ± 10.59  27.72 ± 9.54  −1.03*   Education (% above middle school)  55.3 ± 49.8%  52.4 ± 50.0%  53.8 ± 49.9%  2.9%   Living with partners (%)  82.6 ± 38.0%  78.9 ± 40.8%  80.7 ± 39.5%  3.6%  Maternal health services   Live birth ratio in last 5 years  93.4 ± 16.9%  94.1 ± 16.6%  93.7 ± 16.7%  −0.7%   Institutional delivery (%)  87.5 ± 33.1%  83.7 ± 36.9%  85.6 ± 35.2%  3.7%   Cesarean section for last delivery (%)  15.2 ± 33.9%  12.4 ± 33.0%  13.8 ± 34.5%  2.8%   Any prenatal care visit (%)  95.3 ± 21.3%  85.7 ± 35.0%  90.4 ± 29.5%  9.5%**   Four or more prenatal care visits (%)  87.5 ± 33.1%  79.4 ± 40.5%  83.3 ± 37.3%  8.1%**   Used postnatal care (%)  58.3 ± 49.3%  50.6 ± 50.0%  54.4 ± 49.8%  7.7%**   Ever used family planning methods (%)  52.9 ± 50.0%  51.2 ± 50.0%  52.0 ± 50.0%  1.7%  HIV/AIDS services   Offered HIV test (%)  43.4 ± 49.6%  48.3 ± 50.0%  45.7 ± 49.8%  −4.9%   Received HIV test among those who were offered (%)  87.0 ± 33.7%  88.0 ± 32.6%  87.5 ± 33.1%  −1.0%   Obtained HIV test results for those who received test (%)  89.4 ± 30.9%  77.7 ± 41.7%  83.6 ± 37.0%  11.7%***  Notes: HIV/AIDS, human immunodeficiency virus/acquired immune deficiency syndrome; SD, standard deviation; FCFA, Franc Communauté Financière Africaine. * P < 0.05; ** P < 0.01; *** P < 0.001. Health providers played an important role in influencing mothers to receive testing for HIV/AIDS. As shown in Table 3, health providers asked fewer than half of the mothers to take an HIV/AIDS test during their prenatal visits in both comparison and intervention departments. Most of the mothers invited for testing for HIV/AIDS received the test. However, the percentages of women who were tested and obtained HIV test results differed significantly between the intervention and comparison departments (89 vs 78%, P < 0.001). For children under 5 at baseline, there were no differences in age and gender distribution (Table 4). The percentage of children under 5 who had malnutrition consultation was < 10% in both comparison and intervention departments. Most households (79%) had bed nets at home and most children under 5 (>90%) used a bed net if there was one at their home. About 10% and 20% of children under 5 who had illness did not seek care in the comparison and intervention departments at the baseline, respectively, and the difference was statistically significant (P < 0.001), which indicates there is opportunity for the intervention departments to make improvement. The coverage of bacille Calmette–Guerin (BCG) vaccine for tuberculosis was high (>90%). However, among children aged 1–5, coverage of the third diphtheria, pertussis and tetanus vaccine (DPT) was low with ∼40%. Table 4 Child health services and care at baseline (from household survey) Indicator  Intervention departments (mean ± SD)  Comparison departments (mean ± SD)  Total (mean ± SD)  Difference (intervention vs comparison)  Children’s characteristics   Age (years)  1.57 ± 1.30  1.59 ± 1.27  1.58 ± 1.28  −0.02   Male (%)  53.8 ± 49.9%  53.2 ± 49.9%  53.6 ± 49.9%  0.6%  Use of child care   Had malnutrition consultation (%)  9.3 ± 29.1%  8.2 ± 27.5%  8.7 ± 28.2%  1.1%   Had bed net for child (%)  80.5 ± 39.7%  76.9 ± 42.2%  78.6 ± 41.0%  3.6%   Used bed net in households with bed net (%)  97.4 ± 16.1%  94.0 ± 23.7%  95.7 ± 20.3%  3.3%**   Sought care for children with illness (%)  80.3 ± 39.8%  89.1 ± 31.2%  85.0 ± 35.7%  −8.8%***  Vaccinations   All children aged 12–24 months received BCG vaccination (%)  98.3 ± 12.9%  95.4 ± 21.1%  96.8 ± 17.5%  3.0%***   All children aged 1–5 years received three DPT vaccinations (%)  39.2 ± 48.9%  40.3 ± 49.1%  39.8 ± 49.0%  −1.1%  Indicator  Intervention departments (mean ± SD)  Comparison departments (mean ± SD)  Total (mean ± SD)  Difference (intervention vs comparison)  Children’s characteristics   Age (years)  1.57 ± 1.30  1.59 ± 1.27  1.58 ± 1.28  −0.02   Male (%)  53.8 ± 49.9%  53.2 ± 49.9%  53.6 ± 49.9%  0.6%  Use of child care   Had malnutrition consultation (%)  9.3 ± 29.1%  8.2 ± 27.5%  8.7 ± 28.2%  1.1%   Had bed net for child (%)  80.5 ± 39.7%  76.9 ± 42.2%  78.6 ± 41.0%  3.6%   Used bed net in households with bed net (%)  97.4 ± 16.1%  94.0 ± 23.7%  95.7 ± 20.3%  3.3%**   Sought care for children with illness (%)  80.3 ± 39.8%  89.1 ± 31.2%  85.0 ± 35.7%  −8.8%***  Vaccinations   All children aged 12–24 months received BCG vaccination (%)  98.3 ± 12.9%  95.4 ± 21.1%  96.8 ± 17.5%  3.0%***   All children aged 1–5 years received three DPT vaccinations (%)  39.2 ± 48.9%  40.3 ± 49.1%  39.8 ± 49.0%  −1.1%  Notes: SD, standard deviation; BCG, bacille Calmette–Guerin vaccine for tuberculosis; DPT, diphtheria, pertussis and tetanus vaccine. ** P < 0.01; *** P < 0.001. In general, the perceived quality of health services was good (Table 5). More mothers in the intervention departments felt they were well received by health facilities than those in the comparison departments (92 vs 87%, respectively), and most (89%) felt that health centres had good quality, whether they were in the comparison or intervention departments. However, only about 70% of mothers received medications from health centres during their last medical visit. The rate of receiving medication was lower in intervention departments compared with the comparison departments (65 vs 76%), and this difference was highly statistically significant (P < 0.001). Again, this also indicated potential for the intervention departments to improve medication availability. Another noteworthy indicator was the waiting time of mothers at their last outpatient visit, where mothers in the comparison department spent about 9 min longer than those in the intervention Group (47 vs 38 min), and the difference was statistically significant (P < 0.01). Table 5 Quality of health services at baseline (from household survey) Quality of services  Intervention departments (mean ± SD)  Comparison departments (mean ± SD)  Total (mean ± SD)  Difference (intervention vs comparison)  Waiting time for last visit (minute)  37.8 ± 42.6  47.0 ± 60.0  42.5 ± 52.5  −9.2**  Received medication at visit (received = 1, otherwise = 0)  65.1 ± 47.7%  76.1 ± 42.7%  70.8 ± 45.5%  −11.0%***  Reception (% good reception)  91.7 ± 27.6%  87.5 ± 33.1%  89.5 ± 30.6%  4.3%*  Hygiene (% good hygiene)  81.5 ± 38.8%  80.7 ± 39.5%  81.1 ± 39.2%  0.8%  Quality (% good quality)  89.2 ± 31.1%  88.9 ± 31.5%  89.0 ± 31.3%  0.3%  Quality of services  Intervention departments (mean ± SD)  Comparison departments (mean ± SD)  Total (mean ± SD)  Difference (intervention vs comparison)  Waiting time for last visit (minute)  37.8 ± 42.6  47.0 ± 60.0  42.5 ± 52.5  −9.2**  Received medication at visit (received = 1, otherwise = 0)  65.1 ± 47.7%  76.1 ± 42.7%  70.8 ± 45.5%  −11.0%***  Reception (% good reception)  91.7 ± 27.6%  87.5 ± 33.1%  89.5 ± 30.6%  4.3%*  Hygiene (% good hygiene)  81.5 ± 38.8%  80.7 ± 39.5%  81.1 ± 39.2%  0.8%  Quality (% good quality)  89.2 ± 31.1%  88.9 ± 31.5%  89.0 ± 31.3%  0.3%  Notes: SD, standard deviation; min, minutes. * P < 0.05; ** P < 0.01; *** P < 0.001. Using the DIDs approach without village fixed effects, we found that RBF was associated with improved quality of care, and increased use of curative care (Table 6). More specifically, RBF was associated with a 14.5 percentage point increase in receiving medication at the last visit (P < 0.05), representing a 20.5% increase relative to the baseline average of 70.8%. RBF was associated with a 17.6 percentage point increase in seeking care if a child was sick (P < 0.05), indicating a 39.2% relative increase over the baseline average of 45%. Table 6 Impact of RBF on services (from household surveys) Outcome  n  Without village fixed effects   With village fixed effects       Absolute DIDs  Relative %D  Absolute DIDs  Relative %D  Maternal health services             Institutional delivery (%)  2665  –1.8%  –2.1%  0.4%  0.5%   Prenatal care (%)  2652  –4.4%  –4.9%  –3.2%  –3.5%   3+ prenatal care (%)  2565  1.5%  1.8%  1.6%  1.9%   Postnatal care (%)  2452  4.2%  7.7%  11.2%  20.6%   Used family planning methods (%)  2552  3.5%  6.7%  7.2%  13.8%  HIV/AIDS services             Patient offered HIV test (% of pregnant women)  2406  9.9%  21.6%  11.0%  24.0%   Patient received HIV test when offered (%)  2675  5.6%  15.5%  8.5%  23.6%  Quality of services             Woman received medication at last visit (%)  2644  14.5%*  20.5%  13.3%*  18.8%   Reception (% poor reception)  2502  11.7%*  14.4%  8.4%*  10.4%   Hygiene (% poor hygiene)  2619  12.5%*  15.4%  8.9%  10.4%   Quality (% poor quality)  2577  7.4%*  8.3%  7.1%  9.4%  Child care             Had bed net (% children <5)  3681  –7.3%  –9.3%  –8.8%  –11.2%   Sought curative care (% children in last month)  1886  17.6%*  39.2%  25.3%*  56.3%   Received BCG, children aged 0–23 months (%)  2319  0.7%  0.7%  1.2%  1.2%   Received DPT3, children aged 6–23 months (%)  1282  –19.7%*  –38.2%  –13.3%*  –25.8%  Outcome  n  Without village fixed effects   With village fixed effects       Absolute DIDs  Relative %D  Absolute DIDs  Relative %D  Maternal health services             Institutional delivery (%)  2665  –1.8%  –2.1%  0.4%  0.5%   Prenatal care (%)  2652  –4.4%  –4.9%  –3.2%  –3.5%   3+ prenatal care (%)  2565  1.5%  1.8%  1.6%  1.9%   Postnatal care (%)  2452  4.2%  7.7%  11.2%  20.6%   Used family planning methods (%)  2552  3.5%  6.7%  7.2%  13.8%  HIV/AIDS services             Patient offered HIV test (% of pregnant women)  2406  9.9%  21.6%  11.0%  24.0%   Patient received HIV test when offered (%)  2675  5.6%  15.5%  8.5%  23.6%  Quality of services             Woman received medication at last visit (%)  2644  14.5%*  20.5%  13.3%*  18.8%   Reception (% poor reception)  2502  11.7%*  14.4%  8.4%*  10.4%   Hygiene (% poor hygiene)  2619  12.5%*  15.4%  8.9%  10.4%   Quality (% poor quality)  2577  7.4%*  8.3%  7.1%  9.4%  Child care             Had bed net (% children <5)  3681  –7.3%  –9.3%  –8.8%  –11.2%   Sought curative care (% children in last month)  1886  17.6%*  39.2%  25.3%*  56.3%   Received BCG, children aged 0–23 months (%)  2319  0.7%  0.7%  1.2%  1.2%   Received DPT3, children aged 6–23 months (%)  1282  –19.7%*  –38.2%  –13.3%*  –25.8%  Notes: n denotes the number of observations, DIDs is the results from the difference-in-differences analysis. %D shows the relative change to the average of baseline; HIV/AIDS, human immunodeficiency virus/acquired immune deficiency syndrome; BCG, bacille Calmette–Guerin vaccine for tuberculosis; DPT, diphtheria, pertussis and tetanus vaccine; Coef, coefficient; stat., statistically. * P < 0.05 after Bonferroni correction. In addition, RBF was associated with changes in all the quality indicators used in the household survey. It was estimated that 11.7, 12.5 and 7.4 percentage points more pregnant women felt that they received ‘good’ reception at their last visit and the hygiene and quality of services were better, representing relative increases of 14.4, 15.4 and 8.3%, respectively, compared with the baseline averages. Surprisingly, the RBF was associated with a reduction of 19.7 percentage points in the coverage of DPT3 (P < 0.05). Although RBF showed favourable impacts on the use of more than three prenatal care visits, postnatal visits, use of family planning and BCG vaccination, these impacts were not statistically significant. Among 15 indicators, RBF showed favourable directions on 11 indicators and unfavourable on the remaining 4. This preponderance of favourable results is very unlikely to be due to chance (P < 0.001). Overall, the results from the model with village fixed effects were similar to the model without village fixed effects, although the magnitude of the effects was different for some indicators (e.g. postnatal care). The results from the health facility surveys estimated from the random effects model showed a more favourable impact of RBF on some selected indicators. The RBF scheme was associated with relative increases in curative care (83%), patient referrals (472%), vitamin A distribution (155%), assisted delivery (42%) and HIV/AIDS testing among pregnant women (147%). However, RBF did not improve full immunization among children and anti-tetanus vaccination (VAT2+) among pregnant women (Table 7). Generally, the impacts derived from the health facility survey were much larger than those observed in the household survey. For example, the facility survey showed the number of curative visits increased by 83%, while the household survey showed a 38.9% increase for children receiving curative visits. When compared with the baseline, the services of referrals to hospitals, children receiving vitamin A and HIV/AIDS testing among pregnant women more than doubled. The results from the model with facility fixed effects were quite similar to those from the random effects model. Table 7 Impact of RBF on services (from health facility surveys) Health services  n  Baseline monthly visit  Facility random effects   Facility fixed effects   Absolute DIDs  %D  Absolute DIDs  %D  Curative visits  769  76.2  63.30*  83%  63.55*  83%  Patients referred to hospital  757  1.1  4.97*  472%  4.97*  472%  Children receiving full immunization  633  30.2  −5.42  −18%  −5.37  −18%  Children who received vitamin A  648  22.8  35.26*  155%  33.55*  147%  Pregnant women tested HIV/AIDS  768  3.4  5.04*  147%  5.10*  149%  Assisted births  759  6.8  2.81*  42%  2.80*  41%  Pregnant women vaccinated with VAT2+  678  17.8  2.09  12%  2.21  12%  Median  757  17.8  4.97  83%  4.97  83%  Health services  n  Baseline monthly visit  Facility random effects   Facility fixed effects   Absolute DIDs  %D  Absolute DIDs  %D  Curative visits  769  76.2  63.30*  83%  63.55*  83%  Patients referred to hospital  757  1.1  4.97*  472%  4.97*  472%  Children receiving full immunization  633  30.2  −5.42  −18%  −5.37  −18%  Children who received vitamin A  648  22.8  35.26*  155%  33.55*  147%  Pregnant women tested HIV/AIDS  768  3.4  5.04*  147%  5.10*  149%  Assisted births  759  6.8  2.81*  42%  2.80*  41%  Pregnant women vaccinated with VAT2+  678  17.8  2.09  12%  2.21  12%  Median  757  17.8  4.97  83%  4.97  83%  Notes: n denotes facility quarters, with a maximum of 776; DIDs are the results from difference-in-differences analysis using random-effects models; %D shows the relative change to the baseline; HIV/AIDS, human immunodeficiency virus/acquired immune deficiency syndrome; VAT, tetanus vaccination. * P < 0.05 after Bonferroni correction. Site visits and provider interviews showed frequent delays in incentive payments to health facilities and, at some facilities, a lack of transparency in how incentive payments were allocated within the facility. These delays were primarily due to administrative issues, such as funding rupture, delays in verification and inadequate management knowledge and skills. At the health-facility level, some managers did not capture the essence of RBF and did not inform staff about the RBF programme and how it worked in health facilities. Staff in some intervention health facilities did not know about the implementation of the RBF programme, while other health facilities lacked the capacity to calculate bonuses and link them to the performance of each staff member. Discussion In general, this study shows the potential of this pilot RBF programme to improve MCH services. Both the household and health facility surveys revealed that the implementation of RBF was associated with improvement in some, but not all, incentivized MCH services included in the analysis. The household survey suggested that the major impacts of the RBF programme were on the quality of care, curative care and HIV/AIDS testing among pregnant women. However, the household survey did not convey a significant impact of RBF on prenatal care, postnatal care and family planning. The health facility survey, however, displayed more salient impacts favourable for all indicators except for full immunization among children. Partially due to the favourable results from this pilot study, RoC scaled up the RBF programme to other departments, where the government and the World Bank are co-financing the programme. From the household survey, we found that the perceived quality of services improved consistently using multiple quality indicators. This result is consistent with findings from evaluations of RBF in Rwanda (Basinga et al. 2011; Janssen et al. 2015). In the RBF programme in RoC, quality of care was measured primarily on structural indicators, namely the availability of essential infrastructure (e.g. laboratory equipment, qualification of staff, hygiene, availability of medication etc.). The incentive payment formula weighed quality heavily, with about half of a facility’s payment based on quality. This substantial payment for quality of care provided a strong motivation for health facilities to upgrade health infrastructure. Responding to these quality incentives, health facilities tended to use additional resources from RBF incentives to improve those indicators that could be enhanced most quickly, such as availability of drugs and hygiene. In general, facilities distributed a portion of the proceeds of RBF as bonuses to staff at a facility. Typically, all staff received some share of the proceeds, including the least skilled temporary and non-medical personnel, which may strengthen cohesion and engagement within the health facilities. Another indicator showing improvement in the household survey was curative care for children under 5 years of age. This result parallels findings from a study in Haiti (Zeng et al. 2013). The payment per unit of this service in RoC is $0.40 per visit. When compared with other child services, such as full immunization at $6 per child case (see Table 1), the curative care payment is relatively small. However, full immunization may be restricted by (1) availability of vaccines, (2) limited population, as children under age 2 are the key population for the indicator of full immunization and (3) saturation of the immunization coverage. In RoC, certain vaccines, such as BCG, have high coverage already (Centre Nationale de la Statistique et des Études Économiques (CNSEE) & ICF International 2013), leaving limited room for improvement. The payment rate for assisted delivery, $8 per delivery, is in the top 40% of incentivized services. If health providers respond to incentives, it is expected that the coverage of assisted delivery would increase. Surprisingly, we do not observe a statistically significant increase in assisted delivery from the household survey. Several reasons may explain why there is no improvement in this indicator in spite of high incentives. First, RoC has high baseline coverage of assisted delivery (87%), leaving comparatively less room for improvement. Those women who had not used assisted delivery before may have lived in remote areas that remain hard to reach. Second, scaling up assisted delivery may require resources outside the control of an individual health facility (e.g. training and recruiting new personnel), and require more time to pay off than was available in this 2-year study. Third, the 4-month interruption in RBF payments and its expiration in 2014 may have discouraged long-term investments. In addition to assisted delivery, the coverage of any prenatal care visit, bed nets and receiving DPT3 among children aged 6–23 months had a negative association with the RBF programme, which is unexpected. The same reasons for assisted delivery are applicable to prenatal care and bed nets (e.g. high coverage at the baseline and interruption of RBF implementation) for explaining the negative association. Furthermore, bed nets are generally distributed to pregnant women who visit health facilities regularly for prenatal care in RoC (Koukouikila-Koussounda and Ntoumi 2016), which may explain the unexpected consistent impact of RBF on the coverage of prenatal care, assisted delivery and bed nets. However, the impact of RBF on DPT3 could be more complicated given that the supply of vaccine plays an important role in determining the coverage. DTP3 is often used as a proxy of full immunization. We found that the result from the household survey was consistent with that from the health facility survey, showing the negative impact of RBF on immunization. Further investigation reveals a drop in coverage of immunization in both intervention and comparison groups, with a sharper drop in the comparison group. This may suggest a supply issue of vaccines in RoC in the post-RBF period. The results from the health-facility survey are more favourable than are those from the household survey, which is expected as the household survey uses a whole market analysis approach while the health-facility survey focuses on the public sector. After 2 years of implementation of RBF, the health-facility survey showed that curative visits, hospital referral, vitamin A distribution among children, HIV/AIDS testing among pregnant women and assisted delivery improved substantially. The use of curative care and vaccination is consistent with findings from the household survey. However, there are discrepancies between the health facility and household surveys on assisted delivery, where there is a significant improvement recorded from health-facility survey, but no significant increase recorded in the household survey. Three factors may explain the differences: (1) the RBF programme was implemented in public facilities only, and did not cover private facilities. It is likely that the improved quality at public facilities attracted pregnant women who otherwise would have delivered in private health facilities; (2) some pregnant women in both the comparison and intervention groups could not distinguish licenced health facilities from informal health facilities. If this occurred disproportionally more among pregnant women in the comparison group, there was a risk that the coverage of assisted delivery in the comparison group was inflated much more than that in the intervention group, resulting in no differences in assisted delivery between the two groups. As community engagement is a part of the RBF programme design, it is less likely that community workers refer pregnant women to informal health facilities. Our rate of assisted delivery from the household survey was similar to those from DHSs conducted around the same time in the country (Centre Nationale de la Statistique et des Études Économiques (CNSEE) & ICF International 2013); this consistency helped validate the household survey; and (3) it is also possible that health facilities over-reported their service provision. However, all the health facilities’ data were verified by qualified staff, decreasing the chance of over-reporting. Therefore, the discrepancy in the rates of assisted delivery between the health facility and household surveys is most likely due to the switch in preference from private to public facilities. Nevertheless, the result on assisted delivery needs to be interpreted with caution. According to the RBF programme design, supervisors were supposed to visit and monitor health providers in the RBF programme regularly to reinforce and examine their knowledge of RBF and its impact on the process of care. Site visits revealed initial shortcomings. For example, during regular monitoring and evaluation visits, frontline health facility staff expressed limited understanding of RBF, particularly on how it relates to staff behaviour at facility and community levels, and the overall engagement with patients. This knowledge appeared to improve over time. The relationship between RBF at the organizational level and implementation at staff levels could be improved. Nevertheless, in household surveys about 90% of consumers rated the quality of care as good. Several limitations are worth noting. First, this study uses a quasi-experimental research design with DIDs. Although DIDs are a solid approach to estimate the impact of RBF, neither the intervention nor the comparison departments were selected randomly. Thus, there is a potential selection bias that may affect the evaluation results. Second, in this study we used intention-to-treat analysis. In the intervention group, RBF was first implemented among 73 health facilities that met minimal qualifications, leaving out those whose quality was too poor to have the capacity to implement RBF, which also results in a selection bias. Fortunately, the DIDs approach, to some degree, mitigated such biases. Third, the quality measures in the household survey are more related to patient experience, and less about direct and objective indicators of health process and quality outcomes. Current discussions on development of measures for quality of care in LMICs suggest that incentive payments should be based more on process and outcome quality indicators. Fourth, the evaluation was based on only five departments, so some managerial change at the departmental level during the RBF programme could have affected the results. Although we were not aware of any such change, an unobserved change could still have confounded the findings (Kahneman and Tversky 1979). Last, this study could not link households to health facilities, and thus the household survey potentially captured the impact of both the RBF programme implemented at public health facilities, and any initiatives, if there were, in private and non-governmental organization facilities. However, we are not aware of any large scale health initiatives addressing MCH services in the private and non-governmental organization sector during the RBF implementation period. Although the results were favourable for the RBF programme, they were not consistent across all the key MCH services. RBF implementation was not limited to providing payment incentives only. Rather, it was regarded as a vehicle to implement managerial improvements at administrative and health-centre levels. Successful implementation also requires strong monitoring and evaluation, improved transparency, capacity building and increased autonomy. Future versions of RBF could strengthen several elements to yield more favourable outcomes. For instance, in the current study, RBF was implemented without a component of capacity building related to service delivery. Some staff at health facilities were not aware an RBF programme was ongoing, showing that awareness of this programme among providers was not high. Mobilization of providers from the bottom level (community) is considered essential to guarantee the programme’s success (Falisse et al. 2012; Manongi et al. 2014). There is a need for capacity building at all levels to develop a well-coordinated, integrated network of RBF implementation. In addition, prompt payment or incentives and transparency in their allocation within the facility are critical to the effectiveness of the RBF programme. It is also important to consider the demand-side financing alignment with RBF to generate synergy, simultaneously improving the supply side and stimulating demand in the community for life-saving MCH services (Carrin et al. 2005; Witter et al. 2013). Last, the RBF programme was based entirely on positive incentives with no sanctions for substandard performance. ‘Prospect theory’ has found that averting a loss can be a more powerful motivator than seeking a gain (Kahneman and Tversky 1979). Additionally, while positive incentives require additional public funding, sanctions would save the government money, thereby making RBF more financially sustainable. Disclaimer Views expressed in this article are those of the authors and do not necessarily reflect the views of the authors’ institutions and funder of this study. None of the authors have any financial interest in any product discussed in this article. Acknowledgements We thank Alphonse Indouyi-Ibikoue for assistance with data collection and Clare L. Hurley for editorial assistance. Funding This study was funded by the World Bank. Conflict of interest statement. None declared. References Africa Health Forum. 2013. Results-Based Financing for Health . Washington, DC: The World Bank. Ashir GM, Doctor HV, Afenyadu GY. 2013. Performance based financing and uptake of maternal and child health services in Yobe State, northern Nigeria. Global Journal of Health Science  5: 34– 41. Google Scholar CrossRef Search ADS PubMed  Basinga P, Gertler PJ, Binagwaho A. 2011. Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: an impact evaluation. Lancet  377: 1421– 8. Google Scholar CrossRef Search ADS PubMed  Bonfrer I, Soeters R, Van de Poel E et al.   2014. Introduction of performance-based financing in Burundi was associated with improvements in care and quality. Health Affairs (Millwood)  33: 2179– 87. Google Scholar CrossRef Search ADS   Carrin G, Waelkens MP, Criel B. 2005. Community-based health insurance in developing countries: a study of its contribution to the performance of health financing systems. Tropical Medicine and International Health  10: 799– 811. Google Scholar CrossRef Search ADS PubMed  Centre Nationale de la Statistique et des Études Économiques (CNSEE), ICF International. 2013. Enquête Démographique Et De Santé Du Congo (EDSC-II) 2011-2012 . Calverton, MD: CNSEE et ICF International. Darmstadt GL, Bhutta ZA, Cousens S et al.   2005. Evidence-based, cost-effective interventions: how many newborn babies can we save? Lancet  365: 977– 88. Google Scholar CrossRef Search ADS PubMed  Falisse JB, Meessen B, Ndayishimiye J, Bossuyt M. 2012. Community participation and voice mechanisms under performance-based financing schemes in Burundi. Tropical Medicine and International Health  17: 674– 82. Google Scholar CrossRef Search ADS PubMed  Fretheim A, Witter S, Lindahl AK, Olsen IT. 2012. Performance-based financing in low- and middle-income countries: still more questions than answers. Bulletin of the World Health Organization  90: 559– 559A. Google Scholar CrossRef Search ADS PubMed  Fritsche G,B, Soeters R, Meessen B. 2014. Performance-Based Fianncing Toolkit . Washington, DC: The World Bank. Google Scholar CrossRef Search ADS   Frost JJ, Sonfield A, Zolna MR, Finer LB. 2014. Return on investment: a fuller assessment of the benefits and cost savings of the US publicly funded family planning program. Milbank Quraterly  92: 696– 749. Google Scholar CrossRef Search ADS   Hongoro C, Normand C, 2006. Health workers: Building and motivating the workforce. In: Jamisan D, Breman J, Measham A et al.   (eds). Disease Control Priorities in Developing Countries , 2nd edn. Washington, DC: World Bank. Janssen W, Ngirabega J. d D, Matungwa M, Van Bastelaere S. 2015. Improving quality through performance-based financing in district hospitals in Rwanda between 2006 and 2010: a 5-year experience. Tropical Doctor  45: 27– 35. Google Scholar CrossRef Search ADS PubMed  Kahneman D, Tversky A. 1979. Prospect theory: An analysis of decision under risk. Econometrica  47: 263– 91. Google Scholar CrossRef Search ADS   Koukouikila-Koussounda F, Ntoumi F. 2016. Malaria epidemiological research in the Republic of Congo. Malaria Journal  15: 598. Google Scholar CrossRef Search ADS PubMed  Manongi R, Mushi D, Kessy J, Salome S, Njau B. 2014. Does training on performance based financing make a difference in performance and quality of health care delivery? Health care provider’s perspective in Rungwe Tanzania. BMC Health Services Research  14: 154. Google Scholar CrossRef Search ADS PubMed  Ministry of Health and Population of the Republic of the Congo, World Bank, Cordaid, Memisa. 2014. Final activity report of the second phase of the project on performance based financing. Brazzaville, Republic of the Congo: Ministry of Health and Population of the Republic of the Congo, World Bank, Cordaid, and Memisa. RBFHealth. 2015. Performance-based financing conceptual framework. https://www.rbfhealth.org/resource/performance-based-financing-conceptual-framework, accessed 11 May 2017. Stenberg K, Axelson H, Sheehan P et al.   2014. Advancing social and economic development by investing in women’s and children’s health: a new Global Investment Framework. Lancet  383: 1333– 54. Google Scholar CrossRef Search ADS PubMed  The World Bank. 2015. GDP per Capita . Washington, DC: The World Bank. http://data.worldbank.org/indicator/NY.GDP.PCAP.CD, accessed 30 September 2015. Turcotte-Tremblay AM, Spagnolo J, De Allegri M, Ridde V. 2016. Does performance-based financing increase value for money in low- and middle- income countries? A systematic review. Health Economics Review  6: 30. Google Scholar CrossRef Search ADS PubMed  UNICEF. 2008. UNICEF humanitarian action: Republic of the Congo (Brazzaville) in 2008. https://www.unicef.org/har08/files/har08_Congo_countrychapter.pdf, accessed 30 September 2015. Van de Poel E, Flores G, Ir P, O’Donnell O. 2016. Impact of performance-based Financing in a low-resource setting: a decade of experience in Cambodia. Health Economics  25: 688– 705. Google Scholar CrossRef Search ADS PubMed  Witter S, Toonen J, Meessen B et al.   2013. Performance-based financing as a health system reform: mapping the key dimensions for monitoring and evaluation. BMC Health Services Research  13: 367. Google Scholar CrossRef Search ADS PubMed  World Bank. 2013. Health System Strennthening Project II . Washington, DC: World Bank. World Health Organization. 2010. Health Systems Financing: The Path to Universal Coverage . Geneva, Switzland: World Health Organization. PubMed PubMed  Zeng W, Cros M, Wright KD, Shepard DS. 2013. Impact of performance-based financing on primary health care services in Haiti. Health Policy and Planning  28: 596– 605. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press in association with The London School of Hygiene and Tropical Medicine. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com

Journal

Health Policy and PlanningOxford University Press

Published: Apr 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 12 million articles from more than
10,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Unlimited reading

Read as many articles as you need. Full articles with original layout, charts and figures. Read online, from anywhere.

Stay up to date

Keep up with your field with Personalized Recommendations and Follow Journals to get automatic updates.

Organize your research

It’s easy to organize your research with our built-in tools.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

Monthly Plan

  • Read unlimited articles
  • Personalized recommendations
  • No expiration
  • Print 20 pages per month
  • 20% off on PDF purchases
  • Organize your research
  • Get updates on your journals and topic searches

$49/month

Start Free Trial

14-day Free Trial

Best Deal — 39% off

Annual Plan

  • All the features of the Professional Plan, but for 39% off!
  • Billed annually
  • No expiration
  • For the normal price of 10 articles elsewhere, you get one full year of unlimited access to articles.

$588

$360/year

billed annually
Start Free Trial

14-day Free Trial