TY - JOUR
AU - Fink,, Günther
AB - Abstract This analysis describes specific gaps in the quality of health care in Central Africa and assesses the association between quality of clinical care and mortality at age 2–59 months. Regionally representative facility and household surveys for the Democratic Republic of the Congo, Cameroon and Central African Republic were collected between 2012 and 2016. These data are novel in linking facilities with households in their catchment area. Compliance with diagnostic and danger sign protocols during sick-child visits was observed by trained assessors. We computed facility- and district-level compliance indicators for patients aged 2–59 months and used multivariate multi-level logistic regression models to estimate the association between clinical assessment quality and mortality at age 2–59 months in the catchment areas of the observed facilities. A total of 13 618 live births were analysed and 1818 sick-child visits were directly observed and used to rate 643 facilities. Eight percent of observed visits complied with 80% of basic diagnostic protocols, and 13% of visits fully adhered to select general danger sign protocols. A 10% greater compliance with diagnostic protocols was associated with a 14.1% (adjusted odds ratio (aOR) 95% CI: 0.025–0.244) reduction in the odds of mortality at age 2–59 months; a 10% greater compliance with select general danger sign protocols was associated with a 15.3% (aOR 95% CI: 0.058–0.237) reduction in the same odds. The results of this article suggest that compliance with recommended clinical protocols remains poor in many settings and improvements in mortality at age 2–59 months could be possible if compliance were improved. Child health, quality of care Key Messages The results of this study show remarkably poor compliance with these diagnostic protocols for the diagnosis of children under age 5 with acute health problems. Using a novel data set in three Central African countries, we find that failure to comply with basic diagnostic protocols was associated with substantial excess child mortality. The findings highlight the need for major improvements in the quality of diagnosis and care in low-income settings. Introduction Despite annual mortality reductions of 4.1% over the past 15 years, sub-Saharan Africa remains the region with the highest under-5 mortality rate in the world (UNICEF, 2015). Sustainable Development Goal 3 aims to end preventable deaths of newborns and children under age 5 by 2030, with all countries aiming to reduce under-5 mortality to at most 25 per 1000 live births (United Nations, 2015). Major efforts will be necessary to reach this target in West and Central Africa, where an estimated 98.7 under-5 deaths occurred per 1000 live births in 2015 (You et al., 2015). Both verbal and social autopsies suggest that well over 80% of these deaths occur after children are seen by a modern medical provider (Sodemann et al., 1997; de Savigny et al., 2004; Källander et al., 2008; Rutherford et al., 2009; 2010; Willcox et al., 2018). Although treatment is undoubtedly sought too late in some instances, delays in treatment seeking are unlikely to fully explain the continued high burden of child mortality. While several recent studies highlight the substantial deficiencies in various domains of quality of care, there is very little evidence directly linking the quality of healthcare services to child health outcomes (Rutherford et al., 2010; Fink et al., 2015; Leslie et al., 2016a,b; Gage et al., 2017; Sharma et al., 2017). An important reason why estimates of the empirical relationship between quality of care and health outcomes are rare is the lack of population-level health data that can be directly linked to measures of the quality of clinical care received. In this article, we explore a unique three-country data set with random samples of households in the catchment areas of each facility to assess both the quality of clinical assessments for children under age 5 and the degree to which the quality of clinical care predicts mortality at age 2–59 months in each district’s catchment area. Materials and methods Study population and sampling The data for this study were produced as part of ongoing Results-Based Financing projects supported by the World Bank and the Health Results Innovation Trust Fund. Detailed information on a set of facilities, as well as a random subset of households in the catchment areas of each facility, was collected. In this study, we analysed the baseline surveys from Cameroon, Central African Republic (CAR) and Democratic Republic of Congo (DRC). Survey instruments are available online (World Bank Group, 2014a,b,c; 2015; 2017a,b). Samples were representative at the subnational level: select regions in Cameroon, select prefectures in CAR and select zones areas in DRC. Sampling of facilities and households was done in a three-step sampling procedure. The World Bank selected subnational areas for the Results-Based Financing projects in discussion with the national Ministry of Health. First, facilities were randomly selected in target areas. Second, up to two enumeration areas (EAs) were randomly selected from each catchment area of selected facilities. Third, a random subset of households in the selected EAs was chosen for the household interview. Facility assessments included facility audits, direct observations of clinical consultations, staff interviews and client exit interviews. Household surveys included in-person questionnaires centered around maternal and child health. All household surveys were restricted to households with at least one woman who had been pregnant or delivered a child within 2 years of the survey. Survey-specific sampling designs and the timing of data collection are detailed in Supplementary Table S1. Data access was granted by the World Bank Microdata Library, and data analysis was supported by the respective country research teams. Construction of the pooled samples is detailed in Supplementary Figures S1 and S2 and summarized in Table 1. Table 1 Analytic samples Survey instruments . Analytic sample . N . Direct observation of clinical consultations + client exit interviews Descriptive quality analysis 1818 sick-child visits Direct observation of clinical consultations + household survey Descriptive mortality analysis and regression analysis of births within 3 years in the district catchment area 13 618 live births in 68 district catchment areas Descriptive mortality analysis and regression analysis of births within 5 years in the district catchment area 19 671 live births in 68 district catchment areas Descriptive mortality analysis and regression analysis of births within 3 years in the facility catchment area 6861 live births in 459 facility catchment areas Descriptive mortality analysis and regression analysis of births within 5 years in the facility catchment area 10 032 live births in 459 facility catchment areas Survey instruments . Analytic sample . N . Direct observation of clinical consultations + client exit interviews Descriptive quality analysis 1818 sick-child visits Direct observation of clinical consultations + household survey Descriptive mortality analysis and regression analysis of births within 3 years in the district catchment area 13 618 live births in 68 district catchment areas Descriptive mortality analysis and regression analysis of births within 5 years in the district catchment area 19 671 live births in 68 district catchment areas Descriptive mortality analysis and regression analysis of births within 3 years in the facility catchment area 6861 live births in 459 facility catchment areas Descriptive mortality analysis and regression analysis of births within 5 years in the facility catchment area 10 032 live births in 459 facility catchment areas Open in new tab Table 1 Analytic samples Survey instruments . Analytic sample . N . Direct observation of clinical consultations + client exit interviews Descriptive quality analysis 1818 sick-child visits Direct observation of clinical consultations + household survey Descriptive mortality analysis and regression analysis of births within 3 years in the district catchment area 13 618 live births in 68 district catchment areas Descriptive mortality analysis and regression analysis of births within 5 years in the district catchment area 19 671 live births in 68 district catchment areas Descriptive mortality analysis and regression analysis of births within 3 years in the facility catchment area 6861 live births in 459 facility catchment areas Descriptive mortality analysis and regression analysis of births within 5 years in the facility catchment area 10 032 live births in 459 facility catchment areas Survey instruments . Analytic sample . N . Direct observation of clinical consultations + client exit interviews Descriptive quality analysis 1818 sick-child visits Direct observation of clinical consultations + household survey Descriptive mortality analysis and regression analysis of births within 3 years in the district catchment area 13 618 live births in 68 district catchment areas Descriptive mortality analysis and regression analysis of births within 5 years in the district catchment area 19 671 live births in 68 district catchment areas Descriptive mortality analysis and regression analysis of births within 3 years in the facility catchment area 6861 live births in 459 facility catchment areas Descriptive mortality analysis and regression analysis of births within 5 years in the facility catchment area 10 032 live births in 459 facility catchment areas Open in new tab Quality of care The primary exposure of interest was the quality of clinical assessment in sick-child visits at facilities. As part of the facility survey, trained surveyors followed sick children of consenting caregivers throughout their facility visits and tracked healthcare providers’ diagnostic procedures using a checklist. The facility survey only considered children presenting with a new condition (i.e. not follow-up or routine visit). At the end of the visit, caregivers were invited to complete an exit interview, which solicited information about perceived quality of care, cost, and prescriptions. Based on Integrated Management of Childhood Illness (IMCI) guidelines, we identified 15 procedures that are essential to diagnosis in all sick-child visits for patients aged 2–59 months and within control of the clinician (Box 1). A full description of the selected procedures, underlying survey questions and details of the coding of survey items is provided in Table 2. To allow for minor protocol deviations and measurement error, we coded a visit as compliant with diagnostic protocols if at least 80% of required procedures were completed by the provider. While we would be interested in a sensitivity analysis of higher thresholds that allow for less deviations or measurement error, we are limited by increasingly small data variation at higher threshold levels. Missing information on any procedure was imputed as the mean of other procedures observed in the consultation. We also conducted a sensitivity analysis where missing items were assigned a minimum (0) or maximum (1) value. Table 2 Clinical assessment protocols . Protocol . Original survey question . History taking Inability to drink anything Has the health worker asked if the child can drink or take breast milk? Cough or difficult breathing Has the health worker asked if the child has cough or difficulty breathing? Diarrhoea Has the health worker asked when the patient has diarrhea? Fever Has the health worker asked if the child had a fever for the last 24 hr? Vomiting Has the health worker asked if the child vomits anything he takes? Convulsions Did the health worker ask if the child had convulsions? Ear problems Has the health worker asked if the child has pain or discharge in the ear? Routine examination Weight Has a health worker weighed the child? Temperature Has a health worker taken the child temperature? Or has the health worker checked the temperature if it had not been done by an agent before? Pallor Alternative wording: (1) Has the health worker checked the child's eyes or the palms of the hands, or the soles of the feet (anaemia)? (2) Does the health worker check the palms of the child’s hands, or compare these against the mother’s? (anaemia) Oedema of feet Has the health worker looked at both feet or both ankles (oedema)? Skin turgor Has the health worker checked skin pinch/folds? Ear examination Did the health worker look in his ears? Count respirations Has the health worker checked the respiratory rate? Undressed child for examination Alternative wording: (1) Has the health worker undressed the child? (2) Does the health worker lift shirt? . Protocol . Original survey question . History taking Inability to drink anything Has the health worker asked if the child can drink or take breast milk? Cough or difficult breathing Has the health worker asked if the child has cough or difficulty breathing? Diarrhoea Has the health worker asked when the patient has diarrhea? Fever Has the health worker asked if the child had a fever for the last 24 hr? Vomiting Has the health worker asked if the child vomits anything he takes? Convulsions Did the health worker ask if the child had convulsions? Ear problems Has the health worker asked if the child has pain or discharge in the ear? Routine examination Weight Has a health worker weighed the child? Temperature Has a health worker taken the child temperature? Or has the health worker checked the temperature if it had not been done by an agent before? Pallor Alternative wording: (1) Has the health worker checked the child's eyes or the palms of the hands, or the soles of the feet (anaemia)? (2) Does the health worker check the palms of the child’s hands, or compare these against the mother’s? (anaemia) Oedema of feet Has the health worker looked at both feet or both ankles (oedema)? Skin turgor Has the health worker checked skin pinch/folds? Ear examination Did the health worker look in his ears? Count respirations Has the health worker checked the respiratory rate? Undressed child for examination Alternative wording: (1) Has the health worker undressed the child? (2) Does the health worker lift shirt? ‘Temperature examination’ is considered complete if it is conducted by either (1) a health worker prior to consultation or (2) the healthcare worker during the consultation. Open in new tab Table 2 Clinical assessment protocols . Protocol . Original survey question . History taking Inability to drink anything Has the health worker asked if the child can drink or take breast milk? Cough or difficult breathing Has the health worker asked if the child has cough or difficulty breathing? Diarrhoea Has the health worker asked when the patient has diarrhea? Fever Has the health worker asked if the child had a fever for the last 24 hr? Vomiting Has the health worker asked if the child vomits anything he takes? Convulsions Did the health worker ask if the child had convulsions? Ear problems Has the health worker asked if the child has pain or discharge in the ear? Routine examination Weight Has a health worker weighed the child? Temperature Has a health worker taken the child temperature? Or has the health worker checked the temperature if it had not been done by an agent before? Pallor Alternative wording: (1) Has the health worker checked the child's eyes or the palms of the hands, or the soles of the feet (anaemia)? (2) Does the health worker check the palms of the child’s hands, or compare these against the mother’s? (anaemia) Oedema of feet Has the health worker looked at both feet or both ankles (oedema)? Skin turgor Has the health worker checked skin pinch/folds? Ear examination Did the health worker look in his ears? Count respirations Has the health worker checked the respiratory rate? Undressed child for examination Alternative wording: (1) Has the health worker undressed the child? (2) Does the health worker lift shirt? . Protocol . Original survey question . History taking Inability to drink anything Has the health worker asked if the child can drink or take breast milk? Cough or difficult breathing Has the health worker asked if the child has cough or difficulty breathing? Diarrhoea Has the health worker asked when the patient has diarrhea? Fever Has the health worker asked if the child had a fever for the last 24 hr? Vomiting Has the health worker asked if the child vomits anything he takes? Convulsions Did the health worker ask if the child had convulsions? Ear problems Has the health worker asked if the child has pain or discharge in the ear? Routine examination Weight Has a health worker weighed the child? Temperature Has a health worker taken the child temperature? Or has the health worker checked the temperature if it had not been done by an agent before? Pallor Alternative wording: (1) Has the health worker checked the child's eyes or the palms of the hands, or the soles of the feet (anaemia)? (2) Does the health worker check the palms of the child’s hands, or compare these against the mother’s? (anaemia) Oedema of feet Has the health worker looked at both feet or both ankles (oedema)? Skin turgor Has the health worker checked skin pinch/folds? Ear examination Did the health worker look in his ears? Count respirations Has the health worker checked the respiratory rate? Undressed child for examination Alternative wording: (1) Has the health worker undressed the child? (2) Does the health worker lift shirt? ‘Temperature examination’ is considered complete if it is conducted by either (1) a health worker prior to consultation or (2) the healthcare worker during the consultation. Open in new tab In addition to our broad diagnostic compliance measure, we constructed a narrower measure focusing on healthcare providers’ efforts to collect information on general danger signs. IMCI guidelines require clinicians to solicit a patient history of three general danger signs (continued vomiting, convulsions and inability to drink) and observe the child’s energy level and convulsion status in all sick-child visits (World Health Organization, 2014). We focused the narrower measure of compliance on verbal assessment of the three general danger signs given the difficulty in measuring providers’ assessment of children’s energy level and convulsions status through direct observation. Henceforth, we refer to continued vomiting, convulsions and inability to drink as select general danger signs to distinguish from the set of general danger signs that includes current convulsion status and energy level. Visits were coded as compliant with select general danger sign protocols if information on continued vomiting, convulsions and inability to drink was collected by the healthcare provider. Verifying these signs of severe disease is essential to identify potentially fatal prognoses for immediate referral; WHO uses such compliance as the top priority indicator for the evaluation of IMCI (World Health Organization, 2003). However, evidence from sub-Saharan Africa after implementation of the IMCI strategy suggests that danger signs are regularly missed in fatal cases, and healthcare providers can better identify mild than severe disease (Horwood et al., 2009; Willcox et al., 2018). Both compliance measures were computed for each sick-child visit and then collapsed at the district or facility level for analytical purposes as described in detail below. Districts refer to sub-prefectures in CAR, health districts in DRC and arrondissements in Cameroon. Mortality The primary health outcome of interest was under-5 mortality. Given the restriction of our quality measure to patients aged 2–59 months, we restricted mortality to the same age range. Household surveys asked women with a birth or pregnancy in the last 2 years to report 10-year or full birth histories. To reduce temporal differences between the quality measurement and survival outcomes, we restricted our main analysis to children born in the 3 years prior to the survey. In cases where the month or date of birth was missing, we assumed children were born in the middle of the month and year, respectively, per international guidelines (O’Donnell et al., 2007). In a sensitivity analysis, we assume that children with a missing month or date of birth were born at the beginning or end of the month and year. Children with missing year of birth were omitted from analysis. Covariates To describe the quality of observed sick-child visits, the following facility survey covariates were included in our analysis: survey country, provider cadre, facility level, household asset quintile and reported symptom. Country-specific asset scores were developed from caretaker-reported information using principal components analysis and quintile classification. Facility and healthcare provider types were standardized for comparison across countries. Patient symptom(s) were recorded in both direct observation of clinical consultations and caretaker exit interviews. Importantly, these symptoms have no bearing on the general IMCI protocols considered in this study, which apply to all sick-child visits. In the mortality analysis, we include known predictors of child mortality to improve the precision of our estimates, as well as catchment-level predictors of child mortality to control for potential sources of confounding. Household-level survey covariates include the number of mother’s previous live births, maternal education group (none, primary and secondary) and maternal age (15–19, 20–34 and 35–44 years), while catchment-level covariates include the average asset quintile, portion of households using improved sanitation and the portion of households using improved water source. Covariate selection was restricted by measurements available in the household surveys. Furthermore, we included fixed effects for country-province and year of birth to control for measurement differences across surveys, time trends and province-level characteristics associated with both quality and morality. Statistical analysis The main objective of the project was to use the linked databases to estimate empirical association between quality of care observed at facilities and mortality at age 2–59 months reported in catchment areas served by the same facilities. First, we computed diagnostic and select general danger sign binary compliance indicators for all sick-child visits and presented average compliance results by facility, provider and patient characteristics. We also stratified the probability of death among live births within 3 years of the survey by patient characteristics. To analyse the association between quality of clinical assessment and mortality at age 2–59 months, we estimated an intention-to-treat model in which we regressed individual mortality at age 2–59 months on the unweighted average quality of clinical assessments observed in the catchment area. We estimated a multi-level logistic model with catchment area random effects to account for correlation of the residuals for live-births therein. A more detailed description of the statistical model is provided in the Supplementary Material (Section 1). Given that caregivers do not necessarily seek care at the facility to which they are administratively assigned, we aggregated quality of care at the district level in our primary analysis. Health districts are usually large and, thus, likely to contain all facilities that caregivers will typically access. In our sensitivity analysis, we also estimated the association between quality and health outcomes at the facility level; we expected these associations to be weaker due to caregivers use of facilities outside the area to which they are administratively assigned (i.e. bypassing). Given that quality of care likely changes over time due to provider attrition or behavior change, we restrict mortality to 3 years prior to the survey in our main analysis. While even shorter time windows would be desirable, sample sizes become too small to allow meaningful statistical analysis. In our sensitivity analysis, we present models using longer time windows. In addition, we estimated adjusted models in which we regressed mortality at age 2–59 months on the average compliance with each individual protocol in our sensitivity analyses. All statistical analyses were conducted in Stata/MP 15 (StataCorp, 2017). We cannot rule out the possibility of residual confounding, so we present these results as associational evidence. There is potential for residual confounding if the measured covariates are imprecise or catchment-level confounders are unmeasured altogether. For example, it is possible that poor-quality diagnostic health services collocate with the coverage of preventive health interventions (e.g. vaccination) within provinces. Given the paucity of any preexisting evidence in this area, this associational evidence represents a novel contribution to the study of quality of care and child health outcomes. Results In 516 facilities, 1818 sick-child visits with 643 healthcare providers were directly observed. Table 3 presents patient, provider and facility characteristics of the sick-child visits by survey country. In the pooled sample, over half of the patients were under age 2 at the time of the clinic visit and fever was reported more than twice as often as any other symptom (80%). In the CAR survey, the literacy rate among caretakers (27%) was less than half that in the other two surveys (65% in DRC and 74% in Cameroon). While the dominant site of clinical consultation was primary care facilities across survey countries, the most common healthcare provider varied widely: nurses in DRC (68.6%), health assistant or community health worker in CAR (74.1%) and other health professionals in Cameroon (36.2%). Table 3 Characteristics of sick-child visits observed in the facility survey . DRC . CAR . Cameroon . All . . n . % . n . % . n . % . n . % . Observations 1018 590 210 1818 Providers 325 223 95 643 Facilities 280 211 25 516 Age of patient  2–11 months 222 22 178 30 74 35 474 26  12–23 months 235 23 178 30 55 26 468 26  24–35 months 206 20 108 18 28 13 342 19  36–47 months 163 16 72 12 22 10 257 14  48–59 months 192 19 54 9 31 15 277 15 Sex of patient  Male 568 56 322 55 107 51 997 55  Female 450 44 268 45 103 49 821 45 Literacy of caretaker  Literate 650 64 161 27 156 74 967 53  Illiterate 368 36 429 73 54 26 851 47 Provider cadre  Physician 196 19 6 1 44 21 246 14  Nurse 692 68 120 20 56 27 868 48  Health assistant, community health worker N/A N/A 437 74 34 16 471 26  Others 130 13 27 5 76 36 233 13 Facility level  Secondary 248 24 63 11 40 19 351 19  Primary 770 76 527 89 170 81 1467 81 Reported symptom  Fever 788 87 295 78 104 55 1187 80  Cough, difficulty breathing 326 37 123 27 80 42 529 35  Vomiting 202 22 149 33 N/A N/A 351 26  Diarrhea 169 18 183 39 34 17 386 24  Fatigue 188 23 29 6 N/A N/A 217 17  Others 67 7 38 8 28 14 133 8 . DRC . CAR . Cameroon . All . . n . % . n . % . n . % . n . % . Observations 1018 590 210 1818 Providers 325 223 95 643 Facilities 280 211 25 516 Age of patient  2–11 months 222 22 178 30 74 35 474 26  12–23 months 235 23 178 30 55 26 468 26  24–35 months 206 20 108 18 28 13 342 19  36–47 months 163 16 72 12 22 10 257 14  48–59 months 192 19 54 9 31 15 277 15 Sex of patient  Male 568 56 322 55 107 51 997 55  Female 450 44 268 45 103 49 821 45 Literacy of caretaker  Literate 650 64 161 27 156 74 967 53  Illiterate 368 36 429 73 54 26 851 47 Provider cadre  Physician 196 19 6 1 44 21 246 14  Nurse 692 68 120 20 56 27 868 48  Health assistant, community health worker N/A N/A 437 74 34 16 471 26  Others 130 13 27 5 76 36 233 13 Facility level  Secondary 248 24 63 11 40 19 351 19  Primary 770 76 527 89 170 81 1467 81 Reported symptom  Fever 788 87 295 78 104 55 1187 80  Cough, difficulty breathing 326 37 123 27 80 42 529 35  Vomiting 202 22 149 33 N/A N/A 351 26  Diarrhea 169 18 183 39 34 17 386 24  Fatigue 188 23 29 6 N/A N/A 217 17  Others 67 7 38 8 28 14 133 8 Sample represents those children age 2–59 months seeking curative care services (illness or injury) whose caretakers consented to both direct observation of the clinical consultation and exit interview. Reported symptoms reflect those reported in both the observed clinical consultation and exit interview. Each observation corresponds to a visit in one of the sampled facilities. All means represent unweighted sample averages. Not applicable is abbreviated N/A. Open in new tab Table 3 Characteristics of sick-child visits observed in the facility survey . DRC . CAR . Cameroon . All . . n . % . n . % . n . % . n . % . Observations 1018 590 210 1818 Providers 325 223 95 643 Facilities 280 211 25 516 Age of patient  2–11 months 222 22 178 30 74 35 474 26  12–23 months 235 23 178 30 55 26 468 26  24–35 months 206 20 108 18 28 13 342 19  36–47 months 163 16 72 12 22 10 257 14  48–59 months 192 19 54 9 31 15 277 15 Sex of patient  Male 568 56 322 55 107 51 997 55  Female 450 44 268 45 103 49 821 45 Literacy of caretaker  Literate 650 64 161 27 156 74 967 53  Illiterate 368 36 429 73 54 26 851 47 Provider cadre  Physician 196 19 6 1 44 21 246 14  Nurse 692 68 120 20 56 27 868 48  Health assistant, community health worker N/A N/A 437 74 34 16 471 26  Others 130 13 27 5 76 36 233 13 Facility level  Secondary 248 24 63 11 40 19 351 19  Primary 770 76 527 89 170 81 1467 81 Reported symptom  Fever 788 87 295 78 104 55 1187 80  Cough, difficulty breathing 326 37 123 27 80 42 529 35  Vomiting 202 22 149 33 N/A N/A 351 26  Diarrhea 169 18 183 39 34 17 386 24  Fatigue 188 23 29 6 N/A N/A 217 17  Others 67 7 38 8 28 14 133 8 . DRC . CAR . Cameroon . All . . n . % . n . % . n . % . n . % . Observations 1018 590 210 1818 Providers 325 223 95 643 Facilities 280 211 25 516 Age of patient  2–11 months 222 22 178 30 74 35 474 26  12–23 months 235 23 178 30 55 26 468 26  24–35 months 206 20 108 18 28 13 342 19  36–47 months 163 16 72 12 22 10 257 14  48–59 months 192 19 54 9 31 15 277 15 Sex of patient  Male 568 56 322 55 107 51 997 55  Female 450 44 268 45 103 49 821 45 Literacy of caretaker  Literate 650 64 161 27 156 74 967 53  Illiterate 368 36 429 73 54 26 851 47 Provider cadre  Physician 196 19 6 1 44 21 246 14  Nurse 692 68 120 20 56 27 868 48  Health assistant, community health worker N/A N/A 437 74 34 16 471 26  Others 130 13 27 5 76 36 233 13 Facility level  Secondary 248 24 63 11 40 19 351 19  Primary 770 76 527 89 170 81 1467 81 Reported symptom  Fever 788 87 295 78 104 55 1187 80  Cough, difficulty breathing 326 37 123 27 80 42 529 35  Vomiting 202 22 149 33 N/A N/A 351 26  Diarrhea 169 18 183 39 34 17 386 24  Fatigue 188 23 29 6 N/A N/A 217 17  Others 67 7 38 8 28 14 133 8 Sample represents those children age 2–59 months seeking curative care services (illness or injury) whose caretakers consented to both direct observation of the clinical consultation and exit interview. Reported symptoms reflect those reported in both the observed clinical consultation and exit interview. Each observation corresponds to a visit in one of the sampled facilities. All means represent unweighted sample averages. Not applicable is abbreviated N/A. Open in new tab Among clinical assessments, 7.6% (SD: 26.5%) completed at least 80% of diagnostic protocols and only 14 (0.77%) followed 100% of diagnostic protocols. The proportion of observed sick-child visits in which the provider inquired about all three select general danger signs ranged from 4.3% (SD: 20.3%) in Cameroon to 16.0% (SD: 36.7%) in DRC. Information on compliance with individual procedures using mean and alternative imputation approaches for missing data is provided in Supplementary Figure S3. While this is not an evaluation, it is worth nothing that 37% (N = 679) of providers across all survey countries reported having been trained in IMCI. Figure 1 shows average compliance with diagnostic and select general danger sign protocols by patient, provider and facility characteristics. On average, doctors performed better than lower-level healthcare workers, secondary facilities performed better than primary facilities and wealthier patients received higher-quality assessments than poorer patients. Figure 1 Open in new tabDownload slide Average compliance by patient, provider and facility characteristics in survey year (N = 1818). Sample represents those children age 2–59 months seeking curative care services (illness or injury) whose caretakers consented to both direct observation of the clinical consultation and exit interview. Error bars represent 95% confidence intervals. Each observation corresponds to a visit in one of the sampled facilities. No sample weights were applied Figure 1 Open in new tabDownload slide Average compliance by patient, provider and facility characteristics in survey year (N = 1818). Sample represents those children age 2–59 months seeking curative care services (illness or injury) whose caretakers consented to both direct observation of the clinical consultation and exit interview. Error bars represent 95% confidence intervals. Each observation corresponds to a visit in one of the sampled facilities. No sample weights were applied For our mortality analysis, 13 618 children born within 3 years of the household survey were analysed. Table 4 shows that most children were born to mothers who either had attended primary schooling (41%) or had no education (33%); only 26% of children had a mother who had completed secondary schooling or higher. Nearly a third of children were born to mothers who had given birth to five or more children by the time of the survey. In the pooled sample, 3.0% of live births within 3 years of the household survey had died by the time of the survey. The probability of death at age 2–59 months was highest in CAR (4.3%; SD: 20.3%), followed by Cameroon (2.0%; SD 14.0%) and DRC (2.2%; SD: 14.7%). Table 4 Characteristics of live births within three years of the household survey (N = 13 618) . Live births . Death at age 2–59 months . Probability of death at age 2–59 months (%) . n . % . n . % . Live births 13 618 Deaths 415 Catchment districts 68 68 Province 16 16 Birth timing  2–12 months 6022 44 54 13 0.9  13–24 months 4432 33 148 36 3.3  25–36 months 3164 23 213 51 6.7 Maternal education  None 4530 33 190 46 4.2  Primary 5604 41 163 39 2.9  Secondary and above 3484 26 62 15 1.8 Maternal age  15–19 1933 14 56 13 2.9  20–34 9801 72 303 73 3.1  35–49 1884 14 56 13 3.0 Live births (mother)  0 112 1 3 1 2.7  1 2595 19 51 12 2.0  2 2842 21 77 19 2.7  3 2293 17 81 20 3.5  4 1874 14 62 15 3.3  5+ 3902 29 141 34 3.6 Portion of district households using improved sanitation  0–24% 5576 41 132 32 2.4  25–59% 2693 20 73 18 2.7  50–74% 2546 19 94 23 3.7  75–100% 2803 21 116 28 4.1 Portion of district households using improved water source  0–24% 2725 20 69 17 2.5  25–59% 3561 26 88 21 2.5  50–74% 3681 27 126 30 3.4  75–100% 3651 27 132 32 3.6 Country  Cameroon 3757 28 75 18 2.0  CAR 5861 43 252 61 4.3  DRC 4000 29 88 21 2.2 . Live births . Death at age 2–59 months . Probability of death at age 2–59 months (%) . n . % . n . % . Live births 13 618 Deaths 415 Catchment districts 68 68 Province 16 16 Birth timing  2–12 months 6022 44 54 13 0.9  13–24 months 4432 33 148 36 3.3  25–36 months 3164 23 213 51 6.7 Maternal education  None 4530 33 190 46 4.2  Primary 5604 41 163 39 2.9  Secondary and above 3484 26 62 15 1.8 Maternal age  15–19 1933 14 56 13 2.9  20–34 9801 72 303 73 3.1  35–49 1884 14 56 13 3.0 Live births (mother)  0 112 1 3 1 2.7  1 2595 19 51 12 2.0  2 2842 21 77 19 2.7  3 2293 17 81 20 3.5  4 1874 14 62 15 3.3  5+ 3902 29 141 34 3.6 Portion of district households using improved sanitation  0–24% 5576 41 132 32 2.4  25–59% 2693 20 73 18 2.7  50–74% 2546 19 94 23 3.7  75–100% 2803 21 116 28 4.1 Portion of district households using improved water source  0–24% 2725 20 69 17 2.5  25–59% 3561 26 88 21 2.5  50–74% 3681 27 126 30 3.4  75–100% 3651 27 132 32 3.6 Country  Cameroon 3757 28 75 18 2.0  CAR 5861 43 252 61 4.3  DRC 4000 29 88 21 2.2 Sample represents those live births in catchment areas supported by direct observation of clinical consultation for children age 2–59 months seeking curative care services (illness or injury), regardless of whether the caretaker also consented to exit interview. All means represent unweighted sample averages. Open in new tab Table 4 Characteristics of live births within three years of the household survey (N = 13 618) . Live births . Death at age 2–59 months . Probability of death at age 2–59 months (%) . n . % . n . % . Live births 13 618 Deaths 415 Catchment districts 68 68 Province 16 16 Birth timing  2–12 months 6022 44 54 13 0.9  13–24 months 4432 33 148 36 3.3  25–36 months 3164 23 213 51 6.7 Maternal education  None 4530 33 190 46 4.2  Primary 5604 41 163 39 2.9  Secondary and above 3484 26 62 15 1.8 Maternal age  15–19 1933 14 56 13 2.9  20–34 9801 72 303 73 3.1  35–49 1884 14 56 13 3.0 Live births (mother)  0 112 1 3 1 2.7  1 2595 19 51 12 2.0  2 2842 21 77 19 2.7  3 2293 17 81 20 3.5  4 1874 14 62 15 3.3  5+ 3902 29 141 34 3.6 Portion of district households using improved sanitation  0–24% 5576 41 132 32 2.4  25–59% 2693 20 73 18 2.7  50–74% 2546 19 94 23 3.7  75–100% 2803 21 116 28 4.1 Portion of district households using improved water source  0–24% 2725 20 69 17 2.5  25–59% 3561 26 88 21 2.5  50–74% 3681 27 126 30 3.4  75–100% 3651 27 132 32 3.6 Country  Cameroon 3757 28 75 18 2.0  CAR 5861 43 252 61 4.3  DRC 4000 29 88 21 2.2 . Live births . Death at age 2–59 months . Probability of death at age 2–59 months (%) . n . % . n . % . Live births 13 618 Deaths 415 Catchment districts 68 68 Province 16 16 Birth timing  2–12 months 6022 44 54 13 0.9  13–24 months 4432 33 148 36 3.3  25–36 months 3164 23 213 51 6.7 Maternal education  None 4530 33 190 46 4.2  Primary 5604 41 163 39 2.9  Secondary and above 3484 26 62 15 1.8 Maternal age  15–19 1933 14 56 13 2.9  20–34 9801 72 303 73 3.1  35–49 1884 14 56 13 3.0 Live births (mother)  0 112 1 3 1 2.7  1 2595 19 51 12 2.0  2 2842 21 77 19 2.7  3 2293 17 81 20 3.5  4 1874 14 62 15 3.3  5+ 3902 29 141 34 3.6 Portion of district households using improved sanitation  0–24% 5576 41 132 32 2.4  25–59% 2693 20 73 18 2.7  50–74% 2546 19 94 23 3.7  75–100% 2803 21 116 28 4.1 Portion of district households using improved water source  0–24% 2725 20 69 17 2.5  25–59% 3561 26 88 21 2.5  50–74% 3681 27 126 30 3.4  75–100% 3651 27 132 32 3.6 Country  Cameroon 3757 28 75 18 2.0  CAR 5861 43 252 61 4.3  DRC 4000 29 88 21 2.2 Sample represents those live births in catchment areas supported by direct observation of clinical consultation for children age 2–59 months seeking curative care services (illness or injury), regardless of whether the caretaker also consented to exit interview. All means represent unweighted sample averages. Open in new tab The results from adjusted and unadjusted logistic models of mortality at age 2–59 months are presented in Table 5. In the adjusted models, a shift in the percentage of visits completing at least 80% of basic diagnostic protocols from 0% to 100% (a one unit increase in compliance) was associated with a 78.2% reduction in the odds of mortality at age 2–59 months (adjusted odds ratio (aOR) 95% CI: 0.061–0.779), while a unit increment in select general danger sign compliance was associated with a 80.9% reduction in the odds of mortality (aOR 95% CI: 0.067–0.549). A more conservative 10-percentage point increment in compliance with diagnostic and select general danger sign protocols was associated with a 14.1% and 15.3% decline in the odds of mortality at age 2–59 months, respectively. Substantially increased risk was also observed for mothers under age 20, while large protective effects were found for primary and secondary schooling. Supplementary Table S2a and b presents the results of the main models with alternate date and month of birth imputation assumptions, and Supplementary Table S3a and c presents the results of the main models by survey country. Note that the Cameroon and DRC surveys may be under powered to precisely identify the quality–mortality association. The distribution of the exposure variables among catchment districts is illustrated in Supplementary Figure S4a and b. Table 5 Adjusted and unadjusted logistic models of death at age 2–59 months on quality indices using district catchment areas and three-year birth histories . Death at age 2–59 months (odds ratio) . . Unadjusted . Adjusted . Unadjusted . Adjusted . Disease diagnostic protocol compliance 0.268 0.218 (0.083–0.865) (0.061–0.779) [0.028] [0.019] Select general danger sign compliance 0.245 0.191 (0.092–0.649) (0.067–0.549) [0.005] [0.002] Live births 1.053 1.055 (1.001–1.108) (1.003–1.111) [0.047] [0.038] Maternal age  15–19 1.236 1.245 (0.900–1.696) (0.907–1.709) [0.191] [0.176]  20–34 Ref (1.0) Ref (1.0)  35–49 0.791 0.783 (0.561–1.115) (0.555–1.104) [0.180] [0.163] Maternal education  No schooling Ref (1.0) Ref (1.0)  Primary 0.797 0.809 (0.628–1.012) (0.637–1.027) [0.063] [0.081]  Secondary + 0.668 0.679 (0.473–0.943) (0.481–0.959) [0.022] [0.028] District: asset quintile 0.920 0.928 (0.689–1.229) (0.694–1.241) [0.573] [0.615] District: % improved sanitation 0.314 0.195 (0.035–2.821) (0.021–1.790) [0.301] [0.148] District: % improved water 3.292 4.468 (0.433–24.999) (0.580–34.414) [0.249] [0.151] Constant 0.036 0.069 0.034 0.060 (0.026–0.048) (0.023–0.206) (0.026–0.046) (0.020–0.182) [0.000] [0.000] [0.000] [0.000] Observations 13 618 13 618 13 618 13 618 Number of groups 68 68 68 68 Year of birth FE No Yes No Yes . Death at age 2–59 months (odds ratio) . . Unadjusted . Adjusted . Unadjusted . Adjusted . Disease diagnostic protocol compliance 0.268 0.218 (0.083–0.865) (0.061–0.779) [0.028] [0.019] Select general danger sign compliance 0.245 0.191 (0.092–0.649) (0.067–0.549) [0.005] [0.002] Live births 1.053 1.055 (1.001–1.108) (1.003–1.111) [0.047] [0.038] Maternal age  15–19 1.236 1.245 (0.900–1.696) (0.907–1.709) [0.191] [0.176]  20–34 Ref (1.0) Ref (1.0)  35–49 0.791 0.783 (0.561–1.115) (0.555–1.104) [0.180] [0.163] Maternal education  No schooling Ref (1.0) Ref (1.0)  Primary 0.797 0.809 (0.628–1.012) (0.637–1.027) [0.063] [0.081]  Secondary + 0.668 0.679 (0.473–0.943) (0.481–0.959) [0.022] [0.028] District: asset quintile 0.920 0.928 (0.689–1.229) (0.694–1.241) [0.573] [0.615] District: % improved sanitation 0.314 0.195 (0.035–2.821) (0.021–1.790) [0.301] [0.148] District: % improved water 3.292 4.468 (0.433–24.999) (0.580–34.414) [0.249] [0.151] Constant 0.036 0.069 0.034 0.060 (0.026–0.048) (0.023–0.206) (0.026–0.046) (0.020–0.182) [0.000] [0.000] [0.000] [0.000] Observations 13 618 13 618 13 618 13 618 Number of groups 68 68 68 68 Year of birth FE No Yes No Yes All models include district catchment area random effects. All adjusted models include country-province fixed effects and year of birth dummies. Observation units are live births within 3 years of the survey. Disease diagnostic protocol compliance refers to the proportion of sick-child visits in the catchment district for which at least 80% of 15 protocols were assessed. Select general danger sign compliance refers to the proportion of sick-child visits in the catchment district for which all three select general danger signs were assessed. 95% confidence intervals are in parenthesis, and P-values are in brackets. Open in new tab Table 5 Adjusted and unadjusted logistic models of death at age 2–59 months on quality indices using district catchment areas and three-year birth histories . Death at age 2–59 months (odds ratio) . . Unadjusted . Adjusted . Unadjusted . Adjusted . Disease diagnostic protocol compliance 0.268 0.218 (0.083–0.865) (0.061–0.779) [0.028] [0.019] Select general danger sign compliance 0.245 0.191 (0.092–0.649) (0.067–0.549) [0.005] [0.002] Live births 1.053 1.055 (1.001–1.108) (1.003–1.111) [0.047] [0.038] Maternal age  15–19 1.236 1.245 (0.900–1.696) (0.907–1.709) [0.191] [0.176]  20–34 Ref (1.0) Ref (1.0)  35–49 0.791 0.783 (0.561–1.115) (0.555–1.104) [0.180] [0.163] Maternal education  No schooling Ref (1.0) Ref (1.0)  Primary 0.797 0.809 (0.628–1.012) (0.637–1.027) [0.063] [0.081]  Secondary + 0.668 0.679 (0.473–0.943) (0.481–0.959) [0.022] [0.028] District: asset quintile 0.920 0.928 (0.689–1.229) (0.694–1.241) [0.573] [0.615] District: % improved sanitation 0.314 0.195 (0.035–2.821) (0.021–1.790) [0.301] [0.148] District: % improved water 3.292 4.468 (0.433–24.999) (0.580–34.414) [0.249] [0.151] Constant 0.036 0.069 0.034 0.060 (0.026–0.048) (0.023–0.206) (0.026–0.046) (0.020–0.182) [0.000] [0.000] [0.000] [0.000] Observations 13 618 13 618 13 618 13 618 Number of groups 68 68 68 68 Year of birth FE No Yes No Yes . Death at age 2–59 months (odds ratio) . . Unadjusted . Adjusted . Unadjusted . Adjusted . Disease diagnostic protocol compliance 0.268 0.218 (0.083–0.865) (0.061–0.779) [0.028] [0.019] Select general danger sign compliance 0.245 0.191 (0.092–0.649) (0.067–0.549) [0.005] [0.002] Live births 1.053 1.055 (1.001–1.108) (1.003–1.111) [0.047] [0.038] Maternal age  15–19 1.236 1.245 (0.900–1.696) (0.907–1.709) [0.191] [0.176]  20–34 Ref (1.0) Ref (1.0)  35–49 0.791 0.783 (0.561–1.115) (0.555–1.104) [0.180] [0.163] Maternal education  No schooling Ref (1.0) Ref (1.0)  Primary 0.797 0.809 (0.628–1.012) (0.637–1.027) [0.063] [0.081]  Secondary + 0.668 0.679 (0.473–0.943) (0.481–0.959) [0.022] [0.028] District: asset quintile 0.920 0.928 (0.689–1.229) (0.694–1.241) [0.573] [0.615] District: % improved sanitation 0.314 0.195 (0.035–2.821) (0.021–1.790) [0.301] [0.148] District: % improved water 3.292 4.468 (0.433–24.999) (0.580–34.414) [0.249] [0.151] Constant 0.036 0.069 0.034 0.060 (0.026–0.048) (0.023–0.206) (0.026–0.046) (0.020–0.182) [0.000] [0.000] [0.000] [0.000] Observations 13 618 13 618 13 618 13 618 Number of groups 68 68 68 68 Year of birth FE No Yes No Yes All models include district catchment area random effects. All adjusted models include country-province fixed effects and year of birth dummies. Observation units are live births within 3 years of the survey. Disease diagnostic protocol compliance refers to the proportion of sick-child visits in the catchment district for which at least 80% of 15 protocols were assessed. Select general danger sign compliance refers to the proportion of sick-child visits in the catchment district for which all three select general danger signs were assessed. 95% confidence intervals are in parenthesis, and P-values are in brackets. Open in new tab Table 6 compares the results of sensitivity analyses, in which we expand the timeline analysed from 3 to 5 years (columns 1, 3, 4 and 6) and narrow our measurement of quality from the district to facility level (columns 2, 3, 5 and 6). Supplementary Figure S4c and d illustrates the distribution of facility-level quality. When we expand the time window alone (Table 6, columns 1 and 4), we observe weaker associations between district clinical quality and mortality at age 2–59 months. When we consider the expanded time window and facility-level quality (Table 6, columns 3 and 6), our estimates shrunk further. Among those, however, only the estimated association between select general danger sign compliance and mortality at age 2–59 months (Table 6, column 6) was statistically indistinguishable from a null effect: aOR 95% CI: 0.369–0.961. Table 6 Sensitivity analysis: adjusted logistic models of death at age 2–59 months on quality indices using different spatial coding and time windows . Death at age 2–59 months (odds ratio) . . (1) . (2) . (3) . (4) . (5) . (6) . Catchment area . District . Facility . Facility . District . Facility . Facility . Birth history . 5 years . 3 years . 5 years . 5 years . 3 years . 5 years . Disease diagnostic protocol compliance 0.313 0.805 1.027 (0.110–0.889) (0.324–2.001) (0.542–1.947) [0.029] [0.640] [0.935] Select general danger sign compliance 0.359 0.564 0.596 (0.160–0.802) (0.281–1.129) (0.369–0.961) [0.013] [0.106] [0.034] Constant 0.044 0.092 0.083 0.041 0.095 0.087 (0.015–0.122) (0.030–0.283) (0.032–0.214) (0.015–0.116) (0.031–0.292) (0.034–0.223) [0.000] [0.000] [0.000] [0.000] [0.000] [0.000] Observations 19 671 6511 9682 19 671 6511 9682 Number of groups 68 457 458 68 457 458 . Death at age 2–59 months (odds ratio) . . (1) . (2) . (3) . (4) . (5) . (6) . Catchment area . District . Facility . Facility . District . Facility . Facility . Birth history . 5 years . 3 years . 5 years . 5 years . 3 years . 5 years . Disease diagnostic protocol compliance 0.313 0.805 1.027 (0.110–0.889) (0.324–2.001) (0.542–1.947) [0.029] [0.640] [0.935] Select general danger sign compliance 0.359 0.564 0.596 (0.160–0.802) (0.281–1.129) (0.369–0.961) [0.013] [0.106] [0.034] Constant 0.044 0.092 0.083 0.041 0.095 0.087 (0.015–0.122) (0.030–0.283) (0.032–0.214) (0.015–0.116) (0.031–0.292) (0.034–0.223) [0.000] [0.000] [0.000] [0.000] [0.000] [0.000] Observations 19 671 6511 9682 19 671 6511 9682 Number of groups 68 457 458 68 457 458 All models include catchment area random effects (facility or district), country-province fixed effects and year of birth dummies, while controlling for live births ever born to the mother, maternal age (15–19, 20–34 and 35–44 years), maternal education group (none, primary and secondary), average asset quintile in the catchment area, portion of catchment households using improved sanitation and the portion of catchment households using improved water source. Observation units are live births; 350 live births were dropped from adjusted models using facility catchment areas because no deaths were observed among those born in 2015. Disease diagnostic protocol compliance refers to the proportion of sick-child visits in the catchment area for which at least 80% of 15 protocols were assessed. Select general danger sign compliance refers to the proportion of sick-child visits in the catchment area for which all three select general danger signs were assessed. 95% confidence intervals are in parenthesis, and P-values are in brackets. Open in new tab Table 6 Sensitivity analysis: adjusted logistic models of death at age 2–59 months on quality indices using different spatial coding and time windows . Death at age 2–59 months (odds ratio) . . (1) . (2) . (3) . (4) . (5) . (6) . Catchment area . District . Facility . Facility . District . Facility . Facility . Birth history . 5 years . 3 years . 5 years . 5 years . 3 years . 5 years . Disease diagnostic protocol compliance 0.313 0.805 1.027 (0.110–0.889) (0.324–2.001) (0.542–1.947) [0.029] [0.640] [0.935] Select general danger sign compliance 0.359 0.564 0.596 (0.160–0.802) (0.281–1.129) (0.369–0.961) [0.013] [0.106] [0.034] Constant 0.044 0.092 0.083 0.041 0.095 0.087 (0.015–0.122) (0.030–0.283) (0.032–0.214) (0.015–0.116) (0.031–0.292) (0.034–0.223) [0.000] [0.000] [0.000] [0.000] [0.000] [0.000] Observations 19 671 6511 9682 19 671 6511 9682 Number of groups 68 457 458 68 457 458 . Death at age 2–59 months (odds ratio) . . (1) . (2) . (3) . (4) . (5) . (6) . Catchment area . District . Facility . Facility . District . Facility . Facility . Birth history . 5 years . 3 years . 5 years . 5 years . 3 years . 5 years . Disease diagnostic protocol compliance 0.313 0.805 1.027 (0.110–0.889) (0.324–2.001) (0.542–1.947) [0.029] [0.640] [0.935] Select general danger sign compliance 0.359 0.564 0.596 (0.160–0.802) (0.281–1.129) (0.369–0.961) [0.013] [0.106] [0.034] Constant 0.044 0.092 0.083 0.041 0.095 0.087 (0.015–0.122) (0.030–0.283) (0.032–0.214) (0.015–0.116) (0.031–0.292) (0.034–0.223) [0.000] [0.000] [0.000] [0.000] [0.000] [0.000] Observations 19 671 6511 9682 19 671 6511 9682 Number of groups 68 457 458 68 457 458 All models include catchment area random effects (facility or district), country-province fixed effects and year of birth dummies, while controlling for live births ever born to the mother, maternal age (15–19, 20–34 and 35–44 years), maternal education group (none, primary and secondary), average asset quintile in the catchment area, portion of catchment households using improved sanitation and the portion of catchment households using improved water source. Observation units are live births; 350 live births were dropped from adjusted models using facility catchment areas because no deaths were observed among those born in 2015. Disease diagnostic protocol compliance refers to the proportion of sick-child visits in the catchment area for which at least 80% of 15 protocols were assessed. Select general danger sign compliance refers to the proportion of sick-child visits in the catchment area for which all three select general danger signs were assessed. 95% confidence intervals are in parenthesis, and P-values are in brackets. Open in new tab Supplementary Figure S5 presents further sensitivity analyses for the main adjusted model, in which we consider the relationship between the proportion of district consultations complying with each protocol item and mortality at age 2–59 months separately. For most (11/15) protocols, the relationship is not statistically significant at the 0.05 level. Discussion We used a novel data set from three countries to assess the quality of clinical care received by children under age 5, as well as the relationship between average clinical quality and mortality at age 2–59 months. Overall compliance with clinical assessment guidelines was remarkably poor. Providers completed at least 12 of 15 basic diagnostic protocols (i.e. 80%) in only 8% of observed sick-child visits and assessed all three select general danger signs in only 13%, with relatively small differences across health provider cadres. IMCI training was low in these areas, and we may expect to find different results in similar areas with more trained healthcare workers. Experimental and non-experimental evidence suggests that IMCI and other trainings centered on childhood illness increase provider performance, where performance is defined by compliance with diagnostic protocols or the correct classification of sick children up to 2 years out. Even with these improvements, however, provider performance remained low in study settings (Nguyen et al., 2013; Leslie et al., 2016). Recent literature suggests that poor-quality child health services undermine gains in the utilization of such services, while signaling poor-quality maternal, and newborn health services too (Leslie et al., 2017). More research is needed to identify the drivers of quality disparities across patient profiles suggested by this analysis. The central result of our analysis is that failure to comply with basic diagnostic protocols is associated with substantial excess mortality at age 2–59 months. After controlling for individual, maternal and household, characteristics, we found that children faced 14.1% lower odds of mortality for every 10% increase in compliance with basic diagnostic protocols during sick-child visits at the district level, when compliance is defined as completing 80% of the recommended clinical items. We found similar associations for compliance with select general danger sign protocols, which are central to the assessment of disease severity in IMCI protocols. Although child survival is the focus of this analysis, it is important to note that better case management of non-fatal disease confers other benefits including decreased morbidity, reduced over-prescription, infectious disease control and improved utilization of and satisfaction with the health system (Audo et al., 2005; Maina et al., 2017). Empirical evidence of the impact of clinical quality on child mortality has largely been limited to evaluations of the IMCI strategy; those results have been mixed to-date and fail to isolate the effect of the sub-component to improve case management (Ahmed et al., 2010; Rakha et al., 2013; Gera et al., 2016). Cross-sectional data analyses are rare due to need for households and facilities to coincide geographically and temporally. The only such study we identified relied on strong econometric assumptions in a narrow context (Leslie et al., 2016). Our results suggest that the empirical relationship between the quality of clinical assessment and child health outcomes is indeed strong. As such, health system performance measurement should emphasize quality, in addition to quantity. These findings are particularly consequential to the structure of quality improvement interventions at a time when existing models are yet unproven or falling short. In a wide range of sub-Saharan African settings, resource-intensive models of training and supervision have demonstrated small improvements in compliance with basic clinical protocols in sick-child visits (Leslie et al., 2016). The impact of alternative models, such as performance-based financing, quality improvement collaboratives and a broad range of other stand-alone strategies, on similar measures of quality processes is mixed or modest at best (Das et al., 2016; Rowe et al., 2018; Garcia-Elorrio et al., 2019). New mechanisms for monitoring and improving clinical quality that focus on the identification of life-threatening cases for immediate referral should be explored. The main strength of our study is direct sampling of households in the catchment areas of facilities where the quality of care was observed. Our intent-to-treat estimates take into consideration all behavioral responses of households facing high or low quality of care. By assessing quality at the district level, we mitigated potential bypassing-induced measurement error (Kruk et al., 2009). Districts in the survey settings are large; the average district population is over 50 000 in CAR, around 100 000 in Cameroon, so inter-district bypassing should be small. Empirically, district-level associations were larger in magnitude compared to health facility-level analysis, which suggests that bypassing may indeed attenuate the associations observed at the facility level. Associations observed at the district level may also be attenuated. First, it is possible that mothers underreport or misreport mortality in birth histories (Mahy, 2003). If such errors are more common in high mortality areas, our estimates would be biased towards the null. Second, any changes in quality over time would induce measurement error in the exposure and thus result in attenuation. When we broadened the sample to include more birth cohorts, we indeed found smaller effect sizes. While the observed quality of care in sick-child visits is likely biased upward due to Hawthorne effect, any such effect would be uncorrelated with mortality and not bias our regression results (Leonard and Masatu, 2010). Similar studies omit the first and second observations to minimize bias in our descriptive results, but our data structure does not provide sufficient observations per provider to apply this correction. The results of our study suggest that focused interventions to improve compliance with IMCI guidelines may have great potential to reduce child mortality in the studied area, as well as similar settings. Acknowledgements The authors received no financial support for this analysis. The authors gratefully acknowledge the Health Results Innovation Trust Fund’ financial contributions to data collection for the baseline impact evaluation surveys in Cameroon, Democratic Republic of the Congo and Central African Republic used in this analysis. Operational and technical assistance from local planning authorities, non-governmental organizations and consultants were indispensable to the successful completion of these surveys. The authors also appreciate the clinical expertise provided by Elizabeth Fessler, MSN, WHNP-BC, CNP, in development of this analysis. Supplementary data Supplementary data are available at Health Policy and Planning online. Conflict of interest statement. None declared. Ethical approval. No ethical clearance was required for this secondary data analysis. References Ahmed HM , Mitchell M, Hedt B. 2010 . National implementation of Integrated Management of Childhood Illness (IMCI): policy constraints and strategies . Health Policy 96 : 128 – 33 . Google Scholar Crossref Search ADS PubMed WorldCat Audo MO , Ferguson A, Njoroge PK. 2005 . Quality of health care and its effects in the utilisation of maternal and child health services in Kenya . East African Medical Journal 82 : 547 – 53 . Google Scholar PubMed OpenURL Placeholder Text WorldCat Das A , Gopalan SS, Chandramohan D. 2016 . Effect of pay for performance to improve quality of maternal and child care in low- and middle-income countries: a systematic review . BMC Public Health 16 : 321 . Google Scholar Crossref Search ADS PubMed WorldCat de Savigny D , Mayombana C, Mwageni E et al. 2004 . Care-seeking patterns for fatal malaria in Tanzania . Malaria Journal 3 : 27 . Google Scholar Crossref Search ADS PubMed WorldCat Fink G , Ross R, Hill K et al. 2015 . Institutional deliveries weakly associated with improved neonatal survival in developing countries: evidence from 192 Demographic and Health Surveys . International Journal of Epidemiology 44 : 1879 – 88 . Google Scholar Crossref Search ADS PubMed WorldCat Gage A , Leslie H, Kruk M. 2017 . Does the measure matter? Observed quality of care score and child mortality in a multi-country analysis . Annals of Global Health 83 : 39 . Google Scholar Crossref Search ADS WorldCat Garcia-Elorrio EI , Rowe IS, Teijeiro ME et al. 2019 . The effectiveness of the quality improvement collaborative strategy in low- and middle-income countries: a systematic review and meta-analysis . PLoS One 14 : e0221919 . Google Scholar Crossref Search ADS PubMed WorldCat Gera T , Shah D, Garner P et al. 2016 . Integrated management of childhood illness (IMCI) strategy for children under five . Cochrane Database of Systematic Reviews 2016 (16) . doi:10.1002/14651858.CD010123.pub2. OpenURL Placeholder Text WorldCat Horwood C , Vermaak K, Rollins N et al. 2009 . An evaluation of the quality of IMCI assessments among IMCI trained health workers in South Africa . PLoS One 4 : e5937 . Google Scholar Crossref Search ADS PubMed WorldCat Källander K , Hildenwall H, Waiswa P et al. 2008 . Delayed care seeking for fatal pneumonia in children aged under five years in Uganda: a case-series study . Bulletin of the World Health Organization 86 : 332 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat Kruk ME , Mbaruku G, McCord CW et al. 2009 . Bypassing primary care facilities for childbirth: a population-based study in rural Tanzania . Health Policy and Planning 24 : 279 – 88 . Google Scholar Crossref Search ADS PubMed WorldCat Leonard KL , Masatu MC. 2010 . Using the Hawthorne effect to examine the gap between a doctor’s best possible practice and actual performance . Journal of Development Economics 93 : 226 – 34 . Google Scholar Crossref Search ADS WorldCat Leslie HH , Fink G, Nsona H et al. 2016 a. Obstetric facility quality and newborn mortality in Malawi: a cross-sectional study . PLoS Medicine 13 : e1002151 . Google Scholar Crossref Search ADS PubMed WorldCat Leslie HH , Gage A, Nsona H et al. 2016 b. Training and supervision did not meaningfully improve quality of care for pregnant women or sick children in sub-Saharan Africa . Health Affairs (Project Hope) 35 : 1716 – 24 . Google Scholar Crossref Search ADS PubMed WorldCat Leslie HH , Malata A, Ndiaye Y et al. 2017 . Effective coverage of primary care services in eight high-mortality countries . BMJ Global Health 2 : e000424 . Google Scholar Crossref Search ADS PubMed WorldCat Mahy M. 2003 . Measuring Child Mortality in AIDS-Affected Countries. Workshop on HIV/AIDS and Adult Mortality in Developing Countries. New York. https://www.un.org/en/development/desa/population/events/pdf/expert/5/UNICEF_Paper15.pdf, last accessed 8 May 2020. Maina M , Akech S, Mwaniki P et al. 2017 . Inappropriate prescription of cough remedies among children hospitalised with respiratory illness over the period 2002–2015 in Kenya . Tropical Medicine & International Health 22 : 363 – 9 . Google Scholar Crossref Search ADS WorldCat Nguyen DTK , Leung KK, McIntyre L et al. 2013 . Does Integrated Management of Childhood Illness (IMCI) training improve the skills of health workers? A systematic review and meta-analysis . PLoS One 8 : e66030 . Google Scholar Crossref Search ADS PubMed WorldCat O’Donnell O , Van Doorslaer E, Wagstaff A et al. 2007 . Analyzing Health Equity Using Household Survey Data: A Guide to Techniques and Their Implementation . Washington, DC : The World Bank . Google Scholar Crossref Search ADS Google Scholar Google Preview WorldCat COPAC Rakha MA , Abdelmoneim A-N, Farhoud S et al. 2013 . Does implementation of the IMCI strategy have an impact on child mortality? A retrospective analysis of routine data from Egypt . BMJ Open 3 : e001852 . Google Scholar Crossref Search ADS PubMed WorldCat Rowe AK , Rowe SY, Peters DH et al. 2018 . Effectiveness of strategies to improve health-care provider practices in low-income and middle-income countries: a systematic review . The Lancet Global Health 6 : e1163 – 75 . Google Scholar Crossref Search ADS PubMed WorldCat Rutherford ME , Dockerty JD, Jasseh M et al. 2009 . Access to health care and mortality of children under 5 years of age in the Gambia: a case–control study . Bulletin of the World Health Organization 87 : 216 – 24 . Google Scholar Crossref Search ADS PubMed WorldCat Rutherford ME , Mulholland K, Hill PC. 2010 . How access to health care relates to under-five mortality in sub-Saharan Africa: systematic review . Tropical Medicine & International Health 15 : 508 – 19 . Google Scholar Crossref Search ADS WorldCat Sharma J , Leslie HH, Kundu F et al. 2017 . Poor quality for poor women? Inequities in the quality of antenatal and delivery care in Kenya . PLoS One 12 : e0171236 . Google Scholar Crossref Search ADS PubMed WorldCat Sodemann M , Jakobsen MS, Mølbak K et al. 1997 . High mortality despite good care-seeking behaviour: a community study of childhood deaths in Guinea-Bissau . Bulletin of the World Health Organization 75 : 205 – 12 . Google Scholar PubMed OpenURL Placeholder Text WorldCat StataCorp. 2017 . Stata Statistical Software: Release 15. 15 . College Station, TX : StataCorp LP . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC UNICEF. 2015 . Committing to Child Survival: A Promise Renewed. New York, NY. Available at: https://www.unicef.org/publications/files/APR_2015_9_Sep_15.pdf, last accessed 8 May 2020. United Nations. 2015 . Transforming Our World: The 2030 Agenda for Sustainable Development. https://sustainabledevelopment.un.org/post2015/transformingourworld/publication, last accessed 8 May 2020. Willcox ML , Kumbakumba E, Diallo D et al. 2018 . Circumstances of child deaths in Mali and Uganda: a community-based confidential enquiry. Lancet Global Health 6: e691 – e702 . World Bank Group. 2014 a. Cameroon—Health Results-Based Financing Impact Evaluation 2012, Health Facility Baseline Survey. https://microdata.worldbank.org/index.php/catalog/2047/related-materials, last accessed 8 May 2020. World Bank Group. 2014 b. Cameroon—Health Results-Based Financing Impact Evaluation 2012, Household Baseline Survey. https://microdata.worldbank.org/index.php/catalog/2048/related-materials, last accessed 8 May 2020. World Bank Group. 2014 c. Central African Republic—Health Results-Based Financing Impact Evaluation 2012, Baseline Household Survey. https://microdata.worldbank.org/index.php/catalog/2174/related-materials, last accessed 8 May 2020. World Bank Group. 2015 . Central African Republic—Health Results-Based Financing Impact Evaluation 2012, Health Facility Baseline Survey. https://microdata.worldbank.org/index.php/catalog/2175/related-materials, last accessed 8 May 2020. World Bank Group. 2017 a. Congo, Dem. Rep.—Health Results Based Financing Impact Evaluation 2015, Health Facility Baseline Survey. https://microdata.worldbank.org/index.php/catalog/2825/related-materials, last accessed 8 May 2020. World Bank Group. 2017 b. Congo, Dem. Rep.—Health Results Based Financing Impact Evaluation 2015, Household Baseline Survey. https://microdata.worldbank.org/index.php/catalog/2824/related-materials, last accessed 8 May 2020. World Health Organization. 2003 . Health Facility Survey: Tool to Evaluate the Quality of Care Delivered to Sick Children Attending Outpatient Facilities. https://apps.who.int/iris/handle/10665/42643, last accessed 8 May 2020. World Health Organization. 2014 . Integrated Management of Childhood Illness: Chart Booklet. Switzerland. http://apps.who.int/iris/bitstream/10665/104772/16/9789241506823_Chartbook_eng.pdf, last accessed 8 May 2020. You D , Hug L, Ejdemyr S et al. 2015 . Global, regional, and national levels and trends in under-5 mortality between 1990 and 2015, with scenario-based projections to 2030: a systematic analysis by the UN Inter-agency Group for Child Mortality Estimation . The Lancet 386 : 2275 – 86 . Google Scholar Crossref Search ADS WorldCat © The Author(s) 2020. Published by Oxford University Press in association with The London School of Hygiene and Tropical Medicine. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - Quality of clinical assessment and child mortality: a three-country cross-sectional study
JF - Health Policy and Planning
DO - 10.1093/heapol/czaa048
DA - 2019-07-01
UR - https://www.deepdyve.com/lp/oxford-university-press/quality-of-clinical-assessment-and-child-mortality-a-three-country-Wnio7c70l6
DP - DeepDyve
ER -