Background: Monitoring financial protection against catastrophic health expenditures is important to understand how health financing arrangements in a country protect its population against high costs associated with accessing health services. While catastrophic health expenditures are generally defined to be when household expenditures for health exceed a given threshold of household resources, there is no gold standard with several methods applied to define the threshold and household resources. These different approaches to constructing the indicator might give different pictures of a country’s progress towards financial protection. In order for monitoring to effectively provide policy insight, it is critical to understand the sensitivity of measurement to these choices. Methods: This paper examines the impact of varying two methodological choices by analysing household expenditure data from a sample of 47 countries. We assess sensitivity of cross-country comparisons to a range of thresholds by testing for restricted dominance. We further assess sensitivity of comparisons to different methods for defining household resources (i.e. total expenditure, non-food expenditure and non-subsistence expenditure) by conducting correlation tests of country rankings. Results: We found country rankings are robust to the choice of threshold in a tenth to a quarter of comparisons within the 5–85% threshold range and this increases to half of comparisons if the threshold is restricted to 5–40%, following those commonly used in the literature. Furthermore, correlations of country rankings using different methods to define household resources were moderate to high; thus, this choice makes less difference from a measurement perspective than from an ethical perspective as different definitions of available household resources reflect varying concerns for equity. Conclusions: Interpreting comparisons from global monitoring based on a single threshold should be done with caution as these may not provide reliable insight into relative country progress. We therefore recommend financial protection against catastrophic health expenditures be measured across a range of thresholds using a catastrophic incidence curve as shown in this paper. We further recommend evaluating financial protection in relation to a country’s health financing system arrangements in order to better understand the extent of protection and better inform future policy changes. Keywords: Catastrophic health expenditures, Financial protection, Health financing, Universal health coverage * Correspondence: email@example.com; firstname.lastname@example.org Department of Health Systems Governance and Financing, World Health Organization, 20 Avenue Appia, 1211 Geneva, Switzerland Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 2 of 13 Background making sense of cross-country comparisons to draw There is increasing interest in monitoring the impact of conclusions about the relative performance of health household health expenditures on living standards. This financing systems becomes more challenging. interest is growing because financial protection is a key The objective of this paper is to assess the sensitivity component of universal health coverage (defined as every- of comparisons of country-level estimates of financial one receiving the health services they need and protected protection against catastrophic health expenditures to from financial hardship in doing so), an agreed target for different methodological choices in indicator construc- health in the Sustainable Development Goals (SDGs) . tion. In this analysis, sensitivity is assessed by the extent Global-level monitoring is of particular interest as it allows to which orderings of distributions of financial protec- benchmarking a country’s progress relative to others and tion against catastrophic health expenditures across encourages global dialogue and the exchange of country countries are consistent, irrespective of the threshold, experience. Country-level monitoring is also of particular and correlated, irrespective of the method for defining interest to understand progress over time or differences household resources. We adapted methods to test for across sub-national levels, thereby helping to inform future restricted dominance. These methods have previously policy reforms. The methodological analysis presented here been applied in the measurement of poverty to assess is concerned with monitoring at the global level and sensitivity of poverty incidence rates to the choice of the focuses on comparisons across countries. Regardless of the poverty line , and have more recently been studied level of monitoring, there is need for an indicator that leads as a means to assess the sensitivity of the incidence of to unambiguous assessments of comparative progress. catastrophic health expenditures to the choice of the Monitoring financial protection typically relies on two threshold [8, 13]. This empirical paper is one of the first indicators – catastrophic health expenditures associated to apply such methods to assess the impact of varying with out-of-pocket (OOP) payments for health reducing methodological choices on global monitoring of financial people’s ability to spend on other essential items, and protection. It demonstrates whether these choices mat- impoverishing health expenditures associated with OOP ter, provides new insight into challenges for monitoring, payments for health pushing or further pushing people and recommends a way forward for measuring financial into poverty. Both indicators are thus concerned with protection beyond conventional approaches. the impact of OOP payments, defined as those payments that patients make directly to health providers at the Conceptual underpinnings time of service. They include cost-sharing and informal The concept of financial protection rests on the theoret- payments (in kind and in cash) but exclude payments by ical foundations of insurance and the economic value of a third-party payer . This paper focuses on the former reduced uncertainty or financial risk of being exposed to indicator of catastrophic health expenditures which large healthcare costs [14, 15]. Health insurance, whether monitors when OOP payments as a share of household run by governments, nongovernmental organizations, resources reaches and/or surpasses a certain threshold. communities or commercial companies, seeks to reduce Choices in measuring this relate both to the definition of this risk for the individual; when a country’shealth finan- household available resources (denominator) and to the cing arrangements fail to adequately provide this insur- threshold (percentage) used to determine when the OOP ance function, access to health services will either be share on health is catastrophic. There is no established foregone or privately financed through OOP payments. gold standard for either, with considerable debate over The concern of catastrophic health expenditures is with the last decade. Earlier discussions focused on the the negative impact that OOP payments can have on definition of available household resources [3–5]. More economic well-being, for example when an individual recent discussions concerned the choice of the threshold forgoes consumption of other necessities (e.g. food) to pay [6–8]. In the absence of consensus, studies of cata- for health. It is identified by comparing OOP payments strophic health expenditures frequently present results for health to some definition of household resources and using multiple definitions of household resources and whether these surpass a certain threshold. various thresholds [9–11]. Thus, in measuring catastrophic health expenditures, For global monitoring to be meaningful for policy, it is there are two methodological choices. The first is the important to understand if a country’s performance rela- definition of household resources available to pay for tive to that of another is insensitive to varying methodo- health services. The second is the threshold used to logical choices. Does the assessment that a country has identify health expenditures as catastrophic. higher levels of financial protection than another depend Defining household resources follows two main on the method used to define available household approaches, differing in whether they account for non- resources? Does it also depend on the specific threshold? discretionary spending . In the ‘budget share approach’ If the answer to one and especially to both is yes, then household resources are defined in relation to a household’s Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 3 of 13 total budget without distinguishing spending on necessities. during periods of high and low income [24, 25]. Fur- This approach is easy to understand and requires no thermore, it has been shown that the choice matters further calculation. A further advantage is that it is not less when measuring national incidence rates of cata- dependent on household allocation decisions across discre- strophic health expenditures (as done in this paper) tionary and non-discretionary items. However, it fails to than when measuring inequalities across socio- distinguish between populations who just manage to meet economic groups . We also do not consider the subsistence needs with little or nothing left for discretionary other two additional CTP variants in this paper as expenditures and richer groups who have more latitude in currentlytheyare not ascommonlyusedinthe discretionary spending. measurement of catastrophic health expenditures. The ‘capacity-to-pay (CTP) approach’ addresses this The second methodological choice in measuring protec- limitation, recognising that poorer households spend a tion against catastrophic health expenditure is the threshold higher proportion of available resources on essential items used to define catastrophic OOP payments. Any such than richer households. It thus defines household re- threshold is a normative choice. The choice is based on the sources as net of such spending. The idea is that spending idea that households who are spending above the threshold on other basic necessities should not be considered part of on health are left with a certain balance of their expenditure resources available for health. CTP can be defined in vari- to spend on other essential items [15, 26]. Too low a ous ways but commonly includes a component related to threshold fails to capture a level of spending that causes food spending. One well-established method defines this households to forgo such items. Too high a threshold fails as total expenditures net of all food spending . While to capture small amounts of spending by the poor that are its calculation is simple, a limitation of this method is that nonetheless catastrophic. Catastrophic thresholds in pub- it does not recognise that some food spending is discre- lished studies typically vary between 10% and 40% depend- tionary. Another popular method, proposed by Xu et al. ing on the definition of household resources, with a lower (2003) , approximates the non-discretionary part of threshold used in the budget share method and a higher food spending as average food expenditure per equivalent threshold in CTP methods [9, 27–31]. Typically a single adult across households in the 45th–55th percentile of the threshold is uniformly applied across the population, but it food budget share distribution. When actual food spend- can also vary such that a lower threshold is used for the ing is below this amount, CTP is defined as total expend- poor and a higher threshold for the rich [6, 7]. iture net of actual food spending. Any expenditure above this fixed subsistence expenditure amount is considered discretionary and available for spending on other goods Methods and services, including health. These two CTP methods This analysis relied on household expenditure survey data are conceptually similar but the latter adopts a stricter as- from a sample of 47 countries over 2000–2012 sumption of what is non-discretionary. It could thus be (Additional file 1). This convenience sample was composed argued to more accurately estimate CTP of populations at of nationally representative household survey datasets the bottom of the income distribution. Critics of the Xu which the authors had access to and which had information et al. (2003)  method argue that its definition of subsist- on total consumption expenditure, including on OOP pay- ence expenditure is not based on a normative standard (e.g. ments for health. The dataset represents a diverse spectrum a food consumption basket) and that it can mean that a of countries at different levels of economic development in- poorer household is judged to have greater CTP than a cluding low-, middle- and high-income countries, countries richer one [18, 19]. Further discussions on CTP ap- belonging to all five United Nations regional groups, and proaches, including their conceptual underpinnings, exist countries with diverse financing arrangements ranging elsewhere . from insurance schemes run by governments, nongovern- Other choices can be made in the definition of house- mental organizations, or communities. Data provided infor- hold resources. For example, whether this should be mation on household-level consumption expenditure measured by consumption expenditure or by income which was aggregated into three expenditure variables , whether OOP should be included in the measure (total, food, health). Total expenditure was estimated from or netted out as it does not increase welfare , and monetary and in-kind payments on all goods and services whether other categories of expenditure, such as hous- plus the monetary value of consumption of homemade ing and utilities, should also be considered as essential products. Food expenditure included items purchased and in a CTP approach . Measuring household re- consumed from own production. Health expenditure con- sources using income was not explored in this analysis sisted of OOP payments made by individuals to health pro- as the implications have already been studied elsewhere, viders at the time of service. All data were quality checked and some seminal literature suggests that consumption for missing values of the three aggregated expenditure vari- is the preferable measure given it smooths fluctuations ables and for illogical values (e.g. total expenditure<food Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 4 of 13 expenditure). The frequency of such observations was min- poverty line within the defined interval, one distribution imal and these were dropped from the dataset. of the incidence of poverty is always below another distri- For each household in each country dataset, three bution [12, 32]. In other words, as assessed through statis- health expenditure ratios were constructed as the tical tests, the poverty incidence curves do not cross. share of OOP payments for health in total expend- Analogous to this application of dominance to the iture, total expenditure net of all food expenditure, measurement of poverty, distributions of catastrophic and total expenditure net of subsistence expenditure health expenditures can also be examined for restricted on food (Table 1). dominance. Indeed, measurement of catastrophe is similar To analyse the extent to which country comparisons to that of poverty as both rely on a defined benchmark (a were sensitive to the choice of the catastrophic thresh- poverty line in the case of poverty and a threshold in the old, we adapted a restricted dominance approach de- case of catastrophe), and both are focused on a specific scribed by Flores et al. (see Additional file 2)[8, 13]. The part of the distribution (the lower distribution of income dominance approach was originally developed in the in the case of poverty and the higher share of OOP pay- measurement of inequalities comparing differences be- ments for health in household resources in the case of ca- tween two Lorenz (or concentration) curves to deter- tastrophe). The distributions of catastrophic OOP shares mine if the cumulative distribution of income (or other can also be visualised by plotting incidence rates of cata- variable of interest) is always above the other, indicating strophic health expenditures against a range of thresholds, the more preferred distribution on welfare grounds be- resulting in a curve first referred to by Wagstaff as a ‘cata- cause the degree of inequalities is unambiguously less. strophic spending curve’ . Such a curve corresponds Since then, dominance has been applied in the measure- to a descending cumulative distribution function (CDF) ment of poverty to overcome limitations given that com- and is denoted as 1 − F where F (τ) ≡ OOP_share OOP_share parisons of poverty levels are sensitive to the choice of Prob(OOP_share ≤ τ). the poverty line [32–34]. By examining distributions of Whether comparisons of country-level estimates of cata- income across a specified range of poverty lines, re- strophic health expenditures result in consistent compari- stricted dominance thus allows for ranking distributions sons where one distribution exhibits restricted dominance of poverty levels that are insensitive to the choice of the over the other is assessed through statistical tests (Additional poverty line. Dominance is said to be restricted as it per- file 2). Testing for restricted dominanceisthusvaluableasit tains to part of but not the full income distribution (i.e. enables consistent conclusions to be drawn regarding differ- given the focus is on the poor, particular interest is on ences in financial protection across countries. For restricted the lower part of the distribution). Restricted dominance dominance testing to be applied to a measure, it must hold for poverty can be visualised by plotting on the vertical a minimum of four properties akin to the axioms used in axis the incidence rate for poverty associated with mul- the poverty framework to group poverty indices [32, 36]. tiple poverty lines over a specified range of the income The different measures of financial protection de- distribution which are plotted on the horizontal axis. scribed in Table 1 are (i) focused, insensitive to changes The resulting cumulative distribution function has been above a threshold; (ii) population invariant, insensitive referred to as a ‘poverty incidence curve’ . Compara- to differences in population sizes due to adding an tive assessments of poverty distributions thus exhibit re- exact replicate of a population; (iii) anonymous, in- stricted dominance when, no matter the choice of the sensitive to interchanges in budget share levels; and (iv) Table 1 Measuring catastrophic health expenditures Headcount ratio: Share of the population spending τ% or more of household resources on OOP payments for health mhwh1ðOOP shareh ≥ τÞ m w h h h denotes a household m denotes the number of members of household h w denotes the sampling weight of household h 1() is an indicator function which is equal to 1 if the condition is satisfied and 0 otherwise τ denotes a catastrophic threshold Approach Budget share Capacity-to-pay Method Total expenditure Non-food expenditure Non-subsistence expenditure oop oop oop OOP share exp exp−food exp−se oop=OOP health payments exp=total expenditure food=food expenditure se=subsistence expenditure Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 5 of 13 Pareto-improving, indicating an increase in financial rate of catastrophic health expenditures at threshold τ – protection as household resources increase . referred to by Wagstaff as a ‘catastrophic spending curve’ Our approach for assessing sensitivity through restricted . Figure 1b illustrates a pairwise comparison result- ing in dominance for the country exhibiting a lower dominance consisted of using an intersection-union type CDF (i.e. lower levels of catastrophic health expendi- of test under the null hypothesis of non-dominance tures) and Fig. 1c and 1d resulting in non-dominance between the distributions of the OOP shares on health of A B due to intersections and insignificance. Dominance and ^ ^ two countries. Specifically, H : F −F ¼ 0. OOP share OOP share the type of non-dominance should ultimately be estab- In other words, we tested differences between each coun- lished through statistical tests. try’s share of the population with catastrophic health For each method of constructing OOP shares on health, expenditures conducted at each threshold along their we assessed the frequency and proportion of comparisons CDFs. Following Chen and Duclos (2008)  and Kaur exhibiting dominance (indicating cross-country assess- et al. (1994) , we employed tests based on the mini- ments insensitive to the choice of the threshold) across all min max mum t-statistic approach over τ ∈ [τ ; τ ]ofthe t- possible 2162 pairwise comparisons. A higher proportion ratios of the differences between the catastrophic spending of dominance is preferable as it increases confidence in curve (see Additional file 2). We did not test over the full the reliability of cross-country assessments. We also 0–100% threshold range but over two partial ranges of 5– assessed the frequency and proportion of comparisons 85% and 5–40% with a one percentage point difference resulting in non-dominance (indicating assessments sensi- such that testing occurred for a total of 81 and 36 points, tive to the threshold) due to differences in CDFs found to respectively. Testing was restricted above the lower 5% tail be insignificant and non-dominance due to intersections of the distribution because the concern with catastrophic of CDFs. Finally, we identified the longest continuous health expenditures is for large OOP payments for health threshold range over which observed t-statistics were sig- relative to household resources. In addition, testing along nificant within each pairwise comparison and then aver- the upper tail of the distribution was also restricted: ini- aged this across all comparisons for each method of tially to 85% because of a concern for the power of the test defining household resources. A higher average length in- and need for a sufficient number of observations for con- dicates a longer interval of dominance and suggests that ducting t-tests, and subsequently to 40% because this is the method is less sensitive to the threshold. The length the highest threshold commonly used in the literature. It can also be considered an indirect assessment of the over- is expected that as the range of testing decreases, the like- all power to test for dominance as a longer range of sig- lihood of dominance increases. nificance increases the ability to accept the alternative The null hypothesis of non-dominance was rejected at hypothesis of dominance. the 10% level if the absolute value of all observed t- The sensitivity of cross-country comparisons to statistics was greater than 1.645, the critical value of the methods for defining available household resources was t-distribution. Rejection was at the 10% level to account also assessed. First, we compared the proportion of pair- for fewer observations at tails of the distribution. In wise comparisons resulting in dominance when using these instances, the alternative hypothesis of dominance each method. The higher the proportion of comparisons was not rejected, implying that one country’s headcount resulting in dominance, the less sensitive is that method ratio of catastrophic health expenditures is always statis- for defining household resources compared to another. tically significantly below the other within the range of Second, we computed Spearman’s rank correlation coef- thresholds tested. Furthermore, if the t-statistic was ficient of country rankings across each method. Rather positive[negative], we inferred Country A[B] dominance. than rank countries based on their incidence rate of Failure to reject the null of non-dominance could be catastrophic health expenditures at a single threshold, attributed to either: (i) insignificance, intervals of where we ranked country distributions of OOP shares re- the CDFs were not significantly different (absolute value stricted to part of the catastrophic incidence curve over of the t-statistic less than 1.645) or (ii) intersection, in- the popular 5–40% threshold range. Thus, for each tervals where the CDFs crossed at least once (absolute method, countries were ranked by counting the number value of t-statistic greater than 1.645 and signs of the of pairwise comparisons for which the incidence rate of t-statistic changed for any given pairwise comparison). catastrophic health expenditures in one country was al- Different dominance relationships are illustrated in ways statistically lower than another country over the Fig. 1. Figure 1a shows a descending CDF of catastrophic entire popular threshold range of 5–40%, minus the health expenditures for one country and describes how number of pairwise comparisons for which the incidence the CDF gives the probability of the population spending rate was always statistically higher. If a pairwise com- τ percent or more of household resources on health, parison resulted in non-dominance, it was ignored since where each point of a CDF is equivalent to the incidence it does not allow for an unambiguous ordering of Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 6 of 13 a b c d Fig. 1 Illustration and interpretation of descending cumulative distribution functions for catastrophic health expenditures. All figures show a descending cumulative distribution function of OOP shares on health in household resources (also referred to as a ‘catastrophic incidence curve’). The y-axis represents the proportion of the population whose OOP shares on health in household resources meet or exceed threshold τ, and the x-axis shows the range of catastrophic thresholds τ. Any point on the curve can thus be interpreted as the incidence rate of catastrophic health expenditures for a given threshold. In (a), the cumulative distribution function shows that 15% of the population are spending 25% or more of household resources on OOP payments for health. In (b), Country A is said to exhibit dominance over Country B given its catastrophic incidence curve is always below that of Country A. In other words, the proportion of its population facing catastrophic health expenditures (y-axis) is always lower than Country B, no matter the threshold (x-axis). In (c), Country A and Country B exhibit non-dominance due to intersection given their catastrophic incidence curves intersect at the 12% threshold. This means that the proportion of the population in Country A facing catastrophic health expenditures is lower than Country B for thresholds below 12% but is higher than Country B for thresholds above 12%. In (d), Country A and Country B exhibit non-dominance due to insignificance given their catastrophic incidence curves differ but not to a statistically significant degree. This means that the proportion of the population in Country A facing catastrophic health expenditures differs from the proportion of the population in Country B facing catastrophic health expenditures but the difference is insignificant countries. The higher a country’s rank, the more fre- rate of catastrophic health expenditures relative to an- quently its incidence rates were lower than higher com- other; in contrast, 1697 out of 2162 comparisons resulted pared to other countries. This assessment thus indicates in non-dominance or an inconsistent assessment such the sensitivity of country rankings to using different that, depending on the choice of the threshold, a country’s methods to define household resources. incidence rate was sometimes better and sometimes worse than another. Thus, a country’s assessment of financial Results protection relative to another was sensitive to the choice Across all three methods for measuring catastrophic of the threshold over the 5–85% range. The degree of sen- health expenditures, on average, just over a fifth (21.5%) of sitivity to the threshold varied depending on the method all possible 2162 country comparisons resulted in rejec- for defining household resources. Following the budget tion of the null hypothesis in favour of the alternative hy- share approach where OOP shares on health are con- pothesis of dominance (Table 2). In other words, only 465 structed using total expenditure in the denominator, the of the total 2162 country comparisons resulted in domin- null hypothesis of dominance was rejected for only 10.7% ance or a consistent assessment of a country’sincidence of comparisons (i.e. 232 of 2162 comparisons resulted in Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 7 of 13 Table 2 Analysis of dominance between country distribution functions of OOP shares on health Approach Budget share Capacity-to-pay Method Total expenditure Non-food expenditure Non-subsistence expenditure Threshold range 5–85% 5–40% 5–85% 5–40% 5–85% 5–40% Dominance relationship (frequency (proportion)) Dominance (restricted) 232 (10.7%) 1082 (50.0%) 584 (27.0%) 1202 (55.6%) 582 (26.9%) 1200 (55.5%) Non-dominance due to insignificance 1352 (62.5%) 658 (30.4%) 830 (38.4%) 466 (21.6%) 838 (38.8%) 478 (22.1%) Non-dominance due to intersections 578 (26.7%) 422 (19.5%) 748 (34.6%) 494 (22.9%) 742 (34.3%) 484 (22.4%) Average length of dominance/Power of test 48.9 27.9 60.1 30.0 59.8 30.0 Dominance (restricted): one catastrophic incidence curve is always statistically above[below] another for a specified range of thresholds Non-dominance due to insignificance: catastrophic incidence curves where the difference between curves is not statistically significant Non-dominance due to intersections: catastrophic incidence curves that intersect and where difference between curves are statistically significant Average length of dominance/Power of test: average continuous threshold range over which dominance was observed; considered an indirect assessment of the overall power to test for dominance consistent assessments). This more than doubled when the proportion of cross-country comparisons resulting in using CTP approaches but still remained low, increasing non-dominance due to intersections was slightly lower and to 26.9% using non-subsistence expenditure and 27.0% similar whether using the budget share approach (19.5%) or using non-food expenditure (582 and 584 out of 2162 either CTP approaches (22.9% and 22.4%). comparisons resulted in consistent assessments, respect- Table 2 also shows the average length of a continuous ively). In other words, at least three-quarters of compari- range of threshold points over which significant t-statistics sons were sensitive to the threshold, resulting in were found according to each method for defining house- inconsistent assessments where either of the two countries hold resources. The length of this interval indicates over was found to have higher and lower levels of financial pro- what threshold range comparisons result in consistent as- tection depending on the threshold or where differences sessments and is indicative of the degree of sensitivity of between two countries were not statistically significant. relative country assessments to methods for defining When assessing sensitivity by further restricting dom- household resources in the denominator, as well as the inance testing to the popular 5–40% threshold range, the power of the dominance test to reject the null hypothesis. average proportion of robust assessments increased to Results showed that CTP approaches appeared to be less approximately half of all comparisons. The budget share sensitive and had greater power than the budget share ap- approach resulted in cross-country comparisons robust proach over the 5–85% threshold range as the average to the choice of the threshold 50.0% of the time, com- threshold range over which dominance or consistent re- pared to the two CTP approaches which resulted in ro- sults were observed was always greater (60.1 and 59.8 bust comparisons 55.6% and 55.7% of the time. Thus, threshold points for non-food and non-subsistence, re- when sensitivity was assessed by restricting dominance spectively; compared with 48.9 threshold points for total testing over the 5–40% threshold range, the choice of expenditures). When dominance tests were further re- method for defining available household resources mat- stricted to the 5–40% range, the CTP approaches still ap- tered less as sensitivities were of similar degrees. peared to be less sensitive and to have greater power than As described in the methods section, cross-country com- the budget share method. However, the difference was di- parisons resulting in non-dominance can be attributed to minished as interval lengths were more similar, ranging either catastrophic incidence curves that intersect or curves from 27.9 to 30.0 threshold points. that differ from one another but not to a statistically signifi- Additional file 3 shows results of dominance tests for cant degree. Intersections give inconsistent assessments of each country for each of the three methods. Fig. 2 is which country has statistically higher[lower] levels of finan- shown here as an example, highlighting results for cial protection depending on the threshold. Insignificances, Pakistan. The x-axis shows the range of thresholds for while less informative in that they are unable to find statisti- when the share of OOP payments for health in household cally significant differences between two countries, do not resources can be considered as catastrophic. The y-axis result in contradictions. Dominance testing over the 5–85% shows pairwise country comparisons between Pakistan threshold range indicated that cross-country comparisons and other countries. Solid bars reflect when Pakistan following the budget share approach resulted in a slightly (Country A) exhibited a statistically lower incidence of lower proportion of inconsistent comparisons or non- catastrophic health expenditures) than another Country B. dominance due to intersections (26.7%) than either of the Dashed bars reflect when Country B exhibited lower inci- two CTP approaches (34.6% and 34.3%) (Table 2). When dence of catastrophe than Pakistan (Country A). The testing was further restricted to the 5–40% threshold range, length of these bars reflects thresholds over which t- Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 8 of 13 Fig. 2 (See legend on next page.) Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 9 of 13 (See figure on previous page.) Fig. 2 Example of sensitivity in cross-country comparisons to the choice of the threshold, observed through dominance. Each line represents a pairwise comparison of the incidence rates of catastrophic health expenditures between Country A and Country B. Countries are ordered by decreasing proportion of the population reporting any OOP. Solid bars indicate Country A dominance as it exhibited lower incidence rates of catastrophic health expenditures compared to Country B for the range of thresholds shown on the horizontal axis. Dashed bars indicate Country B dominance as it exhibited lower incidence rates of catastrophic health expenditures compared to Country A for the range of thresholds shown on the horizontal axis. White bars indicate that the difference between incidence rates of catastrophic health expenditures between Country A and Country B were not statistically significant for the range of thresholds shown on the horizontal axis. For any given pairwise comparison, one can therefore observe for which thresholds Country A has higher[lower] incidence rates of catastrophic health expenditures compared to Country B, and whether such assessments are sensitive to the choice of the threshold (i.e. if the type of bars displayed changes) statistics used for assessing dominance were significant. The correlation was very strong between CTP methods Dominance was observed when a bar is continuously using non-food and non-subsistence expenditure (r = .9963, shown over the full threshold range of interest. Non- p < .05), indicating nearly identical assessments of cross- dominance due to intersecting CDFs was observed in lines country comparisons. In comparison, the correlation for with both types of solid and dashed bars, indicating that each of these CTP methods with the budget share method Pakistan exhibited both lower and higher incidence rates of was moderately strong (r = .7226 and r = .7171, p < .05). catastrophe compared to Country B; and non-dominance due to insignificant CDFs was observed in lines with white space, indicating thresholds over which differences be- Discussion tween Pakistan and Country B were insignificant. Figures This study is the one of the first published analyses to thus show the degree of sensitivity to the choice of the investigate the sensitivity of measuring financial protec- threshold. For example, using the budget share method, tion against catastrophic health expenditures to varying comparisons of Pakistan were insensitive to the threshold methodological choices. These choices relate to the with dominance shown by solid bars observed over threshold used to identify health expenditures as cata- seven countries with higher levels of financial protec- strophic, causing a sacrifice of consumption on other es- tion no matter the threshold over the 5–85% range sential items, and to the definition of living standards or and over 28 countries over the 5–40% range. Some household resources available to pay for health services. sensitivities to the threshold were observed in com- This paper is a methodological not a policy analysis and parisons with Ukraine, Turkey, Laos, Rwanda, Cape thus does not attempt to draw policy insight about the Verde, Zambia, and Armenia with Pakistan observed performance of any one country relative to another – the to have higher incidence rates at lower thresholds unit of analysis is methods of measurement rather than (reflected by dashed bars) but lower incidence rates at countries. In order for comparative assessments to be higher thresholds (reflected by solid bars) for all compari- meaningful and to more effectively draw insight for policy, sons except that with Ukraine where the opposite was ob- it is critical to understand how sensitive measurement is served. As seen here and in Additional file 3,the twoCTP to these choices. methods show almost identical profiles or degrees of sensi- Defining the catastrophic threshold requires a choice. tivities to the threshold. While more recent work has attempted to link this When assessing sensitivity of cross-country comparisons choice to disease outcomes or other factors of clinical to methods for defining household resources, all three relevance [38, 39], the choice of the threshold has also methods were highly sensitive with at least three-quarters of been referred to as arbitrarily defined [6, 40–42]. The comparisons dependent on the threshold (Table 2). Over choice would not be especially problematic if compara- the 5–40% range, comparisons were sensitive for approxi- tive assessments were insensitive to the threshold, but mately half of comparisons with similar degrees of sensitivity our results indicated this was not the case. Across all across methods. Sensitivity was also assessed by estimating methods for measuring catastrophic health expenditures, Spearman’s rank correlation coefficients between country country comparisons were robust to the choice of the rankings of financial protection by each method (Table 3). threshold in only a tenth to a quarter of all comparisons Table 3 Correlation of country rankings of catastrophic health expenditure incidence rates over the 5–40% threshold range Total expenditure Non-food expenditure Non-subsistence expenditure Total expenditure 1.0000 Non-food expenditure 0.7226 1.0000 * * Non-subsistence expenditure 0.7171 0.9963 1.0000 Tested using Spearman’s rank correlation coefficient p < 0.05 Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 10 of 13 within the 5–85% threshold range. If comparisons were are sensitive to this methodological choice. In addition, restricted within the 5–40% threshold range, the propor- financial protection has gradations of coverage rather than tion of assessments insensitive to the threshold increased the simplistic protected or not protected categorisation to approximately half of all comparisons across all offered by a single threshold. Measuring catastrophic methods, with slightly more for CTP approaches. These health expenditures using only one point would result in a results signal a challenge for global monitoring given significant loss of information, failing to capture different that sensitivity reduces the ability to confidently draw re- degrees of hardship. The impact of OOP payments for liable conclusions from cross-country comparisons as, health is not discrete but rather the financial burden they depending on the threshold, a country could be assessed impose lies on a continuum from a very low burden where to perform relatively better and worse than another. the impact is marginal, to a moderate burden where the Regarding the choice for how to define household re- impact may render access to some care unaffordable, to a sources, dominance results revealed that the degree of very high burden where OOP payments cause severe finan- sensitivity of cross-country comparisons using either the cial hardship. Thus, measuring across multiple thresholds budget share or the two CTP approaches were all highly offers a more nuanced picture of the varying intensity of fi- sensitive with almost three-quarters of comparisons sen- nancial hardship due to paying for health by a population. sitive over the 5–85% range. It is important to note that We further recommend that financial protection against some of the sensitivity in CTP approaches may be par- catastrophic health expenditures be measured across a tially due to indicator construction. This is linked to dif- range of thresholds using catastrophic incidence curves as ferences in food spending patterns where populations at shown in this paper. Doing so would provide valuable pol- lower levels of socio-economic status in countries at icy insight as different health financing policies will impact lower levels of economic development will spend pro- different levels of OOP payments. For example, a country portionally more on food then those at higher levels. with specific policies for reducing copayments for in- While this by-product of CTP approaches may lead to patient services will provide more financial protection at intersections in catastrophic incidence curves, it is also higher thresholds of catastrophic health expenditures but why such approaches are sometimes preferred as this not necessarily at lower thresholds. Specific policies tai- property does take into account differences in socio- lored to country contexts might therefore explain why economic levels of development and attaches greater cross-country comparisons of financial protection are so concern to equity. When assessing sensitivity over the sensitive to the threshold. It is important to evaluate how 5–40% range, these were sensitive to a similar degree and why the system produces those patterns – analysing across all methods with approximately half of compari- the pathways and interactions between policy interven- sons sensitive, although CTP approaches were slightly tions and health financing arrangements (e.g. prepayment less so. Correlations across country rankings were also and pooling, definition of the benefit package, cost- moderate across those produced by the budget share sharing arrangements, provider incentives) and how these and both CTP methods and were very strong between influence financial protection. Financial protection is likely the two CTP methods. Thus, the choice of the denomin- to improve if fragmentation in the financing system is re- ator and whether non-discretionary expenditures should duced because risks are then increasingly pooled across be considered part of household resources available to the rich and poor and across the healthy and sick; if the pay for health has less practical implications for meas- definition of a benefit package is better defined to meet urement than the choice of the threshold. Furthermore, population health needs; if cost-sharing arrangements no all sensitivity results from CTP methods using non-food longer allow for balanced billing; if referral systems are and non-subsistence expenditure were nearly identical. strengthened; and if perverse incentives inherent in open- Given the two methods are conceptually related, as both ended fee-for-service provider payment methods are exclude some or all food expenditure from the estimate of addressed. Indeed, the value of measuring financial resources available to spend on health, the non-food protection is to not only assess the impact of OOP method is preferable of the two as it is easier to compute payments for health on living standards but also to under- and understand. The choice to define household resources stand how a country’s health financing system performs in following either the budget share or a CTP method repre- terms of fulfilling an insurance function, linking back to sents an important normative choice. This is because CTP the theoretical foundations of financial protection in approaches are typically motivated by an ethical concern health insurance. Indeed, Wagstaff et al. (2018) found that for the poor, given it recognises that poorer households population coverage by a health insurance scheme is not spend a higher proportion of resources on essential items. strongly associated with the incidence rate of catastrophic We recommend that the measurement of financial pro- health expenditures , further warranting investigation tection against catastrophic health expenditures should into design and implementation issues of insurance not be pinned to a single threshold as country assessments schemes for their influence on financial protection. Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 11 of 13 This study relied on a novel application of the domin- impact of methodological choices by empirically estimat- ance approach to studying distributions of OOP shares on ing the impact of varying methodological choices. health across a range of catastrophic thresholds. Other A first limitation of this analysis is that our analysis methodological studies have focused on either the choice examined only a sample set of 47 surveys. It is import- of threshold or definition of living standards. Two studies ant to keep in mind that the purpose of this analysis by Ataguba et al. (2012)  and by Onoka et al. (2011)  was not to conduct a comparative analysis of country examined implications when a single catastrophic thresh- performance but rather to assess sensitivity of country old was applied uniformly and when it was varied to performance to methodological choices, such that increase as a function of income, questioning the assump- country comparisons are not conclusive but illustrative. tion that the threshold should be constant across rich and The time period of surveys between 2002 and 2012 may poor individuals alike. They found that allowing the be considered as broad, however other global studies on threshold to vary increased estimated levels of cata- financial protection presented analyses covering a simi- strophic health expenditures [6, 7]. While these studies larly broad time period of 10-years [43, 44]. examined the application of the threshold, they did not A second limitation is that the analysis relied on sur- address the percentage at which the threshold should be vey instruments that varied in design. While differences initially set. Other publications have focused on the defin- in survey design (e.g. recall period, number of expend- ition of available household resources [4, 27, 30]. Wagstaff iture items) will influence estimates of expenditures, the and van Doorslaer (2003)  developed and compared in- total effect is unclear . Moreover, the same set of dicators following the budget share and CTP approaches, data from varied survey instruments was consistently discussing their underlying theoretical concepts. Xu et al. used in all sensitivity analysis. It should also be noted (2003) [4, 30] further focused on CTP approaches and pro- that these surveys were all developed by national statis- posed a method motivated by a concern for fairness. These tical offices and are systematically used to collect and studies are noteworthy for developing methods popularly categorise information on household expenditure. Such used today, but the evidence base for understanding the information is then consistently used for calculating impact of varying the catastrophic threshold and the defin- consumer price indices and for measuring poverty rates. ition of household resources is still currently limited. Thus, such variations in survey design are also a com- Key strengths and limitations of this analysis merit dis- mon issue in the global monitoring of other indicators, cussion. This study is one of the first analyses of the sen- including poverty which is also derived from the very sitivity of financial protection measures that relies on same income or expenditure surveys. To date, there is household expenditure surveys across many countries. no established method for standardising survey design Previous studies using expenditure surveys were done features in the collection of data on health spending. to produce global estimates or to assess determinants This study explored methodological choices related to of financial protection but did not examine sensitivity conventional indicators of financial protection popularly across methods [30, 31, 43, 44]. Other multi-country used in the literature and also adopted by international studies with a methodological objective relied on monitoring frameworks . Such indicators do not health-specific surveys [8, 19]. Evidence shows that capture the ex-ante value of reduced risk but rather the expenditure surveys provide more accurate estimation ex-post financial burden faced because of the lack of pro- of OOP payments for health relative to other spend- tection. Despite popular use in the empirical literature, ing than if such data were collected from health- these indicators have been criticised as being too narrow specific surveys . [41, 46] and are likely to underestimate the broader ad- A second strength of this study is that it comprehen- verse effects of OOP payments for health given its focus sively examined the impact of methodological choices in on direct medical costs and exclusion of indirect costs both the threshold and definition of household re- such as those related to transport for accessing services, sources. First, sensitivity to the choice of the threshold loss of income due to illness, or coping strategies. was examined using a dominance approach. Second, sensitivity to the choice of the denominator in OOP Conclusions shares on health was examined through comparisons of Global monitoring of financial protection against cata- dominance results and correlation tests of country strophic health expenditures is challenging because there rankings of financial protection based on different are different measurement methods and no established methods for defining household resources. This study gold standard. To understand the challenge this might thus adds to the evidence base in a way not previously pose in global monitoring, we examined the sensitivity done. Given there is no unanimity on a gold standard of cross-country comparisons to threshold points and to nor understanding as to whether these choices ultim- measures of living standards defined by household re- ately matter, we provide new insight regarding the sources. We found moderate to high levels of sensitivity Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 12 of 13 to the threshold such that drawing policy insight from protection and the authors do not have permission to further distribute this data. Datasets analysed within this study are catalogued by the International cross-country comparisons should be done cautiously as Household Survey Network (http://catalog.ihsn.org/index.php/catalog) which interpretations could be limited if based on a single provides links to repositories for accessing micro-data or contact details of threshold point. The sensitivity of comparisons to the national statistical offices to whom requests for access may be made. definition of household resources was also moderate to Authors’ contributions high but was similar across methods, although CTP JH conceived the study as part of her doctoral degree, performed the analysis approaches showed slightly less sensitivity. Hence, from a of all data, and wrote the first draft of the manuscript. GF contributed to the study design and to the drafting of the manuscript. All authors reviewed measurement perspective, our findings clearly demon- successive drafts and read and approved the final manuscript. strate that the choice of the threshold matters most. More valuable insight for policy can be gained by meas- Authors’ information The findings, interpretations, and conclusions expressed in this paper are uring across a range of thresholds using a catastrophic inci- entirely those of the authors as individuals and should not be attributed in dence curve as shown in this paper. This will allow for any manner to the institutions to which each author is affiliated. assessing whether the financial burden is marginal at lower Ethics approval and consent to participate thresholds of OOP shares on health, and/or more severe at This research was conducted as part of the corresponding author’s doctoral higher thresholds. An area for future research is the appli- degree at the London School of Hygiene and Tropical Medicine. Ethics cation of such a measurement approach over time in a sin- approval was granted by the LSHTM Ethics Committee (Ref: 14349). gle country. In addition, such an evaluation should seek to Consent for publication evaluate not only changes in the catastrophic incidence Not applicable. curve but also how these are linked to specific policy in- Competing interests tervention(s) and their interactions with the underlying The authors declare that they have no competing interests. health financing system. Identifying and assessing in which parts of the catastrophic incidence curve changes Publisher’sNote are occurring and how and why these might be linked to Springer Nature remains neutral with regard to jurisdictional claims in specific changes in health financing arrangements can published maps and institutional affiliations. ultimately help improve the design and implementation of Author details related policies. Indeed, the point of measuring 1 Department of Health Systems Governance and Financing, World Health catastrophic health expenditures is not only to monitor Organization, 20 Avenue Appia, 1211 Geneva, Switzerland. World Bank, 3 Chemin Louis-Dunant, 1202 Geneva, Switzerland. London School of Hygiene the impact of OOP payments on living standards, but to and Tropical Medicine, Keppel Street, London WC1E 7HT, United Kingdom. also evaluate how a country’s health financing system can 4 London School of Hygiene and Tropical Medicine, 15-17 Tavistock Place, improve financial protection. London WC1H 9SH, United Kingdom. Received: 26 September 2017 Accepted: 13 March 2018 Endnotes This will occur for those households with similar References levels of total expenditure where actual food spending 1. United Nations. Resolution adopted by the general assembly on 6 July by the poorer household is just below the fixed subsist- 2017: 71/313 work of the statistical commission pertaining to the 2030 agenda for sustainable development (a/71/L.75). New York: United ence expenditure amount and that by the richer house- Nations; 2017. hold is just above. This is because CTP is defined as 2. OECD, Eurostat. WHO. A system of health accounts 2011. Paris: OECD; 2011. total expenditure net of actual food spending when ac- 3. Wagstaff A, van Doorslaer E. Paying for health care: quantifying fairness, catastrophe, and impoverishment, with applications to Vietnam, 1993–98: tual food spending is less than subsistence expenditure. Washington, DC: World Bank Policy Research Working Paper; 2001. 4. Xu K, Klavus J, Kawabata K, Evans DB, Hanvoravongchai P, Ortiz JP. Chapter Additional files 39: household health system contributions and capacity to pay: definitional, empirical and technical challenges. In: Murray CJ, Evans DB, editors. Geneva: World Health Organization; 2003. Additional file 1: List of surveys. (DOCX 17 kb) 5. Wagstaff A. Measuring financial protection in health. Washington, DC: World Additional file 2: Technical note on the application of restricted Bank; 2008. dominance methods to catastrophic health expenditures. (DOCX 21 kb) 6. Ataguba J. Reassessing catastrophic health-care payments with a Nigerian Additional file 3: Results of restricted dominance tests. (DOCX 3131 kb) case study. Health Econ Policy Law. 2012;7(3):309–26. 7. Onoka CA, Onwujekwe OE, Hanson K, Uzochukwu BS. Examining catastrophic health expenditures at variable thresholds using household Acknowledgements consumption expenditure diaries: catastrophic health expenditure in Not applicable. Nigeria. Tropical Med Int Health. 2011;16(10):1334–41. 8. Flores G, O’Donnell O. Robust measurement of financial protection in Funding health. Lausanne: University of Lausanne, HEC-DEEP; mimeo 2018. Not applicable. 9. Su TT, Kouyaté B, Flessa S. Catastrophic household expenditure for health care in a low-income society: a study from Nouna District. Burkina Faso Bull Availability of data and materials World Health Organ. 2006;84(1):21–7. The data supporting the conclusions of this article was made available to the 10. Knaul FM, Wong R, Arreola-Ornelas H, Harvard University, Global Equity authors by a third party for the specific purpose of monitoring financial Initiative, Fundación Mexicana para la Salud, et al. Household spending and Hsu et al. International Journal for Equity in Health (2018) 17:69 Page 13 of 13 impoverishment. Boston: Harvard Global Equity Initiative; Mexican Health 38. Wingfield T, Boccia D, Tovar M, Gavino A, Zevallos K, Montoya R, et al. Foundation; Distributed by Harvard University Press; 2012. Defining catastrophic costs and comparing their importance for adverse 11. Karan A, Yip W, Mahal A. Extending health insurance to the poor in India: tuberculosis outcome with multi-drug resistance: a prospective cohort an impact evaluation of Rashtriya Swasthya Bima Yojana on out of pocket study, Peru. Ruger JP, editor. PLoS Med. 2014;11(7):e1001675. spending for healthcare. Soc Sci Med. 2017;181:83–92. 39. Rudgard WE, Evans CA, Sweeney S, Wingfield T, Lönnroth K, Barreira D, et al. Comparison of two cash transfer strategies to prevent catastrophic costs for 12. Chen W-H. Comparing low income of Canada’s regions a stochastic dominance poor tuberculosis-affected households in low- and middle-income countries: approach. Ottawa: Statistics Canada; 2008. an economic modelling study. Metcalfe JZ, editor. PLoS Med. 2017;14(11): 13. Koch SF. Out-of-pocket payments on health: trends and dominance e1002418. considerations. Pretoria: University of Pretoria; 2015. 40. MurrayCJL,Evans DB,editors.Healthsystems performance assessment: debates, 14. Arrow K. Uncertainty and the welfare economics of medical care. Am Econ methods and empiricism. Geneva: World Health Organization; 2003. Rev. 1963;53:941–73. 41. Moreno-Serra R, Millett C, Smith PC. Towards improved measurement of 15. Berki SE. A look at catastrophic medical expenses and the poor. Health Aff financial protection in health. PLoS Med. 2011;8(9):e1001087. (Millwood). 1986;5(4):138–45. 42. World Health Organization, World Bank. Tracking universal health coverage: 16. O’Donnell O, Wagstaff A, van Doorslaer E, Lindelow M. Analyzing health first global monitoring report. Geneva: World Health Organization; 2015. equity using household survey data. Washington, DC: The World Bank; 2008. 43. Wagstaff A, Flores G, Hsu J, Smitz M-F, Chepynoga K, Buisman LR, et al. 17. World Health Organization. Distribution of health payments and Progress on catastrophic health spending in 133 countries: a retrospective catastrophic expenditures: methodology. Geneva: World Health observational study. Lancet Glob Health. 2018;6(2):e169–79. Organization; 2005. 44. Wagstaff A, Flores G, Smitz M-F, Hsu J, Chepynoga K, Eozenou P. Progress 18. Pal R. Measuring incidence of catastrophic out-of-pocket health expenditure: on impoverishing health spending in 122 countries: a retrospective with application to India. Int J Health Care Finance Econ. 2012;12(1):63–85. observational study. Lancet Glob Health. 2018;6(2):e180–92. 19. Wagstaff A, Eozenou PH-V. CATA meets IMPOV: a unified approach to 45. Heijink R, Xu K, Saksena P, Evans D. Validity and comparability of out-of- measuring financial protection in health. Washington, DC: The World Bank; pocket health expenditure from household surveys: a review of the 2015. p. 1–41. Report No.: WPS6861 literature and current survey instruments. Geneva: World Health 20. World Health Organization, World Bank. Tracking universal health coverage: Organization; 2011. 2017 global monitoring report. Geneva: World Health Organization; 2017. 46. Ruger JP. An alternative framework for analyzing financial protection in 21. Ravallion M. Poverty comparisons: a guide to concepts and methods: World health. PLoS Med. 2012;9(8):e1001294. Bank; 1992. 22. Deaton A, Zaidi S. Guidelines for constructing consumption aggregates for welfare analysis. Washington, DC: World Bank; 2002. 23. Thomson S, Evetovits T, Cylus J, Jakab M. Monitoring financial protection to assess progress towards universal health coverage in Europe. Public Health Panor. 2016;2(3):357–66. 24. Deaton A, Grosh M. Chapter 17: consumption. In: designing household survey questionnaires for developing countries: lessons from ten years of LSMS experience. Washington, DC: World Bank; 2000. 25. Chen S, Ravallion M. The developing world is poorer than we thought, but no less successful in the fight against poverty. Q J Econ. 2010;125(4):1577–625. 26. Wyszewianski L. Financially catastrophic and high-cost cases: definitions, distinctions, and their implications for policy formulation. Inq J Med Care Organ Provis Financ. 1986;23(4):382–94. 27. Wagstaff A, van Doorslaer E. Catastrophe and impoverishment in paying for health care: with applications to Vietnam 1993–1998. Health Econ. 2003; 12(11):921–33. 28. Ranson MK. Reduction of catastrophic health care expenditures by a community-based health insurance scheme in Gujarat, India: current experiences and challenges. Bull World Health Organ. 2002;80(8):613–21. 29. Ekman B. Catastrophic health payments and health insurance: some counterintuitive evidence from one low-income country. Health Policy Amst Neth. 2007;83(2–3):304–13. 30. Xu K, Evans D, Kawabata K, Zeramdini R, Klavus J, Murray C. Household catastrophic health expenditure: a multicountry analysis. Lancet. 2003; 362(9378):111–7. 31. Xu K, Evans DB, Carrin G, Aguilar-Rivera AM, Musgrove P, Evans T. Protecting households from catastrophic health spending. Health Aff (Millwood). 2007; 26(4):972–83. 32. Chen W-H, Duclos J-Y. Testing for poverty dominance: an application to Canada. Stat Can. 2008;42:109–19. Submit your next manuscript to BioMed Central 33. Bennett CJ. Inference for dominance relations. Int Econ Rev. 2013;54(4): 1309–28. and we will help you at every step: 34. Davidson R, Duclos J-Y. Statistical inference for stochastic dominance and • We accept pre-submission inquiries for the measurement of poverty and inequality. Econometrica. 2000;68(6): 1435–64. � Our selector tool helps you to ﬁnd the most relevant journal 35. Wagstaff A. Measuring financial protection in health. In: Smith PC, Mossialos � We provide round the clock customer support E, Papanicolas I, Leatherman S, editors. Performance measurement for health � Convenient online submission system improvement: experiences, challenges and prospects. Cambridge: Cambridge University Press; 2009. � Thorough peer review 36. Zheng B. Poverty orderings. J Econ Surv. 2000;14(4):427–66. � Inclusion in PubMed and all major indexing services 37. Kaur A, Prakasa Rao BLS, Singh H. Testing for second-order stochastic � Maximum visibility for your research dominance of two distributions. Econom Theory. 1994;10(05):849. Submit your manuscript at www.biomedcentral.com/submit
International Journal for Equity in Health – Springer Journals
Published: May 31, 2018
It’s your single place to instantly
discover and read the research
that matters to you.
Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.
All for just $49/month
Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly
Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.
Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.
Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.
All the latest content is available, no embargo periods.
“Hi guys, I cannot tell you how much I love this resource. Incredible. I really believe you've hit the nail on the head with this site in regards to solving the research-purchase issue.”Daniel C.
“Whoa! It’s like Spotify but for academic articles.”@Phil_Robichaud
“I must say, @deepdyve is a fabulous solution to the independent researcher's problem of #access to #information.”@deepthiw
“My last article couldn't be possible without the platform @deepdyve that makes journal papers cheaper.”@JoseServera