Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Development of the Health Insurance Literacy Measure (HILM): Conceptualizing and Measuring Consumer Ability to Choose and Use Private Health Insurance

Development of the Health Insurance Literacy Measure (HILM): Conceptualizing and Measuring... Journal of Health Communication, 19:225–239, 2014 Published with license by Taylor & Francis ISSN: 1081-0730 print=1087-0415 online DOI: 10.1080/10810730.2014.936568 Development of the Health Insurance Literacy Measure (HILM): Conceptualizing and Measuring Consumer Ability to Choose and Use Private Health Insurance KATHRYN A. PAEZ American Institutes for Research, Health and Social Development Program, Rockville, Maryland, USA CORETTA J. MALLERY, HARMONIJOIE NOEL, AND CHRISTOPHER PUGLIESE American Institutes for Research, Health and Social Development Program, Washington, District of Columbia, USA VERONICA E. MCSORLEY AND JENNIFER L. LUCADO American Institutes for Research, Health and Social Development Program, Rockville, Maryland, USA DEEPA GANACHARI American Institutes for Research, Health and Social Development Program, Washington, District of Columbia, USA Understanding health insurance is central to affording and accessing health care in the United States. Efforts to support consumers in making wise purchasing decisions and using health insurance to their advantage would benefit from the development of a valid and reliable measure to assess health insurance literacy. This article reports on the development of the Health Insurance Literacy Measure (HILM), a self- assessment measure of consumers’ ability to select and use private health insurance. The authors developed a conceptual model of health insurance literacy based on formative research and stakeholder guidance. Survey items were drafted using the conceptual model as a guide then tested in two rounds of cognitive interviews. After a field test with 828 respondents, exploratory factor analysis revealed two HILM scales, choosing health insurance and using health insurance, each of which is divided into a confidence subscale and likelihood of behavior subscale. Correlations between the HILM scales and an objective measure of health insurance # Kathryn A. Paez, Coretta J. Mallery, HarmoniJoie Noel, Christopher Pugliese, Veronica E. McSorley, Jennifer L. Lucado, and Deepa Ganachari This is an Open Access article. Non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly attributed, cited, and is not altered, transformed, or built upon in any way, is permitted. The moral rights of the named author(s) have been asserted. Address correspondence to Kathryn A. Paez, American Institutes for Research, Health and Social Development Program, 6003 Executive Boulevard, Suite 3000, Rockville, MD 20852, USA. E-mail: [email protected] 225 226 K. A. Paez et al. knowledge and skills were positive and statistically significant which supports the validity of the measure. Health insurance is one of the most complex and costly products that consumers purchase and use in their lifetime. The state and federal health insurance market- places were designed to improve access to insurance for the uninsured and to sim- plify selecting and purchasing health insurance in the private marketplace. Because many of the consumers shopping in the marketplaces were expected to have limited previous experience with purchasing health insurance and to face barriers such as low literacy, limited English proficiency, and poorer health in comparison with the general population (PwC Health Research Institute, 2012), policymakers aimed to make the shopping experience as easy as possible (Rosenbaum, 2011). The ease of selecting a suitable health insurance plan and making optimal use of health insurance benefits is influenced by a variety of factors: (a) state regulations that limit health insurance products and marketing practices; (b) the complexity of plan benefits; (c) the quality of support received when selecting or using health insurance; and 4) one’s health insurance-related knowledge and skills. In passing the Affordable Care Act (ACA), policymakers took measures to address the first three factors. For example, the law standardizes plan benefits by requiring new plans to provide a comprehensive package of 10 essential benefits; improves preventive care coverage; and restricts the annual dollar limits insurance companies can impose on essential benefits such as hospital stays (Patient Protection and Affordable Care Act, 2010). These requirements protect the consumer and reduce the burden associa- ted with selecting a health plan. The health insurance marketplaces are required by the ACA to provide public outreach and education programs to inform consumers about their health insurance choices in an accurate and impartial way. Navigator and assister roles were created to support consumers in preparing applications to establish eligibility for insurance affordability programs and to enroll consumers in plans. A number of states simplified the selection process for consumers by limit- ing the number of plans each insurer can offer in their health insurance marketplace (Dash, Lucia, Keith, & Monahan, 2013). As federal and state governments moved forward with implementing the ACA, less attention has been placed on the fourth factor facilitating health insurance selection and use—one’s health insurance-related knowledge and skills. Health insurance literacy,the focus of our research, is defined as ‘‘the degree to which individuals have the knowl- edge, ability, and confidence to find and evaluate information about health plans, select the best plan for their own (or their family’s) financial and health circumstances, and use the plan once enrolled’’ (Quincy, 2012, p. 7). Although health insurance literacy overlaps with health literacy, it is distinguished by the need to understand how health insurance benefits are structured, and to understand and estimate cost sharing respon- sibilities. Both health insurance literacy and health literacy require knowledge about health services and one’s health status, and the ability to use this information to make decisions. Just as health literacy is important to achieving better health outcomes (Berk- man, 2011), health insurance literacy is a factor in whether consumers delay or avoid seeking care due to cost (Dorn, 2011; Morgan et al., 2008) or perceive their overall health as poorer (McCormack, 2009; Morgan et al., 2008). We developed and tested a measure to assess health insurance literacy with the goal of shedding light on consumer confidence and their practice of health insurance literate behaviors when selecting and using health insurance. Data generated from use of the Health Insurance Literacy Measure (HILM) is intended to provide empiri- cal data to help answer such questions as (a) what groups need additional outreach support when enrolling in health insurance and using plan benefits to pay for health Development of the HILM 227 services once enrolled; (b) what facets of choosing and using health insurance are most troublesome for consumers, and (c) on what topics and skills should consumer counseling organizations and health literacy experts concentrate their efforts to better assist consumers in selecting and using health insurance. The HILM was tested as a population- assessment of health literacy. Additional testing would be needed for use in counseling consumers. After approval by the institutional review board, the HILM was developed in four stages. Stage 1. We formulated a health insurance literacy conceptual model that is based on a literature review, consultation with a stakeholder group and advisory panel, interviews with key informants, and focus groups with individuals purchasing health insurance in the private market. Stage 2. Building on the conceptual model domains identified in stage one, we selec- ted, generated and refined a large item pool to operationalize each domain. Stage 3. We conducted cognitive testing of items with consumers who may purchase insurance through the marketplaces in two rounds and revised items after each round of testing. Stage 4. We field tested the items beginning with a pilot test to assess item adequacy, followed by a larger scale test using a population-based sample to develop the scales and establish validity of the measure. New Contribution While health literacy has been widely studied and measured, health insurance literacy has not. Although understanding health insurance is central to affording and accessing health care in the United States, studies of consumers and health insurance have lacked a clear definition of health insurance literacy or a conceptual framework. With- out such a conceptual basis, measurement of health insurance literacy cannot be con- sistently applied for comparison of study findings that lead to the identification of evidenced based practices and health policy recommendations. In reviewing the litera- ture, we identified 33 measurement tools related to health literacy, financial literacy, and insurance literacy (Quincy, 2012). None of these measures of health insurance literacy were both tested for reliability and validity and developed for assessing consu- mers purchasing private health insurance. The one tool that was psychometrically tested assessed Medicare beneficiaries’ knowledge of insurance terms and understand- ing of Medicare benefits using excerpts from a pamphlet the Medicare program cre- ated to inform the public (McCormack, et al., 2009). Four surveys assessed facets of health insurance literacy but were not psychometrically tested. One tool assessed Med- icaid beneficiaries’ comprehension of an official, compared with a simplified version of a Medicaid health plan comparison chart (Greene & Peters, 1981). Two surveys, developed as part of the Rand Health Insurance study, assessed families’ perceptions of the services covered by their insurance and knowledge of health insurance benefits (Marquis, 1981). A more recent study reported results of a survey assessing how well Americans understand and believe they understand traditional health insurance (Lowenstein et al., 2013). The Rand and Lowenstein studies concluded that consumers are more likely to understand simplified health insurance structures. Formative Research The development of the measure began with the convening of a Health Insurance Literacy Roundtable where a diverse group of experts from academia, advocacy, 228 K. A. Paez et al. health plans, and private research firms. The Roundtable was charged with explor- ing the need for the measure. Broad consensus on the critical need for the measure was evident and so the charge turned to laying the groundwork for developing a preliminary concept of what a health insurance literate person is able to do (Quincy, 2012). Literature Review We reviewed published and unpublished papers discussing consumers’ understand- ing of health insurance. Consumer understanding of health insurance terminology and cost sharing, and challenges with insurance-related decision making under dif- ferent conditions were some of the topics covered. The literature review findings indicated that consumers struggle with cost and medical terminology (Quincy, 2010, 2011a, 2011b). In addition, consumers with lower numeracy skills are less able to understand their plans (Green et al., 2008; Wood et al., 2011). Literature findings also show that consumers have difficulty calculating their costs when presented with health plan cost structure information (Kutner et al., 2006; Malbon, 2010; Quincy, 2011a). Several studies found that consumers faced with selecting health insurance have difficulty moving into an active purchasing role to weigh complex information (Carman et al., 2010; Greene & Peters, 2009; Quincy, 2010, 2011b; Wood et al., 2011). Moreover, one article showed that consumers have limited ability to objec- tively weigh risk tradeoffs associated with different deductibles and copays against premium costs (Tennyson, 2011). Other research found that consumers’ self-assessed ease of understanding their health plans is not necessarily consistent with their com- prehension (Greene & Peters, 2009). This review and the Health Insurance Literacy Roundtable report (Quincy, 2012) informed the first draft of our conceptual model of health insurance literacy. The model consisted of four domains: health insurance knowledge, information seeking, document literacy and cognitive skills. Key Informant Interviews We interviewed 20 professionals from a variety of organizations that counsel consumers who are shopping for health insurance (Paez, Goldstein, & Ganachari, 2012). Our interview guide encompassed three areas: (a) what do consumers need to know, think about, and do when selecting and using health insurance; (b) what do consumers tend not to know, think about, or do that leads to problems; and (c) how do health insurance benefit designs impact the difficulty of selecting and using insurance. We also asked the counselors to comment on our draft health insurance literacy model. Insurance counselors reported that consumers fail to understand the underlying purpose of health insurance as a hedge against major medical costs. Consumers are unaware of their personal liability should they become seriously ill. Counselors reported that many consumers do not fully assess their needs or the services the plans cover and cost sharing expenses beyond the monthly premium when purchasing insurance. When using health insurance, consumers are often unaware they need to carefully investigate what providers are in-network. Instead, counselors find that consumers assume their primary care provider will attend to verifying the network status of all providers their primary care provider refers them to and that all pro- fessional services received in an in-network hospital are part of their plan’s network. Figuring out their share of medical costs and the insurance share and navigating deductibles are problematic for consumers. The insurance counselors reviewed the model and confirmed that the four domains that we identified were appropriate. Some of the counselors indicated they informally assessed their client’s health insurance literacy in the course of their discussion with them. Several of the Development of the HILM 229 counselors we interviewed did not feel that asking a consumer to complete a paper- and-pencil assessment tool was practical or necessary. Stakeholder Group We convened five meetings of stakeholders during the measure development process. Stakeholders consisted of experts from consumer assistance organizations, acade- mia, advocacy groups, health plans, and the government. Stakeholders referred us to health insurance counselors that would be knowledgeable key informants to inter- view and provided feedback on a draft conceptual model. At the recommendation of the stakeholder group, we added self-efficacy as an underlying domain to the model and the ability to formulate and articulate questions under the information seeking domain. While a discussion took place about the role of computer literacy in health insurance literacy, the group recommended that computer literacy not be included as a conceptual component of the model because it was not a necessary condition for health insurance literacy. In keeping with this recommendation, the stakeholder group believed that health insurance literacy was distinct from health and financial literacy but that the concepts did overlap. The Health Insurance Literacy Conceptual Model We synthesized information gathered from our formative research then revised our health insurance literacy conceptual model (Figure 1). We retained the four domains of health insurance literacy from our draft model: (a) health insurance knowledge; (b) information seeking; (c) document literacy; and (d) cognitive skills and added self-efficacy as an underlying domain. Scale Development To operationalize the domains identified in Figure 1, we developed two pools of items; one pool of self-assessment health insurance literacy items and a second Figure 1. Health insurance literacy conceptual model. 230 K. A. Paez et al. pool of items to objectively test knowledge and skills. The primary purpose of the objective item set was to validate the self-assessment items which were developed as the basis for the final survey which we identify as the HILM. Although we tried to draw the items from the 33 financial literacy, health literacy and health insurance literacy measures identified in the literature review, we found that most of these items were not applicable to our conceptual model of health insurance literacy. When possible, we adapted or modeled items after existing survey questions; however, most items were developed solely for this survey. A subgroup of stakeholders reviewed draft items commenting on their relevance and suggesting additional content for the creation of new items. A technical advisory panel reviewed the conceptual model and the final item pool, providing suggestions for revising some of the items and response categories prior to cognitive testing. After the stakeholder and advisory panel review, we divided the self-assessment and objective knowledge and skills item sets, into two categories, choosing and using health insurance. Many of the objective items applied to either situation. The 58 self-assessment item set measured likelihood of behavior, level of confidence and level of agreement with statements about health insurance purchasing or using practices and abilities. The objective knowledge and skills item set consisted of 30 multiple-choice items that were scored dichotomously as correct or incorrect. Four response choices were provided with ‘‘not sure’’ as one of the options to reduce the likelihood that respondents would guess when they were unsure of an answer. The types of items included selecting the correct definition of a health insurance term, identifying the correct term from a list after reading a description of a concept and selecting the correct response to a scenario. Cognitive Testing We conducted two rounds of cognitive testing with 18 participants in January and February 2013 and 13 participants in March 2013. We recruited participants between the ages of 21 and 64 years old who were currently insured or had been insured at some time in the past five years. We excluded participants who were insured through government-sponsored programs such as Medicare and the Veterans Administration as their primary health insurance since the focus of the measure is literacy associated with private insurance coverage. We included English- speaking participants with a mix of ages, gender, education, private insurance types, and health insurance use. Because the measure was to be a survey, participants had to have basic reading skills. Participants completed the survey and then a trained interviewer asked questions encouraging participants to verbalize their thought process as they reviewed each question. We assessed whether participants interpreted the items as we intended, the adequacy of the response categories, and the variability of the item responses. After the first round of cognitive testing, the self-assessment items were reduced from 51 items grouped into six sections to 33 items organized into five sections. In the first round of testing, we found that few participants could relate to questions about the specific steps to choosing insurance because they did not consciously engage in those activities when comparing plans. We therefore eliminated a level of agreement scale about what people knew when choosing health insurance, for example knowing how to estimate cost. We revised some of the items to either fit a confidence scale or the likelihood of performing certain actions when selecting or using health insurance. We found that participants engaged with their health insurance infrequently enough that responding to a behavior scale about use was difficult for them to relate to so these items were changed to the likelihood of taking a certain action. Items on confidence using health insurance were retained. Development of the HILM 231 In the first round of cognitive testing, the 30 knowledge and skills items were split into two survey versions to make the length more manageable. After the first round of testing, items were discarded or revised so that there was an even distri- bution of level of difficulty ranging from relatively easy to difficult. When two items were similar, we chose the item that was more clearly understood by respondents as we intended. We revised the remaining items by standardizing language to terms that were more familiar to respondents and improving the clarity of wording where respondents had difficult interpreting the question. The second round of testing consisted of one survey containing 33 items self- assessment and 18 objective, knowledge and skills items. During the second round of testing, we collected additional information on participants’ health insurance his- tory such as type of coverage, the number of plans a participant chose from during their open enrollment period, and whether a participant was enrolled in a staff model HMO. After analysis of the second round of testing, we further revised and elimi- nated repetitious or unnecessary items. The final survey for field testing contained 27 self-assessment and 15 knowledge and skills items. Field Test Sample Participants enrolled in GfK’s online panel, a national probability sample of the U.S. adult population, were surveyed in the field test. Our sample consisted of 828 part- icipants, ages 22 through 64, who were uninsured, enrolled in private insurance, or Medicaid. We included participants who were uninsured and covered by Medicaid because some people shift back and forth between private insurance coverage, Medicaid coverage and uninsurance for reasons such as changes in job status or marital status. A minimum sample size of 10 respondents per item was sought to sup- port factor analysis and psychometric analysis (Nunnally & Bernstein, 1994). A small pilot test was conducted with approximately 90 respondents in May 2013 so that we could review missing response patterns and ensure that the programming was accurate. No changes to the survey were needed. The survey was fielded over two weeks beginning mid-June, 2013. E-mail reminders were sent to non-responders on the third day of the field period to enhance survey completion. After the survey administration, a set of study-specific, poststratification weights were constructed to adjust the distribution of our sample back to the U.S. population from which our sample was drawn. Statistical Analysis We examined item frequency patterns and patterns of missing values to ensure that we had items with a range of difficulties and to identify confusion or reluctance to respond to a particular item. We then reverse-coded responses of all the items so that higher scores represent positive alignment with greater health insurance literacy. SAS 9.2 was used for data storage, cleaning, and preliminary analysis. We conducted confirmatory factor analysis and exploratory factor analysis using Mplus 7 to test model fit and to summarize the self-assessment items into a smaller set of composite measures. We tested the fidelity of the item responses to the health insurance literacy conceptual model by conducting confirmatory factor analysis using structural equation modeling. This was not a satisfactory fit; thus, we conducted an exploratory factor analysis, using the principle factor method with squared multiple correlations as initial communality estimates and Geomin oblique rotation (Muthe´n & Muthe´n, 2012). The number of factors was determined by the 232 K. A. Paez et al. eigenvalues and the interpretability of the rotated factor pattern matrix (Muthe´n& Muthe´n, 2012). This resulted in four underlying factors (Table 2). We estimated the Rasch model (Bond and Fox, 2007; Rasch, 1960) to determine the item difficulty, the standard error of measurement in estimating the item dif- ficulty, the point-measure correlation and fit statistics for each of the four factors derived from the exploratory factor analysis. The item difficulty is an estimate of the underlying difficulty calculated by the number of people in our sample that got an item correct. The item difficulty estimates were transformed from the original logit metric to a 0–100 scale for interpretability where 0 represents the lowest poss- ible health insurance literacy level and 100 represents the highest as measured by the set of items. The Rasch model ranks the difficulty of the items in terms of the like- lihood of being endorsed by respondents. The standard error of measurement is the precision of the item difficulty estimation. The point-measure correlation ranges from 1 to 1 and shows the extent to which responses on each item are related to the item difficulty such that higher positive values indicate more discriminating items. Fit statistics such as infit and outfit are used to assess the unidimensionality requirement of the Rasch model. All Rasch analysis was conducted using Winsteps Rasch model software. To assess the validity of the health insurance literacy measure, we performed two sets of analyses using the objective measure of knowledge and skills as the standard. The score for the objective scale was developed by summing of the number of correct responses. We correlated the self-assessment health insurance literacy scales with the objective measure. A strong, positive association would suggest that the health insurance literacy scales were measuring attitudes and behaviors that were related to higher health insurance knowledge and skills. We performed analysis of variance analysis using individual health insurance literacy items and the objective knowledge and skills scale to test whether the different response categories for the self- assessment items adequately differentiated knowledge and skills. Results Among the 937 respondents who consented to take the survey, 828 cases met the qualification criteria. It is important to the psychometric analysis to have racial minorities and those with low levels of education adequately represented in the sam- ple; however, these groups of people were underrepresented in our sample as a result of nonresponse. To correct for nonresponse, we weighted the sample back to the population estimates to balance the distribution in our data. Table 1 presents the demographics of the unweighted and weighted sample. The mean age of respondents in the unweighted sample was 44.1 years (SE¼ 0.43) and slightly more than half were women. Approximately three quarters of those completing the survey were non- Hispanic White, 9.5% were Black and 5% were Hispanic with the remaining falling into the ‘‘other’’ category (10.1%) or identifying as mixed race (2.8%). Approxi- mately one third of respondents reported achieving a high school degree or less and 35.1% earned less than $50,000. The majority of respondents (66.2%) chose their health insurance plan for themselves or their families. Just over 13% of respondents reported visiting a doctor about once per month or more frequently. The confirmatory factor analysis fit statistics indicated our conceptual model did not fit the data. The comparative fit index was slightly under the cutoff of .95 for model fit (Hu & Bentler, 1999) and the root mean square error of approximation value of .11 exceeded the standard of less than .06 (Kenny, 2003) for model fit. Applying exploratory factor analysis, we identified two categories of self-assessment scales, selecting insurance and using insurance, and four total scales. We labeled the selecting health insurance scales confidence choosing and comparing plans, and the Development of the HILM 233 Table 1. Sample characteristics (n¼ 828) Unweighted Weighted Characteristic n % n % Age, mean years (SE) 828 44.1 (0.45) 828 42.5 (0.43) Female 425 51.3 435 52.58 Race=ethnicity White, non-Hispanic 601 72.58 550 66.53 Hispanic 41 4.95 52 6.32 Black 79 9.54 92 11.14 Mixed race 23 2.78 9 1.10 Other 84 10.14 124 14.92 Education Less than high school degree 43 5.19 67 8.06 High school degree 210 25.36 233 28.10 Some college 238 28.74 239 28.86 Bachelor degree or higher 337 40.70 290 34.98 Income Less than $25,000 109 13.16 113 13.64 $25,000 to $49,999 182 21.98 176 21.30 $50,000 to $74,999 151 18.24 156 18.81 $75,000 and greater 386 46.62 383 46.25 Type of insurance (most familiar with?) PPO 2 293 48.51 291 48.25 HMO 1 169 27.98 172 28.45 Other 21 3.48 18 3.02 Not sure 120 19.87 122 20.15 Chooses plan for self or family No 204 33.83 200 33.16 Yes 399 66.17 403 66.84 Physician visit frequency More than once a month 51 6.16 58 1.03 About once a month 63 7.61 67 8.10 A few times a year 397 47.95 380 45.88 One time a year 171 20.65 167 20.14 None in the past year 141 17.03 147 17.80 using health insurance scales confidence using and being proactive (Table 2). In constructing the scales, we eliminated four items with factor loadings under 0.40 (Stevens, 1992) and dropped one item because it double-loaded on two factors. The Rasch analysis results are presented in Table 2. The item difficulty ranges of the scales are as follows: confidence choosing, 41.04 to 60.07; comparing plans, 46.61 to 54.96; confidence using, 45.55 to 59.03; and being proactive, 38.99 to 61.52. Eighteen self-assessment items fell within the acceptable range of 0.5 to 1.5 for infit and outfit statistics. We dropped one item because it had infit and outfit values greater than 1.5. We retained two items that were slightly greater than the 1.5 criteria because they were substantively important to the measure: ‘‘Look to see which doctors and hospitals are covered in each plan?’’ and ‘‘You know most of the things you need to know about using health insurance.’’ The final HILM resulted in 21 self-assessment items. The point-measure correlations ranged from .78 to .90 for the confidence choos- ing scale and .84 to .94 for comparing plans scale (Table 2). The confidence using scale point-measure correlations ranged from .75 to .87 and .72 to .79 for being proactive. The high correlations indicate there are good discriminating items in 234 Table 2. Scale characteristics Factor Item ANOVA Item loading difficulty SEM Infit Outfit PTMEA F (p value) Choosing insurance: Confidence choosing (a¼ .93) How confident would you feel that you ... Know where to go for help if you had trouble affording health insurance 0.84 60.07 .77 1.56 1.87 .78 3.84 (.01) outside of an employer? Know how to estimate what you would have to pay for your health care 0.83 50.04 .78 .97 .97 .86 13.69 (<.01) needs in the next year, not including emergencies? Know what questions to ask so you can choose the best health plan for 0.90 49.15 .76 .66 .63 .90 11.23 (<.01) you? Know where to find the information you need to choose a health plan if 0.94 47.46 .77 .90 .91 .87 10.01 (<.01) you were not offered insurance through an employer? You understand health insurance terms 0.71 44.80 .80 1.06 1.13 .85 21.29 (<.01) You would choose the health plan that is best for you? 0.81 41.04 .78 .81 .79 .87 9.39 (<.01) Choosing insurance: Comparing plans (a¼ .96) When comparing health plans how likely are you to ... Find out if the plans cover unexpected costs such as hospital stays? 0.84 54.96 .83 .94 .88 .91 29.04 (<.01) Understand what you would have to pay for emergency department visits? 0.98 51.49 .83 .63 .56 .94 33.18 (<.01) Understand what you would have to pay for specialist visits? 0.99 51.13 .84 .83 .77 .92 31.01 (<.01) Understand what you would have to pay for prescription drugs? 0.87 49.81 .84 .91 .88 .91 27.96 (<.01) Find out if you have to meet a deductible for health care services? 0.86 49.45 .83 1.00 .97 .90 38.23 (<.01) Look to see which doctors and hospitals are covered in each plan? 0.74 46.61 .84 1.64 1.59 .84 29.84 (<.01) 235 Using insurance: Confidence using (a¼ .93) How confident are you that .. . You know what to do if your health plan refuses to pay for a service you 0.81 59.03 .76 1.09 .99 .83 5.73 (<.01) think should be covered? You know how to figure out your share of the cost for care, after the 0.92 53.52 .78 .78 .79 .87 7.23 (<.01) health plan pays their share? You know what questions to ask your health plan if you have a coverage 0.79 48.78 .82 .82 .78 .86 6.30 (<.01) problem? You know most of the things you need to know about using health 0.44 47.18 .91 1.54 1.56 .75 12.45 (<.01) insurance. You know how to find out what is and is not covered before you receive a 0.83 45.55 .78 .76 .74 .87 8.00 (<.01) health care service? Using insurance: Being proactive (a¼ .80) When using your health insurance plan, how likely are you to .. . Look to member services to tell you what medical services your health plan 0.49 61.52 .92 1.37 1.33 .72 6.21 (<.01) covers? Look into what your health plan will and will not cover before you get 0.55 51.62 .91 .95 .97 .79 4.05 (<.01) health care services? Review the statements you get from your health plan showing what you 0.75 39.11 .98 .74 .79 .79 25.68 (<.01) owe and what they paid for a service? Find out if a doctor is in-network before you see him=her? 0.80 38.99 .97 .88 .78 .75 25.66 (<.01) Note. SEM is the standard error of measurement in estimation of the item difficulty; PTMEA is the point-measure correlation; a is the Cronbach’s alpha coefficient; and ANOVA is analysis of variance. 236 K. A. Paez et al. Table 3. Correlation of self-assessment measure with health insurance knowledge and skills items (n¼ 738) Health insurance literacy Objective measure measure self-assessment scale (knowledge and skills) p Selecting insurance Confidence choosing 0.18 <.01 Comparing health plans 0.37 <.01 Using insurance Confidence using 0.13 <.01 Being proactive 0.32 <.01 the HILM. All Cronbach’s alpha coefficients were greater than .9, indicating high internal consistency (Table 2). In testing the validity of the measure, analysis of variance results showed that the response categories for the 21 self-assessment items adequately differentiated knowledge and skills (Table 2). The health insurance literacy self-assessment scales were positively correlated with the objective knowledge and skills scale (Table 3). The correlation was greatest for comparing plans (.37) and being proactive (.32). The correlation was not as strong for the two scales that measured confidence: confidence choosing (.18) and confidence using (.13). These findings suggest that greater endorsement of confidence and self-reported behaviors measured by the HILM are likely to be related to actual health insurance knowledge and skills. Discussion The success of the federal and state insurance marketplaces hinges in part on consu- mers’ ability to understand health insurance and make informed decisions. Health insurance literacy is one factor that may determine whether consumers select a suit- able health plan and use health insurance to their best advantage. We sought to develop a measure of health insurance literacy to empirically assess consumers’ abil- ity to carry out cognitive tasks and behaviors that are expected of someone who is health insurance literate. The health insurance environment is complex and much work is needed to improve the clarity, accessibility, and usability of materials and tools provided to consumers and to reduce the complexity of the health plan benefits which consumers are expected to navigate. The data generated by the HILM can assist in identifying what facets of health insurance are most confusing to consumers and the characteristics of groups who would benefit from additional support as they are making health insurance purchasing and use decisions. The measure may also be useful as an evaluative tool in the development and implementation of tools and materials informing consumer, and evidence-based interventions to enhance outreach efforts. We tested the HILM among consumers in a range of coverage scenarios to approximate the U.S. population. The Rasch analysis identified a 23-point range in difficulty, indicating that the HILM assesses the middle range of health insurance literacy and may not distinguish between individuals at the high and low extremes. To assess the full range of difficulty, we could add items; however, this may not be necessary for most purposes. In future work, we plan to examine the range of person-level health insurance literacy scores to distinguish low, medium, and high scores and explore how people cluster on the basis of their demographics and level of health insurance literacy. Additional testing would be beneficial if the measure was to be used to assess health insurance literacy as a tool in a consumer counseling setting. Development of the HILM 237 We found that the data did not fit our conceptual model of health insurance literacy but fell into the broader categories of selecting and using health insurance. This indicates that the items are relevant to more than one domain of the four conceptual domains in our model. The tasks associated with selecting and using health insurance are complex and require applying knowledge while performing skills that are interconnected. We found statistically significant correlations between the HILM scale scores and the objective knowledge and skills scores but the correlations were weak for two of the scales, both of which assessed confidence. Our findings indicate that con- fidence may not be a component of health insurance literacy but could be a moder- ator. Self-efficacy may determine whether someone takes advantage of opportunities to purchase health insurance or seeks out needed health care services with confidence about their coverage. Our formative research indicates that consumers who are unsure of their health insurance coverage may hesitate to obtain a service because of their concerns about the out-of-pocket costs. We also observed that personal characteristics and extent of health care utilization may play a role in how confident a person feels with selecting and using health insurance. In developing the HILM, we chose to create a self-assessment measure, lending the HILM for broader use. Our early research indicates that the HILM could be used as a population-based measure to determine where consumer assistance resources should be allocated and to guide content development for navigator, in-person assistance training programs and consumer materials and tools. As the measure is refined in future research and our knowledge of health insurance literacy grows, we anticipate that data from the HILM may play a role in increasing the attention on restructuring health insurance benefits to be more consumer friendly. Acknowledgments and Funding This research was funded by the Missouri Foundation for Health and the American Institutes for Research. The Health Insurance Literacy Roundtable led by Consu- mer’s Union reaffirmed the need for health insurance literacy measure and provided an initial conceptual framework that served as the foundation for this research. We are grateful for the guidance provided by the Technical Expert Panel and the Health Insurance Literacy Stakeholder Group who provided guidance as we developed and tested the health insurance literacy measure. References Berkman, N. D., Sheridan, S. L., Donahue, K. E., Halpern, D. J., & Crotty, K. (2011). Low health insurance literacy and health outcomes: An updated systematic review. Annals of Internal Medicine, 155, 97–107. Bond, T., & Fox, C. M. (2007). Applying the Rasch model (2nd ed.). New York, NY: Routledge. Carman, K. L., Mauer, M., Yegian, J. M., Dardess, P., McGee, J., Evers, M., & Marlo, K. O. (2010). Evidence that consumers are skeptical about evidenced based health care. Health Affairs, 29, 1400–1406. Congressional Budget Office. (2014). Effects on health insurance and the federal budget for the insurance provisions in the Affordable Care Act—May 2013 baseline. Retrieved from http://www.cbo.gov/publication/44190 Dash, S., Lucia, K. W., Keith, K., & Monahan, C. M. (2013). Implementing the Affordable Care Act: Key design decisions for state-based exchanges. Washington, DC: The Commonwealth Fund. Retrieved from http://www.commonwealthfund.org/ Publications/Fund-Reports/2013/Jul/Design-Decisions-for-Exchanges.aspx 238 K. A. Paez et al. Dorn, S. (2011). Implementing national health reform: A five-part strategy for reaching the eligible uninsured. Washington, DC: Urban Institute. Retrieved from http://www. urban.org/publications/412335.html Greene, J., & Peters, E. (2009). Medicaid consumers and informed decision-making. Health Care Financing Review, 30, 25–40. Greenwald, L. M., McCormack, L. A., Uhrig, J. D., & West, N. (2006). Measures and predictors of Medicare knowledge: A review of the literature. Health Care Financing Review, 27, 1–12. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. Kutner, M., Greenberg, E., Jin, Y., & Paulson, C. (2006). The health literacy of America’s adults: Results from the 2003 National Assessment of Adult Literacy (NCES 2006–483). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Lowenstein, G., Friedman, J. Y., McGill, B., Ahmad, S., Linck, S., Sinkula, S., ... Volpp, K. G. (2013). Consumer’s misunderstanding of health insurance. Journal of Health Economics, 32, 850–862. Malbon, J. (2011). America’s Health Insurance Plans & Blue Cross Blue Shield Association: Focus group summary. Seattle, WA: JKM Research. Retrieved from http://www.naic. org/documents/committees_b_consumer_information_110603_ahip_bcbsa_consumer_ testing.pdf Marquis, M. S. (1981). Consumers’ knowledge about their health insurance coverage. Santa Monica, CA: RAND Corporation. McCormack, L., Bann, C., Uhrig, J., Berkman, N., & Rudd, R. (2009). Health insurance literacy of older adults. The Journal of Consumer Affairs, 43, 223–248. Morgan, R. O., Teal, C. R., Hasche, J. C., Petersen, L. A., Byrne, M. M., Paterniti, D. A., & Virnig, B. A. (2008). Does poorer familiarity with Medicare translate into worse access to health care? Journal of the American Geriatrics Society, 56, 2053–2060. Muthe´n, L. K., & Muthe´n, B. O. (2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Authors. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill. Paez, K. A., Goldstein, J., & Ganacheri, D. (2013). Developing a measure of health insurance literacy: Understanding consumers’ ability to choose and use insurance. American Institutes for Research. Retrieved from http://www.air.org/files/Health_Insurance_Literacy_Issues_ Brief_2.25.13.pdf Patient Protection and Affordable Care Act. (2010). Pub. L. No. 111–148, 124 Stat. 119. PwC Health Research Institute. (2012). Health insurance exchanges: Long on options, short on time. Retrieved from http://pwchealth.com/cgi-local/hregister.cgi/reg/pwc-health- insurance-exchanges-impact-and-options.pdf Quincy, L. (2010). Early consumer testing of new health insurance disclosure forms: Washington D.C. Consumer Union & People Talk Research. Retrieved from http:// yourhealthsecurity.org/document/report-early-consumer-testing-of-new-health-insurance- disclosure-forms-consumers-union-and-people-talk-research Quincy, L. (2011a). Early consumer testing of the coverage facts label: A new way of comparing health insurance. Washington, DC: Consumer Union & Kleinman Communication Group. Retrieved from http://consumersunion.org/wp-content/uploads/2013/03/CU_ CFL_Report_FINAL.pdf Quincy, L. (2011b). Early consumer testing of actuarial value concepts. Washington, DC. Consumer Union & Kleinman Communication Group. Retrieved from http:// yourhealthsecurity.org/document/report-early-consumer-testing-of-actuarial-value-concepts Qunicy, L. (2012). Measuring health insurance literacy: A call to action. Retrieved from http:// consumersunion.org/research/measuring-health-insurance-literacy-a-call-to-action Development of the HILM 239 Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research. Chicago, IL: The University of Chicago Press. Rosenbaum, S. (2011). The Patient Protection and Affordable Care Act: Implications for public health policy and practice. Public Health Reports, 126, 130–135. Tennyson, S. (2011). Consumers’ insurance literacy. Terre Haute, IN: Networks Financial Institute at Indiana State University. Retrieved from http://papers.ssrn.com/sol3/ papers.cfm?abstract_id=1896316 Wood, S., Hanoch, Y., Barnes, A., Liu, P. J., Cummings, J., Bhattacharya, C., & Rice, T. (2011). Numeracy and Medicare Part D: The importance of choice and literacy for num- bers in optimizing decision making for Medicare’s prescription drug program. Psychology and Aging, 26, 295–307. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Health Communication Pubmed Central

Development of the Health Insurance Literacy Measure (HILM): Conceptualizing and Measuring Consumer Ability to Choose and Use Private Health Insurance

Loading next page...
 
/lp/pubmed-central/development-of-the-health-insurance-literacy-measure-hilm-H0zuqhG2in

References (61)

Publisher
Pubmed Central
Copyright
Copyright Kathryn A. Paez, Coretta J. Mallery, HarmoniJoie Noel, Christopher Pugliese, Veronica E. McSorley, Jennifer L. Lucado, and Deepa Ganachari
ISSN
1081-0730
eISSN
1087-0415
DOI
10.1080/10810730.2014.936568
Publisher site
See Article on Publisher Site

Abstract

Journal of Health Communication, 19:225–239, 2014 Published with license by Taylor & Francis ISSN: 1081-0730 print=1087-0415 online DOI: 10.1080/10810730.2014.936568 Development of the Health Insurance Literacy Measure (HILM): Conceptualizing and Measuring Consumer Ability to Choose and Use Private Health Insurance KATHRYN A. PAEZ American Institutes for Research, Health and Social Development Program, Rockville, Maryland, USA CORETTA J. MALLERY, HARMONIJOIE NOEL, AND CHRISTOPHER PUGLIESE American Institutes for Research, Health and Social Development Program, Washington, District of Columbia, USA VERONICA E. MCSORLEY AND JENNIFER L. LUCADO American Institutes for Research, Health and Social Development Program, Rockville, Maryland, USA DEEPA GANACHARI American Institutes for Research, Health and Social Development Program, Washington, District of Columbia, USA Understanding health insurance is central to affording and accessing health care in the United States. Efforts to support consumers in making wise purchasing decisions and using health insurance to their advantage would benefit from the development of a valid and reliable measure to assess health insurance literacy. This article reports on the development of the Health Insurance Literacy Measure (HILM), a self- assessment measure of consumers’ ability to select and use private health insurance. The authors developed a conceptual model of health insurance literacy based on formative research and stakeholder guidance. Survey items were drafted using the conceptual model as a guide then tested in two rounds of cognitive interviews. After a field test with 828 respondents, exploratory factor analysis revealed two HILM scales, choosing health insurance and using health insurance, each of which is divided into a confidence subscale and likelihood of behavior subscale. Correlations between the HILM scales and an objective measure of health insurance # Kathryn A. Paez, Coretta J. Mallery, HarmoniJoie Noel, Christopher Pugliese, Veronica E. McSorley, Jennifer L. Lucado, and Deepa Ganachari This is an Open Access article. Non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly attributed, cited, and is not altered, transformed, or built upon in any way, is permitted. The moral rights of the named author(s) have been asserted. Address correspondence to Kathryn A. Paez, American Institutes for Research, Health and Social Development Program, 6003 Executive Boulevard, Suite 3000, Rockville, MD 20852, USA. E-mail: [email protected] 225 226 K. A. Paez et al. knowledge and skills were positive and statistically significant which supports the validity of the measure. Health insurance is one of the most complex and costly products that consumers purchase and use in their lifetime. The state and federal health insurance market- places were designed to improve access to insurance for the uninsured and to sim- plify selecting and purchasing health insurance in the private marketplace. Because many of the consumers shopping in the marketplaces were expected to have limited previous experience with purchasing health insurance and to face barriers such as low literacy, limited English proficiency, and poorer health in comparison with the general population (PwC Health Research Institute, 2012), policymakers aimed to make the shopping experience as easy as possible (Rosenbaum, 2011). The ease of selecting a suitable health insurance plan and making optimal use of health insurance benefits is influenced by a variety of factors: (a) state regulations that limit health insurance products and marketing practices; (b) the complexity of plan benefits; (c) the quality of support received when selecting or using health insurance; and 4) one’s health insurance-related knowledge and skills. In passing the Affordable Care Act (ACA), policymakers took measures to address the first three factors. For example, the law standardizes plan benefits by requiring new plans to provide a comprehensive package of 10 essential benefits; improves preventive care coverage; and restricts the annual dollar limits insurance companies can impose on essential benefits such as hospital stays (Patient Protection and Affordable Care Act, 2010). These requirements protect the consumer and reduce the burden associa- ted with selecting a health plan. The health insurance marketplaces are required by the ACA to provide public outreach and education programs to inform consumers about their health insurance choices in an accurate and impartial way. Navigator and assister roles were created to support consumers in preparing applications to establish eligibility for insurance affordability programs and to enroll consumers in plans. A number of states simplified the selection process for consumers by limit- ing the number of plans each insurer can offer in their health insurance marketplace (Dash, Lucia, Keith, & Monahan, 2013). As federal and state governments moved forward with implementing the ACA, less attention has been placed on the fourth factor facilitating health insurance selection and use—one’s health insurance-related knowledge and skills. Health insurance literacy,the focus of our research, is defined as ‘‘the degree to which individuals have the knowl- edge, ability, and confidence to find and evaluate information about health plans, select the best plan for their own (or their family’s) financial and health circumstances, and use the plan once enrolled’’ (Quincy, 2012, p. 7). Although health insurance literacy overlaps with health literacy, it is distinguished by the need to understand how health insurance benefits are structured, and to understand and estimate cost sharing respon- sibilities. Both health insurance literacy and health literacy require knowledge about health services and one’s health status, and the ability to use this information to make decisions. Just as health literacy is important to achieving better health outcomes (Berk- man, 2011), health insurance literacy is a factor in whether consumers delay or avoid seeking care due to cost (Dorn, 2011; Morgan et al., 2008) or perceive their overall health as poorer (McCormack, 2009; Morgan et al., 2008). We developed and tested a measure to assess health insurance literacy with the goal of shedding light on consumer confidence and their practice of health insurance literate behaviors when selecting and using health insurance. Data generated from use of the Health Insurance Literacy Measure (HILM) is intended to provide empiri- cal data to help answer such questions as (a) what groups need additional outreach support when enrolling in health insurance and using plan benefits to pay for health Development of the HILM 227 services once enrolled; (b) what facets of choosing and using health insurance are most troublesome for consumers, and (c) on what topics and skills should consumer counseling organizations and health literacy experts concentrate their efforts to better assist consumers in selecting and using health insurance. The HILM was tested as a population- assessment of health literacy. Additional testing would be needed for use in counseling consumers. After approval by the institutional review board, the HILM was developed in four stages. Stage 1. We formulated a health insurance literacy conceptual model that is based on a literature review, consultation with a stakeholder group and advisory panel, interviews with key informants, and focus groups with individuals purchasing health insurance in the private market. Stage 2. Building on the conceptual model domains identified in stage one, we selec- ted, generated and refined a large item pool to operationalize each domain. Stage 3. We conducted cognitive testing of items with consumers who may purchase insurance through the marketplaces in two rounds and revised items after each round of testing. Stage 4. We field tested the items beginning with a pilot test to assess item adequacy, followed by a larger scale test using a population-based sample to develop the scales and establish validity of the measure. New Contribution While health literacy has been widely studied and measured, health insurance literacy has not. Although understanding health insurance is central to affording and accessing health care in the United States, studies of consumers and health insurance have lacked a clear definition of health insurance literacy or a conceptual framework. With- out such a conceptual basis, measurement of health insurance literacy cannot be con- sistently applied for comparison of study findings that lead to the identification of evidenced based practices and health policy recommendations. In reviewing the litera- ture, we identified 33 measurement tools related to health literacy, financial literacy, and insurance literacy (Quincy, 2012). None of these measures of health insurance literacy were both tested for reliability and validity and developed for assessing consu- mers purchasing private health insurance. The one tool that was psychometrically tested assessed Medicare beneficiaries’ knowledge of insurance terms and understand- ing of Medicare benefits using excerpts from a pamphlet the Medicare program cre- ated to inform the public (McCormack, et al., 2009). Four surveys assessed facets of health insurance literacy but were not psychometrically tested. One tool assessed Med- icaid beneficiaries’ comprehension of an official, compared with a simplified version of a Medicaid health plan comparison chart (Greene & Peters, 1981). Two surveys, developed as part of the Rand Health Insurance study, assessed families’ perceptions of the services covered by their insurance and knowledge of health insurance benefits (Marquis, 1981). A more recent study reported results of a survey assessing how well Americans understand and believe they understand traditional health insurance (Lowenstein et al., 2013). The Rand and Lowenstein studies concluded that consumers are more likely to understand simplified health insurance structures. Formative Research The development of the measure began with the convening of a Health Insurance Literacy Roundtable where a diverse group of experts from academia, advocacy, 228 K. A. Paez et al. health plans, and private research firms. The Roundtable was charged with explor- ing the need for the measure. Broad consensus on the critical need for the measure was evident and so the charge turned to laying the groundwork for developing a preliminary concept of what a health insurance literate person is able to do (Quincy, 2012). Literature Review We reviewed published and unpublished papers discussing consumers’ understand- ing of health insurance. Consumer understanding of health insurance terminology and cost sharing, and challenges with insurance-related decision making under dif- ferent conditions were some of the topics covered. The literature review findings indicated that consumers struggle with cost and medical terminology (Quincy, 2010, 2011a, 2011b). In addition, consumers with lower numeracy skills are less able to understand their plans (Green et al., 2008; Wood et al., 2011). Literature findings also show that consumers have difficulty calculating their costs when presented with health plan cost structure information (Kutner et al., 2006; Malbon, 2010; Quincy, 2011a). Several studies found that consumers faced with selecting health insurance have difficulty moving into an active purchasing role to weigh complex information (Carman et al., 2010; Greene & Peters, 2009; Quincy, 2010, 2011b; Wood et al., 2011). Moreover, one article showed that consumers have limited ability to objec- tively weigh risk tradeoffs associated with different deductibles and copays against premium costs (Tennyson, 2011). Other research found that consumers’ self-assessed ease of understanding their health plans is not necessarily consistent with their com- prehension (Greene & Peters, 2009). This review and the Health Insurance Literacy Roundtable report (Quincy, 2012) informed the first draft of our conceptual model of health insurance literacy. The model consisted of four domains: health insurance knowledge, information seeking, document literacy and cognitive skills. Key Informant Interviews We interviewed 20 professionals from a variety of organizations that counsel consumers who are shopping for health insurance (Paez, Goldstein, & Ganachari, 2012). Our interview guide encompassed three areas: (a) what do consumers need to know, think about, and do when selecting and using health insurance; (b) what do consumers tend not to know, think about, or do that leads to problems; and (c) how do health insurance benefit designs impact the difficulty of selecting and using insurance. We also asked the counselors to comment on our draft health insurance literacy model. Insurance counselors reported that consumers fail to understand the underlying purpose of health insurance as a hedge against major medical costs. Consumers are unaware of their personal liability should they become seriously ill. Counselors reported that many consumers do not fully assess their needs or the services the plans cover and cost sharing expenses beyond the monthly premium when purchasing insurance. When using health insurance, consumers are often unaware they need to carefully investigate what providers are in-network. Instead, counselors find that consumers assume their primary care provider will attend to verifying the network status of all providers their primary care provider refers them to and that all pro- fessional services received in an in-network hospital are part of their plan’s network. Figuring out their share of medical costs and the insurance share and navigating deductibles are problematic for consumers. The insurance counselors reviewed the model and confirmed that the four domains that we identified were appropriate. Some of the counselors indicated they informally assessed their client’s health insurance literacy in the course of their discussion with them. Several of the Development of the HILM 229 counselors we interviewed did not feel that asking a consumer to complete a paper- and-pencil assessment tool was practical or necessary. Stakeholder Group We convened five meetings of stakeholders during the measure development process. Stakeholders consisted of experts from consumer assistance organizations, acade- mia, advocacy groups, health plans, and the government. Stakeholders referred us to health insurance counselors that would be knowledgeable key informants to inter- view and provided feedback on a draft conceptual model. At the recommendation of the stakeholder group, we added self-efficacy as an underlying domain to the model and the ability to formulate and articulate questions under the information seeking domain. While a discussion took place about the role of computer literacy in health insurance literacy, the group recommended that computer literacy not be included as a conceptual component of the model because it was not a necessary condition for health insurance literacy. In keeping with this recommendation, the stakeholder group believed that health insurance literacy was distinct from health and financial literacy but that the concepts did overlap. The Health Insurance Literacy Conceptual Model We synthesized information gathered from our formative research then revised our health insurance literacy conceptual model (Figure 1). We retained the four domains of health insurance literacy from our draft model: (a) health insurance knowledge; (b) information seeking; (c) document literacy; and (d) cognitive skills and added self-efficacy as an underlying domain. Scale Development To operationalize the domains identified in Figure 1, we developed two pools of items; one pool of self-assessment health insurance literacy items and a second Figure 1. Health insurance literacy conceptual model. 230 K. A. Paez et al. pool of items to objectively test knowledge and skills. The primary purpose of the objective item set was to validate the self-assessment items which were developed as the basis for the final survey which we identify as the HILM. Although we tried to draw the items from the 33 financial literacy, health literacy and health insurance literacy measures identified in the literature review, we found that most of these items were not applicable to our conceptual model of health insurance literacy. When possible, we adapted or modeled items after existing survey questions; however, most items were developed solely for this survey. A subgroup of stakeholders reviewed draft items commenting on their relevance and suggesting additional content for the creation of new items. A technical advisory panel reviewed the conceptual model and the final item pool, providing suggestions for revising some of the items and response categories prior to cognitive testing. After the stakeholder and advisory panel review, we divided the self-assessment and objective knowledge and skills item sets, into two categories, choosing and using health insurance. Many of the objective items applied to either situation. The 58 self-assessment item set measured likelihood of behavior, level of confidence and level of agreement with statements about health insurance purchasing or using practices and abilities. The objective knowledge and skills item set consisted of 30 multiple-choice items that were scored dichotomously as correct or incorrect. Four response choices were provided with ‘‘not sure’’ as one of the options to reduce the likelihood that respondents would guess when they were unsure of an answer. The types of items included selecting the correct definition of a health insurance term, identifying the correct term from a list after reading a description of a concept and selecting the correct response to a scenario. Cognitive Testing We conducted two rounds of cognitive testing with 18 participants in January and February 2013 and 13 participants in March 2013. We recruited participants between the ages of 21 and 64 years old who were currently insured or had been insured at some time in the past five years. We excluded participants who were insured through government-sponsored programs such as Medicare and the Veterans Administration as their primary health insurance since the focus of the measure is literacy associated with private insurance coverage. We included English- speaking participants with a mix of ages, gender, education, private insurance types, and health insurance use. Because the measure was to be a survey, participants had to have basic reading skills. Participants completed the survey and then a trained interviewer asked questions encouraging participants to verbalize their thought process as they reviewed each question. We assessed whether participants interpreted the items as we intended, the adequacy of the response categories, and the variability of the item responses. After the first round of cognitive testing, the self-assessment items were reduced from 51 items grouped into six sections to 33 items organized into five sections. In the first round of testing, we found that few participants could relate to questions about the specific steps to choosing insurance because they did not consciously engage in those activities when comparing plans. We therefore eliminated a level of agreement scale about what people knew when choosing health insurance, for example knowing how to estimate cost. We revised some of the items to either fit a confidence scale or the likelihood of performing certain actions when selecting or using health insurance. We found that participants engaged with their health insurance infrequently enough that responding to a behavior scale about use was difficult for them to relate to so these items were changed to the likelihood of taking a certain action. Items on confidence using health insurance were retained. Development of the HILM 231 In the first round of cognitive testing, the 30 knowledge and skills items were split into two survey versions to make the length more manageable. After the first round of testing, items were discarded or revised so that there was an even distri- bution of level of difficulty ranging from relatively easy to difficult. When two items were similar, we chose the item that was more clearly understood by respondents as we intended. We revised the remaining items by standardizing language to terms that were more familiar to respondents and improving the clarity of wording where respondents had difficult interpreting the question. The second round of testing consisted of one survey containing 33 items self- assessment and 18 objective, knowledge and skills items. During the second round of testing, we collected additional information on participants’ health insurance his- tory such as type of coverage, the number of plans a participant chose from during their open enrollment period, and whether a participant was enrolled in a staff model HMO. After analysis of the second round of testing, we further revised and elimi- nated repetitious or unnecessary items. The final survey for field testing contained 27 self-assessment and 15 knowledge and skills items. Field Test Sample Participants enrolled in GfK’s online panel, a national probability sample of the U.S. adult population, were surveyed in the field test. Our sample consisted of 828 part- icipants, ages 22 through 64, who were uninsured, enrolled in private insurance, or Medicaid. We included participants who were uninsured and covered by Medicaid because some people shift back and forth between private insurance coverage, Medicaid coverage and uninsurance for reasons such as changes in job status or marital status. A minimum sample size of 10 respondents per item was sought to sup- port factor analysis and psychometric analysis (Nunnally & Bernstein, 1994). A small pilot test was conducted with approximately 90 respondents in May 2013 so that we could review missing response patterns and ensure that the programming was accurate. No changes to the survey were needed. The survey was fielded over two weeks beginning mid-June, 2013. E-mail reminders were sent to non-responders on the third day of the field period to enhance survey completion. After the survey administration, a set of study-specific, poststratification weights were constructed to adjust the distribution of our sample back to the U.S. population from which our sample was drawn. Statistical Analysis We examined item frequency patterns and patterns of missing values to ensure that we had items with a range of difficulties and to identify confusion or reluctance to respond to a particular item. We then reverse-coded responses of all the items so that higher scores represent positive alignment with greater health insurance literacy. SAS 9.2 was used for data storage, cleaning, and preliminary analysis. We conducted confirmatory factor analysis and exploratory factor analysis using Mplus 7 to test model fit and to summarize the self-assessment items into a smaller set of composite measures. We tested the fidelity of the item responses to the health insurance literacy conceptual model by conducting confirmatory factor analysis using structural equation modeling. This was not a satisfactory fit; thus, we conducted an exploratory factor analysis, using the principle factor method with squared multiple correlations as initial communality estimates and Geomin oblique rotation (Muthe´n & Muthe´n, 2012). The number of factors was determined by the 232 K. A. Paez et al. eigenvalues and the interpretability of the rotated factor pattern matrix (Muthe´n& Muthe´n, 2012). This resulted in four underlying factors (Table 2). We estimated the Rasch model (Bond and Fox, 2007; Rasch, 1960) to determine the item difficulty, the standard error of measurement in estimating the item dif- ficulty, the point-measure correlation and fit statistics for each of the four factors derived from the exploratory factor analysis. The item difficulty is an estimate of the underlying difficulty calculated by the number of people in our sample that got an item correct. The item difficulty estimates were transformed from the original logit metric to a 0–100 scale for interpretability where 0 represents the lowest poss- ible health insurance literacy level and 100 represents the highest as measured by the set of items. The Rasch model ranks the difficulty of the items in terms of the like- lihood of being endorsed by respondents. The standard error of measurement is the precision of the item difficulty estimation. The point-measure correlation ranges from 1 to 1 and shows the extent to which responses on each item are related to the item difficulty such that higher positive values indicate more discriminating items. Fit statistics such as infit and outfit are used to assess the unidimensionality requirement of the Rasch model. All Rasch analysis was conducted using Winsteps Rasch model software. To assess the validity of the health insurance literacy measure, we performed two sets of analyses using the objective measure of knowledge and skills as the standard. The score for the objective scale was developed by summing of the number of correct responses. We correlated the self-assessment health insurance literacy scales with the objective measure. A strong, positive association would suggest that the health insurance literacy scales were measuring attitudes and behaviors that were related to higher health insurance knowledge and skills. We performed analysis of variance analysis using individual health insurance literacy items and the objective knowledge and skills scale to test whether the different response categories for the self- assessment items adequately differentiated knowledge and skills. Results Among the 937 respondents who consented to take the survey, 828 cases met the qualification criteria. It is important to the psychometric analysis to have racial minorities and those with low levels of education adequately represented in the sam- ple; however, these groups of people were underrepresented in our sample as a result of nonresponse. To correct for nonresponse, we weighted the sample back to the population estimates to balance the distribution in our data. Table 1 presents the demographics of the unweighted and weighted sample. The mean age of respondents in the unweighted sample was 44.1 years (SE¼ 0.43) and slightly more than half were women. Approximately three quarters of those completing the survey were non- Hispanic White, 9.5% were Black and 5% were Hispanic with the remaining falling into the ‘‘other’’ category (10.1%) or identifying as mixed race (2.8%). Approxi- mately one third of respondents reported achieving a high school degree or less and 35.1% earned less than $50,000. The majority of respondents (66.2%) chose their health insurance plan for themselves or their families. Just over 13% of respondents reported visiting a doctor about once per month or more frequently. The confirmatory factor analysis fit statistics indicated our conceptual model did not fit the data. The comparative fit index was slightly under the cutoff of .95 for model fit (Hu & Bentler, 1999) and the root mean square error of approximation value of .11 exceeded the standard of less than .06 (Kenny, 2003) for model fit. Applying exploratory factor analysis, we identified two categories of self-assessment scales, selecting insurance and using insurance, and four total scales. We labeled the selecting health insurance scales confidence choosing and comparing plans, and the Development of the HILM 233 Table 1. Sample characteristics (n¼ 828) Unweighted Weighted Characteristic n % n % Age, mean years (SE) 828 44.1 (0.45) 828 42.5 (0.43) Female 425 51.3 435 52.58 Race=ethnicity White, non-Hispanic 601 72.58 550 66.53 Hispanic 41 4.95 52 6.32 Black 79 9.54 92 11.14 Mixed race 23 2.78 9 1.10 Other 84 10.14 124 14.92 Education Less than high school degree 43 5.19 67 8.06 High school degree 210 25.36 233 28.10 Some college 238 28.74 239 28.86 Bachelor degree or higher 337 40.70 290 34.98 Income Less than $25,000 109 13.16 113 13.64 $25,000 to $49,999 182 21.98 176 21.30 $50,000 to $74,999 151 18.24 156 18.81 $75,000 and greater 386 46.62 383 46.25 Type of insurance (most familiar with?) PPO 2 293 48.51 291 48.25 HMO 1 169 27.98 172 28.45 Other 21 3.48 18 3.02 Not sure 120 19.87 122 20.15 Chooses plan for self or family No 204 33.83 200 33.16 Yes 399 66.17 403 66.84 Physician visit frequency More than once a month 51 6.16 58 1.03 About once a month 63 7.61 67 8.10 A few times a year 397 47.95 380 45.88 One time a year 171 20.65 167 20.14 None in the past year 141 17.03 147 17.80 using health insurance scales confidence using and being proactive (Table 2). In constructing the scales, we eliminated four items with factor loadings under 0.40 (Stevens, 1992) and dropped one item because it double-loaded on two factors. The Rasch analysis results are presented in Table 2. The item difficulty ranges of the scales are as follows: confidence choosing, 41.04 to 60.07; comparing plans, 46.61 to 54.96; confidence using, 45.55 to 59.03; and being proactive, 38.99 to 61.52. Eighteen self-assessment items fell within the acceptable range of 0.5 to 1.5 for infit and outfit statistics. We dropped one item because it had infit and outfit values greater than 1.5. We retained two items that were slightly greater than the 1.5 criteria because they were substantively important to the measure: ‘‘Look to see which doctors and hospitals are covered in each plan?’’ and ‘‘You know most of the things you need to know about using health insurance.’’ The final HILM resulted in 21 self-assessment items. The point-measure correlations ranged from .78 to .90 for the confidence choos- ing scale and .84 to .94 for comparing plans scale (Table 2). The confidence using scale point-measure correlations ranged from .75 to .87 and .72 to .79 for being proactive. The high correlations indicate there are good discriminating items in 234 Table 2. Scale characteristics Factor Item ANOVA Item loading difficulty SEM Infit Outfit PTMEA F (p value) Choosing insurance: Confidence choosing (a¼ .93) How confident would you feel that you ... Know where to go for help if you had trouble affording health insurance 0.84 60.07 .77 1.56 1.87 .78 3.84 (.01) outside of an employer? Know how to estimate what you would have to pay for your health care 0.83 50.04 .78 .97 .97 .86 13.69 (<.01) needs in the next year, not including emergencies? Know what questions to ask so you can choose the best health plan for 0.90 49.15 .76 .66 .63 .90 11.23 (<.01) you? Know where to find the information you need to choose a health plan if 0.94 47.46 .77 .90 .91 .87 10.01 (<.01) you were not offered insurance through an employer? You understand health insurance terms 0.71 44.80 .80 1.06 1.13 .85 21.29 (<.01) You would choose the health plan that is best for you? 0.81 41.04 .78 .81 .79 .87 9.39 (<.01) Choosing insurance: Comparing plans (a¼ .96) When comparing health plans how likely are you to ... Find out if the plans cover unexpected costs such as hospital stays? 0.84 54.96 .83 .94 .88 .91 29.04 (<.01) Understand what you would have to pay for emergency department visits? 0.98 51.49 .83 .63 .56 .94 33.18 (<.01) Understand what you would have to pay for specialist visits? 0.99 51.13 .84 .83 .77 .92 31.01 (<.01) Understand what you would have to pay for prescription drugs? 0.87 49.81 .84 .91 .88 .91 27.96 (<.01) Find out if you have to meet a deductible for health care services? 0.86 49.45 .83 1.00 .97 .90 38.23 (<.01) Look to see which doctors and hospitals are covered in each plan? 0.74 46.61 .84 1.64 1.59 .84 29.84 (<.01) 235 Using insurance: Confidence using (a¼ .93) How confident are you that .. . You know what to do if your health plan refuses to pay for a service you 0.81 59.03 .76 1.09 .99 .83 5.73 (<.01) think should be covered? You know how to figure out your share of the cost for care, after the 0.92 53.52 .78 .78 .79 .87 7.23 (<.01) health plan pays their share? You know what questions to ask your health plan if you have a coverage 0.79 48.78 .82 .82 .78 .86 6.30 (<.01) problem? You know most of the things you need to know about using health 0.44 47.18 .91 1.54 1.56 .75 12.45 (<.01) insurance. You know how to find out what is and is not covered before you receive a 0.83 45.55 .78 .76 .74 .87 8.00 (<.01) health care service? Using insurance: Being proactive (a¼ .80) When using your health insurance plan, how likely are you to .. . Look to member services to tell you what medical services your health plan 0.49 61.52 .92 1.37 1.33 .72 6.21 (<.01) covers? Look into what your health plan will and will not cover before you get 0.55 51.62 .91 .95 .97 .79 4.05 (<.01) health care services? Review the statements you get from your health plan showing what you 0.75 39.11 .98 .74 .79 .79 25.68 (<.01) owe and what they paid for a service? Find out if a doctor is in-network before you see him=her? 0.80 38.99 .97 .88 .78 .75 25.66 (<.01) Note. SEM is the standard error of measurement in estimation of the item difficulty; PTMEA is the point-measure correlation; a is the Cronbach’s alpha coefficient; and ANOVA is analysis of variance. 236 K. A. Paez et al. Table 3. Correlation of self-assessment measure with health insurance knowledge and skills items (n¼ 738) Health insurance literacy Objective measure measure self-assessment scale (knowledge and skills) p Selecting insurance Confidence choosing 0.18 <.01 Comparing health plans 0.37 <.01 Using insurance Confidence using 0.13 <.01 Being proactive 0.32 <.01 the HILM. All Cronbach’s alpha coefficients were greater than .9, indicating high internal consistency (Table 2). In testing the validity of the measure, analysis of variance results showed that the response categories for the 21 self-assessment items adequately differentiated knowledge and skills (Table 2). The health insurance literacy self-assessment scales were positively correlated with the objective knowledge and skills scale (Table 3). The correlation was greatest for comparing plans (.37) and being proactive (.32). The correlation was not as strong for the two scales that measured confidence: confidence choosing (.18) and confidence using (.13). These findings suggest that greater endorsement of confidence and self-reported behaviors measured by the HILM are likely to be related to actual health insurance knowledge and skills. Discussion The success of the federal and state insurance marketplaces hinges in part on consu- mers’ ability to understand health insurance and make informed decisions. Health insurance literacy is one factor that may determine whether consumers select a suit- able health plan and use health insurance to their best advantage. We sought to develop a measure of health insurance literacy to empirically assess consumers’ abil- ity to carry out cognitive tasks and behaviors that are expected of someone who is health insurance literate. The health insurance environment is complex and much work is needed to improve the clarity, accessibility, and usability of materials and tools provided to consumers and to reduce the complexity of the health plan benefits which consumers are expected to navigate. The data generated by the HILM can assist in identifying what facets of health insurance are most confusing to consumers and the characteristics of groups who would benefit from additional support as they are making health insurance purchasing and use decisions. The measure may also be useful as an evaluative tool in the development and implementation of tools and materials informing consumer, and evidence-based interventions to enhance outreach efforts. We tested the HILM among consumers in a range of coverage scenarios to approximate the U.S. population. The Rasch analysis identified a 23-point range in difficulty, indicating that the HILM assesses the middle range of health insurance literacy and may not distinguish between individuals at the high and low extremes. To assess the full range of difficulty, we could add items; however, this may not be necessary for most purposes. In future work, we plan to examine the range of person-level health insurance literacy scores to distinguish low, medium, and high scores and explore how people cluster on the basis of their demographics and level of health insurance literacy. Additional testing would be beneficial if the measure was to be used to assess health insurance literacy as a tool in a consumer counseling setting. Development of the HILM 237 We found that the data did not fit our conceptual model of health insurance literacy but fell into the broader categories of selecting and using health insurance. This indicates that the items are relevant to more than one domain of the four conceptual domains in our model. The tasks associated with selecting and using health insurance are complex and require applying knowledge while performing skills that are interconnected. We found statistically significant correlations between the HILM scale scores and the objective knowledge and skills scores but the correlations were weak for two of the scales, both of which assessed confidence. Our findings indicate that con- fidence may not be a component of health insurance literacy but could be a moder- ator. Self-efficacy may determine whether someone takes advantage of opportunities to purchase health insurance or seeks out needed health care services with confidence about their coverage. Our formative research indicates that consumers who are unsure of their health insurance coverage may hesitate to obtain a service because of their concerns about the out-of-pocket costs. We also observed that personal characteristics and extent of health care utilization may play a role in how confident a person feels with selecting and using health insurance. In developing the HILM, we chose to create a self-assessment measure, lending the HILM for broader use. Our early research indicates that the HILM could be used as a population-based measure to determine where consumer assistance resources should be allocated and to guide content development for navigator, in-person assistance training programs and consumer materials and tools. As the measure is refined in future research and our knowledge of health insurance literacy grows, we anticipate that data from the HILM may play a role in increasing the attention on restructuring health insurance benefits to be more consumer friendly. Acknowledgments and Funding This research was funded by the Missouri Foundation for Health and the American Institutes for Research. The Health Insurance Literacy Roundtable led by Consu- mer’s Union reaffirmed the need for health insurance literacy measure and provided an initial conceptual framework that served as the foundation for this research. We are grateful for the guidance provided by the Technical Expert Panel and the Health Insurance Literacy Stakeholder Group who provided guidance as we developed and tested the health insurance literacy measure. References Berkman, N. D., Sheridan, S. L., Donahue, K. E., Halpern, D. J., & Crotty, K. (2011). Low health insurance literacy and health outcomes: An updated systematic review. Annals of Internal Medicine, 155, 97–107. Bond, T., & Fox, C. M. (2007). Applying the Rasch model (2nd ed.). New York, NY: Routledge. Carman, K. L., Mauer, M., Yegian, J. M., Dardess, P., McGee, J., Evers, M., & Marlo, K. O. (2010). Evidence that consumers are skeptical about evidenced based health care. Health Affairs, 29, 1400–1406. Congressional Budget Office. (2014). Effects on health insurance and the federal budget for the insurance provisions in the Affordable Care Act—May 2013 baseline. Retrieved from http://www.cbo.gov/publication/44190 Dash, S., Lucia, K. W., Keith, K., & Monahan, C. M. (2013). Implementing the Affordable Care Act: Key design decisions for state-based exchanges. Washington, DC: The Commonwealth Fund. Retrieved from http://www.commonwealthfund.org/ Publications/Fund-Reports/2013/Jul/Design-Decisions-for-Exchanges.aspx 238 K. A. Paez et al. Dorn, S. (2011). Implementing national health reform: A five-part strategy for reaching the eligible uninsured. Washington, DC: Urban Institute. Retrieved from http://www. urban.org/publications/412335.html Greene, J., & Peters, E. (2009). Medicaid consumers and informed decision-making. Health Care Financing Review, 30, 25–40. Greenwald, L. M., McCormack, L. A., Uhrig, J. D., & West, N. (2006). Measures and predictors of Medicare knowledge: A review of the literature. Health Care Financing Review, 27, 1–12. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. Kutner, M., Greenberg, E., Jin, Y., & Paulson, C. (2006). The health literacy of America’s adults: Results from the 2003 National Assessment of Adult Literacy (NCES 2006–483). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Lowenstein, G., Friedman, J. Y., McGill, B., Ahmad, S., Linck, S., Sinkula, S., ... Volpp, K. G. (2013). Consumer’s misunderstanding of health insurance. Journal of Health Economics, 32, 850–862. Malbon, J. (2011). America’s Health Insurance Plans & Blue Cross Blue Shield Association: Focus group summary. Seattle, WA: JKM Research. Retrieved from http://www.naic. org/documents/committees_b_consumer_information_110603_ahip_bcbsa_consumer_ testing.pdf Marquis, M. S. (1981). Consumers’ knowledge about their health insurance coverage. Santa Monica, CA: RAND Corporation. McCormack, L., Bann, C., Uhrig, J., Berkman, N., & Rudd, R. (2009). Health insurance literacy of older adults. The Journal of Consumer Affairs, 43, 223–248. Morgan, R. O., Teal, C. R., Hasche, J. C., Petersen, L. A., Byrne, M. M., Paterniti, D. A., & Virnig, B. A. (2008). Does poorer familiarity with Medicare translate into worse access to health care? Journal of the American Geriatrics Society, 56, 2053–2060. Muthe´n, L. K., & Muthe´n, B. O. (2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Authors. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill. Paez, K. A., Goldstein, J., & Ganacheri, D. (2013). Developing a measure of health insurance literacy: Understanding consumers’ ability to choose and use insurance. American Institutes for Research. Retrieved from http://www.air.org/files/Health_Insurance_Literacy_Issues_ Brief_2.25.13.pdf Patient Protection and Affordable Care Act. (2010). Pub. L. No. 111–148, 124 Stat. 119. PwC Health Research Institute. (2012). Health insurance exchanges: Long on options, short on time. Retrieved from http://pwchealth.com/cgi-local/hregister.cgi/reg/pwc-health- insurance-exchanges-impact-and-options.pdf Quincy, L. (2010). Early consumer testing of new health insurance disclosure forms: Washington D.C. Consumer Union & People Talk Research. Retrieved from http:// yourhealthsecurity.org/document/report-early-consumer-testing-of-new-health-insurance- disclosure-forms-consumers-union-and-people-talk-research Quincy, L. (2011a). Early consumer testing of the coverage facts label: A new way of comparing health insurance. Washington, DC: Consumer Union & Kleinman Communication Group. Retrieved from http://consumersunion.org/wp-content/uploads/2013/03/CU_ CFL_Report_FINAL.pdf Quincy, L. (2011b). Early consumer testing of actuarial value concepts. Washington, DC. Consumer Union & Kleinman Communication Group. Retrieved from http:// yourhealthsecurity.org/document/report-early-consumer-testing-of-actuarial-value-concepts Qunicy, L. (2012). Measuring health insurance literacy: A call to action. Retrieved from http:// consumersunion.org/research/measuring-health-insurance-literacy-a-call-to-action Development of the HILM 239 Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research. Chicago, IL: The University of Chicago Press. Rosenbaum, S. (2011). The Patient Protection and Affordable Care Act: Implications for public health policy and practice. Public Health Reports, 126, 130–135. Tennyson, S. (2011). Consumers’ insurance literacy. Terre Haute, IN: Networks Financial Institute at Indiana State University. Retrieved from http://papers.ssrn.com/sol3/ papers.cfm?abstract_id=1896316 Wood, S., Hanoch, Y., Barnes, A., Liu, P. J., Cummings, J., Bhattacharya, C., & Rice, T. (2011). Numeracy and Medicare Part D: The importance of choice and literacy for num- bers in optimizing decision making for Medicare’s prescription drug program. Psychology and Aging, 26, 295–307.

Journal

Journal of Health CommunicationPubmed Central

Published: Oct 14, 2014

There are no references for this article.