The RISE Survey: Developing and Implementing a Valid and Reliable Quantitative Measure of Refugee Integration in the United States

The RISE Survey: Developing and Implementing a Valid and Reliable Quantitative Measure of Refugee... Abstract Integration has been explored through research, policy and practice as a framework for gauging the extent to which refugees successfully navigate the economic, social and cultural dynamics of their new country; however, the definition and assessment of integration still remain elusive. The purpose of this study was to: (i) operationalize an integration framework to create a reliable and valid survey to assess refugee integration, (ii) test the use of a community-based participatory research (CPBR) model (the ‘Community Connector’ (CC) model) with refugees and (iii) gain insights into the integration process of newly resettled refugees with the use of the survey and CC model. The Refugee Integration Survey and Evaluation (RISE) survey was developed and administered orally to 465 newly arrived adult refugees by caseworkers. Follow-up survey data was collected annually over the course of three additional years using the CC model. The survey was deemed reliable and valid. The CC model led to a 70 per cent response rate three years after baseline data collection. On average, the Overall Integration score increased steadily over time. The implications of this study are that a gap in the literature was addressed and the RISE survey can play a key role in understanding the salient integration factors that could be used by receiving countries to promote integration and help ensure the successful resettlement of refugees around the world. Introduction Recent conflicts in the Middle East, primarily Syria, have increased the number of refugees seeking refuge in host countries in Europe and around the world, and have once again thrust a spot light on the significant topic of refugee resettlement and integration. Data from the United Nations High Commissioner for Refugees (UNHCR) estimated that there were a total of 22.5 million refugees worldwide in 2017—the highest level in 20 years (UNHCR 2017). The United States admits a significant number of refugees each year in an effort to support the international goal of providing humanitarian protection to individuals fleeing persecution. Beyond meeting core commitments for providing protection for refugees, absorbing these refugees so that they are supported in contributing to the economic and socio-cultural fabric of receiving countries is a significant challenge for governments and aid agencies. Integration has been explored through research, policy and practice as a framework for gauging the extent to which refugees successfully navigate the economic, social and cultural dynamics of their new country (Ager and Strang 2004). Yet the definition and assessment of integration for individuals remain elusive. Without a means to define and assess integration, it is difficult to identify refugees’ needs and support them with data-informed policies and services during the integration process. Moreover, without data and assessment within the integration framework, it is difficult for refugee resettlement advocates and practitioners to claim a seat at the table in broader community frameworks that might impact refugees, but are not focused on refugees. Literature Review Consensus in the literature about the definition of integration is lacking. Ager and Strang (2008) highlight the confusion and complexity around the notion of refugee integration when they cite a quote by Robinson (1998), which says ‘“integration” is a chaotic concept: a word used by many but understood differently by most’. Currently, the prevailing view is that Immigrant integration is a dynamic, two-way process in which newcomers and the receiving society work together to build secure, vibrant, and cohesive communities … . As an intentional effort, integration engages and transforms all community members, reaping shared benefits and creating a new whole that is greater than the sum of its parts (Petsod et al. 2006: 10). The National Academies of Sciences (2015) defines integration as ‘the extent to which immigrant and refugee populations (within and across generations) come to approximate the status of the native-born population’ (p. 20). This definition leaves room for broad interpretation regarding the specific dimensions along which resettled and native-born populations might be compared. Ager and Strang (2004, 2008) attempted to address the fact that there was no generally accepted definition, theory or model of refugee integration by developing a refugee integration framework. Their conceptual framework included four headings (Means and Markers, Social Connections, Facilitators and Foundation) and, within these headings, anywhere from one to four domains. Under the heading Markers and Means were the domains of Employment, Housing, Education and Health. Grouped together, these domains represent major areas of attainment that are widely recognized as critical factors in the integration process. Under that heading of Social Connections were the domains of Social Bridges, Social Bonds and Social Links. These domains stress the importance of relationships to the understanding of the integration process. There were two domains under the heading Facilitators: Language and Cultural Knowledge, and Safety and Stability, which represent key facilitating factors for the process of integration. Lastly, the only domain under the heading Foundations was Rights and Citizenship, which represents the basis upon which expectations and obligations for the process of integration are established. Ager and Strang developed their conceptual integration framework by conducting documentary and conceptual analysis, fieldwork in settings of refugee settlement, secondary analysis of cross-sectional survey data and verification via discussion at three major seminars. This framework has been used to inform studies of integration both conceptually and methodologically (Atfield et al. 2007; Beirens et al. 2007; Sirriyeh 2007). For this study, the integration framework put forth by Ager and Strang was chosen in an attempt to define integration. This framework was chosen for several reasons. First, the framework speaks to integration outcomes that concern policy-makers and researchers worldwide. Second, the framework was derived empirically, suggesting that the indicators are critical to refugees and members of host communities. Lastly, the framework conceptualized integration more behaviourally, compared to other models that conceptualized it more as a psychological construct (Rudmin 2003). The choosing of the Ager and Strang integration framework helped with the problem of how to define integration, but that did not solve the problem of the scarcity of literature about the assessment of integration, particularly the quantitative, longitudinal assessment of integration. The lack of quantitative, longitudinal data to assess integration hinders efforts to identify key factors that could be used by receiving countries, at both national and local levels, to promote integration and ensure the successful resettlement of refugees around the world. The National Academies of Sciences (2015) report pointedly makes reference to the lack of data regarding integration: The panel was handicapped in its work by the dearth of available longitudinal data to measure immigrant integration. This is a long-standing problem that has become increasingly critical as immigration to the United States has increased and as immigrants have become dispersed throughout the country (National Academies of Sciences 2015: 13). Another prominent report from the Migration Policy Institute (Capps et al. 2015) also cited the lack of data on integration as a hindrance to refugee policy and support: Public doubts and political resistance have emerged in the absence of solid data on the outcomes of resettlement efforts .... A lack of data also makes it difficult to identify problems that may have policy solutions or that may be specific to particular groups of refugees rather than to the resettled population as a whole (Capps et al. 2015: 3). Lastly, in 2010, in a special issue of the Journal of Refugee Studies, guest editors note the lack of longitudinal data available for increasing understanding of refugee integration: There is a general lack of research which examines the long-term experiences of refugees. Potential new insights could be gained into the elements of Ager and Strang’s (2008) framework of integration by mapping out the linkages with future migration paths. Examining the longitudinal experiences of refugees is, therefore, essential when considering refugee integration (Smyth et al. 2010: 413–414). An extensive literature search confirmed the dearth of quantitative, longitudinal data related to refugee integration. Nearly all studies of refugee integration, although revealing, were based on interview (qualitative) data. Quantitative studies found tended to be cross-sectional in nature, often using secondary data, and most were not analysed longitudinally. There were a handful of exceptions, however. Yu et al. (2007) described the patterns of refugees’ economic and socio-cultural integration as portrayed in nationally representative databases. They found the employment earnings of refugees at one and five years after arrival were slightly lower than the family-class immigrants’ earnings and, as expected, much lower than skilled-worker immigrants. Socio-cultural integration was much more of an elusive concept to define and to quantify and it was concluded that more research was needed in this area. Cebulla et al. (2010) mailed surveys to all newly arrived refugees (Survey of New Refugees (SNR)) who arrived in the United Kingdom between December 2005 and March 2007. They sent household surveys at eight, 15 and 21 months after arrival. Surveys included questions related to a range of integration indicators, including English-Language Skills, Employment, Housing and Health. However, initial response rates are not reported, and the response rate at the 21-month follow-up was only 17 per cent. Cheung and Phillimore (2017) reanalysed data from the 2005–07 SNR in order to assess differences in integration outcomes by gender. The authors found significant gender differences in language, self-reported health, ability to budget for household expenses and access to formal social networks and quality housing, with women generally faring worse than men and some inequalities enduring or intensifying over time. The Building a New Life in Australia (BNLA) study is a study that aims to better understand the factors that aid or hinder the successful settlement of humanitarian migrants in Australia, and to provide an evidence base to inform policy and program development (De Maio et al. 2014: p.5). The Australian Institute of Family Studies (AIFS) was commissioned by the Department of Immigration and Border Protection (DIBP) to undertake and manage the BNLA study. A total of 1,500 principal applicants from 11 sites in urban and rural communities are targeted for recruitment into the study. Annual survey data collection began in 2013 and will be completed in 2018. The surveys are collected via telephone interviews and in-person computer-assisted self-interviews (CASI) and are translated into 14 different languages. The surveys cover a variety of integration topics including: housing and neighbourhood, English-language proficiency, education and training, employment and income, immigration experience, health, self-sufficiency, community support, personal resources and life satisfaction, and perceptions of life in Australia. Rioseco et al. (2017) outline the main characteristics of the study and provide an overview of the measures available and potential uses. The first three waves of the BNLA data are currently available to approved users but, to date, no results have been published on the study. Finally, a mixed-methods longitudinal study of refugees and asylum seekers that used the Ager and Strang (2004) integration framework was conducted in Scotland between 2009 and 2013 (Mulvey 2015). A total of 262 respondents completed the main survey and a follow-up survey was administered one year later. There were several significant limitations of this study, however, including significant attrition; the questionnaire being translated into various languages, but administered in a written format, which posed challenges to illiterate participants; and the administration of the survey being conducted by voluntary organizations, which posed protocol problems and prevented the generalizability of the survey sample. The Refugee Integration Survey and Evaluation Ager and Strang (2008), themselves, note the framework’s wider utility and explanatory value now needs to be tested in diverse contexts to gauge whether the proposed structure captures key elements of stakeholder perceptions of what constitutes integration in an appropriately broad range of settings and timeframes (Ager and Strang 2008: p. 185, emphasis added). As such, the first purpose of the Refugee Integration Survey and Evaluation (RISE) study was to operationalize the integration framework conceptualized by Ager and Strang by developing a reliable and valid quantitative measure that could assess refugee integration over time in the United States. The second purpose of the study pertained to understanding the feasibility of applying community-based participatory research (CBPR) principles to collecting the RISE survey data. CBPR involves a commitment to conducting research that shares power with and engages community partners in the research process (Israel et al. 1998). The benefits of using a CBPR approach to survey data collection have been noted by various researchers (Israel et al. 1998; Kruger et al. 2010; Carty et al. 2011; Cohen et al. 2011; Rucinski et al. 2011; Main et al. 2012) and include: the fact that the research can addresses local concerns, more locally relevant data can be obtained, higher cooperation rates and lower refusal rates can result, more insightful interpretation of results can occur and there can be increased community buy-in for the results. The researchers who oversaw this study hypothesized that, in fact, a CBPR model (coined the Community Connector (CC) model in this study) could be adapted for use with the refugee population. The CC model was created by Puma et al. (2016) and was adapted and used in this study in order to leverage the relationships and cultural knowledge of the CCs. The research team hypothesized that the CBPR approach to collecting survey integration data in an urban city in the United States would: (i) increase the survey response rate for an otherwise, at-risk, highly mobile population; (ii) improve the quality of the data collected; and (iii) increase the utility of the data. The third purpose of the study was to learn about refugee integration in the Denver Metro area, since the Colorado Refugee Services Program (CRSP) (in Denver, Colorado) initiated the study. This purpose is only touched upon briefly in this article, but will be the primary focus of a forthcoming article. Methods Survey Development Initial survey development took place from November 2010 through July 2011. The development of the RISE survey instrument encompassed three phases: (i) selection of integration domains and items, (ii) expert panel review and (iii) piloting of the survey instrument with refugees. Ager and Strang’s framework served as the basis for the selection of integration domains (referred in this study as ‘pathways’, because ‘pathways’ better captures the dynamic two-way street of integration) and items; however, the framework was modified slightly. Modifications to the Ager and Strang framework included: the ‘Rights & Citizenship’ domain was converted into a ‘Civic Engagement’ pathway; the ‘Health’ domain was changed to a ‘Physical Health and Well-Being’ pathway because these seemed to better capture the pathways to integration; ‘Child’s Education’ was added as a pathway because it was believed that having children in school could be a ‘touch point’ to mainstream society for care-givers of the children; and the ‘Social Links’ domain was removed from the survey pathways because of the challenges to operationalizing it. These modifications emerged organically from focus groups and feedback from refugees and refugee service professionals. Once pathways were identified, survey items were created. There was intentionality around making the items objective (behaviourally oriented), as opposed to subjective (opinion-/psychologically oriented). An example of an objective item is ‘Within the past year, have you obtained a certificate that qualifies you for a specific kind of job in the U.S.?’ (Yes, No, I Don’t Know). Once an item pool was compiled, it was sent to a panel of three experts to review. Individuals were selected to serve on the panel who had experience in working on state and federal task forces on refugee integration and who had worked with refugees and/or did integration research. Each of the experts was given a list of the items/indicators, and asked to rank them in order of relevance to integration and to edit the list by deleting, revising or adding items that were important to refugee integration. Their summarized reviews were used to refine the draft survey. The resulting 10 pathways assessed in the RISE survey are included in Table 1. Table 1 Ten Survey Pathways and Corresponding Item Concepts Survey pathway Item concepts Employment Employment status Number of jobs and hours worked Pay and benefits Sufficiency of income to meet family needs Education & Training Certification/training/classes received or taken in the U.S. Enrollment in any degree programmes Child’s Education Children’s grade levels in school Volunteer experience at their child(ren)’s schools Visits with child(ren)’s teachers Attendance at school events Child(ren)’s friends from different cultures Physical Health & Well-Being Status of routine physical check-ups Has regular doctor(s) and/or dentist Knowledge of how to make appointments Has health insurance coverage Housing Housing situation (homeless, transitional housing, home owner) Housing type and number of rooms in the dwelling Frequency of moves Social Bonding Interaction with people from same culture/ethnic group Information sources from same culture/ethnic group Attended events from the same culture/ethnic group Social Bridging Interaction with people from different culture/ethnic group Information sources from different culture/ethnic group Attended events from the different culture/ethnic group Language & Cultural Knowledge Self-reported proficiency in English (speaking) Familiarity with U.S. cultural facts (who is the president, where is Congress and the White House located, three sports that are popular in the U.S.) Safety & Stability Feelings of safety at home and outside the home Comfort level with calling on police or the fire department Trust of neighbors Victim of crime or discrimination Civic Engagement Participation in community meetings, clubs or organizations Advocacy or ‘speaking up’ for rights Possession of a driver’s license Applied for green card Desire for U.S. citizenship Survey pathway Item concepts Employment Employment status Number of jobs and hours worked Pay and benefits Sufficiency of income to meet family needs Education & Training Certification/training/classes received or taken in the U.S. Enrollment in any degree programmes Child’s Education Children’s grade levels in school Volunteer experience at their child(ren)’s schools Visits with child(ren)’s teachers Attendance at school events Child(ren)’s friends from different cultures Physical Health & Well-Being Status of routine physical check-ups Has regular doctor(s) and/or dentist Knowledge of how to make appointments Has health insurance coverage Housing Housing situation (homeless, transitional housing, home owner) Housing type and number of rooms in the dwelling Frequency of moves Social Bonding Interaction with people from same culture/ethnic group Information sources from same culture/ethnic group Attended events from the same culture/ethnic group Social Bridging Interaction with people from different culture/ethnic group Information sources from different culture/ethnic group Attended events from the different culture/ethnic group Language & Cultural Knowledge Self-reported proficiency in English (speaking) Familiarity with U.S. cultural facts (who is the president, where is Congress and the White House located, three sports that are popular in the U.S.) Safety & Stability Feelings of safety at home and outside the home Comfort level with calling on police or the fire department Trust of neighbors Victim of crime or discrimination Civic Engagement Participation in community meetings, clubs or organizations Advocacy or ‘speaking up’ for rights Possession of a driver’s license Applied for green card Desire for U.S. citizenship Table 1 Ten Survey Pathways and Corresponding Item Concepts Survey pathway Item concepts Employment Employment status Number of jobs and hours worked Pay and benefits Sufficiency of income to meet family needs Education & Training Certification/training/classes received or taken in the U.S. Enrollment in any degree programmes Child’s Education Children’s grade levels in school Volunteer experience at their child(ren)’s schools Visits with child(ren)’s teachers Attendance at school events Child(ren)’s friends from different cultures Physical Health & Well-Being Status of routine physical check-ups Has regular doctor(s) and/or dentist Knowledge of how to make appointments Has health insurance coverage Housing Housing situation (homeless, transitional housing, home owner) Housing type and number of rooms in the dwelling Frequency of moves Social Bonding Interaction with people from same culture/ethnic group Information sources from same culture/ethnic group Attended events from the same culture/ethnic group Social Bridging Interaction with people from different culture/ethnic group Information sources from different culture/ethnic group Attended events from the different culture/ethnic group Language & Cultural Knowledge Self-reported proficiency in English (speaking) Familiarity with U.S. cultural facts (who is the president, where is Congress and the White House located, three sports that are popular in the U.S.) Safety & Stability Feelings of safety at home and outside the home Comfort level with calling on police or the fire department Trust of neighbors Victim of crime or discrimination Civic Engagement Participation in community meetings, clubs or organizations Advocacy or ‘speaking up’ for rights Possession of a driver’s license Applied for green card Desire for U.S. citizenship Survey pathway Item concepts Employment Employment status Number of jobs and hours worked Pay and benefits Sufficiency of income to meet family needs Education & Training Certification/training/classes received or taken in the U.S. Enrollment in any degree programmes Child’s Education Children’s grade levels in school Volunteer experience at their child(ren)’s schools Visits with child(ren)’s teachers Attendance at school events Child(ren)’s friends from different cultures Physical Health & Well-Being Status of routine physical check-ups Has regular doctor(s) and/or dentist Knowledge of how to make appointments Has health insurance coverage Housing Housing situation (homeless, transitional housing, home owner) Housing type and number of rooms in the dwelling Frequency of moves Social Bonding Interaction with people from same culture/ethnic group Information sources from same culture/ethnic group Attended events from the same culture/ethnic group Social Bridging Interaction with people from different culture/ethnic group Information sources from different culture/ethnic group Attended events from the different culture/ethnic group Language & Cultural Knowledge Self-reported proficiency in English (speaking) Familiarity with U.S. cultural facts (who is the president, where is Congress and the White House located, three sports that are popular in the U.S.) Safety & Stability Feelings of safety at home and outside the home Comfort level with calling on police or the fire department Trust of neighbors Victim of crime or discrimination Civic Engagement Participation in community meetings, clubs or organizations Advocacy or ‘speaking up’ for rights Possession of a driver’s license Applied for green card Desire for U.S. citizenship The survey comprised two types of questions: core items (n = 43), which were scored and which contributed to the pathway scores and to the Overall Integration (OI) score, and auxiliary items (n = 55), which were not scored, but which provided supplemental information about factors related to integration. Although an attempt was made to avoid opinion-based items generally, six out of 43 scored items were opinion questions (e.g. ‘Is your family’s monthly income sufficient to cover rent, food and other expenses necessary for daily living?’ (Too Low, Enough, More than Enough) and ‘Which of the following best describes your English-language skill?’ (I cannot speak English; I speak a little English when shopping and doing other types of business; I can speak English; I can speak English in most Social & Work situations; I am fluent in English). A draft survey was then piloted with different groups of refugees. The first group included five members of the RISE Advisory Committee (RAC) who were refugees from the survey target populations (Burmese, Bhutanese, Iraqi and Somali), had been in the United States for five or more years and had excellent English skills. The RAC members were asked to both take the survey and disseminate it to at least two refugee acquaintances, and then give feedback on the items. Based on feedback from the RAC, further revisions were made. The second group included refugee resettlement agency (formerly called voluntary agency or VOLAG) case managers and their supervisors (most of whom had been refugees when they were admitted to the United States several years previously), as well as the Coordinator of the CRSP at the time (article co-author) and the Integration Partnership Coordinator for the state. They were given versions of the draft survey twice: once to review and share with other refugees of their acquaintance and, second, to test-administer with one another, since case managers would be administering the survey to refugees. Finally, the initial survey administration period, from June through August 2011, was considered to be a piloting period. During this time, even though the data would be used in the study, case managers reported issues that resulted in minor modifications of several survey items. Data Collection Once the survey instrument was created, survey administration commenced. The baseline survey was administered orally by refugee resettlement agency caseworkers in the refugee’s native language (Arabic, Burmese, Karen, Nepali or Somali) at the 90-day post-arrival meeting. All refugees over the age of 18 years old were invited to complete the survey. The survey took approximately 40 minutes to complete and no incentives were given to complete the survey. Surveys were completed for 465 newly arrived adult refugees (out of 480 newly arrived refugees in the 2011–12 fiscal year). In subsequent study years, survey data was collected by ‘CCs’, leaders and reliable members from the target community who were fluent in refugees’ home language and in English and who had sufficient literacy skills and education levels to understand and administer the survey. CCs were trained by the research team to adhere to strict study protocols and Institutional Review Board (IRB) human subjects requirements. Post-Administration Survey Tweaking After the first year of survey administration, the survey psychometrics were further scrutinized via two approaches: (i) a translation and back-translation approach and (ii) cognitive interviewing approach. A firm that had not translated the survey initially was hired to retranslate the survey from English into Arabic, Burmese, Karen, Nepali and Somali, using professional, certified translators. Once that was done, study researchers asked different translators (also certified) to back-translate the newly translated Year 2 survey from the target language back into English. Due to budget constraints, not all survey items were examined in this way. A total of 71 per cent of core (scored) items (35 out of 49) and other seemingly problematic supplemental items were back-translated. The 35 core items were back-translated into five different languages, which resulted in a total of 175 back-translated items. Across all languages, 13 per cent of the items were found to contain minor differences between the Year 1 and Year 2 translations. The same items were not flagged across languages; different items were problematic in different languages. Karen appeared to have the least problems with translation (n = 1 item) and Somali appeared to have the most problems with translation (n = 8 items). All translation errors were corrected. Cognitive interviews were completed in order to understand whether various items were being understood and interpreted by the refugees as the research team intended (face/content validity). The cognitive interview is a verbal interview technique that attempts to review non-observable understanding that motivates behaviour (Willis 1999). The RISE cognitive interviews were conducted by a researcher (manuscript lead author), with the assistance of an interpreter, and with one or two refugees at a time. It should be noted that the interpreter, who was also a CC, was also used as a ‘key informant’ throughout the cognitive interviews because of his/her familiarity with the refugees’ language and culture, as well as the RISE survey items. The cognitive interviews aimed to understand: (i) how refugees understood particular items on the RISE survey (e.g. What did refugees believe these items are asking? What do certain words and phrases mean to respondents?) and (ii) in what ways items could be reframed to elicit the information intended. Participants were a convenience sample of refugees who had been in the United States for 90–120 days at the time the cognitive interviews were conducted. Attempts were made to balance for gender (and age in some cases) within each language group. A total of nine refugees participated (six from Bhutan (spoke Nepali), three from Burma (two spoke Burmese, one spoke Karen)). More Nepali speakers were selected due to their prevalence in our sample. Twelve core survey items (about 25 per cent of scored items) were identified for the cognitive interviews, as well as four supplemental items. The cognitive interviews took place in April 2013 in the community-use room at an apartment building, in which many of the refugees lived, and in a close-by apartment of an interpreter (when the community-use room was unavailable). For each selected item, the following protocol was generally followed. First, the interpreter read the question and asked the participant to respond to the question. Second, each refugee was then asked what he or she believed was being asked by the question. In some cases, the researcher asked refugees to define specific terms or phrases in the question. Third, after the interpreter related refugees’ responses, the researcher may have asked the interpreter, as well as refugee(s), for clarification, using spontaneous probes. Finally, the researcher then explained (through the help of the interpreter) the intention behind the question, which the interpreter would relay to refugees. Once refugees understood the intention, they were asked how questions might have been phrased to more clearly/adequately reflect the intention. Interviews lasted approximately one hour. All interviews were recorded, transcribed and analysed. Cognitive interview results suggested a number of changes in the wording of particular survey items—for example, removal of the term ‘license’ from a question about acquisition of job certification, because of the confusion with ‘driver’s licenses’; removal of the term ‘degree program’ and replacing it with ‘community college or college’ because participants did not understand the term; replacing the term ‘other than your family’ with ‘who are not related to you’ in the social bonding questions about time spent with others, due to confusion about what ‘family’ meant in this context; removal of the item ‘do you trust your neighbors?’ due to confusion as to what ‘trust’ and what ‘neighbors’ meant. Changes strengthened the instrument; however, the most noteworthy finding was that nearly all respondents understood the items as intended, reassuring the research team of the validity of the survey. Developing a reliable and valid refugee integration survey was the primary focus of RISE for the first two years of the project. By the end of Year 3 (the second year of data collection—Baseline +2), the RISE team concluded the study of the psychometric properties of the survey instrument (although minor changes to the survey were made in later years of survey administration, such as the addition of items that probed for challenges related to transportation). In Years 4 and 5 (Baseline +3 and Baseline +4), the primarily focus of the study shifted from the reliability and validity of the survey to findings that pertained to refugees, particularly in terms of progress along various integration pathways and for integration overall. Data Analysis Techniques To address the first purpose of the study, reliability was assessed quantitatively by examining internal consistency of the latent construct of ‘overall integration’. Additionally, researchers examined one-year test–retest reliability across survey administrations. Face and content validity were assessed qualitatively via expert panel reviews of items and the cognitive interviews (described above). Construct validity (convergent and divergent validity) was assessed quantitatively by running bivariate correlations between the various pathways across time. To address the second purpose of the study pertaining to the utility of a CBPR approach to collecting the RISE survey data, the survey response rates were tracked over time. Additionally, the trend in the OI score over time was examined as both a proxy of quality (and usefulness) of the survey results (Purpose 2) and to gain insight into refugee integration in Denver, Colorado, over time (Purpose 3). The OI score (each year) was created by summing core pathway scores (Yes = 1 point and No = 0 points). This meant that pathways with more items contributed more to the OI score than others, but the pathways that did contribute more to the OI score (e.g. Language and Cultural Knowledge) and those that contributed very little (e.g. Housing) were consistent with the literature and what was learned from the focus groups that had been conducted. The pathway scores were created by reverse-scoring the items that needed to be reverse-scored (so that a higher value was viewed as more positive for integration) and then summing all items within the individual pathways. Any missing values, which was a nominal amount, were assigned a score of zero so that the pathway items could be summed and the maximum sample could be preserved for the analyses. Descriptive cut-points were determined based on the analysis of the frequency distribution of scores during the first two years of data collection. The research team noted natural breaks in the distribution curve and assigned low, medium and high integration categories accordingly. Results In this section, results are organized according to the three purposes of the study: creation of a reliable and valid survey to assess refugee integration (using Ager and Strang’s framework); test the efficacy of a CBPR model in data collection for a vulnerable, difficult-to-access refugee population; gain insights into the integration process of newly resettled refugees in Denver, Colorado. Purpose 1: Integration Survey Reliability and Validity Internal consistency (Cronbach’s alpha) measures whether scores on similar items are related (internally consistent). Another way to think about internal consistency is it is the extent to which all of the items of a test measure the same latent variable. Because the latent construct of interest was OI, the research team examined internal consistency for the OI variable (n = 10 pathways) at Baseline (α = 0.63) and at 1-Year (Baseline +1) (α = 0.60). Although Cronbach’s alpha was calculated for OI, it should be emphasized that the latent construct of integration was not operationalized as a psychological construct. It was measured more as a checklist of behaviours, which, when aggregated, led to ‘pathways’ (to integration) and, when these ‘pathways’ were aggregated, yielded an OI score. Chronbach’s alphas calculated at Baseline and at 1-Year follow-up indicate that the integration ‘pathways’ were related to each other at both time points, and that the pathways were not completely redundant in the information that they were contributing to the OI construct. In other words, each pathway contributed unique information to the measurement of OI, which is what is desired and which is what would be expected given that some of the pathways (e.g. Health and Well-Being and Child Education) would not theoretically be related to each other. Internal consistency was not measured within pathways because: (i) pathways are tallies of key behaviours (individual items) related to a particular facet of integration (rather than physiological constructs) and (ii) several pathways did not have a large enough number of items to warrant an internal consistency reliability analysis (e.g. Educational Training, Housing, Social Bonding and Social Bridging all had three items or fewer). Test–retest reliability was assessed across Baseline and 1-Year follow-up (Baseline +1). Results are presented in Table 2. Test–retest correlations ranged from 0.17 to 0.72. The correlation for OI at Baseline and ‘Baseline +1’ (r = 0.60) is moderate to high and reflects the fact that there is some stability in refugees’ scores over time; however, there is also variability in the growth of these scores over time. The range of the pathway scores is consistent with this notion. Table 2 Descriptive Statistics and Test–Retest Reliability (r) for Pathway and Overall Integration Scores Domain Baseline Mean (SD) 1-Year follow-up Mean (SD) Pearson correlation (r) Employment and Economic Self-Sufficiency 1.1 (2.0) 3.2 (2.9) 0.30** Education and Training 0.4 (0.7) 0.6 (0.6) 0.39** Child(ren)’s Education 0.4 (0.8) 0.4 (1.0) 0.35** Physical Health and Well-Being 2.2 (1.1) 1.8 (1.0) – Housing 19% 87% McNemar Test*** Social Bonding 1.5 (1.1) 2.5 (0.9) 0.17** Social Bridging 0.8 (1.0) 1.7 (1.3) 0.24** Language and Cultural Knowledge 4.6 (3.3) 6.4 (3.9) 0.72** Safety and Stability 3.3 (1.2) 3.8 (0.6) 0.24** Civic Engagement 0.1 (0.4) 1.4 (0.7) 0.22** Overall Integration 14.6 (6.9) 22.4 (8.1) 0.60** Domain Baseline Mean (SD) 1-Year follow-up Mean (SD) Pearson correlation (r) Employment and Economic Self-Sufficiency 1.1 (2.0) 3.2 (2.9) 0.30** Education and Training 0.4 (0.7) 0.6 (0.6) 0.39** Child(ren)’s Education 0.4 (0.8) 0.4 (1.0) 0.35** Physical Health and Well-Being 2.2 (1.1) 1.8 (1.0) – Housing 19% 87% McNemar Test*** Social Bonding 1.5 (1.1) 2.5 (0.9) 0.17** Social Bridging 0.8 (1.0) 1.7 (1.3) 0.24** Language and Cultural Knowledge 4.6 (3.3) 6.4 (3.9) 0.72** Safety and Stability 3.3 (1.2) 3.8 (0.6) 0.24** Civic Engagement 0.1 (0.4) 1.4 (0.7) 0.22** Overall Integration 14.6 (6.9) 22.4 (8.1) 0.60** *p < 0.5; **p < 0.01; ***p < 0.001;—N/A because all newly arrived refugees saw a physician during their initial health screening, which biased the baseline results. Table 2 Descriptive Statistics and Test–Retest Reliability (r) for Pathway and Overall Integration Scores Domain Baseline Mean (SD) 1-Year follow-up Mean (SD) Pearson correlation (r) Employment and Economic Self-Sufficiency 1.1 (2.0) 3.2 (2.9) 0.30** Education and Training 0.4 (0.7) 0.6 (0.6) 0.39** Child(ren)’s Education 0.4 (0.8) 0.4 (1.0) 0.35** Physical Health and Well-Being 2.2 (1.1) 1.8 (1.0) – Housing 19% 87% McNemar Test*** Social Bonding 1.5 (1.1) 2.5 (0.9) 0.17** Social Bridging 0.8 (1.0) 1.7 (1.3) 0.24** Language and Cultural Knowledge 4.6 (3.3) 6.4 (3.9) 0.72** Safety and Stability 3.3 (1.2) 3.8 (0.6) 0.24** Civic Engagement 0.1 (0.4) 1.4 (0.7) 0.22** Overall Integration 14.6 (6.9) 22.4 (8.1) 0.60** Domain Baseline Mean (SD) 1-Year follow-up Mean (SD) Pearson correlation (r) Employment and Economic Self-Sufficiency 1.1 (2.0) 3.2 (2.9) 0.30** Education and Training 0.4 (0.7) 0.6 (0.6) 0.39** Child(ren)’s Education 0.4 (0.8) 0.4 (1.0) 0.35** Physical Health and Well-Being 2.2 (1.1) 1.8 (1.0) – Housing 19% 87% McNemar Test*** Social Bonding 1.5 (1.1) 2.5 (0.9) 0.17** Social Bridging 0.8 (1.0) 1.7 (1.3) 0.24** Language and Cultural Knowledge 4.6 (3.3) 6.4 (3.9) 0.72** Safety and Stability 3.3 (1.2) 3.8 (0.6) 0.24** Civic Engagement 0.1 (0.4) 1.4 (0.7) 0.22** Overall Integration 14.6 (6.9) 22.4 (8.1) 0.60** *p < 0.5; **p < 0.01; ***p < 0.001;—N/A because all newly arrived refugees saw a physician during their initial health screening, which biased the baseline results. Face and content validity were assessed qualitatively via expert panel reviews of items and the cognitive interviews (described above). Construct validity (convergent and divergent) was assessed quantitatively by running bivariate correlations between the various pathways at Baseline (Table 3) and 1-Year (Baseline +1) (Table 4). Table 3 ‘Pathway’ Construct Validity (Convergent and Divergent): Bivariate Correlations between the Various Pathways at Baseline Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.14** 1.00 Child_Edu –0.10* –0.16** 1.00 Health_WB 0.00 0.15** 0.14** 1.00 Housing 0.15** –0.04 0.03 0.08 1.00 Soc_Bond 0.17** 0.11** 0.06 0.20** 0.25** 1.00 Soc_Bridg 0.15** 0.16** 0.08 –0.01 0.22** 0.46** 1.00 Lang_CK 0.30** 0.36** 0.00 0.15** 0.21** 0.45** 0.54** 1.00 Safety 0.20** 0.20** 0.011* 0.38** 0.12** 0.41** 0.34** 0.55** 1.00 Civic_Eng 0.17** 0.05 0.07 0.09* 0.10* 0.23** 0.13** 0.32** 0.12** 1.00 Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.14** 1.00 Child_Edu –0.10* –0.16** 1.00 Health_WB 0.00 0.15** 0.14** 1.00 Housing 0.15** –0.04 0.03 0.08 1.00 Soc_Bond 0.17** 0.11** 0.06 0.20** 0.25** 1.00 Soc_Bridg 0.15** 0.16** 0.08 –0.01 0.22** 0.46** 1.00 Lang_CK 0.30** 0.36** 0.00 0.15** 0.21** 0.45** 0.54** 1.00 Safety 0.20** 0.20** 0.011* 0.38** 0.12** 0.41** 0.34** 0.55** 1.00 Civic_Eng 0.17** 0.05 0.07 0.09* 0.10* 0.23** 0.13** 0.32** 0.12** 1.00 Table 3 ‘Pathway’ Construct Validity (Convergent and Divergent): Bivariate Correlations between the Various Pathways at Baseline Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.14** 1.00 Child_Edu –0.10* –0.16** 1.00 Health_WB 0.00 0.15** 0.14** 1.00 Housing 0.15** –0.04 0.03 0.08 1.00 Soc_Bond 0.17** 0.11** 0.06 0.20** 0.25** 1.00 Soc_Bridg 0.15** 0.16** 0.08 –0.01 0.22** 0.46** 1.00 Lang_CK 0.30** 0.36** 0.00 0.15** 0.21** 0.45** 0.54** 1.00 Safety 0.20** 0.20** 0.011* 0.38** 0.12** 0.41** 0.34** 0.55** 1.00 Civic_Eng 0.17** 0.05 0.07 0.09* 0.10* 0.23** 0.13** 0.32** 0.12** 1.00 Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.14** 1.00 Child_Edu –0.10* –0.16** 1.00 Health_WB 0.00 0.15** 0.14** 1.00 Housing 0.15** –0.04 0.03 0.08 1.00 Soc_Bond 0.17** 0.11** 0.06 0.20** 0.25** 1.00 Soc_Bridg 0.15** 0.16** 0.08 –0.01 0.22** 0.46** 1.00 Lang_CK 0.30** 0.36** 0.00 0.15** 0.21** 0.45** 0.54** 1.00 Safety 0.20** 0.20** 0.011* 0.38** 0.12** 0.41** 0.34** 0.55** 1.00 Civic_Eng 0.17** 0.05 0.07 0.09* 0.10* 0.23** 0.13** 0.32** 0.12** 1.00 Table 4 ‘Pathway’ Construct Validity (Convergent and Divergent): Bivariate Correlations between the Various Pathways at Year 1 Follow-Up (Baseline +1) Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.25** 1.00 Child_Edu –0.13* –0.01 1.00 Health_WB –0.29** –0.17** 0.10* 1.00 Housing 0.24** 0.04 –0.23** –0.19** 1.00 Soc_Bond 0.20** 0.27** –0.08 –0.12** 0.19* 1.00 Soc_Bridg 0.50** 0.47** –0.09 –0.32** 0.26** 0.50** 1.00 Lang_CK 0.54** 0.51** –0.03 –0.37** 0.25** 0.37** 0.79** 1.00 Safety 0.15** 0.20** –0.02 –0.08 0.13* 0.37** 0.37** 0.31** 1.00 Civic_Eng 0.36** 0.22** 0.09 –0.09 0.09 0.05** 0.31** 0.45** 0.15** 1.00 Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.25** 1.00 Child_Edu –0.13* –0.01 1.00 Health_WB –0.29** –0.17** 0.10* 1.00 Housing 0.24** 0.04 –0.23** –0.19** 1.00 Soc_Bond 0.20** 0.27** –0.08 –0.12** 0.19* 1.00 Soc_Bridg 0.50** 0.47** –0.09 –0.32** 0.26** 0.50** 1.00 Lang_CK 0.54** 0.51** –0.03 –0.37** 0.25** 0.37** 0.79** 1.00 Safety 0.15** 0.20** –0.02 –0.08 0.13* 0.37** 0.37** 0.31** 1.00 Civic_Eng 0.36** 0.22** 0.09 –0.09 0.09 0.05** 0.31** 0.45** 0.15** 1.00 Table 4 ‘Pathway’ Construct Validity (Convergent and Divergent): Bivariate Correlations between the Various Pathways at Year 1 Follow-Up (Baseline +1) Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.25** 1.00 Child_Edu –0.13* –0.01 1.00 Health_WB –0.29** –0.17** 0.10* 1.00 Housing 0.24** 0.04 –0.23** –0.19** 1.00 Soc_Bond 0.20** 0.27** –0.08 –0.12** 0.19* 1.00 Soc_Bridg 0.50** 0.47** –0.09 –0.32** 0.26** 0.50** 1.00 Lang_CK 0.54** 0.51** –0.03 –0.37** 0.25** 0.37** 0.79** 1.00 Safety 0.15** 0.20** –0.02 –0.08 0.13* 0.37** 0.37** 0.31** 1.00 Civic_Eng 0.36** 0.22** 0.09 –0.09 0.09 0.05** 0.31** 0.45** 0.15** 1.00 Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.25** 1.00 Child_Edu –0.13* –0.01 1.00 Health_WB –0.29** –0.17** 0.10* 1.00 Housing 0.24** 0.04 –0.23** –0.19** 1.00 Soc_Bond 0.20** 0.27** –0.08 –0.12** 0.19* 1.00 Soc_Bridg 0.50** 0.47** –0.09 –0.32** 0.26** 0.50** 1.00 Lang_CK 0.54** 0.51** –0.03 –0.37** 0.25** 0.37** 0.79** 1.00 Safety 0.15** 0.20** –0.02 –0.08 0.13* 0.37** 0.37** 0.31** 1.00 Civic_Eng 0.36** 0.22** 0.09 –0.09 0.09 0.05** 0.31** 0.45** 0.15** 1.00 Convergent validity measures constructs that theoretically should be related to each other and divergent validity measures constructs that should not theoretically be related to each other. These are measured by correlations. Correlations between theoretically similar constructs should be high, while correlations between theoretically dissimilar constructs should be low. In general, the correlations in Table 3 are lower than in Table 4 because there was much less variability in the ‘pathway’ scores at Baseline than at 1-Year follow-up (Baseline +1). Overall, though, the pattern of correlations does reflect what would be expected: constructs that are theoretically similar (e.g. Social Bridging and Language and Cultural Knowledge) are highly correlated (r = 0.79 at Baseline +1) and constructs that are theoretically dissimilar (e.g. Education and Training and Housing) have low or no correlation (r = 0.04 at Baseline +1). Many of the constructs are moderately correlated, reflecting some similarity in the pattern of scores (e.g. Social Bonding with Social Bridging (r = 0.50 at Baseline +1), Language and Cultural Knowledge (r = 0.37 at Baseline +1), Safety and Security (r = 0.37 at Baseline +1) and Civic Engagement (r = 0.31 at Baseline +1). Purpose 2: Application of CBPR for High Response Rates and Data Quality The response rates for the study were as follows: Baseline (B) +1 = 76 per cent; Baseline +2 = 73 per cent; and Baseline +3 = 70 per cent (Figure 1). During Baseline +3, the last year of data collection (2014–15), CCs administered 327 RISE surveys, which reflected 96 per cent retention from the prior year. Figure 1 View largeDownload slide Response Rate to RISE Survey Over Time Figure 1 View largeDownload slide Response Rate to RISE Survey Over Time Because surveys were administered orally in respondents’ native language, nearly all surveys were completed in their entirety, with very few ‘Don’t Know/Refused’ responses. Additionally, focus groups revealed that many of the refugees completed the survey because they felt that doing so would create better integration experiences for those refugees resettled in the future. Readers should keep in mind that the 327 surveys administered at Baseline +3 reflect approximately 70 per cent of the entire population of eligible, consenting adult refugees in the ethnic groups targeted for the study, which is rare for a survey project. Purpose 3: Gain Insight into the Integration Process for Newly Resettled Refugees The relationships between the pathways featured in Table 4 provided the first most significant insight into the integration process of newly resettled refugees. Second, Figure 2 shows OI for all refugees in the study over time. OI, on average, increased steadily from Baseline through Baseline +3 (B = 14.6; Baseline +1 = 22.4; Baseline +2 = 24.8; Baseline +3 = 30.2). Figure 2 View largeDownload slide Mean Integration Scores Over Time with Integration Categories Figure 2 View largeDownload slide Mean Integration Scores Over Time with Integration Categories Using the profile of OI scores at Baseline +1, researchers applied cut-points to distinguish low, medium and high integration scores (low = 1–16; medium = 17–24; high = 25–44). The distribution of the OI score over time is included in Figure 3. Figure 3 View largeDownload slide Frequency and Distribution of Integration Scores at Each RISE Administration Figure 3 View largeDownload slide Frequency and Distribution of Integration Scores at Each RISE Administration The distribution of OI scores over time are reflected by the following lines: dotted black line (Baseline), solid purple line (Baseline +1), brown line with intermittent breaks (Baseline +2) and thick red line (Baseline +3). The graph shows that refugee scores have shifted towards higher ranges each successive year, with most refugees falling into the high integration score category at Baseline +3. Comparing the red line (Baseline +3) to the fainter lines (cohort scores from prior years) shows how integration has steadily moved from low towards high integration scores across the four survey administrations. The percentage of respondents in each category was also tracked over time. At Baseline +1, 30 per cent of individuals with low integration scores shifted to medium or high integration scores. At Baseline +2, another 11 per cent shifted from low integration scores to medium or high integration scores, leaving 21 per cent of the population as having low integration scores at Baseline +2, compared to 63 per cent at Baseline. At Baseline +3, only 5 per cent of refugees were categorized as having low integration scores, 18 per cent were categorized as having medium integration scores and the vast majority (76 per cent) were categorized as having high integration scores. Figure 3 also shows this gradual shift of the cohort from low to high integration scores with each administration. Conclusion The RISE project was designed to fill a gap in the literature on refugee integration by operationalizing an integration framework, then creating a statistically reliable and valid survey to assess integration. The study also tested the efficacy of using a CBPR approach for collecting data in refugee communities. Finally, the study yielded data on the integration progress of an entire cohort of newly arrived refugees during the first four years of resettlement. RISE researchers believe that the survey instrument is both reliable and valid, based on evidence from both quantitative and qualitative analyses. Internal consistency (Cronbach’s alpha) and test–retest reliability were examined. The OI variable exibited respectable internal consistency at Baseline (α = 0.63) and at 1-Year (Baseline +1) (α = 0.60), given that the latent construct of integration is not a psychological construct in the way it was measured, but rather a checklist of behaviours. Test–retest correlations ranged from 0.17 to 0.72. The translation and back-translation approach and the cognitive interviews confirmed the meaning of the survey items (content validity). Construct validity was established by examining the pattern of correlations of the constructs over time. Constructs that were conceptually similar based on empirical evidence from the study or prior research (e.g. Social Bridging and Language and Cultural Knowledge) and were highly correlated (r = 0.79 at Baseline +1) and constructs that were theoretically dissimilar, or not associated in the context of integration, (e.g. Education and Training and Housing) had no correlation (r = 0.04 at Baseline +1), which is what was expected. Once the psychometric prosperities of the survey were deemed acceptable, a data-collection approach based on the principles of CBPR was used. The premise of the CC model was that refugee communities are impervious to outsiders, but ‘tightly knit’ from an insider’s perspective. Many refugees were resettled in Denver because they had family and friends already present, and these social networks created familiarity and safety in a new and ambiguous environment. The CCs were themselves refugees and, as such, embedded members of the refugee communities. They had broad and deep knowledge of individuals, families and social networks, and were bilingual in the languages of the target populations and in English. Often, the CCs were social connectors within their own communities. This enabled them to make contact in unexpected ways with refugee survey participants (e.g. finding the whereabouts of one refugee at an uncle’s wedding) that would have been unavailable to an outside researcher. The CC model is credited for the success of the survey data-collection efforts—namely the high response rates from Baseline (77, 73 and 70 per cent, respectively) and the 95 per cent annual retention from Baseline +1 through Baseline +3. This CBPR model also resulted in the collection of high-quality data (lack of missing data and production of meaningful data). The commitment of the CCs to this survey project was also paramount to the success of this survey project. The RISE research team believes that CCs’ strong commitment was due to the following reasons: (i) the CCs felt validated and appreciated—the study team acknowledged and celebrated the knowledge and wisdom of the CCs and, as such, gave the CCs a tremendous amount of respect, which minimized barriers that might have emerged using conventional, hierarchical research approaches; (ii) the CCs were integrally involved in all aspects of the study, including survey and research design, data collection and feedback on findings—which fostered the development of a sense of ownership of and investment in the project; and (iii) the CCs felt personally responsible for helping others in their community, and they viewed the survey project as a way to do this. Despite the successes related to the implementation of the CC model, challenges with this approach certainly arose. The CC model was effective, but not efficient. First, it required continuous communication with CCs, although their chaotic schedules and lack of access to/use of e-mail and phones (at times) made this communication and scheduling challenging. Second, retention of respondents within the target communities depended almost entirely on the ability of RISE research team to identify socially connected and reliable CCs. Due to the inability of the research team to identify and retain reliable Iraqi and Somali CCs, data collection with these groups was discontinued after the second survey administration. The creation of a sound integration measurement tool and the use of the CC model enabled the collection of high-quality, quantitative, longitudinal data, which in turn allowed researchers to get a ground-level perspective on the integration process for a cohort of refugees in Denver, Colorado. RISE data confirmed what professionals had suspected—refugees as a whole make steady progress towards integration during the first four years after arrival. As a result of RISE, data could be used to provide insights into refugee integration that were not previously available. RISE data spoke to differences in integration based on certain demographic characteristics (e.g. age) and the role of the family during integration. A future article will address distinct pathway profiles that emerged among those with high versus low integration scores. It was hypothesized that the CC model would increase the utility of the data and, in fact, we are starting to see that the data is being used in various ways. For example, the CRSP made note of the finding that social bridging can greatly influence integration and is working with a contractor to modify their programmes to include aspects of social bridging. The state refugee coordinators have built some of the findings into their messaging, as well as the pathway indicators into other aspects of their work. Welcoming America, a non-profit, non-partisan organization, has used the findings to promote the assets that refugees bring to the receiving communities. There are still many other ways in which the results of the RISE study can be used to support work with refugees. One limitation of the RISE study is that it focused on integration only on the refugee side of the equation and, as such, did not provide data related to the receiving community. As noted previously, the prevailing view is that integration involves a dynamic process between immigrant and refugees and the receiving community, so future research on the assessment of integration as it relates to the receiving community would be a valuable contribution to the field. Another limitation was that the OI score (each year) was created by summing core pathway scores. This meant that pathways with more items contributed more to the OI score than others. Although the pathways that did contribute more to the OI score (e.g. Language and Cultural Knowledge) and those that contributed very little (e.g. Housing) were consistent with the literature and what was learned from the focus groups, this does impact the calculation of the OI score, so the interpreter of such scores must keep this in mind. A third limitation is that the test–retest reliability was recorded across years, rather than within a more constricted time frame. This can create ‘noise’ in the results, since growth can occur between administrations. Other publications have included one-year test–retest reliability (Glenn and Klonsky 2011; Bruce et al. 2014; Scolaro et al. 2017), but further research should confirm these findings. Another limitation is that convergent validity was established using the integration pathways themselves; however, it was argued that the 10 pathways do not comprise a latent psychological construct. Another measure of integration was not separately administered to the participating refugees in this study, so the pathways themselves could only be used as a proxy to establishing construct validity. Lastly, the research team’s modification of Ager and Strang’s (2004) framework introduces some discontinuity in the research literature. The RISE Civic Engagement pathway is similar to Ager and Strang’s Social Links domain. Ager and Strang’s Citizenship and Rights domain has no equivalent in the RISE instrument. These differences can be assessed and reconciled over the course of further research. The RISE survey can and should continue to evolve to strengthen (and possibly reduce) pathway items and adapt items to be relevant in other contexts and communities. Pathway items can be integrated into other instruments in order to promote consistency in research related to refugee integration. This work confirmed that the integration approach is critical for gaining a full understanding of the complex, multifaceted experience of refugee resettlement. It is our hope that the RISE survey will be used by researchers and refugee service professionals to improve the understanding of and effectiveness in promoting refugee integration through research, policy and practice. Acknowledgements This publication was made possible with support from the United States Department of Health and Human Services, Office of Refugee Resettlement (ORR) through a grant to the Colorado Department of Human Services, Colorado Refugee Service Program CRSP (PO IHA CRSP1123064). CRSP contracted with Quality Evaluations Designs (PI: Dr Gary Lichtenstein) to conduct the RISE study. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the U.S. Department of Health and Human Services or CRSP. The authors wish to extend a sincere thank you to Dr Martin Tombari, Dr Laurie Bennett and Mrs Maggie Miller, all of whom were on the study team and whose contributions were invaluable to this study and the contents of this article. Additionally, the authors wish to thank Mr Joseph Wissman-Horther and Ms Kit Taintor for their support and guidance but, most importantly, the authors would like to extend a very warm thank you to the resettlement agencies in metro Denver, the CCs and all of the survey respondents, without whom, this project would not have been possible. References AGER A. , STRANG A. ( 2004 ) Indicators of Integration: Final Report . Home Office Development and Practice Report #28. London : Home Office , http://webarchive.nationalarchives.gov.uk/20110218135832/http:/rds.homeoffice.gov.uk/rds/pdfs04/dpr28.pdf (accessed 18 February 2017). AGER A. , STRANG A. ( 2008 ) ‘Understanding Integration: A Conceptual Framework’ . Journal of Refugee Studies 1 ( 2 ): 166 – 191 . Google Scholar CrossRef Search ADS ATFIELD G. , BRAHMBHATT K. , O’TOOLE T. ( 2007 ) Refugees’ Experiences of Integration . London, UK : University of Birmingham and the Refugee Council . BEIRENS H. , HUGHES N. , HEK R. , SPICER N. ( 2007 ) ‘ Preventing Social Exclusion of Refugee and Asylum Seeking Children: Building New Networks’ . Social Policy and Society 6 : 219 – 229 . Google Scholar CrossRef Search ADS BRUCE J. , ECHEMENDIA R. , MEEUWISSE W. , COMNER P. , SISCO A. ( 2014 ) ‘ 1 Year Test–Retest Reliability of ImPACT in Professional Ice Hockey Players’ . Clinical Neurophychology 28 ( 1 ): 14 – 25 . Google Scholar CrossRef Search ADS CAPPS R. , NEWLAND K. , FRATZKE S. , GROVES S. , AUCLAIR G. , FIX M. , MCHUGH M. ( 2015 ) The Integration Outcomes of U.S. Refugees: Successes and Challenges . Washington, DC : Migration Policy Institute . CARTY D. C. , KRUGER D. J. , TURNER T. M. , CAMPBELL B. , DELONEY E. H. , LEWIS Y. ( 2011 ) ‘ Racism, Health Status, and Birth Outcomes: Results of a Participatory Community-Based Intervention and Health Survey’ . Journal of Urban Health: Bulletin of the New York Academy of Medicine 88 ( 1 ): 84 – 96 . Google Scholar CrossRef Search ADS PubMed CEBULLA A. , DANIEL M. , ZURAWAN A. ( 2010 ) Spotlight on Refugee Integration: Findings from the Survey of New Refugees in the United Kingdom: Report #37 . London : Home Office . CHEUNG S. , PHILLIMORE J. ( 2017 ) ‘ Gender and Refugee Integration: A Quantitative Analysis of Integration and Social Policy Outcomes’ . Journal of Social Policy 46 ( 2 ): 211 – 230 . Google Scholar CrossRef Search ADS COHEN A. , LOPEZ A. , MALLOY N. , MORELLO-FROSCH R. ( 2011 ) ‘ Our Environment, Our Health: A Community-Based Participatory Environmental Health Survey in Richmond, California’ . Health Education and Behavior 39 : 198 – 209 . Google Scholar CrossRef Search ADS PubMed DE MAIO J. , SILBERT M. , JENKINSON R. , SMART D. ( 2014 ) ‘Building a New Life in Australia: Introducing the Longitudinal Study of Humanitarian Migrants’ . Family Matters 94 : 5 – 14 . GLENN C. R. , KLONSKY D. E. ( 2011 ) ‘ One-Year Test–Retest Reliability of the Inventory of Statements about Self-Injury (ISAS)’ . Assessment 18 ( 3 ): 375 – 378 . Google Scholar CrossRef Search ADS PubMed ISRAEL B. , SCHULZ A. , PARKER E. , BECKER A. ( 1998 ) ‘ Review of Community-Based Research: Assessing Partnership Approaches to Improve Public Health’ . Annual Review of Public Health 19 ( 1 ): 169 – 177 . Google Scholar CrossRef Search ADS KRUGER D. J. , HAMACHER L. , STRUGAR-FRITSCH D. , SHIREY L. , RENDA E. , ZIMMERMAN M. ( 2010 ) ‘ Facilitating the Development of a County Health Coverage Plan with Data from Community-Based Health Survey’ . Journal of Public Health Policy 31 ( 2 ): 199 – 211 . Google Scholar CrossRef Search ADS PubMed MAIN D. S. , WARE G. , IWASAKI P. G. , BURRY M. , STEINER E. , FEDDIE K. , HAVERHALS L. ( 2012 ) ‘Taking Neighborhood Health to Heart (TNH2H): Building a Community-Based Participatory Data System’ . Preventing Chronic Disease 9 : 110058 . http://dx.doi.org/10.5121/ijcseit.2012.2605 MULVEY G. ( 2015 ) ‘ Refugee Integration Policy: The Effects of UK Policy-Making on Refugees in Scotland’ . Journal of Social Policy 44 ( 2 ): 357 – 375 . Google Scholar CrossRef Search ADS NATIONAL ACADEMIES OF SCIENCES, ENGINEERING,MEDICINE ( 2015 ) ‘The Integration of Immigrants into American Society: Panel on the Integration of Immigrants into American Society’. In Waters M. C. , Pnieau M. G. (eds) Committee on Population, Division of Behavioral and Social Sciences and Education . Washington, DC : The National Academies Press . PETSOD D. , WANG T. , MCGARVEY C. ( 2006 ) Investing in Our Communities: Strategies for Immigrant Integration: A Toolkit for Grantmakers. Grantmakers Concerned with Immigrants and Refugees, https://www.gcir.org/sites/default/files/resources/GCIR%20Toolkit.pdf (accessed 18 February 2017). PUMA J. E. , BELANSKY E. S. , GARCIA R. , SCARBRO S. , WILLIFORD D. , MARSHALL J. ( 2016 ) ‘A Community-Engaged Approach to Collecting Rural Health Surveillance Data’ . Journal of Rural Health 00 : 1 – 9 . RIOSECO P. , DE MAIO J. , HOANG C. ( 2017 ) ‘ The Building a New Life in Australia (BNLA) Dataset: A Longitudinal Study of Humanitarian Migrants in Australia’ . Australian Economic Review 50 ( 3 ): 356 – 362 . Google Scholar CrossRef Search ADS ROBINSON V. ( 1998 ) ‘Defining and Measuring Successful Refugee Integration’. Proceedings of ECRE International Conference on Integration of Refugees in Europe, Antwerp, Brussels: ECRE. RUCINSKI D. , DAVIS R. , GOMEZ G. , FLORES J. , PEREZ I. , ZANONI J. ( 2011 ) ‘ An Approach to Assessing Data Quality in an Area Probability Survey Using CBPR’ . Health Promotion and Practice 12 ( 5 ): 704 – 712 . Google Scholar CrossRef Search ADS RUDMIN F. W. ( 2003 ) ‘Critical History of the Acculturation Psychology of Assimilation, Separation, Integration, and Marginalization’ . Review of General Psychology 7 ( 1 ): 3 – 37 . Google Scholar CrossRef Search ADS SCOLARO MOSERA R. , SCHATZC P. , GROSNERD E. , KOLLIASE K. ( 2017 ) ‘One Year Test–Retest Reliability of Neurocognitive Baseline Scores in 10- to 12-Year Olds’ . Applied Neurospsychology: Child 6 ( 2 ): 166 – 171 . Google Scholar CrossRef Search ADS SIRRIYEH A. ( 2007 ) ‘Young Asylum Seekers’ Conceptions of “Home” at a Time of Transition to Adulthood’. Disaporas, Migration and Identities Working Paper 6, Arts and Humanities Research Board/University of Leeds, http://web.archive.org/web/20150215055840/http://www.diasporas.ac.uk/ (accessed 18 February 2017). SMYTH G. , STEWART E. , DE LOMBA E. ( 2010 ) ‘ Introduction: Critical Reflections on Refugee Integration: Lessons from International Perspectives’ . Journal of Migration Studies 23 ( 4 ): 411 – 414 . UNITED NATIONS HIGH COMMISSIONER FOR REFUGEES (UNHCR) ( 2017 ) Refugee Figures, http://www.unhcr.org/uk/figures-at-a-glance.html (accessed 22 October 2017). WILLIS G. B. ( 1999 ) Cognitive Interviewing: A ‘How-To’ Guide. Presented at the annual meeting of the American Statistical Association. Alexandria. VA: Research Triangle Institute, http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=71CC295F5AF0F2D2C4B4804BF5CE5C9C?doi=10.1.1.469.6024&rep=rep1&type=pdf (accessed 18 February 2017). YU S. , OUELLET E. , WARMINGTON A. ( 2007 ) ‘Refugee Integration in Canada: A Survey of Empirical Evidence and Existing Services’ . Journal on Refugees 24 ( 2 ): 17 – 34 . © The Author(s) 2018. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Refugee Studies Oxford University Press

The RISE Survey: Developing and Implementing a Valid and Reliable Quantitative Measure of Refugee Integration in the United States

Loading next page...
 
/lp/ou_press/the-rise-survey-developing-and-implementing-a-valid-and-reliable-tjYhLaw39q
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com
ISSN
0951-6328
eISSN
1471-6925
D.O.I.
10.1093/jrs/fex047
Publisher site
See Article on Publisher Site

Abstract

Abstract Integration has been explored through research, policy and practice as a framework for gauging the extent to which refugees successfully navigate the economic, social and cultural dynamics of their new country; however, the definition and assessment of integration still remain elusive. The purpose of this study was to: (i) operationalize an integration framework to create a reliable and valid survey to assess refugee integration, (ii) test the use of a community-based participatory research (CPBR) model (the ‘Community Connector’ (CC) model) with refugees and (iii) gain insights into the integration process of newly resettled refugees with the use of the survey and CC model. The Refugee Integration Survey and Evaluation (RISE) survey was developed and administered orally to 465 newly arrived adult refugees by caseworkers. Follow-up survey data was collected annually over the course of three additional years using the CC model. The survey was deemed reliable and valid. The CC model led to a 70 per cent response rate three years after baseline data collection. On average, the Overall Integration score increased steadily over time. The implications of this study are that a gap in the literature was addressed and the RISE survey can play a key role in understanding the salient integration factors that could be used by receiving countries to promote integration and help ensure the successful resettlement of refugees around the world. Introduction Recent conflicts in the Middle East, primarily Syria, have increased the number of refugees seeking refuge in host countries in Europe and around the world, and have once again thrust a spot light on the significant topic of refugee resettlement and integration. Data from the United Nations High Commissioner for Refugees (UNHCR) estimated that there were a total of 22.5 million refugees worldwide in 2017—the highest level in 20 years (UNHCR 2017). The United States admits a significant number of refugees each year in an effort to support the international goal of providing humanitarian protection to individuals fleeing persecution. Beyond meeting core commitments for providing protection for refugees, absorbing these refugees so that they are supported in contributing to the economic and socio-cultural fabric of receiving countries is a significant challenge for governments and aid agencies. Integration has been explored through research, policy and practice as a framework for gauging the extent to which refugees successfully navigate the economic, social and cultural dynamics of their new country (Ager and Strang 2004). Yet the definition and assessment of integration for individuals remain elusive. Without a means to define and assess integration, it is difficult to identify refugees’ needs and support them with data-informed policies and services during the integration process. Moreover, without data and assessment within the integration framework, it is difficult for refugee resettlement advocates and practitioners to claim a seat at the table in broader community frameworks that might impact refugees, but are not focused on refugees. Literature Review Consensus in the literature about the definition of integration is lacking. Ager and Strang (2008) highlight the confusion and complexity around the notion of refugee integration when they cite a quote by Robinson (1998), which says ‘“integration” is a chaotic concept: a word used by many but understood differently by most’. Currently, the prevailing view is that Immigrant integration is a dynamic, two-way process in which newcomers and the receiving society work together to build secure, vibrant, and cohesive communities … . As an intentional effort, integration engages and transforms all community members, reaping shared benefits and creating a new whole that is greater than the sum of its parts (Petsod et al. 2006: 10). The National Academies of Sciences (2015) defines integration as ‘the extent to which immigrant and refugee populations (within and across generations) come to approximate the status of the native-born population’ (p. 20). This definition leaves room for broad interpretation regarding the specific dimensions along which resettled and native-born populations might be compared. Ager and Strang (2004, 2008) attempted to address the fact that there was no generally accepted definition, theory or model of refugee integration by developing a refugee integration framework. Their conceptual framework included four headings (Means and Markers, Social Connections, Facilitators and Foundation) and, within these headings, anywhere from one to four domains. Under the heading Markers and Means were the domains of Employment, Housing, Education and Health. Grouped together, these domains represent major areas of attainment that are widely recognized as critical factors in the integration process. Under that heading of Social Connections were the domains of Social Bridges, Social Bonds and Social Links. These domains stress the importance of relationships to the understanding of the integration process. There were two domains under the heading Facilitators: Language and Cultural Knowledge, and Safety and Stability, which represent key facilitating factors for the process of integration. Lastly, the only domain under the heading Foundations was Rights and Citizenship, which represents the basis upon which expectations and obligations for the process of integration are established. Ager and Strang developed their conceptual integration framework by conducting documentary and conceptual analysis, fieldwork in settings of refugee settlement, secondary analysis of cross-sectional survey data and verification via discussion at three major seminars. This framework has been used to inform studies of integration both conceptually and methodologically (Atfield et al. 2007; Beirens et al. 2007; Sirriyeh 2007). For this study, the integration framework put forth by Ager and Strang was chosen in an attempt to define integration. This framework was chosen for several reasons. First, the framework speaks to integration outcomes that concern policy-makers and researchers worldwide. Second, the framework was derived empirically, suggesting that the indicators are critical to refugees and members of host communities. Lastly, the framework conceptualized integration more behaviourally, compared to other models that conceptualized it more as a psychological construct (Rudmin 2003). The choosing of the Ager and Strang integration framework helped with the problem of how to define integration, but that did not solve the problem of the scarcity of literature about the assessment of integration, particularly the quantitative, longitudinal assessment of integration. The lack of quantitative, longitudinal data to assess integration hinders efforts to identify key factors that could be used by receiving countries, at both national and local levels, to promote integration and ensure the successful resettlement of refugees around the world. The National Academies of Sciences (2015) report pointedly makes reference to the lack of data regarding integration: The panel was handicapped in its work by the dearth of available longitudinal data to measure immigrant integration. This is a long-standing problem that has become increasingly critical as immigration to the United States has increased and as immigrants have become dispersed throughout the country (National Academies of Sciences 2015: 13). Another prominent report from the Migration Policy Institute (Capps et al. 2015) also cited the lack of data on integration as a hindrance to refugee policy and support: Public doubts and political resistance have emerged in the absence of solid data on the outcomes of resettlement efforts .... A lack of data also makes it difficult to identify problems that may have policy solutions or that may be specific to particular groups of refugees rather than to the resettled population as a whole (Capps et al. 2015: 3). Lastly, in 2010, in a special issue of the Journal of Refugee Studies, guest editors note the lack of longitudinal data available for increasing understanding of refugee integration: There is a general lack of research which examines the long-term experiences of refugees. Potential new insights could be gained into the elements of Ager and Strang’s (2008) framework of integration by mapping out the linkages with future migration paths. Examining the longitudinal experiences of refugees is, therefore, essential when considering refugee integration (Smyth et al. 2010: 413–414). An extensive literature search confirmed the dearth of quantitative, longitudinal data related to refugee integration. Nearly all studies of refugee integration, although revealing, were based on interview (qualitative) data. Quantitative studies found tended to be cross-sectional in nature, often using secondary data, and most were not analysed longitudinally. There were a handful of exceptions, however. Yu et al. (2007) described the patterns of refugees’ economic and socio-cultural integration as portrayed in nationally representative databases. They found the employment earnings of refugees at one and five years after arrival were slightly lower than the family-class immigrants’ earnings and, as expected, much lower than skilled-worker immigrants. Socio-cultural integration was much more of an elusive concept to define and to quantify and it was concluded that more research was needed in this area. Cebulla et al. (2010) mailed surveys to all newly arrived refugees (Survey of New Refugees (SNR)) who arrived in the United Kingdom between December 2005 and March 2007. They sent household surveys at eight, 15 and 21 months after arrival. Surveys included questions related to a range of integration indicators, including English-Language Skills, Employment, Housing and Health. However, initial response rates are not reported, and the response rate at the 21-month follow-up was only 17 per cent. Cheung and Phillimore (2017) reanalysed data from the 2005–07 SNR in order to assess differences in integration outcomes by gender. The authors found significant gender differences in language, self-reported health, ability to budget for household expenses and access to formal social networks and quality housing, with women generally faring worse than men and some inequalities enduring or intensifying over time. The Building a New Life in Australia (BNLA) study is a study that aims to better understand the factors that aid or hinder the successful settlement of humanitarian migrants in Australia, and to provide an evidence base to inform policy and program development (De Maio et al. 2014: p.5). The Australian Institute of Family Studies (AIFS) was commissioned by the Department of Immigration and Border Protection (DIBP) to undertake and manage the BNLA study. A total of 1,500 principal applicants from 11 sites in urban and rural communities are targeted for recruitment into the study. Annual survey data collection began in 2013 and will be completed in 2018. The surveys are collected via telephone interviews and in-person computer-assisted self-interviews (CASI) and are translated into 14 different languages. The surveys cover a variety of integration topics including: housing and neighbourhood, English-language proficiency, education and training, employment and income, immigration experience, health, self-sufficiency, community support, personal resources and life satisfaction, and perceptions of life in Australia. Rioseco et al. (2017) outline the main characteristics of the study and provide an overview of the measures available and potential uses. The first three waves of the BNLA data are currently available to approved users but, to date, no results have been published on the study. Finally, a mixed-methods longitudinal study of refugees and asylum seekers that used the Ager and Strang (2004) integration framework was conducted in Scotland between 2009 and 2013 (Mulvey 2015). A total of 262 respondents completed the main survey and a follow-up survey was administered one year later. There were several significant limitations of this study, however, including significant attrition; the questionnaire being translated into various languages, but administered in a written format, which posed challenges to illiterate participants; and the administration of the survey being conducted by voluntary organizations, which posed protocol problems and prevented the generalizability of the survey sample. The Refugee Integration Survey and Evaluation Ager and Strang (2008), themselves, note the framework’s wider utility and explanatory value now needs to be tested in diverse contexts to gauge whether the proposed structure captures key elements of stakeholder perceptions of what constitutes integration in an appropriately broad range of settings and timeframes (Ager and Strang 2008: p. 185, emphasis added). As such, the first purpose of the Refugee Integration Survey and Evaluation (RISE) study was to operationalize the integration framework conceptualized by Ager and Strang by developing a reliable and valid quantitative measure that could assess refugee integration over time in the United States. The second purpose of the study pertained to understanding the feasibility of applying community-based participatory research (CBPR) principles to collecting the RISE survey data. CBPR involves a commitment to conducting research that shares power with and engages community partners in the research process (Israel et al. 1998). The benefits of using a CBPR approach to survey data collection have been noted by various researchers (Israel et al. 1998; Kruger et al. 2010; Carty et al. 2011; Cohen et al. 2011; Rucinski et al. 2011; Main et al. 2012) and include: the fact that the research can addresses local concerns, more locally relevant data can be obtained, higher cooperation rates and lower refusal rates can result, more insightful interpretation of results can occur and there can be increased community buy-in for the results. The researchers who oversaw this study hypothesized that, in fact, a CBPR model (coined the Community Connector (CC) model in this study) could be adapted for use with the refugee population. The CC model was created by Puma et al. (2016) and was adapted and used in this study in order to leverage the relationships and cultural knowledge of the CCs. The research team hypothesized that the CBPR approach to collecting survey integration data in an urban city in the United States would: (i) increase the survey response rate for an otherwise, at-risk, highly mobile population; (ii) improve the quality of the data collected; and (iii) increase the utility of the data. The third purpose of the study was to learn about refugee integration in the Denver Metro area, since the Colorado Refugee Services Program (CRSP) (in Denver, Colorado) initiated the study. This purpose is only touched upon briefly in this article, but will be the primary focus of a forthcoming article. Methods Survey Development Initial survey development took place from November 2010 through July 2011. The development of the RISE survey instrument encompassed three phases: (i) selection of integration domains and items, (ii) expert panel review and (iii) piloting of the survey instrument with refugees. Ager and Strang’s framework served as the basis for the selection of integration domains (referred in this study as ‘pathways’, because ‘pathways’ better captures the dynamic two-way street of integration) and items; however, the framework was modified slightly. Modifications to the Ager and Strang framework included: the ‘Rights & Citizenship’ domain was converted into a ‘Civic Engagement’ pathway; the ‘Health’ domain was changed to a ‘Physical Health and Well-Being’ pathway because these seemed to better capture the pathways to integration; ‘Child’s Education’ was added as a pathway because it was believed that having children in school could be a ‘touch point’ to mainstream society for care-givers of the children; and the ‘Social Links’ domain was removed from the survey pathways because of the challenges to operationalizing it. These modifications emerged organically from focus groups and feedback from refugees and refugee service professionals. Once pathways were identified, survey items were created. There was intentionality around making the items objective (behaviourally oriented), as opposed to subjective (opinion-/psychologically oriented). An example of an objective item is ‘Within the past year, have you obtained a certificate that qualifies you for a specific kind of job in the U.S.?’ (Yes, No, I Don’t Know). Once an item pool was compiled, it was sent to a panel of three experts to review. Individuals were selected to serve on the panel who had experience in working on state and federal task forces on refugee integration and who had worked with refugees and/or did integration research. Each of the experts was given a list of the items/indicators, and asked to rank them in order of relevance to integration and to edit the list by deleting, revising or adding items that were important to refugee integration. Their summarized reviews were used to refine the draft survey. The resulting 10 pathways assessed in the RISE survey are included in Table 1. Table 1 Ten Survey Pathways and Corresponding Item Concepts Survey pathway Item concepts Employment Employment status Number of jobs and hours worked Pay and benefits Sufficiency of income to meet family needs Education & Training Certification/training/classes received or taken in the U.S. Enrollment in any degree programmes Child’s Education Children’s grade levels in school Volunteer experience at their child(ren)’s schools Visits with child(ren)’s teachers Attendance at school events Child(ren)’s friends from different cultures Physical Health & Well-Being Status of routine physical check-ups Has regular doctor(s) and/or dentist Knowledge of how to make appointments Has health insurance coverage Housing Housing situation (homeless, transitional housing, home owner) Housing type and number of rooms in the dwelling Frequency of moves Social Bonding Interaction with people from same culture/ethnic group Information sources from same culture/ethnic group Attended events from the same culture/ethnic group Social Bridging Interaction with people from different culture/ethnic group Information sources from different culture/ethnic group Attended events from the different culture/ethnic group Language & Cultural Knowledge Self-reported proficiency in English (speaking) Familiarity with U.S. cultural facts (who is the president, where is Congress and the White House located, three sports that are popular in the U.S.) Safety & Stability Feelings of safety at home and outside the home Comfort level with calling on police or the fire department Trust of neighbors Victim of crime or discrimination Civic Engagement Participation in community meetings, clubs or organizations Advocacy or ‘speaking up’ for rights Possession of a driver’s license Applied for green card Desire for U.S. citizenship Survey pathway Item concepts Employment Employment status Number of jobs and hours worked Pay and benefits Sufficiency of income to meet family needs Education & Training Certification/training/classes received or taken in the U.S. Enrollment in any degree programmes Child’s Education Children’s grade levels in school Volunteer experience at their child(ren)’s schools Visits with child(ren)’s teachers Attendance at school events Child(ren)’s friends from different cultures Physical Health & Well-Being Status of routine physical check-ups Has regular doctor(s) and/or dentist Knowledge of how to make appointments Has health insurance coverage Housing Housing situation (homeless, transitional housing, home owner) Housing type and number of rooms in the dwelling Frequency of moves Social Bonding Interaction with people from same culture/ethnic group Information sources from same culture/ethnic group Attended events from the same culture/ethnic group Social Bridging Interaction with people from different culture/ethnic group Information sources from different culture/ethnic group Attended events from the different culture/ethnic group Language & Cultural Knowledge Self-reported proficiency in English (speaking) Familiarity with U.S. cultural facts (who is the president, where is Congress and the White House located, three sports that are popular in the U.S.) Safety & Stability Feelings of safety at home and outside the home Comfort level with calling on police or the fire department Trust of neighbors Victim of crime or discrimination Civic Engagement Participation in community meetings, clubs or organizations Advocacy or ‘speaking up’ for rights Possession of a driver’s license Applied for green card Desire for U.S. citizenship Table 1 Ten Survey Pathways and Corresponding Item Concepts Survey pathway Item concepts Employment Employment status Number of jobs and hours worked Pay and benefits Sufficiency of income to meet family needs Education & Training Certification/training/classes received or taken in the U.S. Enrollment in any degree programmes Child’s Education Children’s grade levels in school Volunteer experience at their child(ren)’s schools Visits with child(ren)’s teachers Attendance at school events Child(ren)’s friends from different cultures Physical Health & Well-Being Status of routine physical check-ups Has regular doctor(s) and/or dentist Knowledge of how to make appointments Has health insurance coverage Housing Housing situation (homeless, transitional housing, home owner) Housing type and number of rooms in the dwelling Frequency of moves Social Bonding Interaction with people from same culture/ethnic group Information sources from same culture/ethnic group Attended events from the same culture/ethnic group Social Bridging Interaction with people from different culture/ethnic group Information sources from different culture/ethnic group Attended events from the different culture/ethnic group Language & Cultural Knowledge Self-reported proficiency in English (speaking) Familiarity with U.S. cultural facts (who is the president, where is Congress and the White House located, three sports that are popular in the U.S.) Safety & Stability Feelings of safety at home and outside the home Comfort level with calling on police or the fire department Trust of neighbors Victim of crime or discrimination Civic Engagement Participation in community meetings, clubs or organizations Advocacy or ‘speaking up’ for rights Possession of a driver’s license Applied for green card Desire for U.S. citizenship Survey pathway Item concepts Employment Employment status Number of jobs and hours worked Pay and benefits Sufficiency of income to meet family needs Education & Training Certification/training/classes received or taken in the U.S. Enrollment in any degree programmes Child’s Education Children’s grade levels in school Volunteer experience at their child(ren)’s schools Visits with child(ren)’s teachers Attendance at school events Child(ren)’s friends from different cultures Physical Health & Well-Being Status of routine physical check-ups Has regular doctor(s) and/or dentist Knowledge of how to make appointments Has health insurance coverage Housing Housing situation (homeless, transitional housing, home owner) Housing type and number of rooms in the dwelling Frequency of moves Social Bonding Interaction with people from same culture/ethnic group Information sources from same culture/ethnic group Attended events from the same culture/ethnic group Social Bridging Interaction with people from different culture/ethnic group Information sources from different culture/ethnic group Attended events from the different culture/ethnic group Language & Cultural Knowledge Self-reported proficiency in English (speaking) Familiarity with U.S. cultural facts (who is the president, where is Congress and the White House located, three sports that are popular in the U.S.) Safety & Stability Feelings of safety at home and outside the home Comfort level with calling on police or the fire department Trust of neighbors Victim of crime or discrimination Civic Engagement Participation in community meetings, clubs or organizations Advocacy or ‘speaking up’ for rights Possession of a driver’s license Applied for green card Desire for U.S. citizenship The survey comprised two types of questions: core items (n = 43), which were scored and which contributed to the pathway scores and to the Overall Integration (OI) score, and auxiliary items (n = 55), which were not scored, but which provided supplemental information about factors related to integration. Although an attempt was made to avoid opinion-based items generally, six out of 43 scored items were opinion questions (e.g. ‘Is your family’s monthly income sufficient to cover rent, food and other expenses necessary for daily living?’ (Too Low, Enough, More than Enough) and ‘Which of the following best describes your English-language skill?’ (I cannot speak English; I speak a little English when shopping and doing other types of business; I can speak English; I can speak English in most Social & Work situations; I am fluent in English). A draft survey was then piloted with different groups of refugees. The first group included five members of the RISE Advisory Committee (RAC) who were refugees from the survey target populations (Burmese, Bhutanese, Iraqi and Somali), had been in the United States for five or more years and had excellent English skills. The RAC members were asked to both take the survey and disseminate it to at least two refugee acquaintances, and then give feedback on the items. Based on feedback from the RAC, further revisions were made. The second group included refugee resettlement agency (formerly called voluntary agency or VOLAG) case managers and their supervisors (most of whom had been refugees when they were admitted to the United States several years previously), as well as the Coordinator of the CRSP at the time (article co-author) and the Integration Partnership Coordinator for the state. They were given versions of the draft survey twice: once to review and share with other refugees of their acquaintance and, second, to test-administer with one another, since case managers would be administering the survey to refugees. Finally, the initial survey administration period, from June through August 2011, was considered to be a piloting period. During this time, even though the data would be used in the study, case managers reported issues that resulted in minor modifications of several survey items. Data Collection Once the survey instrument was created, survey administration commenced. The baseline survey was administered orally by refugee resettlement agency caseworkers in the refugee’s native language (Arabic, Burmese, Karen, Nepali or Somali) at the 90-day post-arrival meeting. All refugees over the age of 18 years old were invited to complete the survey. The survey took approximately 40 minutes to complete and no incentives were given to complete the survey. Surveys were completed for 465 newly arrived adult refugees (out of 480 newly arrived refugees in the 2011–12 fiscal year). In subsequent study years, survey data was collected by ‘CCs’, leaders and reliable members from the target community who were fluent in refugees’ home language and in English and who had sufficient literacy skills and education levels to understand and administer the survey. CCs were trained by the research team to adhere to strict study protocols and Institutional Review Board (IRB) human subjects requirements. Post-Administration Survey Tweaking After the first year of survey administration, the survey psychometrics were further scrutinized via two approaches: (i) a translation and back-translation approach and (ii) cognitive interviewing approach. A firm that had not translated the survey initially was hired to retranslate the survey from English into Arabic, Burmese, Karen, Nepali and Somali, using professional, certified translators. Once that was done, study researchers asked different translators (also certified) to back-translate the newly translated Year 2 survey from the target language back into English. Due to budget constraints, not all survey items were examined in this way. A total of 71 per cent of core (scored) items (35 out of 49) and other seemingly problematic supplemental items were back-translated. The 35 core items were back-translated into five different languages, which resulted in a total of 175 back-translated items. Across all languages, 13 per cent of the items were found to contain minor differences between the Year 1 and Year 2 translations. The same items were not flagged across languages; different items were problematic in different languages. Karen appeared to have the least problems with translation (n = 1 item) and Somali appeared to have the most problems with translation (n = 8 items). All translation errors were corrected. Cognitive interviews were completed in order to understand whether various items were being understood and interpreted by the refugees as the research team intended (face/content validity). The cognitive interview is a verbal interview technique that attempts to review non-observable understanding that motivates behaviour (Willis 1999). The RISE cognitive interviews were conducted by a researcher (manuscript lead author), with the assistance of an interpreter, and with one or two refugees at a time. It should be noted that the interpreter, who was also a CC, was also used as a ‘key informant’ throughout the cognitive interviews because of his/her familiarity with the refugees’ language and culture, as well as the RISE survey items. The cognitive interviews aimed to understand: (i) how refugees understood particular items on the RISE survey (e.g. What did refugees believe these items are asking? What do certain words and phrases mean to respondents?) and (ii) in what ways items could be reframed to elicit the information intended. Participants were a convenience sample of refugees who had been in the United States for 90–120 days at the time the cognitive interviews were conducted. Attempts were made to balance for gender (and age in some cases) within each language group. A total of nine refugees participated (six from Bhutan (spoke Nepali), three from Burma (two spoke Burmese, one spoke Karen)). More Nepali speakers were selected due to their prevalence in our sample. Twelve core survey items (about 25 per cent of scored items) were identified for the cognitive interviews, as well as four supplemental items. The cognitive interviews took place in April 2013 in the community-use room at an apartment building, in which many of the refugees lived, and in a close-by apartment of an interpreter (when the community-use room was unavailable). For each selected item, the following protocol was generally followed. First, the interpreter read the question and asked the participant to respond to the question. Second, each refugee was then asked what he or she believed was being asked by the question. In some cases, the researcher asked refugees to define specific terms or phrases in the question. Third, after the interpreter related refugees’ responses, the researcher may have asked the interpreter, as well as refugee(s), for clarification, using spontaneous probes. Finally, the researcher then explained (through the help of the interpreter) the intention behind the question, which the interpreter would relay to refugees. Once refugees understood the intention, they were asked how questions might have been phrased to more clearly/adequately reflect the intention. Interviews lasted approximately one hour. All interviews were recorded, transcribed and analysed. Cognitive interview results suggested a number of changes in the wording of particular survey items—for example, removal of the term ‘license’ from a question about acquisition of job certification, because of the confusion with ‘driver’s licenses’; removal of the term ‘degree program’ and replacing it with ‘community college or college’ because participants did not understand the term; replacing the term ‘other than your family’ with ‘who are not related to you’ in the social bonding questions about time spent with others, due to confusion about what ‘family’ meant in this context; removal of the item ‘do you trust your neighbors?’ due to confusion as to what ‘trust’ and what ‘neighbors’ meant. Changes strengthened the instrument; however, the most noteworthy finding was that nearly all respondents understood the items as intended, reassuring the research team of the validity of the survey. Developing a reliable and valid refugee integration survey was the primary focus of RISE for the first two years of the project. By the end of Year 3 (the second year of data collection—Baseline +2), the RISE team concluded the study of the psychometric properties of the survey instrument (although minor changes to the survey were made in later years of survey administration, such as the addition of items that probed for challenges related to transportation). In Years 4 and 5 (Baseline +3 and Baseline +4), the primarily focus of the study shifted from the reliability and validity of the survey to findings that pertained to refugees, particularly in terms of progress along various integration pathways and for integration overall. Data Analysis Techniques To address the first purpose of the study, reliability was assessed quantitatively by examining internal consistency of the latent construct of ‘overall integration’. Additionally, researchers examined one-year test–retest reliability across survey administrations. Face and content validity were assessed qualitatively via expert panel reviews of items and the cognitive interviews (described above). Construct validity (convergent and divergent validity) was assessed quantitatively by running bivariate correlations between the various pathways across time. To address the second purpose of the study pertaining to the utility of a CBPR approach to collecting the RISE survey data, the survey response rates were tracked over time. Additionally, the trend in the OI score over time was examined as both a proxy of quality (and usefulness) of the survey results (Purpose 2) and to gain insight into refugee integration in Denver, Colorado, over time (Purpose 3). The OI score (each year) was created by summing core pathway scores (Yes = 1 point and No = 0 points). This meant that pathways with more items contributed more to the OI score than others, but the pathways that did contribute more to the OI score (e.g. Language and Cultural Knowledge) and those that contributed very little (e.g. Housing) were consistent with the literature and what was learned from the focus groups that had been conducted. The pathway scores were created by reverse-scoring the items that needed to be reverse-scored (so that a higher value was viewed as more positive for integration) and then summing all items within the individual pathways. Any missing values, which was a nominal amount, were assigned a score of zero so that the pathway items could be summed and the maximum sample could be preserved for the analyses. Descriptive cut-points were determined based on the analysis of the frequency distribution of scores during the first two years of data collection. The research team noted natural breaks in the distribution curve and assigned low, medium and high integration categories accordingly. Results In this section, results are organized according to the three purposes of the study: creation of a reliable and valid survey to assess refugee integration (using Ager and Strang’s framework); test the efficacy of a CBPR model in data collection for a vulnerable, difficult-to-access refugee population; gain insights into the integration process of newly resettled refugees in Denver, Colorado. Purpose 1: Integration Survey Reliability and Validity Internal consistency (Cronbach’s alpha) measures whether scores on similar items are related (internally consistent). Another way to think about internal consistency is it is the extent to which all of the items of a test measure the same latent variable. Because the latent construct of interest was OI, the research team examined internal consistency for the OI variable (n = 10 pathways) at Baseline (α = 0.63) and at 1-Year (Baseline +1) (α = 0.60). Although Cronbach’s alpha was calculated for OI, it should be emphasized that the latent construct of integration was not operationalized as a psychological construct. It was measured more as a checklist of behaviours, which, when aggregated, led to ‘pathways’ (to integration) and, when these ‘pathways’ were aggregated, yielded an OI score. Chronbach’s alphas calculated at Baseline and at 1-Year follow-up indicate that the integration ‘pathways’ were related to each other at both time points, and that the pathways were not completely redundant in the information that they were contributing to the OI construct. In other words, each pathway contributed unique information to the measurement of OI, which is what is desired and which is what would be expected given that some of the pathways (e.g. Health and Well-Being and Child Education) would not theoretically be related to each other. Internal consistency was not measured within pathways because: (i) pathways are tallies of key behaviours (individual items) related to a particular facet of integration (rather than physiological constructs) and (ii) several pathways did not have a large enough number of items to warrant an internal consistency reliability analysis (e.g. Educational Training, Housing, Social Bonding and Social Bridging all had three items or fewer). Test–retest reliability was assessed across Baseline and 1-Year follow-up (Baseline +1). Results are presented in Table 2. Test–retest correlations ranged from 0.17 to 0.72. The correlation for OI at Baseline and ‘Baseline +1’ (r = 0.60) is moderate to high and reflects the fact that there is some stability in refugees’ scores over time; however, there is also variability in the growth of these scores over time. The range of the pathway scores is consistent with this notion. Table 2 Descriptive Statistics and Test–Retest Reliability (r) for Pathway and Overall Integration Scores Domain Baseline Mean (SD) 1-Year follow-up Mean (SD) Pearson correlation (r) Employment and Economic Self-Sufficiency 1.1 (2.0) 3.2 (2.9) 0.30** Education and Training 0.4 (0.7) 0.6 (0.6) 0.39** Child(ren)’s Education 0.4 (0.8) 0.4 (1.0) 0.35** Physical Health and Well-Being 2.2 (1.1) 1.8 (1.0) – Housing 19% 87% McNemar Test*** Social Bonding 1.5 (1.1) 2.5 (0.9) 0.17** Social Bridging 0.8 (1.0) 1.7 (1.3) 0.24** Language and Cultural Knowledge 4.6 (3.3) 6.4 (3.9) 0.72** Safety and Stability 3.3 (1.2) 3.8 (0.6) 0.24** Civic Engagement 0.1 (0.4) 1.4 (0.7) 0.22** Overall Integration 14.6 (6.9) 22.4 (8.1) 0.60** Domain Baseline Mean (SD) 1-Year follow-up Mean (SD) Pearson correlation (r) Employment and Economic Self-Sufficiency 1.1 (2.0) 3.2 (2.9) 0.30** Education and Training 0.4 (0.7) 0.6 (0.6) 0.39** Child(ren)’s Education 0.4 (0.8) 0.4 (1.0) 0.35** Physical Health and Well-Being 2.2 (1.1) 1.8 (1.0) – Housing 19% 87% McNemar Test*** Social Bonding 1.5 (1.1) 2.5 (0.9) 0.17** Social Bridging 0.8 (1.0) 1.7 (1.3) 0.24** Language and Cultural Knowledge 4.6 (3.3) 6.4 (3.9) 0.72** Safety and Stability 3.3 (1.2) 3.8 (0.6) 0.24** Civic Engagement 0.1 (0.4) 1.4 (0.7) 0.22** Overall Integration 14.6 (6.9) 22.4 (8.1) 0.60** *p < 0.5; **p < 0.01; ***p < 0.001;—N/A because all newly arrived refugees saw a physician during their initial health screening, which biased the baseline results. Table 2 Descriptive Statistics and Test–Retest Reliability (r) for Pathway and Overall Integration Scores Domain Baseline Mean (SD) 1-Year follow-up Mean (SD) Pearson correlation (r) Employment and Economic Self-Sufficiency 1.1 (2.0) 3.2 (2.9) 0.30** Education and Training 0.4 (0.7) 0.6 (0.6) 0.39** Child(ren)’s Education 0.4 (0.8) 0.4 (1.0) 0.35** Physical Health and Well-Being 2.2 (1.1) 1.8 (1.0) – Housing 19% 87% McNemar Test*** Social Bonding 1.5 (1.1) 2.5 (0.9) 0.17** Social Bridging 0.8 (1.0) 1.7 (1.3) 0.24** Language and Cultural Knowledge 4.6 (3.3) 6.4 (3.9) 0.72** Safety and Stability 3.3 (1.2) 3.8 (0.6) 0.24** Civic Engagement 0.1 (0.4) 1.4 (0.7) 0.22** Overall Integration 14.6 (6.9) 22.4 (8.1) 0.60** Domain Baseline Mean (SD) 1-Year follow-up Mean (SD) Pearson correlation (r) Employment and Economic Self-Sufficiency 1.1 (2.0) 3.2 (2.9) 0.30** Education and Training 0.4 (0.7) 0.6 (0.6) 0.39** Child(ren)’s Education 0.4 (0.8) 0.4 (1.0) 0.35** Physical Health and Well-Being 2.2 (1.1) 1.8 (1.0) – Housing 19% 87% McNemar Test*** Social Bonding 1.5 (1.1) 2.5 (0.9) 0.17** Social Bridging 0.8 (1.0) 1.7 (1.3) 0.24** Language and Cultural Knowledge 4.6 (3.3) 6.4 (3.9) 0.72** Safety and Stability 3.3 (1.2) 3.8 (0.6) 0.24** Civic Engagement 0.1 (0.4) 1.4 (0.7) 0.22** Overall Integration 14.6 (6.9) 22.4 (8.1) 0.60** *p < 0.5; **p < 0.01; ***p < 0.001;—N/A because all newly arrived refugees saw a physician during their initial health screening, which biased the baseline results. Face and content validity were assessed qualitatively via expert panel reviews of items and the cognitive interviews (described above). Construct validity (convergent and divergent) was assessed quantitatively by running bivariate correlations between the various pathways at Baseline (Table 3) and 1-Year (Baseline +1) (Table 4). Table 3 ‘Pathway’ Construct Validity (Convergent and Divergent): Bivariate Correlations between the Various Pathways at Baseline Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.14** 1.00 Child_Edu –0.10* –0.16** 1.00 Health_WB 0.00 0.15** 0.14** 1.00 Housing 0.15** –0.04 0.03 0.08 1.00 Soc_Bond 0.17** 0.11** 0.06 0.20** 0.25** 1.00 Soc_Bridg 0.15** 0.16** 0.08 –0.01 0.22** 0.46** 1.00 Lang_CK 0.30** 0.36** 0.00 0.15** 0.21** 0.45** 0.54** 1.00 Safety 0.20** 0.20** 0.011* 0.38** 0.12** 0.41** 0.34** 0.55** 1.00 Civic_Eng 0.17** 0.05 0.07 0.09* 0.10* 0.23** 0.13** 0.32** 0.12** 1.00 Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.14** 1.00 Child_Edu –0.10* –0.16** 1.00 Health_WB 0.00 0.15** 0.14** 1.00 Housing 0.15** –0.04 0.03 0.08 1.00 Soc_Bond 0.17** 0.11** 0.06 0.20** 0.25** 1.00 Soc_Bridg 0.15** 0.16** 0.08 –0.01 0.22** 0.46** 1.00 Lang_CK 0.30** 0.36** 0.00 0.15** 0.21** 0.45** 0.54** 1.00 Safety 0.20** 0.20** 0.011* 0.38** 0.12** 0.41** 0.34** 0.55** 1.00 Civic_Eng 0.17** 0.05 0.07 0.09* 0.10* 0.23** 0.13** 0.32** 0.12** 1.00 Table 3 ‘Pathway’ Construct Validity (Convergent and Divergent): Bivariate Correlations between the Various Pathways at Baseline Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.14** 1.00 Child_Edu –0.10* –0.16** 1.00 Health_WB 0.00 0.15** 0.14** 1.00 Housing 0.15** –0.04 0.03 0.08 1.00 Soc_Bond 0.17** 0.11** 0.06 0.20** 0.25** 1.00 Soc_Bridg 0.15** 0.16** 0.08 –0.01 0.22** 0.46** 1.00 Lang_CK 0.30** 0.36** 0.00 0.15** 0.21** 0.45** 0.54** 1.00 Safety 0.20** 0.20** 0.011* 0.38** 0.12** 0.41** 0.34** 0.55** 1.00 Civic_Eng 0.17** 0.05 0.07 0.09* 0.10* 0.23** 0.13** 0.32** 0.12** 1.00 Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.14** 1.00 Child_Edu –0.10* –0.16** 1.00 Health_WB 0.00 0.15** 0.14** 1.00 Housing 0.15** –0.04 0.03 0.08 1.00 Soc_Bond 0.17** 0.11** 0.06 0.20** 0.25** 1.00 Soc_Bridg 0.15** 0.16** 0.08 –0.01 0.22** 0.46** 1.00 Lang_CK 0.30** 0.36** 0.00 0.15** 0.21** 0.45** 0.54** 1.00 Safety 0.20** 0.20** 0.011* 0.38** 0.12** 0.41** 0.34** 0.55** 1.00 Civic_Eng 0.17** 0.05 0.07 0.09* 0.10* 0.23** 0.13** 0.32** 0.12** 1.00 Table 4 ‘Pathway’ Construct Validity (Convergent and Divergent): Bivariate Correlations between the Various Pathways at Year 1 Follow-Up (Baseline +1) Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.25** 1.00 Child_Edu –0.13* –0.01 1.00 Health_WB –0.29** –0.17** 0.10* 1.00 Housing 0.24** 0.04 –0.23** –0.19** 1.00 Soc_Bond 0.20** 0.27** –0.08 –0.12** 0.19* 1.00 Soc_Bridg 0.50** 0.47** –0.09 –0.32** 0.26** 0.50** 1.00 Lang_CK 0.54** 0.51** –0.03 –0.37** 0.25** 0.37** 0.79** 1.00 Safety 0.15** 0.20** –0.02 –0.08 0.13* 0.37** 0.37** 0.31** 1.00 Civic_Eng 0.36** 0.22** 0.09 –0.09 0.09 0.05** 0.31** 0.45** 0.15** 1.00 Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.25** 1.00 Child_Edu –0.13* –0.01 1.00 Health_WB –0.29** –0.17** 0.10* 1.00 Housing 0.24** 0.04 –0.23** –0.19** 1.00 Soc_Bond 0.20** 0.27** –0.08 –0.12** 0.19* 1.00 Soc_Bridg 0.50** 0.47** –0.09 –0.32** 0.26** 0.50** 1.00 Lang_CK 0.54** 0.51** –0.03 –0.37** 0.25** 0.37** 0.79** 1.00 Safety 0.15** 0.20** –0.02 –0.08 0.13* 0.37** 0.37** 0.31** 1.00 Civic_Eng 0.36** 0.22** 0.09 –0.09 0.09 0.05** 0.31** 0.45** 0.15** 1.00 Table 4 ‘Pathway’ Construct Validity (Convergent and Divergent): Bivariate Correlations between the Various Pathways at Year 1 Follow-Up (Baseline +1) Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.25** 1.00 Child_Edu –0.13* –0.01 1.00 Health_WB –0.29** –0.17** 0.10* 1.00 Housing 0.24** 0.04 –0.23** –0.19** 1.00 Soc_Bond 0.20** 0.27** –0.08 –0.12** 0.19* 1.00 Soc_Bridg 0.50** 0.47** –0.09 –0.32** 0.26** 0.50** 1.00 Lang_CK 0.54** 0.51** –0.03 –0.37** 0.25** 0.37** 0.79** 1.00 Safety 0.15** 0.20** –0.02 –0.08 0.13* 0.37** 0.37** 0.31** 1.00 Civic_Eng 0.36** 0.22** 0.09 –0.09 0.09 0.05** 0.31** 0.45** 0.15** 1.00 Employ Educ_ Train Child_ Edu Health_ WB Housing Soc_ Bond Soc_ Bridg Lang_ CK Safety Civic_ Eng Employ 1.00 Educ_Train 0.25** 1.00 Child_Edu –0.13* –0.01 1.00 Health_WB –0.29** –0.17** 0.10* 1.00 Housing 0.24** 0.04 –0.23** –0.19** 1.00 Soc_Bond 0.20** 0.27** –0.08 –0.12** 0.19* 1.00 Soc_Bridg 0.50** 0.47** –0.09 –0.32** 0.26** 0.50** 1.00 Lang_CK 0.54** 0.51** –0.03 –0.37** 0.25** 0.37** 0.79** 1.00 Safety 0.15** 0.20** –0.02 –0.08 0.13* 0.37** 0.37** 0.31** 1.00 Civic_Eng 0.36** 0.22** 0.09 –0.09 0.09 0.05** 0.31** 0.45** 0.15** 1.00 Convergent validity measures constructs that theoretically should be related to each other and divergent validity measures constructs that should not theoretically be related to each other. These are measured by correlations. Correlations between theoretically similar constructs should be high, while correlations between theoretically dissimilar constructs should be low. In general, the correlations in Table 3 are lower than in Table 4 because there was much less variability in the ‘pathway’ scores at Baseline than at 1-Year follow-up (Baseline +1). Overall, though, the pattern of correlations does reflect what would be expected: constructs that are theoretically similar (e.g. Social Bridging and Language and Cultural Knowledge) are highly correlated (r = 0.79 at Baseline +1) and constructs that are theoretically dissimilar (e.g. Education and Training and Housing) have low or no correlation (r = 0.04 at Baseline +1). Many of the constructs are moderately correlated, reflecting some similarity in the pattern of scores (e.g. Social Bonding with Social Bridging (r = 0.50 at Baseline +1), Language and Cultural Knowledge (r = 0.37 at Baseline +1), Safety and Security (r = 0.37 at Baseline +1) and Civic Engagement (r = 0.31 at Baseline +1). Purpose 2: Application of CBPR for High Response Rates and Data Quality The response rates for the study were as follows: Baseline (B) +1 = 76 per cent; Baseline +2 = 73 per cent; and Baseline +3 = 70 per cent (Figure 1). During Baseline +3, the last year of data collection (2014–15), CCs administered 327 RISE surveys, which reflected 96 per cent retention from the prior year. Figure 1 View largeDownload slide Response Rate to RISE Survey Over Time Figure 1 View largeDownload slide Response Rate to RISE Survey Over Time Because surveys were administered orally in respondents’ native language, nearly all surveys were completed in their entirety, with very few ‘Don’t Know/Refused’ responses. Additionally, focus groups revealed that many of the refugees completed the survey because they felt that doing so would create better integration experiences for those refugees resettled in the future. Readers should keep in mind that the 327 surveys administered at Baseline +3 reflect approximately 70 per cent of the entire population of eligible, consenting adult refugees in the ethnic groups targeted for the study, which is rare for a survey project. Purpose 3: Gain Insight into the Integration Process for Newly Resettled Refugees The relationships between the pathways featured in Table 4 provided the first most significant insight into the integration process of newly resettled refugees. Second, Figure 2 shows OI for all refugees in the study over time. OI, on average, increased steadily from Baseline through Baseline +3 (B = 14.6; Baseline +1 = 22.4; Baseline +2 = 24.8; Baseline +3 = 30.2). Figure 2 View largeDownload slide Mean Integration Scores Over Time with Integration Categories Figure 2 View largeDownload slide Mean Integration Scores Over Time with Integration Categories Using the profile of OI scores at Baseline +1, researchers applied cut-points to distinguish low, medium and high integration scores (low = 1–16; medium = 17–24; high = 25–44). The distribution of the OI score over time is included in Figure 3. Figure 3 View largeDownload slide Frequency and Distribution of Integration Scores at Each RISE Administration Figure 3 View largeDownload slide Frequency and Distribution of Integration Scores at Each RISE Administration The distribution of OI scores over time are reflected by the following lines: dotted black line (Baseline), solid purple line (Baseline +1), brown line with intermittent breaks (Baseline +2) and thick red line (Baseline +3). The graph shows that refugee scores have shifted towards higher ranges each successive year, with most refugees falling into the high integration score category at Baseline +3. Comparing the red line (Baseline +3) to the fainter lines (cohort scores from prior years) shows how integration has steadily moved from low towards high integration scores across the four survey administrations. The percentage of respondents in each category was also tracked over time. At Baseline +1, 30 per cent of individuals with low integration scores shifted to medium or high integration scores. At Baseline +2, another 11 per cent shifted from low integration scores to medium or high integration scores, leaving 21 per cent of the population as having low integration scores at Baseline +2, compared to 63 per cent at Baseline. At Baseline +3, only 5 per cent of refugees were categorized as having low integration scores, 18 per cent were categorized as having medium integration scores and the vast majority (76 per cent) were categorized as having high integration scores. Figure 3 also shows this gradual shift of the cohort from low to high integration scores with each administration. Conclusion The RISE project was designed to fill a gap in the literature on refugee integration by operationalizing an integration framework, then creating a statistically reliable and valid survey to assess integration. The study also tested the efficacy of using a CBPR approach for collecting data in refugee communities. Finally, the study yielded data on the integration progress of an entire cohort of newly arrived refugees during the first four years of resettlement. RISE researchers believe that the survey instrument is both reliable and valid, based on evidence from both quantitative and qualitative analyses. Internal consistency (Cronbach’s alpha) and test–retest reliability were examined. The OI variable exibited respectable internal consistency at Baseline (α = 0.63) and at 1-Year (Baseline +1) (α = 0.60), given that the latent construct of integration is not a psychological construct in the way it was measured, but rather a checklist of behaviours. Test–retest correlations ranged from 0.17 to 0.72. The translation and back-translation approach and the cognitive interviews confirmed the meaning of the survey items (content validity). Construct validity was established by examining the pattern of correlations of the constructs over time. Constructs that were conceptually similar based on empirical evidence from the study or prior research (e.g. Social Bridging and Language and Cultural Knowledge) and were highly correlated (r = 0.79 at Baseline +1) and constructs that were theoretically dissimilar, or not associated in the context of integration, (e.g. Education and Training and Housing) had no correlation (r = 0.04 at Baseline +1), which is what was expected. Once the psychometric prosperities of the survey were deemed acceptable, a data-collection approach based on the principles of CBPR was used. The premise of the CC model was that refugee communities are impervious to outsiders, but ‘tightly knit’ from an insider’s perspective. Many refugees were resettled in Denver because they had family and friends already present, and these social networks created familiarity and safety in a new and ambiguous environment. The CCs were themselves refugees and, as such, embedded members of the refugee communities. They had broad and deep knowledge of individuals, families and social networks, and were bilingual in the languages of the target populations and in English. Often, the CCs were social connectors within their own communities. This enabled them to make contact in unexpected ways with refugee survey participants (e.g. finding the whereabouts of one refugee at an uncle’s wedding) that would have been unavailable to an outside researcher. The CC model is credited for the success of the survey data-collection efforts—namely the high response rates from Baseline (77, 73 and 70 per cent, respectively) and the 95 per cent annual retention from Baseline +1 through Baseline +3. This CBPR model also resulted in the collection of high-quality data (lack of missing data and production of meaningful data). The commitment of the CCs to this survey project was also paramount to the success of this survey project. The RISE research team believes that CCs’ strong commitment was due to the following reasons: (i) the CCs felt validated and appreciated—the study team acknowledged and celebrated the knowledge and wisdom of the CCs and, as such, gave the CCs a tremendous amount of respect, which minimized barriers that might have emerged using conventional, hierarchical research approaches; (ii) the CCs were integrally involved in all aspects of the study, including survey and research design, data collection and feedback on findings—which fostered the development of a sense of ownership of and investment in the project; and (iii) the CCs felt personally responsible for helping others in their community, and they viewed the survey project as a way to do this. Despite the successes related to the implementation of the CC model, challenges with this approach certainly arose. The CC model was effective, but not efficient. First, it required continuous communication with CCs, although their chaotic schedules and lack of access to/use of e-mail and phones (at times) made this communication and scheduling challenging. Second, retention of respondents within the target communities depended almost entirely on the ability of RISE research team to identify socially connected and reliable CCs. Due to the inability of the research team to identify and retain reliable Iraqi and Somali CCs, data collection with these groups was discontinued after the second survey administration. The creation of a sound integration measurement tool and the use of the CC model enabled the collection of high-quality, quantitative, longitudinal data, which in turn allowed researchers to get a ground-level perspective on the integration process for a cohort of refugees in Denver, Colorado. RISE data confirmed what professionals had suspected—refugees as a whole make steady progress towards integration during the first four years after arrival. As a result of RISE, data could be used to provide insights into refugee integration that were not previously available. RISE data spoke to differences in integration based on certain demographic characteristics (e.g. age) and the role of the family during integration. A future article will address distinct pathway profiles that emerged among those with high versus low integration scores. It was hypothesized that the CC model would increase the utility of the data and, in fact, we are starting to see that the data is being used in various ways. For example, the CRSP made note of the finding that social bridging can greatly influence integration and is working with a contractor to modify their programmes to include aspects of social bridging. The state refugee coordinators have built some of the findings into their messaging, as well as the pathway indicators into other aspects of their work. Welcoming America, a non-profit, non-partisan organization, has used the findings to promote the assets that refugees bring to the receiving communities. There are still many other ways in which the results of the RISE study can be used to support work with refugees. One limitation of the RISE study is that it focused on integration only on the refugee side of the equation and, as such, did not provide data related to the receiving community. As noted previously, the prevailing view is that integration involves a dynamic process between immigrant and refugees and the receiving community, so future research on the assessment of integration as it relates to the receiving community would be a valuable contribution to the field. Another limitation was that the OI score (each year) was created by summing core pathway scores. This meant that pathways with more items contributed more to the OI score than others. Although the pathways that did contribute more to the OI score (e.g. Language and Cultural Knowledge) and those that contributed very little (e.g. Housing) were consistent with the literature and what was learned from the focus groups, this does impact the calculation of the OI score, so the interpreter of such scores must keep this in mind. A third limitation is that the test–retest reliability was recorded across years, rather than within a more constricted time frame. This can create ‘noise’ in the results, since growth can occur between administrations. Other publications have included one-year test–retest reliability (Glenn and Klonsky 2011; Bruce et al. 2014; Scolaro et al. 2017), but further research should confirm these findings. Another limitation is that convergent validity was established using the integration pathways themselves; however, it was argued that the 10 pathways do not comprise a latent psychological construct. Another measure of integration was not separately administered to the participating refugees in this study, so the pathways themselves could only be used as a proxy to establishing construct validity. Lastly, the research team’s modification of Ager and Strang’s (2004) framework introduces some discontinuity in the research literature. The RISE Civic Engagement pathway is similar to Ager and Strang’s Social Links domain. Ager and Strang’s Citizenship and Rights domain has no equivalent in the RISE instrument. These differences can be assessed and reconciled over the course of further research. The RISE survey can and should continue to evolve to strengthen (and possibly reduce) pathway items and adapt items to be relevant in other contexts and communities. Pathway items can be integrated into other instruments in order to promote consistency in research related to refugee integration. This work confirmed that the integration approach is critical for gaining a full understanding of the complex, multifaceted experience of refugee resettlement. It is our hope that the RISE survey will be used by researchers and refugee service professionals to improve the understanding of and effectiveness in promoting refugee integration through research, policy and practice. Acknowledgements This publication was made possible with support from the United States Department of Health and Human Services, Office of Refugee Resettlement (ORR) through a grant to the Colorado Department of Human Services, Colorado Refugee Service Program CRSP (PO IHA CRSP1123064). CRSP contracted with Quality Evaluations Designs (PI: Dr Gary Lichtenstein) to conduct the RISE study. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the U.S. Department of Health and Human Services or CRSP. The authors wish to extend a sincere thank you to Dr Martin Tombari, Dr Laurie Bennett and Mrs Maggie Miller, all of whom were on the study team and whose contributions were invaluable to this study and the contents of this article. Additionally, the authors wish to thank Mr Joseph Wissman-Horther and Ms Kit Taintor for their support and guidance but, most importantly, the authors would like to extend a very warm thank you to the resettlement agencies in metro Denver, the CCs and all of the survey respondents, without whom, this project would not have been possible. References AGER A. , STRANG A. ( 2004 ) Indicators of Integration: Final Report . Home Office Development and Practice Report #28. London : Home Office , http://webarchive.nationalarchives.gov.uk/20110218135832/http:/rds.homeoffice.gov.uk/rds/pdfs04/dpr28.pdf (accessed 18 February 2017). AGER A. , STRANG A. ( 2008 ) ‘Understanding Integration: A Conceptual Framework’ . Journal of Refugee Studies 1 ( 2 ): 166 – 191 . Google Scholar CrossRef Search ADS ATFIELD G. , BRAHMBHATT K. , O’TOOLE T. ( 2007 ) Refugees’ Experiences of Integration . London, UK : University of Birmingham and the Refugee Council . BEIRENS H. , HUGHES N. , HEK R. , SPICER N. ( 2007 ) ‘ Preventing Social Exclusion of Refugee and Asylum Seeking Children: Building New Networks’ . Social Policy and Society 6 : 219 – 229 . Google Scholar CrossRef Search ADS BRUCE J. , ECHEMENDIA R. , MEEUWISSE W. , COMNER P. , SISCO A. ( 2014 ) ‘ 1 Year Test–Retest Reliability of ImPACT in Professional Ice Hockey Players’ . Clinical Neurophychology 28 ( 1 ): 14 – 25 . Google Scholar CrossRef Search ADS CAPPS R. , NEWLAND K. , FRATZKE S. , GROVES S. , AUCLAIR G. , FIX M. , MCHUGH M. ( 2015 ) The Integration Outcomes of U.S. Refugees: Successes and Challenges . Washington, DC : Migration Policy Institute . CARTY D. C. , KRUGER D. J. , TURNER T. M. , CAMPBELL B. , DELONEY E. H. , LEWIS Y. ( 2011 ) ‘ Racism, Health Status, and Birth Outcomes: Results of a Participatory Community-Based Intervention and Health Survey’ . Journal of Urban Health: Bulletin of the New York Academy of Medicine 88 ( 1 ): 84 – 96 . Google Scholar CrossRef Search ADS PubMed CEBULLA A. , DANIEL M. , ZURAWAN A. ( 2010 ) Spotlight on Refugee Integration: Findings from the Survey of New Refugees in the United Kingdom: Report #37 . London : Home Office . CHEUNG S. , PHILLIMORE J. ( 2017 ) ‘ Gender and Refugee Integration: A Quantitative Analysis of Integration and Social Policy Outcomes’ . Journal of Social Policy 46 ( 2 ): 211 – 230 . Google Scholar CrossRef Search ADS COHEN A. , LOPEZ A. , MALLOY N. , MORELLO-FROSCH R. ( 2011 ) ‘ Our Environment, Our Health: A Community-Based Participatory Environmental Health Survey in Richmond, California’ . Health Education and Behavior 39 : 198 – 209 . Google Scholar CrossRef Search ADS PubMed DE MAIO J. , SILBERT M. , JENKINSON R. , SMART D. ( 2014 ) ‘Building a New Life in Australia: Introducing the Longitudinal Study of Humanitarian Migrants’ . Family Matters 94 : 5 – 14 . GLENN C. R. , KLONSKY D. E. ( 2011 ) ‘ One-Year Test–Retest Reliability of the Inventory of Statements about Self-Injury (ISAS)’ . Assessment 18 ( 3 ): 375 – 378 . Google Scholar CrossRef Search ADS PubMed ISRAEL B. , SCHULZ A. , PARKER E. , BECKER A. ( 1998 ) ‘ Review of Community-Based Research: Assessing Partnership Approaches to Improve Public Health’ . Annual Review of Public Health 19 ( 1 ): 169 – 177 . Google Scholar CrossRef Search ADS KRUGER D. J. , HAMACHER L. , STRUGAR-FRITSCH D. , SHIREY L. , RENDA E. , ZIMMERMAN M. ( 2010 ) ‘ Facilitating the Development of a County Health Coverage Plan with Data from Community-Based Health Survey’ . Journal of Public Health Policy 31 ( 2 ): 199 – 211 . Google Scholar CrossRef Search ADS PubMed MAIN D. S. , WARE G. , IWASAKI P. G. , BURRY M. , STEINER E. , FEDDIE K. , HAVERHALS L. ( 2012 ) ‘Taking Neighborhood Health to Heart (TNH2H): Building a Community-Based Participatory Data System’ . Preventing Chronic Disease 9 : 110058 . http://dx.doi.org/10.5121/ijcseit.2012.2605 MULVEY G. ( 2015 ) ‘ Refugee Integration Policy: The Effects of UK Policy-Making on Refugees in Scotland’ . Journal of Social Policy 44 ( 2 ): 357 – 375 . Google Scholar CrossRef Search ADS NATIONAL ACADEMIES OF SCIENCES, ENGINEERING,MEDICINE ( 2015 ) ‘The Integration of Immigrants into American Society: Panel on the Integration of Immigrants into American Society’. In Waters M. C. , Pnieau M. G. (eds) Committee on Population, Division of Behavioral and Social Sciences and Education . Washington, DC : The National Academies Press . PETSOD D. , WANG T. , MCGARVEY C. ( 2006 ) Investing in Our Communities: Strategies for Immigrant Integration: A Toolkit for Grantmakers. Grantmakers Concerned with Immigrants and Refugees, https://www.gcir.org/sites/default/files/resources/GCIR%20Toolkit.pdf (accessed 18 February 2017). PUMA J. E. , BELANSKY E. S. , GARCIA R. , SCARBRO S. , WILLIFORD D. , MARSHALL J. ( 2016 ) ‘A Community-Engaged Approach to Collecting Rural Health Surveillance Data’ . Journal of Rural Health 00 : 1 – 9 . RIOSECO P. , DE MAIO J. , HOANG C. ( 2017 ) ‘ The Building a New Life in Australia (BNLA) Dataset: A Longitudinal Study of Humanitarian Migrants in Australia’ . Australian Economic Review 50 ( 3 ): 356 – 362 . Google Scholar CrossRef Search ADS ROBINSON V. ( 1998 ) ‘Defining and Measuring Successful Refugee Integration’. Proceedings of ECRE International Conference on Integration of Refugees in Europe, Antwerp, Brussels: ECRE. RUCINSKI D. , DAVIS R. , GOMEZ G. , FLORES J. , PEREZ I. , ZANONI J. ( 2011 ) ‘ An Approach to Assessing Data Quality in an Area Probability Survey Using CBPR’ . Health Promotion and Practice 12 ( 5 ): 704 – 712 . Google Scholar CrossRef Search ADS RUDMIN F. W. ( 2003 ) ‘Critical History of the Acculturation Psychology of Assimilation, Separation, Integration, and Marginalization’ . Review of General Psychology 7 ( 1 ): 3 – 37 . Google Scholar CrossRef Search ADS SCOLARO MOSERA R. , SCHATZC P. , GROSNERD E. , KOLLIASE K. ( 2017 ) ‘One Year Test–Retest Reliability of Neurocognitive Baseline Scores in 10- to 12-Year Olds’ . Applied Neurospsychology: Child 6 ( 2 ): 166 – 171 . Google Scholar CrossRef Search ADS SIRRIYEH A. ( 2007 ) ‘Young Asylum Seekers’ Conceptions of “Home” at a Time of Transition to Adulthood’. Disaporas, Migration and Identities Working Paper 6, Arts and Humanities Research Board/University of Leeds, http://web.archive.org/web/20150215055840/http://www.diasporas.ac.uk/ (accessed 18 February 2017). SMYTH G. , STEWART E. , DE LOMBA E. ( 2010 ) ‘ Introduction: Critical Reflections on Refugee Integration: Lessons from International Perspectives’ . Journal of Migration Studies 23 ( 4 ): 411 – 414 . UNITED NATIONS HIGH COMMISSIONER FOR REFUGEES (UNHCR) ( 2017 ) Refugee Figures, http://www.unhcr.org/uk/figures-at-a-glance.html (accessed 22 October 2017). WILLIS G. B. ( 1999 ) Cognitive Interviewing: A ‘How-To’ Guide. Presented at the annual meeting of the American Statistical Association. Alexandria. VA: Research Triangle Institute, http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=71CC295F5AF0F2D2C4B4804BF5CE5C9C?doi=10.1.1.469.6024&rep=rep1&type=pdf (accessed 18 February 2017). YU S. , OUELLET E. , WARMINGTON A. ( 2007 ) ‘Refugee Integration in Canada: A Survey of Empirical Evidence and Existing Services’ . Journal on Refugees 24 ( 2 ): 17 – 34 . © The Author(s) 2018. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Journal

Journal of Refugee StudiesOxford University Press

Published: Jan 18, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off