Assessing the reliability and validity of the Danish version of Organizational Readiness for Implementing Change (ORIC)

Assessing the reliability and validity of the Danish version of Organizational Readiness for... Background: Organizational change initiatives in health care frequently achieve only partial implementation success. Understanding an organizational readiness for change (ORC) may be a way to develop more effective and efficient change strategies. Denmark, like many countries, has begun a major system-wide structural reform which involves considerable changes in service delivery. Due to the lack of a validated Danish instrument, we aimed to translate and validate a Danish version of the Organizational Readiness for Implementing Change (ORIC) questionnaire. It measures if organizational members are confident in their collective commitment towards and ability (efficacy)to implement organizational change. ORIC is concise, grounded in theory, and designed, but not yet validated among employees in a real hospital setting. Methods: The 12-item ORIC instrument was translated into Danish and back-translated to English. Employees (N =284) at a hospital department facing a major organizational change in the Central Denmark Region completed the questionnaire. Face and content validity was ascertained. Exploratory factor analysis (EFA) and a confirmatory factor analysis (CFA) were used to assess construct validity. Reliability was assessed with Cronbach’s alpha. Item response theory (Rasch analysis) was used to determine item and person reliability. Results: Response rate was 72%. A two factor (commitment and efficacy), 11-item scale, of the Danish language ORIC was shown to be valid (CFI = .95, RMSEA = .067, and CMNI/DF = 2.32) and reliable (Cronbach’s alpha 0.88) in a health care setting. Item response analysis confirmed acceptable person and item separation reliability. Conclusions: Our version of ORIC showed acceptable validity and reliability as an instrument for measuring readiness for implementing organizational change in a Danish-speaking health care population. For health care managers interested in evaluating their organizations and tailor change strategies, ORIC’s brevity and theoretical underpinnings could make it an appealing and feasible tool to develop more successful change efforts. Keywords: Organizational readiness for change, Validation study, Translation, Change management, Implementation, Questionnaire Background success [1, 2]. Understanding the degree to which The optimism that signals the start of so many change organizational members and organizations are “ready” to efforts in health care seems somewhat misguided when implement a specific change has been suggested as a way decades of research suggest that organizational change forward. Multiple definitions and multiple instruments to initiatives frequently achieve only partial implementation measure organizational readiness for change exist [3]; however, there is no gold standard. Recently, a focus on the supra-individual level has emerged [4], and an organi- * Correspondence: Marie.storkholm@ki.se zation’s readiness for change has been defined as “ashared Department of Learning, Informatics, Management and Ethics, Medical Management Centre, Karolinska Institutet, Tomtebodavägen 18A, 171 77 psychological state in which organizational members feel Stockholm, Sweden committed to implementing an organizational change and Department of Obstetrics and Gynecology, Aarhus University Hospital, confident in their collective abilities to do so” [5]. The Aarhus, Denmark Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Storkholm et al. Implementation Science (2018) 13:78 Page 2 of 7 supra-individual level, i.e., the collective level above the Methods individuals in the organizations, is important to study as Participants and procedures collective behavior change is often required to implement This study was conducted at the Department of Obstetrics multiple changes, such as in staffing, workflow, and and Gynecology (OB/GYN) at Aarhus University Hospital organizational structures [5]. (AUH). To improve efficiency, integration, and coordin- Commitment to change can be defined as the mindset ation of health care services as required by the national that binds an individual to the course of action deemed reform, AUH started a hospital merger, downsizing of necessary for the successful implementation of a change beds, and construction of a new hospital. Hospital initiative [6]. Weiner proposes that organizational management decided that the OB/GYN department members whose commitment to change is based on should reduce beds by 36% and their budget by 10%, “want to” rather than “have to” or “ought to” display not mainly by reducing nursing staff. The efficacy require- only more cooperative behavior (e.g., volunteering for ments were addressed through an improvement process problem-solving teams), but also championing behavior that led to the implementation of changes at the level of (e.g., promoting the value of the change to others) [5, 7]. care pathways for individual medical conditions and at the Change efficacy refers to organization members’ shared organizational level. belief in their “collective capabilities to organize and Organizational members were identified through em- execute the courses of action involved in change imple- ployee lists provided by department managers. We cross- mentation” [8]. It reflects the amount of knowledge checked the list with department mailing lists to ensure all available about what to do and how to do it, i.e., it is a employees had been identified. We administrated the function of members’ cognitive appraisal of three factors questionnaire electronically via SurveyXact (Aarhus/ of implementation capability: the resources (including Denmark) and distributed it via email to all staff and man- time) available, task demands, and the situation the agers in the OB/GYN department (n = 403) in June 2014. organization faces [4, 9]. The ability to measure how an Informed consent was obtained electronically; only partic- organization’s members perceive these two constructs ipants who gave their consent were able to answer the may, ultimately, be used by health care leaders to develop questionnaire. Reminders were sent out through Septem- more effective and efficient change strategies [4]. ber. The response rate was 72%. Key characteristics of the Like many countries, Denmark has begun a major respondents are presented in Table 1. system-wide structural reform to improve efficiency and the integration and coordination of health care [10]. The regions, hospitals, and clinical departments need to Measures make considerable changes in service delivery, which re- The “Organizational Readiness for Implementing Change” quires the coordinated and collective actions of multiple (ORIC) questionnaire contains 12 items that correspond organizational members to succeed. As part of a case to the domains of commitment and efficacy. It uses a study (not reported in this Short Report) to explore how 5-point Likert scale (1 = strongly disagree and 5 = strongly a clinical department has managed this challenge [11], agree). To facilitate analysis, we grouped and labeled the we wanted to use a validated instrument to test the items according to the domain they addressed (Table 2). organization’s readiness for implementing change. As we could not find a Danish instrument, we chose the Table 1 Baseline characteristics of the study participants, no. “Organizational Readiness for Implementing Change” (%) unless otherwise indicated (n = 284) (ORIC) questionnaire developed by Shea et al. as it is a Variable No. (%) promising, reliable, and valid instrument designed for Gender, female 264 (93.0) health care, even though factor analysis has revealed some Profession limitations and it has not yet been validated in an actual Nurse and Licensed Practical Nurse 109 (38.4) hospital setting [4]. In addition to undergoing a thorough Midwife 95 (33.5) psychometric assessment, ORIC is unique in comparison Physician 51 (18.0) to other instruments [3]as it is grounded in theory, targets the supra-individual level, and its brevity suits the Medical secretary 21 (7.4) busy health care context [4]. Thus, a Danish translation of Others 8 (2.8) this instrument could both become a helpful instrument Age, years, mean (SD) 45, 9 (10.6) for Danish health care managers aiming to tailor Manager, yes 15 (5.3) implementation strategies in different health care settings Permanent employed, yes 239 (84.2) [12]. In this Short Report, we assess the reliability and Length of employment, median (IQR) 18 (8.3, 27.4) validity of a Danish version of the ORIC instrument in a hospital setting. SD standard deviation, IQR interquartile range Storkholm et al. Implementation Science (2018) 13:78 Page 3 of 7 Table 2 The original English version of the ORIC (12 items) as presented in [4]. The Danish translation can be found in the supplementary file Item number Item description Change efficacy (7 items) E1 People who work here feel confident that the organization can get people invested in implementing this change E2 People who work here feel confident that they can keep track of progress in implementing this change E3 People who work here feel confident that the organization can support people as they adjust to this change E4 People who work here feel confident that they can keep the momentum going in implementing this change E5 People who work here feel confident that they can handle the challenges that might arise in implementing this change. E6 People who work here feel confident that they can coordinate tasks so that implementation goes smoothly E7 People who work here feel confident that they can manage the politics of implementing this change Change commitment (5 items) C1 People who work here are committed to implementing this change. C2 People who work here will do whatever it takes to implement this change C3 People who work here want to implement this change C4 People who work here are determined to implement this change C5 People who work here are motivated to implement this change The numbering (E1-E7 and C1-C5) corresponds to the The number of factors to retain was determined using additional file in the Shea article [4]. parallel analysis and eigenvalue-greater-than-one deci- sion rule [15]. Following the principal axis FA, a CFA Translation was performed to validate the resulting constructs from The ORIC questionnaire was translated through the stand- the EFA. The use of CFA to investigate the construct ard approach of “forward” and “back” translation [13, 14]. validity of hypothesis-based testing instruments adds a Forward translation was done by a native Danish speaker. level of statistical precision. Maximum Likelihood Esti- Back-translation was done by an independent bilingual mation was used to fit the CFA model. A full dataset (English/Danish) native English speaker. The translations was used with no missing value. Our sample size of 284 (see Additional file 1) were compared, discussed, and then met the criteria of Myers et al. that includes as follows: refined. As recommended by Weiner, [5]the questionnaire N ≥ 200, ratio of N to the number of variables in a model was modified through the inclusion of an introductory (p), N/p ≥ 10 [16]. description of the organizational change and to specific We employed the commonly reported indexes [14, 17, 18] item sets so that it was clear what was meant by the phrase to assess the fitness of the model: chi-square/df, CFI and “this change”. root mean square error of approximation (RMSEA). The following cut-off values were used as the level of acceptance: Statistical analysis CFI equal to or greater than 0.90, [19] RMSEA equal to or Face and content validity of the translated questionnaire less than 0.08 [20], and CMIN/DF < 3. [17, 21]We consid- was assessed by Danish researchers working with ered Cronbach’s alpha values of 0.7 as acceptable for internal organizational psychology and behavior research and consistency. The level of significance was specified at 0.05. who had conducted numerous studies within health care Following the classical item analysis, Andrich’s exten- in Denmark and abroad. sion of the Rasch measurement model for rating scale Statistical analyses included Cronbach’s alpha to examine data was used [22] to evaluate the measurement proper- the reliability of the instrument, exploratory factor analysis ties of the ORIC. Rasch is often considered to be an item (EFA) using principal axis factor analysis to evaluate dimen- response theory (IRT) model. IRT is an important sionality followed by a confirmatory factor analysis (CFA) method of assessing the validity of measurement scales to investigate constructs validity. We hypothesized that the that is still underutilized [23]. It describes the properties two-factor representation efficacy and commitment of of the items in the scale, and respondents’ answers to Weiner [4] would be replicated in this study. the individual items, item and person parameters, model Principal-axis factor analysis (FA) was chosen, because fit statistics, and differential item functioning (DIF). The change commitment and change efficacy are interre- fit of response categories, items, and persons to the lated, although independent, facets of organizational expectations of the model is evaluated by quality control readiness [4, 5]. fit statistics. These are infit mean square error and outfit Storkholm et al. Implementation Science (2018) 13:78 Page 4 of 7 mean square error [24]. For response categories, items, model 1 showed the following values: CFI = .838, and persons, a fit (infit and outfit) value of 1.0 is perfect RMSEA = .113, and CMIN/DF = 4.796. For model 1, the fit to the Rasch model. Fit values less than 1.0 indicate chi-square test was significant (p < 0.001) and the other less variation in responses than the model expected criteria for model fit were not met. The main difficulty while fit values greater than 1.0 indicate greater variabil- presented in model 1 was that items E1 (Efficacy 1) and ity in responses than expected. The fit values for items C1 (Commitment 1) had low standardized regression should be between 0.5 and 1.5 for an item to have good weights and the model did not fit as reflected by the fit fit to the model [24]. Rasch analysis computes two indices. These items showed low factor loadings in both reliability coefficients for the measurement of person the EFA and CFA to warrant consideration of exclusion separation statistics—and model and real reliability to in the CFA. estimate the overall reliability of the scale. Reliability The results showed that model 3 has acceptable model value larger than 80% associated with separation index fit with CFI = .95, RMSEA = .067, and CMIN/DF = 2.32 greater than 2 is acceptable. Winsteps 4.0 was used for (Table 3). However, chi-square value for the overall the Rasch model analyses [24]. SPSS 24 and Amos 24 model fit was significant (p < .001) suggesting a lack of were used for all other analyses (SPSS, IBM, Chicago). fit between the hypothesized model and the data. We The level of significance was specified at 0.05. ignored this due to the sensitivity of chi-square in large samples as the sample size for study is greater than 200 Results [18, 25]. The assumption of normality was fulfilled. Face and content validity was confirmed for all items A two factor (commitment and efficacy), 11-item scale, with the exception of one. This referred to the word of the Danish language ORIC was shown to be valid. “commitment”, which is difficult to translate, as there is The constructs produced in the a priori model coincide no Danish word that fully captures the English meaning with the constructs contextually available in the original used in item C1. Furthermore, there were minor com- article [4]. ments regarding translations, such as the word “people” The Rasch analysis shows that the scale has acceptable in all the items, which in the translation referred to the person and item separation reliabilities but the assump- employees in the department, who were exposed to the tion of unidimensionality was not satisfied, indicating that change implementation. more than one latent construct is measured by a set of 12 The overall Cronbach’s alpha was 0.88. Efficacy is a ORIC items. The unidimensionality test is a Winsteps seven-item factor (Cronbach’s α = 0.87) and commitment principal components analysis (PCA) of the standardized a five-item factor (Cronbach’s α = 0.75). Two factors were response residuals (the responses that are misfitting). The produced from the analysis: commitment and efficacy. eigenvalue of the first contrast must be > 2.0 to suggest EFA results for the ORIC scale found two factors with that a second dimension exists within the misfitting an eigenvalue > 1. The screen plot also indicated the two responses. The eigenvalue of the first contrast is 2.22. The factors. Most items loaded onto their respective factors three items with the strongest positive loadings on the first (not presented). Items loaded on efficacy ranged between contrast were item C3 (.68), C4 (.67), and C5 (.57). The 0.54 and 0.76 and on commitment between 0.66 and 0.70. two items with the strongest negative loadings on the first CFA examined five models to evaluate the best fit for contrast were item E3 (− .58) and item E2 (− .47). the overall data (Table 3). All items displayed statistically All items have acceptable fit (Table 5). To evaluate if the significant factor loadings on their respective factors. Danish item calibrations differ by subgroups, a Model 1 includes all the items without any co-variance. Mantel-Hanzel DIF analysis was conducted. No items with Most of the loadings exhibited values between 0.62 and significantly different difficulty calibrations in age group, 0.86 for the factor efficacy (Table 4). The CFA results of sex, management role, and profession were identified. Table 3 Results of the CFA by model and indices Model Absolute fit (RMSEA) Incremental fit (CFI) Parsimonious fit (Chisq/df) Model 1 .113 .838 4.796 Model 2 .117 .841 5.000 Model 3 .067 .950 2.320 Model 4 .074 .943 2.603 Model 5 .071 .953 2.476 Model 1—all items without co-variance of the two factors; model 2—E1 is removed; model 3—E1 removed and the two factors were allowed to correlate; model 4—C1 removed and the two factors were allowed to correlate; model 5—C1 and E1 removed and the two factors were allowed to correlate RMSEA root mean square error of approximation, CFI confirmatory fit index, Chisq/df chi-square/degrees of freedom Storkholm et al. Implementation Science (2018) 13:78 Page 5 of 7 Table 4 Standardized regression weights of the items from the demonstrated two correlated factors. However, the confirmatory factor analysis (CFA) metrics in our study were slightly different. By removing Items Model 1 Model 2 Model 3 Model 5 Item E1 and allowing the factors to correlate, we identi- fied a good fit of an 11-item model with a high factor E1 .644 ––– loading. Shea et al. [4] obtained a good fit model by E2 .724 .719 .712 .713 excluding two efficacy items. E3 .763 .740 .730 .729 Our ability to construct a model with a better fit may E4 .729 .731 .738 .740 be due to the fact that our study was conducted in a E5 .635 .661 .666 .669 hospital department facing a real organizational change E6 .721 .743 .740 .742 effort. Thus, our study among hospital employees adds to Shea et al.’s study that validated the instrument E7 .667 .665 .669 .670 among graduate and undergraduate students and NGO’s C1 .402 .402 .416 – staff [4]. Like in the original article [4], item E1 (People C2 .461 .461 .488 .488 who work here feel confident that the organization can C3 .692 .692 .662 .673 get people invested in implementing this change) was C4 .707 .707 .716 .728 excluded as it cross loaded on the commitment factor. C5 .789 .789 .782 .777 The key phrase in item E1 is “invested in the change”.In E efficacy, C commitment Danish, the word invested was replaced with engaged because the phrase “investment” is in Danish exclusively associated with money, which may have contributed to For the Danish ORIC model, reliability is .86 and real the cross loading. reliability is .99 with a separation index of 2.47 and 9.69 Recent studies using the English-language ORIC either respectively. The true reliability of the Danish ORIC had a small sample or surveyed only one representative of model is somewhere between these two values. Reliability each workplace studied [26, 27], which challenge Weiner’s value larger than 80% associated with separation index conceptualization of a collective measurement [5]. In both greater than 2 is acceptable [24]. studies, the finding partly indicated that individual levels of commitment or efficacy can have been captured instead Discussion of the overall organizational levels, even though both stud- With a response rate of 72%, an 11-item scale of a Danish ies used the word “We” in their questionnaire [26, 27]. In translation of the ORIC was shown to be valid and reliable our study, we used a translation of the phrase “People when tested in a hospital setting. All items had acceptable who work here”, in accordance with the original version fit in the IRT analysis and we did not observe significant instead of “We”. We have not explored the implications of DIF in subgroups of age group, sex, management role, or this difference. Another limitation is that we surveyed one profession. clinical department, which may reduce generalizability. Corresponding to the theory and previous psychomet- Future studies should focus on comparing different ric assessment [4, 5], factor analysis of the instrument settings and determine the predictive value of ORIC through the use of organizational measures for instance on change efforts and performance outcomes. A further Table 5 Item statistics of the Danish version of ORIC limitation is that we did not perform a concurrent validity Item Difficulty Standard error Infit Outfit assessment, to compare the ORIC results with other readi- C2 – .66 .08 1.47 1.43 ness measures. C3 – .46 .08 1.39 1.39 A strength of this study is the use of both classical test theory and IRT. IRT offers a number of advantages by C1 – 1.90 .10 1.25 1.23 modeling the relationship of individual items to the E1 – .03 .08 .98 1.00 construct measured. It provides a richer description of the E5 .20 .08 .97 .95 performance of each item and greater detail on a measure’s C4 – .75 .09 .96 .96 precision compared to classical test theory, where a single C5 – .39 .08 .93 .89 estimate, such as Cronbach’s α, describes a measure’s E7 .69 .08 .88 .89 reliability [23]. Thus, our study supports the English version as reliable and valid for health care settings. In the Danish E6 .80 .08 .88 .86 context, the translated instrument can be used to further E3 1.08 .08 .87 .87 investigate the strategies hospitals employ to manage the E2 .93 .08 .81 .80 current system-wide reform and the related changes in E4 .49 .08 .76 .75 service delivery. Storkholm et al. Implementation Science (2018) 13:78 Page 6 of 7 Conclusion Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in The Danish version of ORIC showed acceptable reliability published maps and institutional affiliations. and validity as an instrument in a Danish-speaking popu- lation. It also confirms the validity and reliability of the Author details Department of Learning, Informatics, Management and Ethics, Medical ORIC instrument for hospital settings. Its brevity and Management Centre, Karolinska Institutet, Tomtebodavägen 18A, 171 77 theoretical underpinnings could make it an appealing and 2 Stockholm, Sweden. Department of Obstetrics and Gynecology, Aarhus feasible tool for health care managers interested in evalu- University Hospital, Aarhus, Denmark. Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Stockholm, Sweden. ating their organizations and to tailor change strategies that better match staff views of the organization’sreadi- Received: 27 February 2018 Accepted: 22 May 2018 ness for implementing change. Additional file References 1. Shortell S, Bennett C, Byck G. Assessing the impact of continuous quality improvement on clinical practice: what it will take to accelerate progress. Additional file 1: Organizational Readiness for Implementing Change Milbank Q 1998;76:593–624, 510. http://www.pubmedcentral.nih.gov/ (ORIC)—Danish version. (DOC 46 kb) articlerender.fcgi?artid=2751103&tool=pmcentrez&rendertype=abstract. Accessed 22 Apr 2014. 2. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of Abbreviations innovations in service organizations: systematic review and SPSS, IBM: Statistical Package for the Social Sciences, International Business recommendations. Milbank Q. 2004;82:581–629. https://doi.org/10.1111/j. Machines Corporation; AUH: Aarhus University Hospital; CFA: Confirmatory 0887-378X.2005.340_1.x. factor analysis; CFI: Confirmatory fit index; DIF: Differential item functioning; 3. Weiner BJ, Amick H, S-YD L. Conceptualization and measurement of EFA: Exploratory factor analysis; FA: Factor analysis; IRT: Item response theory; organizational readiness for change: a review of the literature in health OB/GYN: Obstetrics and gynecology; ORIC: Organizational Readiness for services research and other fields. Med Care Res Rev. 2008;65:379–436. Implementing Change; PA: Parallel analysis; RMSEA: Root mean square error https://doi.org/10.1177/1077558708317802. of approximation 4. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new Acknowledgements measure. Implement Sci. 2014;9. https://doi.org/10.1186/1748-5908-9-7. MS was financially supported by Aarhus University Hospital, Institute of 5. Weiner BJ. A theory of organizational readiness for change. Implement Sci. Clinical Medicine at Aarhus University, and Central Region Denmark. The 2009;4:67. https://doi.org/10.1186/1748-5908-4-67. funders had no involvement in the study design; in the collection, analysis, 6. Meyer JP, Stanley DJ, Herscovitch L, Topolnytsky L. Affective, continuance, and interpretation of the data; in the writing of the report; and in the and normative commitment to the organization: a meta-analysis of decision to submit the paper for publication. antecedents, correlates, and consequences. J Vocat Behav. 2002;61:20–52. https://doi.org/10.1006/jvbe.2001.1842. 7. Herscovitch L, Meyer JP. Commitment to organizational change: extension Funding of a three-component model. J Appl Psychol. 2002;87:474–87. https://doi. MS was financially supported by Aarhus University Hospital, Institute of org/10.1037//0021-9010.87.3.474. Clinical Medicine at Aarhus University, and Central Region Denmark. The 8. Bandura A. Self-efficacy: The exercise of control. New York: Freeman; 1997. funders had no involvement in the study design; in the collection, analysis, 9. Weiner BJ, Lewis MA, Linnan LA. Using organization theory to and interpretation of the data; in the writing of the report; and in the understand the determinants of effective implementation of worksite decision to submit the paper for publication. health promotion programs. Health Educ Res. 2009;24:292–305. https:// doi.org/10.1093/her/cyn019. Availability of data and materials 10. Rudkjøbing A, Olejaz M, Birk HO, Nielsen AJ, Hernández-quevedo C, The dataset collected and analyzed during this study is available from the Published A. Integrated care: a Danish perspective. BMJ. 2012;345:4451. corresponding author on reasonable request. 11. Storkholm MH, Mazzocato P, Savage M, Savage C. Money’s (not) on my mind: a qualitative study of how staff and managers understand health care’s triple aim. BMC Health Serv Res. 2017;17:98. https://doi.org/10. Authors’ contributions 1186/s12913-017-2052-3. MS, PM, and CS designed the study. MS recruited participants and 12. Jordan C, Hoefer RA. Reliability and validity in quantitative collected the data. MS and MT conducted the analyses, which were measurement. In: Thyer BA, editor. Social work research methods. reviewed by PM and CS. MS, PM, MT, and CS drafted and revised the Thousand Oaks, CA: Sage; 2001. manuscript. All authors read, contributed to, and approved the final 13. Behling O, Law KS. Translating questionnaires and other research version. PM and CS were the PIs. instruments. London: Sage; 2000. 14. Rode N. Translation of measurement instruments and their reliability: an Ethics approval and consent to participate example of job-related affective well-being scale. Metod Zv. 2005;2:15–26. Ethical approval was deemed not to be necessary by the Central Denmark 15. Kaiser HF. The application of electronic computers to factor analysis. Educ Region Ethic Review Board. Participants were informed about the purpose of Psychol Meas. 1960;20:141–51. the survey and that they could withdraw their participation at any point. They 16. Myers ND, Ahn S, Jin Y. Sample size and power estimates for a confirmatory received contact information should they have questions or desire that their factor analytic model in exercise and sport. Res Q Exerc Sport. 2011;82:412– data be withdrawn. Following a standard praxis in Denmark, the information 23. https://doi.org/10.1080/02701367.2011.10599773. provided was written out in the introduction to the questionnaire. Informed 17. Marsh HW, Hocevar D. Application of confirmatory factor analysis to the consent was received from all participants before they were able to answer the study of self-concept: first- and higher order factor models and their ORIC survey. The data were kept in a password-protected computer and was invariance across groups. Psychol Bull. 1985;97:562–82. anonymized before the statistical analysis. 18. Hair J, Black W, Babin B, Anderson R. Multivariate Data Analysis: Global Edition. 7th Edition. Upper Saddle River: Pearson Higher Education; 2010. Competing interests 19. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990; The authors declare that they have no competing interests. 107:238–46. https://doi.org/10.1037/0033-2909.107.2.238. Storkholm et al. Implementation Science (2018) 13:78 Page 7 of 7 20. Browne M, Cudeck R. Alternative ways of assessing model fit. In: Bollen K, Long J, editors. Testing structural equation models. Newbury Park, CA: Sage; 1993. 21. Kline R. Principles and practice of structural equation modeling. NY: Guilford press; 1998. 22. Andrich D. Controversy and the Rasch model. Med Care 2004;42 Supplement:I-7. https://doi.org/10.1097/01.mlr.0000103528.48582.7c. 23. Nguyan TH, Han H-R, Kim MT, Chan KS. An introduction to item response theory for patient-reported outcome measurement. Patient. 2014;7:23–35. 24. Linacre J. A User’s guide to WINSTEPS. In: Rasch-model computer programs. Program Manual for 4.0.0. Chicago. http://www.winsteps.com/winman/ copyright.htm. 25. Jöreskog KG, Sörbom D. LISREL 8: user’s reference guide. 2nd ed. Lincolnwood: Scientific Software International; 1996. 26. Sanders KA, Wolcott MD, McLaughlin JE, D’Ostroph A, Shea CM, Pinelli NR. Organizational readiness for change: Preceptor perceptions regarding early immersion of student pharmacists in health-system practice. Res Soc Adm Pharm. 2017. https://doi.org/10.1016/j.sapharm.2017.03.004. 27. Hannon PA, Helfrich CD, Chan KG, Allen CL, Hammerback K, Kohn MJ, et al. Development and pilot test of the workplace readiness questionnaire, a theory-based instrument to measure small workplaces’ readiness to implement wellness programs. Am J Health Promot. 2017;31:67–75. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Implementation Science Springer Journals

Assessing the reliability and validity of the Danish version of Organizational Readiness for Implementing Change (ORIC)

Free
7 pages
Loading next page...
 
/lp/springer_journal/assessing-the-reliability-and-validity-of-the-danish-version-of-Q0bfU0i05d
Publisher
BioMed Central
Copyright
Copyright © 2018 by The Author(s).
Subject
Medicine & Public Health; Health Promotion and Disease Prevention; Health Administration; Health Informatics; Public Health
eISSN
1748-5908
D.O.I.
10.1186/s13012-018-0769-y
Publisher site
See Article on Publisher Site

Abstract

Background: Organizational change initiatives in health care frequently achieve only partial implementation success. Understanding an organizational readiness for change (ORC) may be a way to develop more effective and efficient change strategies. Denmark, like many countries, has begun a major system-wide structural reform which involves considerable changes in service delivery. Due to the lack of a validated Danish instrument, we aimed to translate and validate a Danish version of the Organizational Readiness for Implementing Change (ORIC) questionnaire. It measures if organizational members are confident in their collective commitment towards and ability (efficacy)to implement organizational change. ORIC is concise, grounded in theory, and designed, but not yet validated among employees in a real hospital setting. Methods: The 12-item ORIC instrument was translated into Danish and back-translated to English. Employees (N =284) at a hospital department facing a major organizational change in the Central Denmark Region completed the questionnaire. Face and content validity was ascertained. Exploratory factor analysis (EFA) and a confirmatory factor analysis (CFA) were used to assess construct validity. Reliability was assessed with Cronbach’s alpha. Item response theory (Rasch analysis) was used to determine item and person reliability. Results: Response rate was 72%. A two factor (commitment and efficacy), 11-item scale, of the Danish language ORIC was shown to be valid (CFI = .95, RMSEA = .067, and CMNI/DF = 2.32) and reliable (Cronbach’s alpha 0.88) in a health care setting. Item response analysis confirmed acceptable person and item separation reliability. Conclusions: Our version of ORIC showed acceptable validity and reliability as an instrument for measuring readiness for implementing organizational change in a Danish-speaking health care population. For health care managers interested in evaluating their organizations and tailor change strategies, ORIC’s brevity and theoretical underpinnings could make it an appealing and feasible tool to develop more successful change efforts. Keywords: Organizational readiness for change, Validation study, Translation, Change management, Implementation, Questionnaire Background success [1, 2]. Understanding the degree to which The optimism that signals the start of so many change organizational members and organizations are “ready” to efforts in health care seems somewhat misguided when implement a specific change has been suggested as a way decades of research suggest that organizational change forward. Multiple definitions and multiple instruments to initiatives frequently achieve only partial implementation measure organizational readiness for change exist [3]; however, there is no gold standard. Recently, a focus on the supra-individual level has emerged [4], and an organi- * Correspondence: Marie.storkholm@ki.se zation’s readiness for change has been defined as “ashared Department of Learning, Informatics, Management and Ethics, Medical Management Centre, Karolinska Institutet, Tomtebodavägen 18A, 171 77 psychological state in which organizational members feel Stockholm, Sweden committed to implementing an organizational change and Department of Obstetrics and Gynecology, Aarhus University Hospital, confident in their collective abilities to do so” [5]. The Aarhus, Denmark Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Storkholm et al. Implementation Science (2018) 13:78 Page 2 of 7 supra-individual level, i.e., the collective level above the Methods individuals in the organizations, is important to study as Participants and procedures collective behavior change is often required to implement This study was conducted at the Department of Obstetrics multiple changes, such as in staffing, workflow, and and Gynecology (OB/GYN) at Aarhus University Hospital organizational structures [5]. (AUH). To improve efficiency, integration, and coordin- Commitment to change can be defined as the mindset ation of health care services as required by the national that binds an individual to the course of action deemed reform, AUH started a hospital merger, downsizing of necessary for the successful implementation of a change beds, and construction of a new hospital. Hospital initiative [6]. Weiner proposes that organizational management decided that the OB/GYN department members whose commitment to change is based on should reduce beds by 36% and their budget by 10%, “want to” rather than “have to” or “ought to” display not mainly by reducing nursing staff. The efficacy require- only more cooperative behavior (e.g., volunteering for ments were addressed through an improvement process problem-solving teams), but also championing behavior that led to the implementation of changes at the level of (e.g., promoting the value of the change to others) [5, 7]. care pathways for individual medical conditions and at the Change efficacy refers to organization members’ shared organizational level. belief in their “collective capabilities to organize and Organizational members were identified through em- execute the courses of action involved in change imple- ployee lists provided by department managers. We cross- mentation” [8]. It reflects the amount of knowledge checked the list with department mailing lists to ensure all available about what to do and how to do it, i.e., it is a employees had been identified. We administrated the function of members’ cognitive appraisal of three factors questionnaire electronically via SurveyXact (Aarhus/ of implementation capability: the resources (including Denmark) and distributed it via email to all staff and man- time) available, task demands, and the situation the agers in the OB/GYN department (n = 403) in June 2014. organization faces [4, 9]. The ability to measure how an Informed consent was obtained electronically; only partic- organization’s members perceive these two constructs ipants who gave their consent were able to answer the may, ultimately, be used by health care leaders to develop questionnaire. Reminders were sent out through Septem- more effective and efficient change strategies [4]. ber. The response rate was 72%. Key characteristics of the Like many countries, Denmark has begun a major respondents are presented in Table 1. system-wide structural reform to improve efficiency and the integration and coordination of health care [10]. The regions, hospitals, and clinical departments need to Measures make considerable changes in service delivery, which re- The “Organizational Readiness for Implementing Change” quires the coordinated and collective actions of multiple (ORIC) questionnaire contains 12 items that correspond organizational members to succeed. As part of a case to the domains of commitment and efficacy. It uses a study (not reported in this Short Report) to explore how 5-point Likert scale (1 = strongly disagree and 5 = strongly a clinical department has managed this challenge [11], agree). To facilitate analysis, we grouped and labeled the we wanted to use a validated instrument to test the items according to the domain they addressed (Table 2). organization’s readiness for implementing change. As we could not find a Danish instrument, we chose the Table 1 Baseline characteristics of the study participants, no. “Organizational Readiness for Implementing Change” (%) unless otherwise indicated (n = 284) (ORIC) questionnaire developed by Shea et al. as it is a Variable No. (%) promising, reliable, and valid instrument designed for Gender, female 264 (93.0) health care, even though factor analysis has revealed some Profession limitations and it has not yet been validated in an actual Nurse and Licensed Practical Nurse 109 (38.4) hospital setting [4]. In addition to undergoing a thorough Midwife 95 (33.5) psychometric assessment, ORIC is unique in comparison Physician 51 (18.0) to other instruments [3]as it is grounded in theory, targets the supra-individual level, and its brevity suits the Medical secretary 21 (7.4) busy health care context [4]. Thus, a Danish translation of Others 8 (2.8) this instrument could both become a helpful instrument Age, years, mean (SD) 45, 9 (10.6) for Danish health care managers aiming to tailor Manager, yes 15 (5.3) implementation strategies in different health care settings Permanent employed, yes 239 (84.2) [12]. In this Short Report, we assess the reliability and Length of employment, median (IQR) 18 (8.3, 27.4) validity of a Danish version of the ORIC instrument in a hospital setting. SD standard deviation, IQR interquartile range Storkholm et al. Implementation Science (2018) 13:78 Page 3 of 7 Table 2 The original English version of the ORIC (12 items) as presented in [4]. The Danish translation can be found in the supplementary file Item number Item description Change efficacy (7 items) E1 People who work here feel confident that the organization can get people invested in implementing this change E2 People who work here feel confident that they can keep track of progress in implementing this change E3 People who work here feel confident that the organization can support people as they adjust to this change E4 People who work here feel confident that they can keep the momentum going in implementing this change E5 People who work here feel confident that they can handle the challenges that might arise in implementing this change. E6 People who work here feel confident that they can coordinate tasks so that implementation goes smoothly E7 People who work here feel confident that they can manage the politics of implementing this change Change commitment (5 items) C1 People who work here are committed to implementing this change. C2 People who work here will do whatever it takes to implement this change C3 People who work here want to implement this change C4 People who work here are determined to implement this change C5 People who work here are motivated to implement this change The numbering (E1-E7 and C1-C5) corresponds to the The number of factors to retain was determined using additional file in the Shea article [4]. parallel analysis and eigenvalue-greater-than-one deci- sion rule [15]. Following the principal axis FA, a CFA Translation was performed to validate the resulting constructs from The ORIC questionnaire was translated through the stand- the EFA. The use of CFA to investigate the construct ard approach of “forward” and “back” translation [13, 14]. validity of hypothesis-based testing instruments adds a Forward translation was done by a native Danish speaker. level of statistical precision. Maximum Likelihood Esti- Back-translation was done by an independent bilingual mation was used to fit the CFA model. A full dataset (English/Danish) native English speaker. The translations was used with no missing value. Our sample size of 284 (see Additional file 1) were compared, discussed, and then met the criteria of Myers et al. that includes as follows: refined. As recommended by Weiner, [5]the questionnaire N ≥ 200, ratio of N to the number of variables in a model was modified through the inclusion of an introductory (p), N/p ≥ 10 [16]. description of the organizational change and to specific We employed the commonly reported indexes [14, 17, 18] item sets so that it was clear what was meant by the phrase to assess the fitness of the model: chi-square/df, CFI and “this change”. root mean square error of approximation (RMSEA). The following cut-off values were used as the level of acceptance: Statistical analysis CFI equal to or greater than 0.90, [19] RMSEA equal to or Face and content validity of the translated questionnaire less than 0.08 [20], and CMIN/DF < 3. [17, 21]We consid- was assessed by Danish researchers working with ered Cronbach’s alpha values of 0.7 as acceptable for internal organizational psychology and behavior research and consistency. The level of significance was specified at 0.05. who had conducted numerous studies within health care Following the classical item analysis, Andrich’s exten- in Denmark and abroad. sion of the Rasch measurement model for rating scale Statistical analyses included Cronbach’s alpha to examine data was used [22] to evaluate the measurement proper- the reliability of the instrument, exploratory factor analysis ties of the ORIC. Rasch is often considered to be an item (EFA) using principal axis factor analysis to evaluate dimen- response theory (IRT) model. IRT is an important sionality followed by a confirmatory factor analysis (CFA) method of assessing the validity of measurement scales to investigate constructs validity. We hypothesized that the that is still underutilized [23]. It describes the properties two-factor representation efficacy and commitment of of the items in the scale, and respondents’ answers to Weiner [4] would be replicated in this study. the individual items, item and person parameters, model Principal-axis factor analysis (FA) was chosen, because fit statistics, and differential item functioning (DIF). The change commitment and change efficacy are interre- fit of response categories, items, and persons to the lated, although independent, facets of organizational expectations of the model is evaluated by quality control readiness [4, 5]. fit statistics. These are infit mean square error and outfit Storkholm et al. Implementation Science (2018) 13:78 Page 4 of 7 mean square error [24]. For response categories, items, model 1 showed the following values: CFI = .838, and persons, a fit (infit and outfit) value of 1.0 is perfect RMSEA = .113, and CMIN/DF = 4.796. For model 1, the fit to the Rasch model. Fit values less than 1.0 indicate chi-square test was significant (p < 0.001) and the other less variation in responses than the model expected criteria for model fit were not met. The main difficulty while fit values greater than 1.0 indicate greater variabil- presented in model 1 was that items E1 (Efficacy 1) and ity in responses than expected. The fit values for items C1 (Commitment 1) had low standardized regression should be between 0.5 and 1.5 for an item to have good weights and the model did not fit as reflected by the fit fit to the model [24]. Rasch analysis computes two indices. These items showed low factor loadings in both reliability coefficients for the measurement of person the EFA and CFA to warrant consideration of exclusion separation statistics—and model and real reliability to in the CFA. estimate the overall reliability of the scale. Reliability The results showed that model 3 has acceptable model value larger than 80% associated with separation index fit with CFI = .95, RMSEA = .067, and CMIN/DF = 2.32 greater than 2 is acceptable. Winsteps 4.0 was used for (Table 3). However, chi-square value for the overall the Rasch model analyses [24]. SPSS 24 and Amos 24 model fit was significant (p < .001) suggesting a lack of were used for all other analyses (SPSS, IBM, Chicago). fit between the hypothesized model and the data. We The level of significance was specified at 0.05. ignored this due to the sensitivity of chi-square in large samples as the sample size for study is greater than 200 Results [18, 25]. The assumption of normality was fulfilled. Face and content validity was confirmed for all items A two factor (commitment and efficacy), 11-item scale, with the exception of one. This referred to the word of the Danish language ORIC was shown to be valid. “commitment”, which is difficult to translate, as there is The constructs produced in the a priori model coincide no Danish word that fully captures the English meaning with the constructs contextually available in the original used in item C1. Furthermore, there were minor com- article [4]. ments regarding translations, such as the word “people” The Rasch analysis shows that the scale has acceptable in all the items, which in the translation referred to the person and item separation reliabilities but the assump- employees in the department, who were exposed to the tion of unidimensionality was not satisfied, indicating that change implementation. more than one latent construct is measured by a set of 12 The overall Cronbach’s alpha was 0.88. Efficacy is a ORIC items. The unidimensionality test is a Winsteps seven-item factor (Cronbach’s α = 0.87) and commitment principal components analysis (PCA) of the standardized a five-item factor (Cronbach’s α = 0.75). Two factors were response residuals (the responses that are misfitting). The produced from the analysis: commitment and efficacy. eigenvalue of the first contrast must be > 2.0 to suggest EFA results for the ORIC scale found two factors with that a second dimension exists within the misfitting an eigenvalue > 1. The screen plot also indicated the two responses. The eigenvalue of the first contrast is 2.22. The factors. Most items loaded onto their respective factors three items with the strongest positive loadings on the first (not presented). Items loaded on efficacy ranged between contrast were item C3 (.68), C4 (.67), and C5 (.57). The 0.54 and 0.76 and on commitment between 0.66 and 0.70. two items with the strongest negative loadings on the first CFA examined five models to evaluate the best fit for contrast were item E3 (− .58) and item E2 (− .47). the overall data (Table 3). All items displayed statistically All items have acceptable fit (Table 5). To evaluate if the significant factor loadings on their respective factors. Danish item calibrations differ by subgroups, a Model 1 includes all the items without any co-variance. Mantel-Hanzel DIF analysis was conducted. No items with Most of the loadings exhibited values between 0.62 and significantly different difficulty calibrations in age group, 0.86 for the factor efficacy (Table 4). The CFA results of sex, management role, and profession were identified. Table 3 Results of the CFA by model and indices Model Absolute fit (RMSEA) Incremental fit (CFI) Parsimonious fit (Chisq/df) Model 1 .113 .838 4.796 Model 2 .117 .841 5.000 Model 3 .067 .950 2.320 Model 4 .074 .943 2.603 Model 5 .071 .953 2.476 Model 1—all items without co-variance of the two factors; model 2—E1 is removed; model 3—E1 removed and the two factors were allowed to correlate; model 4—C1 removed and the two factors were allowed to correlate; model 5—C1 and E1 removed and the two factors were allowed to correlate RMSEA root mean square error of approximation, CFI confirmatory fit index, Chisq/df chi-square/degrees of freedom Storkholm et al. Implementation Science (2018) 13:78 Page 5 of 7 Table 4 Standardized regression weights of the items from the demonstrated two correlated factors. However, the confirmatory factor analysis (CFA) metrics in our study were slightly different. By removing Items Model 1 Model 2 Model 3 Model 5 Item E1 and allowing the factors to correlate, we identi- fied a good fit of an 11-item model with a high factor E1 .644 ––– loading. Shea et al. [4] obtained a good fit model by E2 .724 .719 .712 .713 excluding two efficacy items. E3 .763 .740 .730 .729 Our ability to construct a model with a better fit may E4 .729 .731 .738 .740 be due to the fact that our study was conducted in a E5 .635 .661 .666 .669 hospital department facing a real organizational change E6 .721 .743 .740 .742 effort. Thus, our study among hospital employees adds to Shea et al.’s study that validated the instrument E7 .667 .665 .669 .670 among graduate and undergraduate students and NGO’s C1 .402 .402 .416 – staff [4]. Like in the original article [4], item E1 (People C2 .461 .461 .488 .488 who work here feel confident that the organization can C3 .692 .692 .662 .673 get people invested in implementing this change) was C4 .707 .707 .716 .728 excluded as it cross loaded on the commitment factor. C5 .789 .789 .782 .777 The key phrase in item E1 is “invested in the change”.In E efficacy, C commitment Danish, the word invested was replaced with engaged because the phrase “investment” is in Danish exclusively associated with money, which may have contributed to For the Danish ORIC model, reliability is .86 and real the cross loading. reliability is .99 with a separation index of 2.47 and 9.69 Recent studies using the English-language ORIC either respectively. The true reliability of the Danish ORIC had a small sample or surveyed only one representative of model is somewhere between these two values. Reliability each workplace studied [26, 27], which challenge Weiner’s value larger than 80% associated with separation index conceptualization of a collective measurement [5]. In both greater than 2 is acceptable [24]. studies, the finding partly indicated that individual levels of commitment or efficacy can have been captured instead Discussion of the overall organizational levels, even though both stud- With a response rate of 72%, an 11-item scale of a Danish ies used the word “We” in their questionnaire [26, 27]. In translation of the ORIC was shown to be valid and reliable our study, we used a translation of the phrase “People when tested in a hospital setting. All items had acceptable who work here”, in accordance with the original version fit in the IRT analysis and we did not observe significant instead of “We”. We have not explored the implications of DIF in subgroups of age group, sex, management role, or this difference. Another limitation is that we surveyed one profession. clinical department, which may reduce generalizability. Corresponding to the theory and previous psychomet- Future studies should focus on comparing different ric assessment [4, 5], factor analysis of the instrument settings and determine the predictive value of ORIC through the use of organizational measures for instance on change efforts and performance outcomes. A further Table 5 Item statistics of the Danish version of ORIC limitation is that we did not perform a concurrent validity Item Difficulty Standard error Infit Outfit assessment, to compare the ORIC results with other readi- C2 – .66 .08 1.47 1.43 ness measures. C3 – .46 .08 1.39 1.39 A strength of this study is the use of both classical test theory and IRT. IRT offers a number of advantages by C1 – 1.90 .10 1.25 1.23 modeling the relationship of individual items to the E1 – .03 .08 .98 1.00 construct measured. It provides a richer description of the E5 .20 .08 .97 .95 performance of each item and greater detail on a measure’s C4 – .75 .09 .96 .96 precision compared to classical test theory, where a single C5 – .39 .08 .93 .89 estimate, such as Cronbach’s α, describes a measure’s E7 .69 .08 .88 .89 reliability [23]. Thus, our study supports the English version as reliable and valid for health care settings. In the Danish E6 .80 .08 .88 .86 context, the translated instrument can be used to further E3 1.08 .08 .87 .87 investigate the strategies hospitals employ to manage the E2 .93 .08 .81 .80 current system-wide reform and the related changes in E4 .49 .08 .76 .75 service delivery. Storkholm et al. Implementation Science (2018) 13:78 Page 6 of 7 Conclusion Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in The Danish version of ORIC showed acceptable reliability published maps and institutional affiliations. and validity as an instrument in a Danish-speaking popu- lation. It also confirms the validity and reliability of the Author details Department of Learning, Informatics, Management and Ethics, Medical ORIC instrument for hospital settings. Its brevity and Management Centre, Karolinska Institutet, Tomtebodavägen 18A, 171 77 theoretical underpinnings could make it an appealing and 2 Stockholm, Sweden. Department of Obstetrics and Gynecology, Aarhus feasible tool for health care managers interested in evalu- University Hospital, Aarhus, Denmark. Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Stockholm, Sweden. ating their organizations and to tailor change strategies that better match staff views of the organization’sreadi- Received: 27 February 2018 Accepted: 22 May 2018 ness for implementing change. Additional file References 1. Shortell S, Bennett C, Byck G. Assessing the impact of continuous quality improvement on clinical practice: what it will take to accelerate progress. Additional file 1: Organizational Readiness for Implementing Change Milbank Q 1998;76:593–624, 510. http://www.pubmedcentral.nih.gov/ (ORIC)—Danish version. (DOC 46 kb) articlerender.fcgi?artid=2751103&tool=pmcentrez&rendertype=abstract. Accessed 22 Apr 2014. 2. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of Abbreviations innovations in service organizations: systematic review and SPSS, IBM: Statistical Package for the Social Sciences, International Business recommendations. Milbank Q. 2004;82:581–629. https://doi.org/10.1111/j. Machines Corporation; AUH: Aarhus University Hospital; CFA: Confirmatory 0887-378X.2005.340_1.x. factor analysis; CFI: Confirmatory fit index; DIF: Differential item functioning; 3. Weiner BJ, Amick H, S-YD L. Conceptualization and measurement of EFA: Exploratory factor analysis; FA: Factor analysis; IRT: Item response theory; organizational readiness for change: a review of the literature in health OB/GYN: Obstetrics and gynecology; ORIC: Organizational Readiness for services research and other fields. Med Care Res Rev. 2008;65:379–436. Implementing Change; PA: Parallel analysis; RMSEA: Root mean square error https://doi.org/10.1177/1077558708317802. of approximation 4. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new Acknowledgements measure. Implement Sci. 2014;9. https://doi.org/10.1186/1748-5908-9-7. MS was financially supported by Aarhus University Hospital, Institute of 5. Weiner BJ. A theory of organizational readiness for change. Implement Sci. Clinical Medicine at Aarhus University, and Central Region Denmark. The 2009;4:67. https://doi.org/10.1186/1748-5908-4-67. funders had no involvement in the study design; in the collection, analysis, 6. Meyer JP, Stanley DJ, Herscovitch L, Topolnytsky L. Affective, continuance, and interpretation of the data; in the writing of the report; and in the and normative commitment to the organization: a meta-analysis of decision to submit the paper for publication. antecedents, correlates, and consequences. J Vocat Behav. 2002;61:20–52. https://doi.org/10.1006/jvbe.2001.1842. 7. Herscovitch L, Meyer JP. Commitment to organizational change: extension Funding of a three-component model. J Appl Psychol. 2002;87:474–87. https://doi. MS was financially supported by Aarhus University Hospital, Institute of org/10.1037//0021-9010.87.3.474. Clinical Medicine at Aarhus University, and Central Region Denmark. The 8. Bandura A. Self-efficacy: The exercise of control. New York: Freeman; 1997. funders had no involvement in the study design; in the collection, analysis, 9. Weiner BJ, Lewis MA, Linnan LA. Using organization theory to and interpretation of the data; in the writing of the report; and in the understand the determinants of effective implementation of worksite decision to submit the paper for publication. health promotion programs. Health Educ Res. 2009;24:292–305. https:// doi.org/10.1093/her/cyn019. Availability of data and materials 10. Rudkjøbing A, Olejaz M, Birk HO, Nielsen AJ, Hernández-quevedo C, The dataset collected and analyzed during this study is available from the Published A. Integrated care: a Danish perspective. BMJ. 2012;345:4451. corresponding author on reasonable request. 11. Storkholm MH, Mazzocato P, Savage M, Savage C. Money’s (not) on my mind: a qualitative study of how staff and managers understand health care’s triple aim. BMC Health Serv Res. 2017;17:98. https://doi.org/10. Authors’ contributions 1186/s12913-017-2052-3. MS, PM, and CS designed the study. MS recruited participants and 12. Jordan C, Hoefer RA. Reliability and validity in quantitative collected the data. MS and MT conducted the analyses, which were measurement. In: Thyer BA, editor. Social work research methods. reviewed by PM and CS. MS, PM, MT, and CS drafted and revised the Thousand Oaks, CA: Sage; 2001. manuscript. All authors read, contributed to, and approved the final 13. Behling O, Law KS. Translating questionnaires and other research version. PM and CS were the PIs. instruments. London: Sage; 2000. 14. Rode N. Translation of measurement instruments and their reliability: an Ethics approval and consent to participate example of job-related affective well-being scale. Metod Zv. 2005;2:15–26. Ethical approval was deemed not to be necessary by the Central Denmark 15. Kaiser HF. The application of electronic computers to factor analysis. Educ Region Ethic Review Board. Participants were informed about the purpose of Psychol Meas. 1960;20:141–51. the survey and that they could withdraw their participation at any point. They 16. Myers ND, Ahn S, Jin Y. Sample size and power estimates for a confirmatory received contact information should they have questions or desire that their factor analytic model in exercise and sport. Res Q Exerc Sport. 2011;82:412– data be withdrawn. Following a standard praxis in Denmark, the information 23. https://doi.org/10.1080/02701367.2011.10599773. provided was written out in the introduction to the questionnaire. Informed 17. Marsh HW, Hocevar D. Application of confirmatory factor analysis to the consent was received from all participants before they were able to answer the study of self-concept: first- and higher order factor models and their ORIC survey. The data were kept in a password-protected computer and was invariance across groups. Psychol Bull. 1985;97:562–82. anonymized before the statistical analysis. 18. Hair J, Black W, Babin B, Anderson R. Multivariate Data Analysis: Global Edition. 7th Edition. Upper Saddle River: Pearson Higher Education; 2010. Competing interests 19. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990; The authors declare that they have no competing interests. 107:238–46. https://doi.org/10.1037/0033-2909.107.2.238. Storkholm et al. Implementation Science (2018) 13:78 Page 7 of 7 20. Browne M, Cudeck R. Alternative ways of assessing model fit. In: Bollen K, Long J, editors. Testing structural equation models. Newbury Park, CA: Sage; 1993. 21. Kline R. Principles and practice of structural equation modeling. NY: Guilford press; 1998. 22. Andrich D. Controversy and the Rasch model. Med Care 2004;42 Supplement:I-7. https://doi.org/10.1097/01.mlr.0000103528.48582.7c. 23. Nguyan TH, Han H-R, Kim MT, Chan KS. An introduction to item response theory for patient-reported outcome measurement. Patient. 2014;7:23–35. 24. Linacre J. A User’s guide to WINSTEPS. In: Rasch-model computer programs. Program Manual for 4.0.0. Chicago. http://www.winsteps.com/winman/ copyright.htm. 25. Jöreskog KG, Sörbom D. LISREL 8: user’s reference guide. 2nd ed. Lincolnwood: Scientific Software International; 1996. 26. Sanders KA, Wolcott MD, McLaughlin JE, D’Ostroph A, Shea CM, Pinelli NR. Organizational readiness for change: Preceptor perceptions regarding early immersion of student pharmacists in health-system practice. Res Soc Adm Pharm. 2017. https://doi.org/10.1016/j.sapharm.2017.03.004. 27. Hannon PA, Helfrich CD, Chan KG, Allen CL, Hammerback K, Kohn MJ, et al. Development and pilot test of the workplace readiness questionnaire, a theory-based instrument to measure small workplaces’ readiness to implement wellness programs. Am J Health Promot. 2017;31:67–75.

Journal

Implementation ScienceSpringer Journals

Published: Jun 5, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off