Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Assessing Implementation Fidelity of a School-Based Crisis Prevention Program with an Ex-Post-Facto Design: The NETWASS FOI Assessment System

Assessing Implementation Fidelity of a School-Based Crisis Prevention Program with an... This research paper presents an operationalization procedure for measuring fidelity of implementation (FOI) of a school-based crisis prevention program. The implementation literature recommends that program developers specify core components of an intervention that are directly related to a program’s theory of change and need to be implemented with high FOI. This approach allows stakeholders some flexibility to adapt a program to individual circumstances yet helps assure that the intended outcomes are achieved. We trained 3,473 school staff in 98 German schools in the NETWorks Against School Shootings (NETWASS) program. Following the CORE cycle, we conceptualized 12 core components and operationalized relevant FOI dimensions of dosage, quality, adherence. and responsiveness. FOI was measured ex post facto, i.e after program implementation was completed, and separately for three distinct stages of implementation. Finally, we identified theoretical cut-offs for high fidelity on 15 measures using 91 items derived from an existing data set. Results indicate a high FOI across all schools for the dimensions of dosage and quality. Regarding responsiveness, high FOI was found for intervention compliance at t1 and program acceptance at the follow-up. Participant engagement during the trainings was measured separately and remained below our threshold. Adherence to 10 out of 12 core components was high. After training, school staff reported sufficient theoretical knowledge and were sensitized to recognize students in trouble, but actual case evaluation skills left room for improvement.The resulting FOI Assessment System requires validation by empirical research. Multi-level statistical modelling is necessary to test the hyothesized relationships between FOI per stage and outcomes, and obtain empirical validation of the hypothesized core components. Despite the methodological weaknesses of applying the CORE cycle ex post facto, it seems to be a feasible strategy to assess dimensions of FOI. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal of Developmental Science IOS Press

Assessing Implementation Fidelity of a School-Based Crisis Prevention Program with an Ex-Post-Facto Design: The NETWASS FOI Assessment System

Download PDF
14 pages

Loading next page...
 
/lp/ios-press/assessing-implementation-fidelity-of-a-school-based-crisis-prevention-sSabJ8TGRJ
Publisher
IOS Press
Copyright
Copyright © 2020 © 2020 – IOS Press and the authors. All rights reserved
ISSN
2192-001X
DOI
10.3233/DEV-200293
Publisher site
See Article on Publisher Site

Abstract

This research paper presents an operationalization procedure for measuring fidelity of implementation (FOI) of a school-based crisis prevention program. The implementation literature recommends that program developers specify core components of an intervention that are directly related to a program’s theory of change and need to be implemented with high FOI. This approach allows stakeholders some flexibility to adapt a program to individual circumstances yet helps assure that the intended outcomes are achieved. We trained 3,473 school staff in 98 German schools in the NETWorks Against School Shootings (NETWASS) program. Following the CORE cycle, we conceptualized 12 core components and operationalized relevant FOI dimensions of dosage, quality, adherence. and responsiveness. FOI was measured ex post facto, i.e after program implementation was completed, and separately for three distinct stages of implementation. Finally, we identified theoretical cut-offs for high fidelity on 15 measures using 91 items derived from an existing data set. Results indicate a high FOI across all schools for the dimensions of dosage and quality. Regarding responsiveness, high FOI was found for intervention compliance at t1 and program acceptance at the follow-up. Participant engagement during the trainings was measured separately and remained below our threshold. Adherence to 10 out of 12 core components was high. After training, school staff reported sufficient theoretical knowledge and were sensitized to recognize students in trouble, but actual case evaluation skills left room for improvement.The resulting FOI Assessment System requires validation by empirical research. Multi-level statistical modelling is necessary to test the hyothesized relationships between FOI per stage and outcomes, and obtain empirical validation of the hypothesized core components. Despite the methodological weaknesses of applying the CORE cycle ex post facto, it seems to be a feasible strategy to assess dimensions of FOI.

Journal

International Journal of Developmental Science – IOS Press

Published: May 2, 2020

References

  • Can evidence-based prevention programs be sustained in community practice settings? The Early Risers’ advanced-stage effectiveness trial
    August,
  • Leaking and death-threats by students: A study in German schools
    Bondü,
  • A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research
    Borrelli,
  • The Oregon multidimensional treatment foster care model: Features, outcomes, and progress in dissemination
    Chamberlain,
  • Program integrity in primary and early secondary prevention: Are implementation effects out of control?
    Dane,
  • How do implementation efforts relate to program adherence? Examining the role of organizational, implementer, and program factors
    Dariotis,
  • Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework
    Domitrovich,
  • Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation
    Durlak,
  • A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings
    Dusenbury,
  • Quality of implementation: Developing measures crucial to understanding the diffusion of preventive interventions
    Dusenbury,
  • Strategies for enhancing the adoption of school-based prevention programs. Lessons learned from the Blueprints for Violence Prevention replications of the Life Skills Training Program
    Fagan,
  • Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of parent management training
    Forgatch,
  • School shootings as organizational deviance
    Fox,
  • Quality of school-based prevention programs: Results from a national survey
    Gottfredson,
  • Assessing fidelity of implementation (FOI) for school-based mindfulness and yoga interventions: A systematic review
    Gould,
  • Assessing fidelity of core components in a mindfulness and yoga intervention for urban youth: Applying the CORE Process
    Gould,
  • Studying rare events through qualitative case studies
    Harding,
  • Transporting efficacious treatments to field settings: The link between supervisory practices and therapist fidelity in MST programs
    Henggeler,