Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Validity of self-assessment in a quality improvement collaborative in Ecuador

Validity of self-assessment in a quality improvement collaborative in Ecuador ObjectivesHealth care quality improvement (QI) efforts commonly use self-assessment to measure compliance with quality standards. This study investigates the validity of self-assessment of quality indicators.DesignCross sectional.SettingA maternal and newborn care improvement collaborative intervention conducted in health facilities in Ecuador in 2005.ParticipantsFour external evaluators were trained in abstracting medical records to calculate six indicators reflecting compliance with treatment standards.InterventionsAbout 30 medical records per month were examined at 12 participating health facilities for a total of 1875 records. The same records had already been reviewed by QI teams at these facilities (self-assessment).Main Outcome MeasuresOverall compliance, agreement (using the Kappa statistic), sensitivity and specificity were analyzed. We also examined patterns of disagreement and the effect of facility characteristics on levels of agreement.ResultsExternal evaluators reported compliance of 69–90%, while self-assessors reported 71–92%, with raw agreement of 71–95% and Kappa statistics ranging from fair to almost perfect agreement. Considering external evaluators as the gold standard, sensitivity of self-assessment ranged from 90 to 99% and specificity from 48 to 86%. Simpler indicators had fewer disagreements. When disagreements occurred between self-assessment and external valuators, the former tended to report more positive findings in five of six indicators, but this tendency was not of a magnitude to change program actions. Team leadership, understanding of the tools and facility size had no overall impact on the level of agreement.ConclusionsWhen compared with external evaluation (gold standard), self-assessment was found to be sufficiently valid for tracking QI team performance. Sensitivity was generally higher than specificity. Simplifying indicators may improve validity. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal for Quality in Health Care Oxford University Press

Validity of self-assessment in a quality improvement collaborative in Ecuador

Download PDF
7 pages

Loading next page...
 
/lp/oxford-university-press/validity-of-self-assessment-in-a-quality-improvement-collaborative-in-DQqruwTd6u

References (16)

Publisher
Oxford University Press
Copyright
© The Author 2011. Published by Oxford University Press in association with the International Society for Quality in Health Care; all rights reserved
Subject
Papers
ISSN
1353-4505
eISSN
1464-3677
DOI
10.1093/intqhc/mzr057
pmid
21840942
Publisher site
See Article on Publisher Site

Abstract

ObjectivesHealth care quality improvement (QI) efforts commonly use self-assessment to measure compliance with quality standards. This study investigates the validity of self-assessment of quality indicators.DesignCross sectional.SettingA maternal and newborn care improvement collaborative intervention conducted in health facilities in Ecuador in 2005.ParticipantsFour external evaluators were trained in abstracting medical records to calculate six indicators reflecting compliance with treatment standards.InterventionsAbout 30 medical records per month were examined at 12 participating health facilities for a total of 1875 records. The same records had already been reviewed by QI teams at these facilities (self-assessment).Main Outcome MeasuresOverall compliance, agreement (using the Kappa statistic), sensitivity and specificity were analyzed. We also examined patterns of disagreement and the effect of facility characteristics on levels of agreement.ResultsExternal evaluators reported compliance of 69–90%, while self-assessors reported 71–92%, with raw agreement of 71–95% and Kappa statistics ranging from fair to almost perfect agreement. Considering external evaluators as the gold standard, sensitivity of self-assessment ranged from 90 to 99% and specificity from 48 to 86%. Simpler indicators had fewer disagreements. When disagreements occurred between self-assessment and external valuators, the former tended to report more positive findings in five of six indicators, but this tendency was not of a magnitude to change program actions. Team leadership, understanding of the tools and facility size had no overall impact on the level of agreement.ConclusionsWhen compared with external evaluation (gold standard), self-assessment was found to be sufficiently valid for tracking QI team performance. Sensitivity was generally higher than specificity. Simplifying indicators may improve validity.

Journal

International Journal for Quality in Health Care – Oxford University Press

Published: Dec 12, 2011

Keywords: quality measurement audit quality indicators statistical methods reproductive health hospital care general medicine

There are no references for this article.