Access the full text.
Sign up today, get DeepDyve free for 14 days.
J. Dawson, I. Boller, H. Doll, G. Lavis, R. Sharp, P. Cooke, C. Jenkinson (2012)
Responsiveness of the Manchester-Oxford Foot Questionnaire (MOXFQ) compared with AOFAS, SF-36 and EQ-5D assessments following foot or ankle surgery.The Journal of bone and joint surgery. British volume, 94 2
E. Roos, S. Brandsson, J. Karlsson (2001)
Validation of the Foot and Ankle Outcome Score for Ankle Ligament ReconstructionFoot & Ankle International, 22
M. Liang (1995)
Evaluating measurement responsiveness.The Journal of rheumatology, 22 6
J. Dawson, I. Boller, H. Doll, G. Lavis, R. Sharp, P. Cooke, C. Jenkinson (2011)
The MOXFQ patient-reported questionnaire: assessment of data quality, reliability and validity in relation to foot and ankle surgery.Foot, 21 2
D. Streiner, G. Norman (2008)
Health measurement scales
F. Naal, F. Impellizzeri, P. Rippstein (2010)
Which Are the Most Frequently Used Outcome Instruments in Studies on Total Ankle Arthroplasty?Clinical Orthopaedics and Related Research®, 468
Alpesh Patel, Derek Donegan, T. Albert (2007)
The 36‐Item Short FormJournal of the American Academy of Orthopaedic Surgeons, 15
S. Weiner, M. Scarborough, R. Griend (1996)
Resection Arthrodesis of the Knee with an Intercalary Allograft*The Journal of Bone & Joint Surgery, 78
P. Dolan (1997)
Modeling valuations for EuroQol health states.Medical care, 35 11
(1978)
Statistical power analysis for the behavioral sciences
A. Williams (1990)
EuroQol : a new facility for the measurement of health-related quality of lifeHealth Policy, 16
Kim Wamper, I. Sierevelt, R. Poolman, M. Bhandari, D. Haverkamp (2010)
The Harris hip score: Do ceiling effects limit its usefulness in orthopedics?Acta Orthopaedica, 81
J. Bland, Douglas Altman (1986)
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENTThe Lancet, 327
E. Budiman‐Mak, K. Conrad, K. Roach (1991)
The Foot Function Index: a measure of foot pain and disability.Journal of clinical epidemiology, 44 6
MC and ÅC: study design, data collection, literature search, analysis of data, and manuscript preparation. JÅN: statistical analysis. MK: study design and manuscript preparation
I. Groot, M. Favejee, M. Reijman, J. Verhaar, C. Terwee (2008)
The Dutch version of the knee injury and osteoarthritis outcome score: A validation studyHealth and Quality of Life Outcomes, 6
J. Lau, Nizar Mahomed, L. Schon (2005)
Results of an Internet Survey Determining the Most Frequently Used Ankle Scores by AOFAS MembersFoot & Ankle International, 26
J. Dawson, R. Fitzpatrick, D. Murray, A. Carr (1996)
Questionnaire on the perceptions of patients about total hip replacement.The Journal of bone and joint surgery. British volume, 78 2
H. Kitaoka, I. Alexander, R. Adelaar, J. Nunley, M. Myerson, M. Sanders (1994)
Clinical Rating Systems for the Ankle-Hindfoot, Midfoot, Hallux, and Lesser ToesFoot & Ankle International, 15
Francis Guillemin, C. Bombardier, D. Beaton (1993)
Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines.Journal of clinical epidemiology, 46 12
(2006)
Validaiton of the Eruoqol questionnaire in cardiac rehabilitation Heart, 92
Altınay Karatepe, R. Günaydın, T. Kaya, Uğur Karlıbaş, Gülriz Özbek (2009)
Validation of the Turkish version of the foot and ankle outcome scoreRheumatology International, 30
(1990)
EuroQol—a new facility for the measurement of health-related quality of life. The EuroQol Group, 16
M. Sullivan, J. Karlsson, J. Ware (1995)
The Swedish SF-36 Health Survey--I. Evaluation of data quality, scaling assumptions, reliability and construct validity across general populations in Sweden.Social science & medicine, 41 10
A. Maher, T. Kilmartin (2011)
Patient reported outcomes following the combined rotation scarf and Akin's osteotomies in 71 consecutive cases.Foot, 21 1
A. Marinozzi, N. Martinelli, M. Panascì, F. Cancilleri, E. Franceschetti, B. Vincenzi, A. Martino, V. Denaro (2009)
Italian translation of the Manchester-Oxford Foot Questionnaire, with re-assessment of reliability and validityQuality of Life Research, 18
Jill Dawson, Helen Doll, Jane Coffey, Crispin Jenkinson (2007)
Responsiveness and minimally important change for the Manchester-Oxford foot questionnaire (MOXFQ) compared with AOFAS and SF-36 assessments following surgery for hallux valgus.Osteoarthritis and cartilage, 15 8
Robert Trevethan (2010)
Evaluation of two self-referent foot health instruments.Foot, 20 4
T. Harrison, E. Fawzy, F. Dinah, S. Palmer (2010)
Prospective assessment of dorsal cheilectomy for hallux rigidus using a patient-reported outcome score.The Journal of foot and ankle surgery : official publication of the American College of Foot and Ankle Surgeons, 49 3
D. Streiner, G. Norman (1989)
Health Measurement Scales: A practical guide to thier development and use
A. Hosman, Rhett Mason, T. Hobbs, A. Rothwell (2007)
A New Zealand national joint registry review of 202 total ankle replacements followed for up to 6 yearsActa Orthopaedica, 78
Gavin Button, S. Pinney (2004)
A Meta-analysis of Outcome Rating Scales in Foot and Ankle Surgery: Is There a Valid, Reliable, and Responsive System?Foot & Ankle International, 25
B. Schweikert, H. Hahmann, R. Leidl (2005)
Validation of the EuroQol questionnaire in cardiac rehabilitationHeart, 92
C. Terwee, S. Bot, M. Boer, D. Windt, D. Knol, J. Dekker, L. Bouter, H. Vet (2007)
Quality criteria were proposed for measurement properties of health status questionnaires.Journal of clinical epidemiology, 60 1
J. Dawson, Jane Coffey, H. Doll, G. Lavis, P. Cooke, M. Herron, C. Jenkinson (2006)
A patient-based questionnaire to assess outcomes of foot surgery: Validation in the context of surgery for hallux valgusQuality of Life Research, 15
N. SooHoo, Michael Shuler, L. Fleming (2003)
Evaluation of the Validity of the AOFAS Clinical Rating Systems by Correlation to the SF-36Foot & Ankle International, 24
L. Walford, G. Dahlberg (1940)
Statistical Methods for Medical and Biological StudentsThe Indian Medical Gazette, 75
M. Richter, S. Zech, J. Geerling, M. Frink, K. Knobloch, C. Krettek (2006)
A new foot and ankle outcome score: Questionnaire based, subjective, Visual-Analogue-Scale, validated and computerizedFoot and Ankle Surgery, 12
A. Maher, T. Kilmartin (2010)
Patient-reported outcomes: a new direction for podiatric surgery?
A. Bremander, I. Petersson, E. Roos (2003)
Validation of the Rheumatoid and Arthritis Outcome Score (RAOS) for the lower extremityHealth and Quality of Life Outcomes, 1
Acta Orthopaedica 2012; 83 (2): 197–203 197 Validity, reliability, and responsiveness of a self-reported foot and ankle score (SEFAS) 1, 2 2 2 2 Maria Cöster , Magnus K Karlsson , Jan-Åke Nilsson , and Åke Carlsson 1 2 Department of Orthopedics, Kalmar Hospital, Kalmar; Department of Clinical Sciences and Orthopaedics, Lund University, Skåne University Hospital in Malmö, Malmö, Sweden Correspondence: mariaco@ltkalmar.se Submitted 11-04-04. Accepted 11-10-29 Background and purpose A questionnaire was introduced by replacement (TAR) as an amendment to medically recorded the New Zealand Arthroplasty Registry for use when evaluat- joint-specific data and it proved to be useful, particularly in ing the outcome of total ankle replacement surgery. We evalu- the prediction of failures (Hosman et al. 2007). However, the ated the reliability, validity, and responsiveness of the modified original version of the questionnaire has not been validated. Swedish version of the questionnaire (SEFAS) in patients with Already existing self-administrated foot and ankle scores osteoarthritis or inflammatory arthritis before and/or after their contain numerous questions and can be complicated to use. ankle was replaced or fused. For osteoarthritis and inflammatory arthritis of the ankle, Patients and methods The questionnaire was translated into there are few validated instruments and they are seldom Swedish and cross-culturally adapted according to a standard- used (Budiman-Mak et al. 1991, Button and Pinney 2004, ized procedure. It was sent to 135 patients with ankle arthritis Naal et al. 2010). None can be regarded as the gold standard. who were scheduled for or had undergone surgery, together with The generic, self-administered questionnaires short form 36 the foot and ankle outcome score (FAOS), the short form 36 (SF- (SF-36) (Sullivan et al. 1995, Patel et al. 2007) and EuroQol 36) score, and the EuroQol (EQ-5D) score. Construct validity was (EQ-5D) (EuroQol Group 1990) are useful when evaluating evaluated with Spearman’s correlation coefficient when com- general health, but they may be less effective when evaluating paring SEFAS with FAOS, SF-36, and EQ-5D, content validity joint-specific disability. by calculating floor and ceiling effects, test-retest reliability with Thus, there is a need for a simple, self-administered and intraclass correlation coefficient (ICC), internal consistency with ankle-specific score that is capable of evaluating pain and Cronbach’s alpha (n = 62), agreement by Bland-Altman plot, and functional status in patients with osteoarthritis and inflamma- responsiveness by effect size and standardized response mean (n tory arthritis of the ankle, and the outcome of surgical inter- = 37). ventions—not least when collecting data for national surgical Results For construct validity, we correlated SEFAS with the registers. We therefore assessed the validity, the reliability, other scores and 70% or more of our predefined hypotheses con- and the responsiveness of the modified Swedish version of cerning correlations could be confirmed. There were no floor or the New Zealand total ankle replacement questionnaire, here ceiling effects. ICC was 0.92 (CI 95%: 0.88–0.95), Cronbach’s called the self-reported foot and ankle score (SEFAS), in rela- alpha 0.96, effect size was 1.44, and the standardized response tion to 3 established self-administered scoring systems. The mean was 1.00. reason for choosing the foot and ankle outcome score (FAOS) Interpretation SEFAS is a self-reported foot and ankle score for comparison was that this region-specific score is the only with good validity, reliability and responsiveness, indicating that one available in Swedish and the reason for choosing the the score can be used to evaluate patients with osteoarthritis or generic scores SF-36 and EQ-5D was because they are widely inflammatory arthritis of the ankle and outcome of surgery. used. A self-administered ankle questionnaire based on the vali- Patients and methods dated Oxford-12 questionnaire for total hip replacement has The self-reported foot and ankle score (SEFAS) been constructed by the New Zealand National Joint Regis- try. The aim was to collect patient-based data after total ankle SEFAS is based on the New Zealand total ankle questionnaire Open Access - This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the source is credited. DOI 10.3109/17453674.2012.657579 198 Acta Orthopaedica 2012; 83 (2): 197–203 (Hosman et al. 2007) that was originally derived from the vali- answer options are provided, and each question is rated on a dated Oxford-12 hip questionnaire (Dawson et al. 1996). 8 of scale from 0 to 4. A score is calculated for each subscale after the 12 questions are the same as in the original questionnaire, which raw scores for each subscale are transformed to a scale while 4 of the questions have been replaced with foot and ranging from 0 to 100 and presented graphically as the FAOS ankle-specific questions. The score covers different constructs profile. The minimum possible total score for each subscale is that are not reported separately in subscales: pain, function 0 points, a condition that represents the most severe disability, including limitations of function, and other symptoms. Each whereas the maximum of 100 points represents normal ankle of the 12 multiple-choice questions is scored from 0 to 4; thus, function. Missing data were assessed according to the user’s 0 total points represents the most severe disability and 48 guide for FAOS (2003) (www.koos.nu). represents normal function. The New Zealand National Joint The SF-36 score Registry adopted this new scoring system in 2007, as did the Swedish Ankle Registry. SF-36 is a score that is calculated from a validated generic When the Swedish version of the questionnaire was cre- questionnaire that contains 36 items (Sullivan et al. 1995, ated, it was translated into Swedish according to a standard- Patel et al. 2007). The score was developed for measuring ized cross-cultural adaption procedure (Guillemin et al. health-related quality of life and is not especially related to the 1993). The English questionnaire was forward translated disease under consideration. The score is widely used when into Swedish by 2 independent medically educated, native evaluating patients with different diseases, including muscu- Swedish speakers. The Swedish version was then backward loskeletal disorders. SF-36 measures 8 different dimensions of translated into English by a native English-speaking profes- health. The minimum possible total score of SF-36 is 0 points, sional translator with no knowledge of the original version. which represents the most severe disability, whereas 100 The original English version was then compared with the points represents the best possible health status. Finally, from backward-translated version, and if there was any discrep- the 8-dimension scores, 2 summary scales are calculated: 1 for ancy, these questions were made clearer by the 2 Swedish physical health and 1 for mental health. translators in a final Swedish version. This final version was The EuroQol (EQ-5D) score then given to 10 patients with different hindfoot disabilities. They were asked to complete the questionnaire and were also The EQ-5D score is calculated from a self-administered ques- asked if they understood all the questions. None of the volun- tionnaire developed for measuring health outcome, and like teers reported any difficulty in completing the questionnaire. the SF-36 score, does not specifically address foot and ankle Regarding 8 of the questions in the original score, we had to disability (EuroQol Group 1990, Dolan 1997). The score is make an important change in the text. Thus, in order to make applicable to a wide range of health conditions and treatments, it possible to use the questionnaire preoperatively and after and is specifically designed as a complement to other quality- procedures other than ankle replacement, “the ankle operated of-life measures such as the SF-36 (Schweikert et al. 2006). on” had to be changed to the Swedish equivalent of “the ankle The questionnaire covers 5 different dimensions. The mini- in question”. mum possible total score of EQ-5D is 0.0 points, a condition We adopted the following approach in cases of incompletely that represents the most severe disability. EQ-5D also includes answered questionnaires in the SEFAS: (1) when results from a visual analog scale (VAS) that assesses the general health 2 or more boxes were missing, the questionnaire was dis- state, ranging from 0 to 100. The worst possible health state is regarded; (2) when the result from 1 box was missing, the 0 and the best possible health state is 100. mean result of the remaining 11 boxes was used; (3) when the Patients patients gave 2 answers for 1 question, the worse outcome was recorded; and (4) when the patients had put a mark between 2 All questionnaires described above were sent to 135 patients boxes, the worse outcome was recorded. registered in the Swedish Ankle Registry due to planned and/ The New Zealand total ankle questionnaire can be found at or accomplished replacement or fusion of the ankle joint. This www.cdhb.govt.nz/NJR and the self-reported foot and ankle included 74 women and 61 men with a median age of 63 years questionnaire (SEFAS) in Swedish and English can be found (26–85), during the period February 2008 to January 2010. at www.swedankle.se. Primary total ankle replacement (TAR) was planned and/or performed in 101 patients, total ankle revision in 9, and pri- The foot and ankle outcome score (FAOS) mary ankle fusion in 25. The index diagnosis was rheumatoid The FAOS score is calculated from a patient-administrated arthritis in 27 cases, idiopathic or posttraumatic osteoarthritis 42-item questionnaire developed for foot and ankle-related in 90 cases, and “miscellaneous” in 18 cases. disability (Roos et al. 2001). The FAOS covers 5 dimensions Informed written consent was obtained from the partici- that are reported separately: (1) pain, (2) other symptoms, pants. The ethics committee of Lund University, Sweden (3) activities of daily living, (4) function in sport and recre- approved the study (2009/698) and it was performed in line ation, and (5) ankle-related quality of life (QoL). Standardized with the Helsinki Declaration. Acta Orthopaedica 2012; 83 (2): 197–203 199 Evaluation of the scores median age of 64(26–85) years to be included in the reliability Validity (n = 135). Validity is an estimate of how well a score testing. We used intraclass correlation coefficient (ICC) with actually measures what it is supposed to measure. Criterion a two-way mixed model to evaluate test-retest reliability. The validity compares a new score with a gold standard but this was ICC is considered to be good at 0.70 and above (Streiner and not applicable in our evaluation, as there is no gold standard Norman 2008). However, reliability is sometimes also reported for evaluation of foot and ankle disability. Construct validity from a wider perspective, to include internal consistency as an concerns the extent to which a score relates to other scores estimate of the extent to which the specific questions within a consistent with theoretically derived hypotheses (de Groot et score are correlated to each other and therefore measure the al. 2008). In the absence of a gold standard, the validity in our same thing. When we evaluated reliability as internal consis- study was expressed in terms of construct validity, calculated tency, we used the first questionnaire that was answered by the with the Spearman’s correlation coefficient. For the validity 62 patients described above. To test internal consistency, we test we compared the SEFAS with FAOS, SF-36 and EQ-5D. used Cronbach’s α (CA).We used the widely accepted cutoff We took account of the fact that pain and function are the two for CA at 0.70 and considered it to be good when it was 0.70 most important symptoms for the patients and that pain and or higher (Streiner and Norman 2008) function therefore are the constructs of interest in each score. Agreement (n = 62). Agreement is an estimate of the mea- For convergent validity, we formulated 5 hypotheses. The cor- surement error of a score. When we evaluated agreement, relation between SEFAS and FAOS subscales pain, activities we used the 2 sets of questionnaires from the 62 patients of daily living (ADL), and symptoms, for SEFAS and SF-36 described above and prepared the data as Bland-Altman plots BP and PF should be ≥ 0.60. We also hypothesized that that (Bland and Altman 1986, Button and Pinney 2004). These SEFAS would show stronger correlation with FAOS pain and plots show the difference between the SEFAS scores in the ADL than with SF-36 BP and PF. We formulated 3 hypotheses 2 questionnaires answered by the same patient (Bland and concerned discriminant validity: that the correlation between Altman 1986). Intraindividual variability of the functional SEFAS and SF-36 GH, SF-36 RP, and the summary scale measures was expressed as standard error of a single determi- in SF-36 mental health should be ≤ 0.30. We hypothesized nation (S ), and is shown together with the coefficient of method that all the other correlations between SEFAS and SF-36, variation (in%) for all the scores in Table 1. The formula used the EQ-5D, and FAOS sport and recreation and quality of was S = √(∑d /(2n)), where d is the difference between method i i life should be between 0.30 and 0.60. For evaluation of the the ith paired measurement and n is the number of differences construct of interest, i.e. the pain and function in SEFAS, we (Dahlberg 1940). related the pain-specific and function-specific questions sepa- Responsiveness (n = 37). Responsiveness is an estimate of rately to specific subscales in the other scores. how well a questionnaire detects changes over time or changes Floor and ceiling effects (n = 135). Floor and ceiling effects due to an intervention. When we evaluated responsiveness, we show the proportion of individuals who achieve the highest or included 37 patients (22 women) with a median age of 65 (24– lowest possible numeric value of a score and are considered 80) years who had answered the questionnaires just before and present when more than 15% of the individuals achieve these median 6 months (5–7) after replacement or fusion of their values. Floor and ceiling effects can be used when evaluating ankle. Only 20 of the 37 patients had completed the FAOS content validity. A high floor and ceiling effect could make twice, due to the fact that this questionnaire was removed from it difficult to distinguish patients from each other and also to Swedish Ankle Register in 2011. To test responsiveness, we measure changes in patients after intervention (Terwee et al. used effect size (ES) and standardized response mean (SRM). 2007, Wamper et al. 2010). Effect size is calculated by taking the difference between the Reliability (n = 62). Reliability is an estimate of the repro- means before and after treatment and dividing it by the stan- ducibility of a score, and can be measured in different ways. dard deviation of the same measure before treatment. Cohen In this study we evaluated test-retest reliability i.e. how well defined an effect size of 0.20 as small, one of 0.50 as moder - a score produces the same outcome when the questionnaire is ate, and of 0.80 or greater as large (Cohen 1978). Standard given to the same individual on separate occasions but close to response mean is calculated by taking the difference between each other in time. For this evaluation, 78 consecutive patients the means before and after treatment and dividing it by the were asked to answer and the questionnaires were sent to them standard deviation of the change. SRM values are generally twice, about 6 months after surgery, by post with a postage- lower than the corresponding ES values (Liang 1995). paid return envelope. The second questionnaire was sent as Statistics soon as they had returned the first one. A maximum of 31 days was allowed to elapse between the dates of response, and the Statistical calculations were performed with SPSS software median time was 10 days. In the second round, 3 question- version 17.0. The statistics related to validity, reproducibility, naires were incompletely filled in, 4 were returned more than reliability, agreement and responsiveness are described under 4 weeks after the first questionnaire was returned, and 9 were each paragraph above. We calculated the confidence interval not returned at all. This left 62 patients (40 women) with a for the correlations according to Fisher’s z-transformation. 200 Acta Orthopaedica 2012; 83 (2): 197–203 Table 1. Validity, reliability, and measurement errors of the different scores. Correlation analyses comparing SEFAS and the other scores. Data are presented as mean with 95% CI or standard deviation (SD), and as proportions (%) Validity Reliability Agreement Floor and Intraclass Spearman Rho ceiling correlation – SEFAS versus effects Test Retest coefficient (ICC) Cronbach’s S method Questionnaire (95% CI) (%) mean (SD) mean (SD) (95%CI) α Number 135 135 62 62 62 62 62 SEFAS 1 0 29 (9.6) 29 (9.9) 0.92 (0.87–0.95) 0.96 2.7 (15%) FAOS – – – – Pain 0.82 (0.76–0.88) 4.4 71 (21) 69 (22) 0.89 (0.82–0.93) 0.94 7.4 (11%) Symptom 0.50 (0.37–0.63) 0 60 (16) 62 (15) 0.84 (0.75–0.90) 0.92 6.1 (10%) ADL 0.77 (0.70–0.83) 1.5 77 (19) 77 (19) 0.96 (0.94–0.98) 0.98 3.7 (5%) Sport/Recreation 0.42 (0.27–0.56) 34 24 (26) 24 (25) 0.78 (0.66–0.86) 0.88 12 (49%) Quality of life 0.82 (0.76–0.88) 8.2 51 (25) 53 (25) 0.92 (0.87–0.95) 0.96 7.1 (14%) EQ–5D 0.76 (0.68–0.83) 8.1 0.72 (0.22) 0.67 (0.24) 0.80 (0.68–0.87) 0.89 0.1 (16%) Visual analog scale (VAS) 0.65 (0.53–0.75) 1.5 69 (19) 69 (20) 0.96 (0.94–0.98) 0.98 9.1 (13%) SF–36 – – – – Physical functioning (PF) 0.64 (0.53–0.74) 1.4 54 (23) 53 (23) 0.92 (0.88–0.95) 0.96 6.4 (12%) Role limitations, physical (RP) 0.30 (0.14–0.46) 62 40 (43) 40 (43) 0.89 (0.82–0.93) 0.94 13 (32%) Bodily pain (BP) 0.76 (0.68–0.83) 4.4 57 (24) 54 (23) 0.87 (0.79–0.92) 0.93 8.5 (15%) General health (GH) 0.17 (0.00–0.34) 3 67 (22) 67 (23) 0.93 (0.89–0.96) 0.97 5.9 (9%) Vitality (VT) 0.46 (0.31–0.59) 1.5 61 (21) 61 (24) 0.88 (0.80–0.92) 0.93 8.0 (13%) Social functioning (SF) 0.42 (0.28–0.57) 33 82 (23) 82 (22) 0.72 (0.57–0.82) 0.84 12 (14%) Role limitation, emotional (RE) 0.31 (0.15–0.46) 76 68 (43) 69 (42) 0.76 (0.64–0.85) 0.87 17 (25%) Mental health (MH) 0.38 (0.22–0.52) 8.9 80 (17) 81 (16) 0.83 (0.74–0.90) 0.91 6.1 (12%) Physical 0.51 (0.37–0.64) – 38 (11) 37 (11) 0.88 (0.80–0.92) 0.93 3.8 (10%) Mental 0.30 (0.13–0.45) – 53 (14) 54 (13) 0.77 (0.64–0.86) 0.87 6.4 (12%) S and the coefficient of variation for all the scores are method Results given in Table 1. The construct validity analyses, including the Spearman cor- The responsiveness analysis, including effect size (ES) and relation coefficients, are presented in Table 1. SEFAS mainly standardized response mean (SRM), is presented in Table 2. measures pain and function, and as expected we found the The ES for SEFAS was 1.44 and the SRM was 1.00. highest correlations between SEFAS and the subscales in FAOS and SF-36 that measure similar constructs. 70% or more of our predefined hypotheses could be confirmed. We Discussion also found higher correlations with FAOS pain and ADL than with SF-36 BP and PF, as expected. Concerning discriminant This study shows that the SEFAS self-reported foot and ankle validity, the correlation between SEFAS and SF-36 GH, SF-36 score has good validity, reliability, and responsiveness, which RP, and the summary scale in SF-36 mental health were low. could be used to evaluate osteoarthritis or inflammatory arthri- The correlations coefficients between the pain-specific ques- tis of the ankle both before and after surgical intervention. For tions in SEFAS and the FAOS subscale pain and SF-36 BP evaluation of overall validity of an outcome instrument, sev- were 0.81 and 0.75, respectively. The correlation coefficients eral clinimetric properties should be of sufficient quality (Bre- between the function-specific questions in SEFAS and FAOS mander et al. 2003, Terwee et al. 2007). We found that these subscale ADL and SF-36 BF were 0.68 and 0.50, respectively. properties of the SEFAS were comparable with those of the The content validity analysis, including floor and ceiling ankle specific-score FAOS. The FAOS is a widely used foot effects, is presented in Table 1. None of the patients had the and ankle-specific score that has been translated to several lan- highest possible or the lowest possible numeric value in the guages (Goksel Karatepe et al. 2009), but to our knowledge SEFAS; i.e., there was no floor or ceiling effect. The reliability has only been validated for ankle ligament reconstructions analysis, including the test-retest and the interclass correlation (Roos et al. 2001). The correlation between SEFAS and the coefficient (ICC), is also presented in Table 1. The ICC for FAOS subscale for sport and recreation was low, as the FAOS SEFAS was 0.92 (95% CI: 0.88–0.95) and the Cronbach’s α may better capture sports-specific deficits while the SEFAS was 0.96. reflects everyday activity. There was also lower correlation The agreement analysis (including Bland-Altman plots) is with the FAOS symptoms subscale, which includes various shown in the Figure. The measurement error analyses with unspecific phenomena. Acta Orthopaedica 2012; 83 (2): 197–203 201 Dierence between SEAS scores Dierence between FAOS_total Dierence between EQ-5D 20 0.8 0.6 0.4 0.2 0.0 -20 -5 -40 -0.2 -10 -60 -0.4 -15 -80 -20 -0.6 -100 01020304050 0 100 200 300 400 500 0.00 0.50 1.00 Mean of SEAS scores Mean of EQ-5D Mean of FAOS_total Dierence between VAS Dierence between SF-36 Physical Dierence between SF-36 Mental 25 20 40 -5 -10 -10 -10 -20 -15 -20 -20 -30 10 30 50 70 10 20 30 40 50 10 30 50 70 Mean of VAS Mean of SF-36 Physical Mean of SF-36 Mental Agreement in 62 patients for the SEFAS, FAOS_total, EQ-5D, VAS, and SF-36 Physical and Mental, presented as Bland-Altman plots (Bland and Altman 1986). Table 2. Responsiveness expressed as effect size (ES) and standard response mean (SRM) calculated for 37 patients before and 6 months after ankle arthroplasty operation. Due to missing answers, numbers are also shown n Preoperatively Postoperatively Effect size Standard response (mean) (mean) (ES) mean (SRM) SEFAS 35 17 27 1.44 1 FAOS Pain 19 43 68 1.78 0.94 Symptom 20 46 52 0.47 0.55 ADL 19 53 73 1.36 0.89 Sport/Recreation 20 17 24 0.37 0.26 Quality of life 20 27 48 1.38 0.79 EQ-5D 36 0.4 0.6 0.93 0.81 VAS 29 50 67 0.65 0.53 SF-36 Physical functioning (PF) 36 36 50 0.67 0.6 Role limitation, physical (RP) 35 28 36 0.23 0.15 Bodily pain (BP) 34 30 54 1.25 0.68 General health (GH) 33 70 69 –0.04 –0.06 Vitality (VT) 34 53 57 0.14 0.13 Social functioning (SF) 33 75 82 0.31 0.28 Role limitation, emotional (RE) 32 53 72 0.44 0.33 Mental health (MH) 34 79 80 0.07 0.08 Physical 30 31 36 0.66 0.49 Mental 30 51 54 0.26 0.23 In contrast to SEFAS and FAOS, SF-36 is a generic instru- a variety of general diseases, and is therefore often used in ment that measures health-related quality of life. This instru- the process of validation of new scores. As expected we found ment is widely used and validated for outcome assessment in both convergent and divergent validity when comparing the 202 Acta Orthopaedica 2012; 83 (2): 197–203 ankle-specific SEFAS with this generic SF-36. when discussing the Harris hip score. In contrast to the find- EQ-5D is another generic score used in numerous orthope- ings in the other instruments that we evaluated, SEFAS did not dic and other studies, and for various indications. In the pres- show floor or ceiling effects. Our study population was highly ent study we found a high correlation between the ankle-spe- selected, consisting of patients with ankle osteoarthritis, which cific SEFAS and EQ-5D, reflecting the fact that osteoarthritis could be a reason for these results. and inflammatory arthritis in the ankle have an effect on qual- The test-retest reliability was good for all the questionnaires, ity of life. with an ICC of > 0.70. The internal consistency was also good The foot function index (FFI) is another validated, self- for all scores, with Cronbach’s α values above 0.70. However, reported foot questionnaire that was originally validated in some authors have pointed out that a value of Cronbach’s α patients with rheumatoid arthritis (Budiman-Mak et al. 1991), that is too high may be a problem, indicating that different but it was revised and there are now several versions in differ- questions in the questionnaire capture the same symptom or ent languages. A number of problems have been apparent with deficits. In this respect, a Cronbach’s α of 0.96 in the SEFAS the FFI score (Trevethan 2010). There has also been a report may be too high to be ideal (Terwee et al. 2007). inferring that this score is of less value in patients undergoing Reliability and agreement are both concepts concerning ankle replacement (Naal et al. 2010). estimation of the reproducibility of different instruments The subjective visual-analog scale of the foot and ankle (Terwee et al. 2007). Agreement includes estimation of the (VAS FA) is a recently reported and validated questionnaire absolute measurement error, i.e. the deviation of one measure- that shows good correlation with the SF-36 (Richter et al. ment from another, while reliability estimates how well differ- 2006). However, since the reliability, content validity, and ent patients can be distinguished from each other, when taking responsiveness of the VAS FA have (to our knowledge) never the measurement error into account. The ICC and Cronbach’s been evaluated, this ought to be done before the instrument is α are the most frequently used parameters when estimat- introduced for the evaluation of ankle disability. None of the ing reliability, while a variety of parameters have been used above-mentioned scores have been translated into Swedish. when describing agreement (Streiner and Norman 2008). We In addition to the scores evaluated in this report, there are used the Bland-Altman plot (Bland and Altman 1986, Terwee clinician-based ankle-specific scores. The difference between et al. 2007). Bland-Altman plot indicated that there is good self-reported or patient-reported outcome and clinician-based agreement between responses to SEFAS questionnaires when scores is basically the fact that the clinician-based score is answered more than once. dependent on anyone who examines the patient, and of course The responsiveness is an important consideration when the assessment can be more or less subjective. The Kofoed and estimating the effect of an intervention such as arthrodesis Mazur scores are 2 clinician-based scores that (to our knowl- or arthroplasty of the ankle joint. In this study, the SEFAS edge) have never been validated (Button and Pinney 2004, questionnaire showed good responsiveness, calculated with Naal et al. 2010). The American orthopaedic foot and ankle both ES and SRM as we expected, but the sample size was score (AOFAS) (Kitaoka et al. 1994, Lau et al. 2005) is without somewhat low—which is a limitation. Also, the other region- doubt the most commonly used instrument, but it has the dis- specific score FAOS shows good responsiveness, which is one advantage of not only including patient-related information but of the known advantages of region-specific scores. also of requiring a professional clinical examination. In prac- The strengths of our study include the structural evaluation tice, this precludes this score and the above-mentioned scores of a new self-administrated ankle-specific score against other from being used in large registry studies. The AOFAS has commonly used scores (both foot and ankle-specific) and also been the subject of other criticism. SooHoo et al. (2003) generic scores regarding reliability, validity, and responsive- reported poor construct validity when comparing AOFAS with ness. One of the limitations of our study is that we did not SF-36 in patients with foot and ankle disability, lower than include the satisfacion-rate of the patient. It would also have when SF-36 was compared with scores evaluating shoulder, been advantageous if patients with different ankle diagnoses knee, and upper extremity disability. Another concern is that and foot disorders had been evaluated in the same manner as clinician-based scores do not adequately take into account the separate cohorts, as our inferences can now only be applied to patient’s point of view. These problems were summarized by patients with ankle arthritis and only to surgical intervention Naal et al. (2010), who reported that self-reported outcome with arthroplasty or arthrodesis. instruments allow a more complete estimation of the patient’s We conclude that SEFAS is a valuable self-administrated health status and of other issues relevant to the patient. questionnaire for evaluating patients with osteoarthritis and Floor and ceiling effects must also be evaluated when intro- inflammatory arthritis of the ankle and the outcome of ankle ducing new scores, as a ceiling effect makes it impossible surgery. It could, for example, be a suitable tool for patient- to grade improvements after interventions. This estimate is related outcome measures (PROMs) in connection with most important when evaluating registry data for clinicians national registries. Further studies ought to be conducted to and healthcare politicians when allocating resources to spe- determine whether SEFAS can also be used to estimate the cific interventions—as pointed out by Wamper et al. (2010) function of patients with all kinds of foot and ankle disorders. Acta Orthopaedica 2012; 83 (2): 197–203 203 MC and ÅC: study design, data collection, literature search, analysis of data, Hosman A H, Mason R B, Hobbs T, Rothwell A G. A New Zealand national and manuscript preparation. JÅN: statistical analysis. MK: study design and joint registry review of 202 total ankle replacements followed for up to 6 manuscript preparation. years. Acta Orthop 2007; 78 (5): 584-91. Kitaoka H B, Alexander I J, Adelaar R S, Nunley J A, Myerson M S, Sand- ers M. Clinical rating systems for the ankle-hindfoot, midfoot, hallux, and lesser toes. Foot Ankle Int 1994; 15 (7): 349-53. This study was supported by grants from the Swedish Medical Research Lau J T, Mahomed N M, Schon L C. Results of an Internet survey determining Council, the Skåne and Kalmar Regional Grants, the Swedish Society of the most frequently used ankle scores by AOFAS members. Foot Ankle Int Medicine, the Swedish Society for Medical Research, and the Swedish Asso- 2005; 26 (6): 479-82. ciation of Local Authorities and Regions. Liang M H. Evaluating measurement responsiveness. J Rheumatol 1995; 22 (6) : 1191-2. Naal F D, Impellizzeri F M, Rippstein PF. Which are the most frequently Bland J M, Altman D G. Statistical methods for assessing agreement between used outcome instruments in studies on total ankle arthroplasty? Clin two methods of clinical measurement. Lancet 1986; 1 (8476): 307-10. Orthop 2010; (468) (3): 815-26. Bremander A B, Petersson I F, Roos E M. Validation of the Rheumatoid and Patel A A, Donegan D, Albert T. The 36-item short form. J Am Acad Orthop Arthritis Outcome Score (RAOS) for the lower extremity. Health Qual Life Surg 2007; 15 (2): 126-34. Outcomes 2003; 1: 55. Richter M, Zech S, Geerling J, Frink M, Knobloch K, Krettek C. A new foot Budiman-Mak E, Conrad K J, Roach KE. The Foot Function Index: a mea- and ankle outcome score: Questionnaire based, subjective, Visual-Ana- sure of foot pain and disability. J Clin Epidemiol 1991; 44 (6): 561-70. logue-Scale, validated and computerized. Foot and Ankle Surgery 2006; Button G, Pinney S. A meta-analysis of outcome rating scales in foot and 12 (4): 191-9. ankle surgery: is there a valid, reliable, and responsive system? Foot Ankle Roos E M, Brandsson S, Karlsson J. Validation of the foot and ankle outcome Int 2004; 25 (8): 521-5. score for ankle ligament reconstruction. Foot Ankle Int 2001; 22 (10): 788- Cohen J. Statistical power analysis for the behavioral sciences. Academic 94. Press, New York. 1978. Schweikert B, Hahmann H, Leidl R. Validaiton of the Eruoqol questionnaire in cardiac rehabilitation Heart 2006; 92 (1): 62-7. Dahlberg G. Statistical methods for medical and biological students. London: George Allen and Unwin Ltd, 1940. SooHoo N F, Shuler M, Fleming L L. Evaluation of the validity of the AOFAS Clinical Rating Systems by correlation to the SF-36. Foot Ankle Int 2003; Dawson J, Fitzpatrick R, Carr A, Murray D. Questionnaire on the perceptions 24 (1): 50-5. of patients about total hip replacement. J Bone Joint Surg (Br) 1996; 78 (2): 185-90. Streiner D, Norman G. Health measurement scales: a practical guide to their developement and use. New York: Oxford University Press, 2008. de Groot I B, Favejee M M, Reijman M, Verhaar J A N, Terwee C B. The Dutch version of the knee injury and osteoarthtitis outcome score: A vali- Sullivan M, Karlsson J, Ware J E, Jr. The Swedish SF-36 Health Survey. I. dation study. Health and Quality of Life Outcomes 2008; 6: 16 Evaluation of data quality, scaling assumptions, reliability and construct validity across general populations in Sweden. Soc Sci Med 1995; 41 (10): Dolan P. Modeling valuations for EuroQol health states. Med Care 1997; 35: 1349-58. 1095-108. Terwee C B, Bot S D, de Boer M R, van der Windt D A, Knol D L, Dekker J, EuroQol Group. EuroQol—a new facility for the measurement of health- et al. Quality criteria were proposed for measurement properties of health related quality of life. The EuroQol Group. Health Policy 1990; 16 (3): status questionnaires. J Clin Epidemiol 2007; 60 (1): 34-42. 199-208. Goksel Karatepe A, Gunaydin R, Kaya T, Karlibas U, Ozbek G. Validation Trevethan R. Evaluation of two self-referent foot health instruments. Foot of the Turkish version of the foot and ankle outcome score. Rheumatol Int (Edinb) 2010; 20 (4): 101-8. 2009; 30 (2): 169-73. Wamper K E, Sierevelt I N, Poolman R W, Bhandari M, Haverkamp D. The Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health- Harris hip score: Do ceiling effects limit its usefulness in orthopedics? Acta related quality of life measures: literature review and proposed guidelines. Orthop 2010; 81 (6): 703-7. J Clin Epidemiol 1993; 46 (12): 1417-32.
Acta Orthopaedica – Taylor & Francis
Published: Apr 1, 2012
You can share this free article with as many people as you like with the url below! We hope you enjoy this feature!
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.