Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Are distal radius fracture classifications reproducible? Intra and interobserver agreement

Are distal radius fracture classifications reproducible? Intra and interobserver agreement João Carlos Belloti Are distal radius fracture Marcel Jun Sugawara Tamaoki Carlos Eduardo da classifi cations reproducible? Intra Silveira Franciozi João Baptista Gomes dos Santos and interobserver agreement Daniel Balbachevsky Department of Orthopedics and Traumatology, Universidade Federal de Eduardo Chap Chap São Paulo — Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil Walter Manna Albertoni Flávio Faloppa INTRODUCTION tem has been used to defi ne surgical fixation ABSTRACT Distal radius fractures have an approxi- methods, but its accuracy and reproducibility CONTEXT AND OBJECTIVE: Various classi fica- mate incidence of 1:10,000 people and rep- for identifying the four fragments on con- tion systems have been proposed for fractures resent 16% of skeletal and 74% of forearm ventional x-rays have not been validated yet of the distal radius, but the reliability of these fractures. They are more prevalent among by clinical trials, and the system still presents classifications is seldom addressed. For a fracture classification to be useful, it must females and present a progressive increase in disagreements. provide prognostic signi ficance, interobserver complications with age, as osteopenia and The Arbeitsgemeinschaft für Osteo- reliability and intraobserver reproducibility. The osteoporosis become more prevalent. The synthesefragen/Association for the Study of aim here was to evaluate the intraobserver and interobserver agreement of distal radius fracture most common trauma mechanism is falling Internal Fixation (AO/ASIF) rating system classifi cations. 3 over onto the hand. The characteristics of was created in 1986 and reviewed in 1990. It DESIGN AND SETTING: This was a validation such fractures (trace location, possible joint considers bone injury severity and is a basis study on interobserver and intraobserver reli- involvement, comminution and degree of soft- for treatment and results evaluation. There are ability. It was developed in the Department of part lesion) are directly related to the force of three basic lesion types in this system: extra-ar- Orthopedics and Traumatology, Universidade Federal de São Paulo — Escola Paulista the trauma, wrist angle at the moment of the ticular, partial articular and complete articular. de Medicina. 2 trauma and bone health. The three groups are organized into increasing METHOD: X-rays from 98 cases of displaced dis- Systems have been developed to help order of severity of morphological complexity, tal radius fracture were evaluated by v fi e observ- surgeons in classifying fractures into differ- treatment diffi culty and prognostics. It is one ers: one third-year orthopedic resident (R3), one ent and clinically useful groups for treatment of the most complete ratings available, but sixth-year undergraduate medical student (UG6), one radiologist physician (XRP), one orthopedic de n fi ition. Colles, Smith, Pouteau, and others its intra and interobserver reproducibility has trauma specialist (OT) and one orthopedic hand described fracture morphology with a view been a problem when evaluating groups and surgery specialist (OHS). The radiographs were 3-5 11,12 to treatment. With the advent of radiology, subgroups. classi fied on three different occasions (times it became possible to describe injuries more The Universal rating system described T1, T2 and T3) using the Universal (Cooney), Arbeitsgemeinschaft für Osteosynthesefragen/ precisely, including both the degree of dis- by Cooney is characterized by simplicity, Association for the Study of Internal Fixation placement and the presence of joint injuries. classifying fractures as intra or extra-articular, (AO/ASIF), Frykman and Fernández classifi ca- In 1951, Garland and Werley created a clas- displacement present or absent, and according tions. The kappa coeffi cient ( κ) was applied to assess the degree of agreement. sifi cation based on the presence or absence of to the degree of stability and possibilities of joint involvement, metaphyseal comminution reduction. It thus acts as a guide for treatment RESULTS: Among the three occasions, the highest mean intraobserver k was observed in the Univer- and/or angular deformity. In 1959, Lindstrom patterns. sal classifi cation (0.61), followed by Fernández expanded these criteria to six groups, describ- The rating system proposed by Fernández (0.59), Frykman (0.55) and AO/ASIF (0.49). ing the fragment displacement in further is based on the trauma mechanism. This rat- The interobserver agreement was unsatisfactory in all class ic fi ations. The Fernández class ic fi ation detail, along with joint involvement. ing was created to be practical, predict stabil- showed the best agreement (0.44) and the worst In 1967, Frykman established a rating ity, check on associated fractures of the ulna was the Frykman classifi cation (0.26). system that considered the radiocarpal and/or styloid process, identify equivalent lesions in CONCLUSION: The low agreement levels ob- distal radius-ulna joints, and also the presence children and make general recommendations served in this study suggest that there is still no or absence of the ulnar styloid. Even thus, this for treatment. classifi cation method with high reproducibility. was a limited rating system: it did not consider To be considered good, a rating system KEY WORDS: Colles’ fracture. Radius fractures. factors like the extent of fragment displace- must be valid, reliable and reproducible. Classifi cation. Reproducibility of results. Valida- tion studies. ment, presence or absence of comminution Furthermore, an ideal rating system should and instability factors. standardize a trustworthy communication lan- In 1984, Melone published a rating sys- guage that provides guidelines for treatment, tem for distal radius joint fractures based on indicates the possibilities of complications, four parts: radius styloid, radius shaft, dorsal evaluates fracture stability and enables frac- fragment and palmar radius. This rating sys- ture prognosis. This ideal system should also Sao Paulo Med J. 2008;126(3):180-5. ORIGINAL ARTICLE 181 provide a mechanism that allows comparison an illustrated brochure showing descriptions of studies to evaluate intraobserver and interob- of the results obtained with treatments un- degrees and types of injury. server reliability and reproducibility. The dertaken on similar fractures in other centers, At the first evaluation (time T1), all the kappa agreement coefc fi ient provides a parallel reported at different times in the literature. x-rays were assessed in numerical sequence. rating of the agreement among the observers Variation in evaluators’ expertise may Three weeks later, at the second evaluation that is randomly correct. Kappa values range have influenced evaluations carried out on (time T2), the initial x-ray order was ran- from -1 to +1; values between -1 and 0 indicate intraobserver and interobserver agreement. domly changed to generate a new sequence. that the observed agreement was lower than Studies have shown that less experienced A further randomization of the sequence was what was randomly expected, 0 indicates the observers attain lower rates of intraobserver performed for the third evaluation (time T3), random agreement level, and +1 indicates total 12,16 17 agreement than do expert physicians. after six weeks. The x-rays were scanned and agreement. In general, kappa values of less However, in a comparison of one group in analyzed in computers. Data were collected than 0.5 are considered unsatisfactory; values which the observers were more experienced in on spreadsheets and the kappa (κ) coefficient between 0.5 and 0.75 are considered satisfac- rating assessments with another group whose was used to assess agreements. tory and appropriate, and values above 0.75 expertise was lower, no signic fi ant difference in k was applied using the method proposed are considered excellent. 12 17 interobserver agreement was found. It would by Fleiss et al. , and the random expected This project was approved by the Research also be expected that, as observers study and agreement calculation described by Scott Ethics Committee of Unifesp-EPM, under become accustomed to using a given rating and Cohen was also used. The latter two No. 1076-06, on August 4, 2006. system, the agreement between them, and methods enable calculation of agreements within their own observations, would increase. for multiple (more than two) observers with RESULTS Yet, it was observed that repeated application, regard to evaluations of nominal variances. Out of the initial 98 fractures, eight were i.e. at different moments in time, of the same They have therefore frequently been used in excluded: four presented poor quality x-rays rating system, had no impact on intraobserver and interobserver reproducibility. Table 1. Intraobserver kappa values between the three times (T1, T2 and T3) Considering the high prevalence of these Classification kinds of fracture and the need to properly Observer and reproducibly classify them, we developed Universal AO/ASIF Frykman Fernández the present study. Its aim was to evaluate the OHS 0.6568 0.6362 0.6375 0.7115 reproducibility of the four most widely used OT 0.6452 0.634 0.6632 0.5274 rating systems in our field. XRP 0.3513 0.3111 0.3099 0.2882 R3 0.7589 0.4835 0.549 0.7412 UG6 0.6406 0.3751 0.5829 0.6812 OBJECTIVE Mean kappa 0.61056 0.48798 0.5485 0.5899 This objective of this study was to evaluate R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand the intraobserver and interobserver agreement surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association of the Universal, AO/ASIF, Frykman and for the Study of Internal Fixation. Fernández rating systems for fractures with regard to displacement of the radius distal Table 2. Intraobserver kappa values between times T1 and T2 extremity. Classification Observer Universal AO/ASIF Frykman Fernández MATERIAL AND METHODS OHS 0.6914 0.6284 0.6121 0.738 This was a ratings reproducibility study OT 0.5784 0.623 0.6075 0.4933 using the kappa index. Ninety-eight displaced XRP 0.2478 0.2144 0.2971 0.2638 distal radius fractures in 96 patients over the R3 0.7089 0.4341 0.6076 0.7699 age of 40 years who had been treated at the UG6 0.5821 0.385 0.5036 0.626 Hand Institute of Universidade Federal de São Mean kappa 0.56172 0.45698 0.52558 0.5782 Paulo — Escola Paulista de Medicina (Unifesp- R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand EPM) were retrospectively evaluated from the surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association for the Study of Internal Fixation. radiographic archives. Five observers were involved: one third-year orthopedic resident Table 3. Intraobserver kappa values between times T2 and T3 (R3), one sixth-year undergraduate medical Classification student (UG6), one radiologist physician Observer Universal AO/ASIF Frykman Fernández (XRP), one orthopedic trauma specialist (OT) OHS 0.6597 0.7076 0.6896 0.7721 and one orthopedic hand surgery specialist OT 0.6905 0.6618 0.7112 0.4433 (OHS). These observers used four classic fi ation XRP 0.4523 0.4381 0.3116 0.3775 systems to label each case using simple x-rays R3 0.8909 0.6927 0.5397 0.8023 in two incidence planes (posteroanterior and UG6 0.6615 0.5504 0.7024 0.8117 lateral to the wrist). The classic fi ations used Mean kappa 0.67098 0.61012 0.5909 0.64138 were the Universal (Cooney), AO/ASIF, Fryk- R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand man and Fernández, and these were previously surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association presented and explained to the evaluators, with for the Study of Internal Fixation. Sao Paulo Med J. 2008;126(3):180-5. 182 and another four presented x-rays produced Table 4. Interobserver kappa values at each time (T1, T2, T3) with the forearm immobilized in plaster. Thus, Classification Time the sample size was reduced to 90 fractures. Universal AO/ASIF Frykman Fernández The highest mean intraobserver κ, tak- T1 0.3963 0.2702 0.2427 0.34 ing all three observation times, was from the T2 0.4004 0.2988 0.2589 0.4087 Universal classification ( κ = 0.61), followed T3 0.4118 0.3117 0.2608 0.4344 by Fernández (κ = 0.59), Frykman (κ = 0.55) AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/ Association for the Study of Internal Fixation. and AO/ASIF (κ = 0.49) (Table 1). Evaluation of the intraobserver k between Table 5. Analysis of interobserver kappa values for pairs at time 1 (T1) the times T1 and T2 showed that the highest mean was from the Fernández classifica- Classification Pairs tion (κ = 0.58), followed by the Universal Universal AO/ASIF Frykman Fernández (κ = 0.56), and the lowest mean was from the OHS versus OT 0.5429 0.4911 0.4007 0.3654 AO/ASIF (κ = 0.46) (Table 2). OHS versus XRP 0.2097 0.1635 0.1733 0.3036 Between times T2 and T3, the mean intrao- OHS versus R3 0.5465 0.4532 0.2999 0.6849 bserver κ was greater, ranging from κ = 0.59 for OHS versus UG6 0.3683 0.1644 0.0954 0.4304 the Frykman classic fi ation to κ = 0.67 for the OT versus XRP 0.3463 0.2158 0.2468 0.2186 Universal classic fi ation (Table 3). OT versus R3 0.594 0.4272 0.4173 0.3505 All the mean interobserver k values for the OT versus UG6 0.4453 0.2431 0.2225 0.3188 classic fi ations were higher at time 3, such that XRP versus R3 0.2689 0.1497 0.2513 0.1548 they were (in decreasing order) Fernández XRP versus UG6 0.1958 0.2158 0.1686 0.0584 κ = 0.44, Universal κ = 0.41, AO/ASIF R3 versus UG6 0.5726 0.2872 0.24 0.6029 κ = 0.31 and Frykman κ = 0.26 (Table 4). R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association Evaluation of the interobserver k by com- for the Study of Internal Fixation. paring pairs of observers at time 1 showed that the highest agreement was between the ob- servers R3 and UG6 (0.60) in the Fernández Table 6. Analysis of interobserver kappa values for pairs at time 2 (T2) classification. On the other hand, the lowest Classification Pairs agreement was between XRP and UG6 (0.06), Universal AO/ASIF Frykman Fernández in the same classification system (Table 5). OHS versus OT 0.5751 0.4889 0.399 0.4102 At time 2, the highest agreement was OHS versus XRP 0.2338 0.2191 0.1373 0.3286 obtained between OHS and R3 (0.77) in the OHS versus R3 0.4934 0.4602 0.308 0.7691 Fernández classification, while the lowest was OHS versus UG6 0.556 0.3695 0.1495 0.4989 between XRP and R3 (0.12) in the AO/ASIF OT versus XRP 0.2769 0.1808 0.2444 0.2737 system (Table 6). OT versus R3 0.5285 0.5285 0.4117 0.4193 At time 3, the highest κ was between OT versus UG6 0.5428 0.3342 0.2949 0.4153 OHS and R3 (0.6) in the Fernández classic fi a - XRP versus R3 0.2112 0.1174 0.2225 0.3397 tion, while the lowest was between XRP and XRP versus UG6 0.2187 0.1272 0.2748 0.205 UG6 (0.1) in the AO/ASIF system (Table 7). R3 versus UG6 0.4368 0.3646 0.2422 0.4926 R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association DISCUSSION for the Study of Internal Fixation. The four classification systems evaluated in the present study were chosen because they are the ones that are most widely studied Table 7. Analysis of interobserver kappa values for pairs at time 3 (T3) and used in our field to classify distal radius Classification fractures. Pairs Universal AO/ASIF Frykman Fernández In the Frykman classification, the general OHS versus OT 0.5441 0.5382 0.4512 0.4781 mean kappa value for intraobserver agreement OHS versus XRP 0.2692 0.2175 0.297 0.3492 was satisfactory (0.55), although the radiolo- OHS versus R3 0.532 0.4985 0.3277 0.6471 gist physician (XRP) presented an unsatisfac- OHS versus UG6 0.5011 0.4618 0.1265 0.5337 tory value (0.31) that was far from the other OT versus XRP 0.3481 0.222 0.4361 0.293 four observers. After recalculating the intrao- OT versus R3 0.5491 0.4306 0.2638 0.4564 bserver kappa without the medical student OT versus UG6 0.5142 0.3328 0.2312 0.4769 (UG6) and the orthopedic resident (R3), who XRP versus R3 0.2471 0.1551 0.1759 0.3381 were less experienced evaluators, the kappa XRP versus UG6 0.2244 0.1005 0.2337 0.3505 value decreased to 0.54. This showed that the R3 versus UG6 0.5118 0.4143 0.2105 0.534 professional’s expertise level had no signic fi ant R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand impact on the intraobserver agreement. Vari- surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association for the Study of Internal Fixation. ance analysis between the observation times Sao Paulo Med J. 2008;126(3):180-5. 183 showed that UG6 presented relatively high exclusively based on radiographic criteria: groups and subgroups (nine types) and the variance (0.51 to 0.70) that was 39% greater involvement or non-involvement of the radio- mean intraobserver value was unsatisfactory than among the other observers. This probably carpal joint, presence or absence of dislocation, (0.49). There was a significant difference be - resulted from the learning process required fracture reducibility and stability. The biggest tween the values for the more experienced to become accustomed to this classification difc fi ulty found in applying this classic fi ation observers (OHS κ = 0.64 and OT κ = 0.64) system. This assumption is reinforced by the was in assessing the degree of instability of and those for the less experienced ones observation that there was relatively lower the fracture. The literature did not demon- (R3 κ = 0.4835 and UG6 κ = 0.3751). This variance among the more experienced observ- strate any consensus regarding the best way to suggests that the expertise level had an influ - ers at the same times. This suggests that the predict specic fi instability criteria on the initial ence. Only the XRP observer presented a value observer’s conditioning and knowledge, spe- x-ray, and there are several studies with discor- at odds with what was expected (κ = 0.34) 23-26 cific to the Frykman system, had a significant dant results concerning such criteria. for the more experienced evaluators. When impact on the reproducibility obtained. It In the Universal classic fi ation, the average the intraobserver kappa was recalculated is important to make it clear that the profes- intraobserver index was satisfactory (0.61056). without the less experienced observers (R3 sional expertise level was different from the When the intraobserver kappa was recalcu- and UG6), there was an increase in κ to level of experience relating to the classic fi ation. lated without the less experienced observers 0.53, which reinforces the hypothesis that the The k-value for the intraobserver agreement (R3 and UG6), there was a reduction in kappa professional expertise level had a significant in the Frykman classification evaluated by to 0.5511. This demonstrated that the degree impact on the intraobserver agreement. The Andersen et al. in 1996 was 0.48. In 1998, of expertise did not influence the results, analysis of variation between the times T1 and Illarramendi et al. in 1998 found κ = 0.61, since an increased kappa would be expected T3 demonstrated that the UG6 observer (less and in 2003, Oliveira Filho et al. found κ = when excluding the less experienced evalu- experienced) had an increase in agreement 0.55. These coefficients reported in literature ators. On the other hand, analysis of how of 43%, XRP increased by 59.6%) and R3 were in line with the results from the present the agreement evolved from time T1 to M3 increased by 104.3%. This demonstrated that study (κ = 0.55). showed that UG6 presented an increase of conditioning to the classic fi ation had a signifi - With regard to the observer’s experience, 13%, which was lower than what was observed cant impact on intraobserver reproducibility, the study published by Oliveira Filho et al. for R3 (increase of 25.7%) and XRP (increase particularly among the individuals with less had similar conclusions to ours, thus demon- of 82.5%). The intraobserver agreement for expertise in using it. strating the positive effect of expertise on the the Universal classification was also satisfac - In the literature, we saw that κ ranged 16 10,12,16,22 agreement rate. tory in another study, which found κ = 0.54. from 0.37 to 0.60 in different studies, The interobserver agreement rate for However, that study demonstrated that the thus suggesting that the intraobserver repro- the Frykman classification was unsatisfac - observer’s experience was a factor that modi- ducibility of AO/ASIF should be close to 0.5. tory, albeit with a progressive increase from fied the agreement. In the present study, the mean κ was 0.48, T1 (0.2427) to T3 (0.2608). However, this The interobserver agreement for the with a range from 0.31 to 0.63. It was only in increase was relatively lower than what was ob- Universal classification was unsatisfactory, the study by Andersen et al. , that the profes- served from the other classification systems. but presented a progressive increase from T1 sional expertise level had no signic fi ant impact The analysis showed that, in comparison (0.3963) to T3 (0.4118). The XRP evaluator on intraobserver reproducibility. This could with the most experienced observers (OHS presented a lower agreement rate than what be explained by the presence of radiologist and OT), the XRP observer presented lower would be expected. However, we found that and orthopedist observers who were working agreement rates. This suggests that although this observer’s agreement rate increased in in similar fields and frequently applied the the XRP observer had professional experience relation to the OT and OHS evaluators. This AO/ASIF classification, in the same way as in 12,16,22 with radiographic evaluations, this observer suggests that conditioning to the Universal our study. In the other studies, expertise was not using these classification methods classification (i.e. the evolution from T1 to played a modifying role in relation to intrao- routinely. This demonstrates that professional T3) was a factor that acted positively on the bserver reproducibility. experience of radiographic evaluation is not, reproducibility. The interobserver agreement for the AO/ on its own, a determining factor for a higher The same difficulty described above for ASIF classic fi ation was also unsatisfactory, but agreement rate using these classification sys - the Universal classification was found in the presented progressive increase from T1 (0.27) tems. We also saw this when analyzing the AO/ASIF application, even considering that to T3 (0.31). The XRP evaluator presented other classification methods. in the latter, evaluation of the comminution increased agreement with OT and OHS, by In our study, the interobserver repro- location is extremely important for defining 0.8% and 3.0% respectively, and the UG6 ducibility of the Frykman classification was the groups. It is possible that this difficulty observer presented increased agreement with unsatisfactory (0.26 at T3), and the k value is the limiting factor for unsatisfactory agree- OT and OHS, by 36.9% and 180.9%. This was relatively lower than found in the studies ment rates that have been found in previous suggests that conditioning was a factor acting 10 22 10,16 by Andersen et al. and Illarramendi et al. , studies. positively towards interobserver reproduc- which presented k of 0.35 and 0.43 respec- Assuming that the presence and location ibility, particularly for the less experienced tively. Our unsatisfactory result from the of comminution are determining variables individuals. 10,12,16,22 Frykman classification probably results from with regard to fracture stability, thereby den fi i - In the literature, we saw that the the low agreement rate between XRP and the tively guiding the therapy, detailed investiga- interobserver κ ranged from 0.3 to 0.5. This other evaluators. tion of the reproducibility of these variables suggests that the AO/ASIF kappa is close to The Universal classic fi ation evaluates the on the radiograph becomes necessary. 0.4, implying unsatisfactory reproducibility. following variables of distal radius fractures, In the AO/ASIF classification, we used It also suggests that the professional expertise Sao Paulo Med J. 2008;126(3):180-5. 184 level had no impact on interobserver reproduc- there was a progressive increase in agreement still no classification method that is fully ibility in this classification, in the same way as between T1 (κ = 0.34) and T3 (κ = 0.44). reproducible. seen in our study. This was relatively greater than what was seen The best interobserver reproducibility rate In the Fernández classic fi ation, the mean in the other classifications. This suggests that was observed in the Fernández classification intraobserver κ was satisfactory (κ = 0.59). the conditioning in this classification had a (0.43) and the worst was in the Frykman When the intraobserver kappa was recalculated greater impact on reproducibility than did the classification (0.26). The intraobserver re - without the less experienced observers, there conditioning in other classifications. producibility was satisfactory in the Universal was a reduction in κ (0.51), thus demonstrat- It is important to mention that the present (0.61), Fernández (0.59) and Frykman (0.55) ing that professional expertise did not have any study was limited to evaluating the agree- classic fi ations, and it was unsatisfactory in the inu fl ence on intraobserver agreement. Likewise, ment between the observers’ opinions. The AO/ASIF classification (0.49). professional experience was not seen to have any study was unable to measure the accuracy of Implications for positive inu fl ence on interobserver agreement each observer’s opinion. To clarify the accuracy further research between the times T1 and T3. issue, studies in which clinical-radiographic Conditioning (through evolution from diagnoses made by each observer were com- There is a need to perform new studies T1 to T3) was seen to be a factor acting pared with an examination result or a standard aimed at clarifying which classification vari - positively on intraobserver reproducibility, procedure, i.e. one with high sensitivity and ables present the highest disagreement rates for the Fernández classification. There are no specificity, would be needed in order to prove between observers, with consequent limits equivalent studies on this classification in the the proposed diagnosis. to reproducibility. In the continuing search literature, which makes it impossible to make for an ideal classification, prospective studies comparisons with the present results. CONCLUSIONS to describe which variables can predict the Regarding the interobserver agreement The agreement rates observed in the instability factors in such fractures through for this classification, it could be seen that present study show that currently there is radiographic examination are necessary. REFERENCES 1. Pires PR. Fraturas do rádio distal. In: Sociedade Brasileira de 13. Cooney WP. Fractures of the distal radius. A modern 24. Lafontaine M, Hardy D, Delince P. Stability assessment of distal Ortopedia e Traumatologia. Traumatologia ortopédica. Rio de treatment-based classification. Orthop Clin North Am. radius fractures. Injury. 1989;20(4):208-10. Janeiro: Revinter; 2004. 1993;24(2):211-6. 25. Abbaszadegan H, Jonsson U, von Sivers K. Prediction 2. Falch JA. Epidemiology of fractures of the distal forearm in 14. Fernández DL. Fractures of the distal radius: operative treatment. of instability of Colles’ fractures. Acta Orthop Scand. Oslo, Norway. Acta Orthop Scand. 1983;54(2):291-5. Instr Course Lect. 1993;42:73-88. 1989;60(6):646-50. 3. Colles A. On the fracture of the carpal extremity of the radius. 15. Martin JS, Marsh JL. Current classification of fractures. Ratio - 26. Nesbitt KS, Failla JM, Les C. Assessment of instabil- Edinb Med Surg J. 1814;10:181. Clin Orthop Relat Res. nale and utility. Radiol Clin North Am. 1997;35(3):491-506. ity factors in adult distal radius fractures. J Hand Surg [Am]. 2006;445:5-7. 16. Oliveira Filho OM, Belangero WD, Teles JBM. Fraturas do 2004;29(6):1128-38. 4. Smith RW. A treatise on fractures in the vicinity of joints and on rádio distal: avaliação das classificações. Rev Assoc Med Bras 27. Lill CA, Goldhahn J, Albrecht A, Eckstein F, Gatska C, certain forms of accidental and congenital dislocations. Dublin: (1992). 2004;50(1):55-61. Schneider E. Impact of bone density on distal radius fracture Hodges & Smith; 1847. 17. Fleiss JL, Slakter MJ, Fischman SL, Park MH, Chilton patterns and comparison between five different fracture clas - 5. Pouteau C. Contenant quelques ree fl xions sur quelques fractures NW. Inter-examiner reliability in caries trials. J Dent Res. sifications. J Orthop Trauma. 2003;17(4):271-8. de l’avant-bras, sur le luxations incomplètes du poignet et sur 1979;58(2):604-9. le diastasis. In: Pouteau C. Oeuvres Posthumes de M. Pouteau. 18. Scott WA. Reliability of content analysis: the case of nominal Paris: Ph.D. Pierres; 1783. p. 251-66. scale coding. Public Opinion Quarterly. 1955;19:321-5. 6. Gartland JJ Jr, Werley CW. Evaluation of healed Colles’ fractures. Available from: http://poq.oxfordjournals.org/cgi/content/sum- J Bone Joint Surg Am. 1951;33-A(4):895-907. mary/19/3/321. Accessed in 2008 (Apr 7). 7. Lindstrom A. Fractures of the distal end of radius. A clinical 19. Cohen J. A coefficient of agreement for nominal scales. Educ and statistical study of end results. Acta Orthop Scand Suppl. Psychol Meas. 1960;20(3):37-46. Available from: http://www. 1959;41:1-118. garfield.library.upenn.edu/classics1986/A1986AXF2600001. 8. Frykman G. Fracture of distal radius including sequelae--shoul- pdf. Accessed in 2008 (Apr 7). der-hand-n fi ger syndrome, disturbance in the distal radio-ulnar 20. Martin JS, Marsh JL, Bonar SK, DeCoster TA, Found EM, joint and impairment of nerve function. A clinical and experi- Brandser EA. Assessment of the AO/ASIF fracture classic fi ation mental study. Acta Orthop Scand. 1967;Suppl 108:3+. for the distal tibia. J Orthop Trauma. 1997;11(7):477-83. 9. Melone CP Jr. Distal radius fractures: patterns of articular 21. Belloti JC, Santos JB, Atallah AN, Albertoni WM, Faloppa F. fragmentation. Orthop Clin North Am. 1993;24(2):239-53. Fractures of the distal radius (Colles’ fracture). Sao Paulo Med 10. Andersen DJ, Blair WF, Steyers CM Jr, Adams BD, el-Khouri J. 2007;125(3):132-8. GY, Brandser EA. Classification of distal radius fractures: an 22. Illarramendi A, González Della Valle A, Segal E, De Carli P, analysis of interobserver reliability and intraobserver reproduc- Maignon G, Gallucci G. Evaluation of simplified Frykman and ibility. J Hand Surg [Am]. 1996;21(4):574-82. AO classifications of fractures of the distal radius. Assessment 11. Burstein AH. Fracture classification systems: do they work and of interobserver and intraobserver agreement. Int Orthop. Sources of funding: None are they useful? J Bone Joint Surg Am. 1993;75(12):1743-4. 1998;22(2):111-5. Conflict of interest: None 12. Kreder HJ, Hanel DP, McKee M, Jupiter J, McGillivary G, 23. Handoll HH, Madhok R. Closed reduction methods for treating Date of first submission: October 15, 2007 Swiontkowski MF. Consistency of AO fracture classic fi ation for distal radial fractures in adults. Cochrane Database Syst Rev. Last received: April 17, 2008 the distal radius. J Bone Joint Surg Br. 1996;78(5):726-31. 2003;(1):CD003763. Accepted: April 18, 2008. Sao Paulo Med J. 2008;126(3):180-5. 185 AUTHOR INFORMATION RESUMO João Carlos Belloti, MD, MSc, PhD. Attending physician in As classificações das fraturas do rádio distal são reprodutíveis? Concordância intra e in - the Traumatology Sector, Department of Orthopedics and terobservadores Traumatology, Universidade Federal de São Paulo — Escola CONTEXTO E OBJETIVO: Para que as classificações das fraturas possam ser úteis, elas devem prover o Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil. prognóstico, apresentar concordância interobservador e reprodutibilidade intraobservador. O objetivo foi Marcel Jun Sugawara Tamaoki. Third-year resident in the avaliar a concordância intra e interobservadores das classificações das fraturas do rádio distal. Department of Orthopedics and Traumatology, Universi- dade Federal de São Paulo — Escola Paulista de Medicina TIPO DE ESTUDO E LOCAL: Estudo de validação (concordância intra e interobservadores), desenvolvido no (Unifesp-EPM), São Paulo, Brazil. Departamento de Ortopedia e Traumatologia da Universidade Federal de São Paulo — Escola Paulista de Carlos Eduardo da Silveira Franciozi. Third-year resident Medicina (Unifesp-EPM), São Paulo, Brasil. in the Department of Orthopedics and Traumatology, MÉTODO: Foram avaliadas 90 fraturas do rádio distal com desvio por meio de radiografias por cinco Universidade Federal de São Paulo — Escola Paulista de observadores (um médico residente de Ortopedia do terceiro ano, um graduando do sexto ano de medicina, Medicina (Unifesp-EPM), São Paulo, Brazil. um médico radiologista, um ortopedista especializado em trauma e um ortopedista especializado em João Baptista Gomes dos Santos, MD, PhD. Adjunct profes- cirurgia da mão) em três momentos diferentes, empregando as classic fi ações Universal (Cooney), AO/ASIF sor and head of Hand Surgery Clinic, Department of (Osteosynthesfragen/Association for the Study of Internal Fixation), Frykman e Fernández. Aplicou-se o Orthopedics and Traumatology, Universidade Federal de coeficiente de concordância kappa ( κ) para avaliação das classificações. São Paulo — Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil. RESULTADOS: O maior κ intraobservador médio, se considerarmos os três momentos, foi da classificação Universal (κ = 0,61), seguida da Fernández (κ = 0,59), Frykman (κ = 0,55) e AO/ASIF (κ = 0,49). Daniel Balbachevsky, MD. Attending physician in the Trauma- A concordância interobservador foi insatisfatória em todas as classic fi ações. A classic fi ação de Fernández tology Sector, Department of Orthopedics and Traumatol- ogy, Universidade Federal de São Paulo — Escola Paulista mostrou a melhor concordância (κ = 0,44) e a pior foi a de Frykman (κ = 0,26). de Medicina (Unifesp-EPM), São Paulo, Brazil. CONCLUSÃO: Os baixos níveis de concordância observados neste estudo sugerem que atualmente ainda Eduardo Chap Chap. Sixth-year undergraduate, Universidade não há um método de classificação plenamente reprodutível. Federal de São Paulo — Escola Paulista de Medicina PALAVRAS-CHAVE: Fratura de Colles. Fraturas do rádio. Classificação. Reprodutibilidade dos testes. (Unifesp-EPM), São Paulo, Brazil. Estudos de validação. Walter Manna Albertoni, MD, PhD. Titular professor of the Department of Orthopedics and Traumatology, Universi- dade Federal de São Paulo — Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil. Flávio Faloppa, MD, PhD. Titular professor and head of the Department of Orthopedics and Traumatology, Universi- dade Federal de São Paulo — Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil. Address for correspondence: João Carlos Belloti Rua Borges de Lagoa, 783 — 5 andar São Paulo (SP) — Brasil —CEP 04038-032 Tel./Fax (+55 11) 5571-6621 E-mail: [email protected] Copyright © 2008, Associação Paulista de Medicina Sao Paulo Med J. 2008;126(3):180-5. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Sao Paulo Medical Journal Unpaywall

Are distal radius fracture classifications reproducible? Intra and interobserver agreement

Download PDF
6 pages

Loading next page...
 
/lp/unpaywall/are-distal-radius-fracture-classifications-reproducible-intra-and-iNrV7HAw1g

References (33)

Publisher
Unpaywall
ISSN
1516-3180
DOI
10.1590/s1516-31802008000300008
Publisher site
See Article on Publisher Site

Abstract

João Carlos Belloti Are distal radius fracture Marcel Jun Sugawara Tamaoki Carlos Eduardo da classifi cations reproducible? Intra Silveira Franciozi João Baptista Gomes dos Santos and interobserver agreement Daniel Balbachevsky Department of Orthopedics and Traumatology, Universidade Federal de Eduardo Chap Chap São Paulo — Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil Walter Manna Albertoni Flávio Faloppa INTRODUCTION tem has been used to defi ne surgical fixation ABSTRACT Distal radius fractures have an approxi- methods, but its accuracy and reproducibility CONTEXT AND OBJECTIVE: Various classi fica- mate incidence of 1:10,000 people and rep- for identifying the four fragments on con- tion systems have been proposed for fractures resent 16% of skeletal and 74% of forearm ventional x-rays have not been validated yet of the distal radius, but the reliability of these fractures. They are more prevalent among by clinical trials, and the system still presents classifications is seldom addressed. For a fracture classification to be useful, it must females and present a progressive increase in disagreements. provide prognostic signi ficance, interobserver complications with age, as osteopenia and The Arbeitsgemeinschaft für Osteo- reliability and intraobserver reproducibility. The osteoporosis become more prevalent. The synthesefragen/Association for the Study of aim here was to evaluate the intraobserver and interobserver agreement of distal radius fracture most common trauma mechanism is falling Internal Fixation (AO/ASIF) rating system classifi cations. 3 over onto the hand. The characteristics of was created in 1986 and reviewed in 1990. It DESIGN AND SETTING: This was a validation such fractures (trace location, possible joint considers bone injury severity and is a basis study on interobserver and intraobserver reli- involvement, comminution and degree of soft- for treatment and results evaluation. There are ability. It was developed in the Department of part lesion) are directly related to the force of three basic lesion types in this system: extra-ar- Orthopedics and Traumatology, Universidade Federal de São Paulo — Escola Paulista the trauma, wrist angle at the moment of the ticular, partial articular and complete articular. de Medicina. 2 trauma and bone health. The three groups are organized into increasing METHOD: X-rays from 98 cases of displaced dis- Systems have been developed to help order of severity of morphological complexity, tal radius fracture were evaluated by v fi e observ- surgeons in classifying fractures into differ- treatment diffi culty and prognostics. It is one ers: one third-year orthopedic resident (R3), one ent and clinically useful groups for treatment of the most complete ratings available, but sixth-year undergraduate medical student (UG6), one radiologist physician (XRP), one orthopedic de n fi ition. Colles, Smith, Pouteau, and others its intra and interobserver reproducibility has trauma specialist (OT) and one orthopedic hand described fracture morphology with a view been a problem when evaluating groups and surgery specialist (OHS). The radiographs were 3-5 11,12 to treatment. With the advent of radiology, subgroups. classi fied on three different occasions (times it became possible to describe injuries more The Universal rating system described T1, T2 and T3) using the Universal (Cooney), Arbeitsgemeinschaft für Osteosynthesefragen/ precisely, including both the degree of dis- by Cooney is characterized by simplicity, Association for the Study of Internal Fixation placement and the presence of joint injuries. classifying fractures as intra or extra-articular, (AO/ASIF), Frykman and Fernández classifi ca- In 1951, Garland and Werley created a clas- displacement present or absent, and according tions. The kappa coeffi cient ( κ) was applied to assess the degree of agreement. sifi cation based on the presence or absence of to the degree of stability and possibilities of joint involvement, metaphyseal comminution reduction. It thus acts as a guide for treatment RESULTS: Among the three occasions, the highest mean intraobserver k was observed in the Univer- and/or angular deformity. In 1959, Lindstrom patterns. sal classifi cation (0.61), followed by Fernández expanded these criteria to six groups, describ- The rating system proposed by Fernández (0.59), Frykman (0.55) and AO/ASIF (0.49). ing the fragment displacement in further is based on the trauma mechanism. This rat- The interobserver agreement was unsatisfactory in all class ic fi ations. The Fernández class ic fi ation detail, along with joint involvement. ing was created to be practical, predict stabil- showed the best agreement (0.44) and the worst In 1967, Frykman established a rating ity, check on associated fractures of the ulna was the Frykman classifi cation (0.26). system that considered the radiocarpal and/or styloid process, identify equivalent lesions in CONCLUSION: The low agreement levels ob- distal radius-ulna joints, and also the presence children and make general recommendations served in this study suggest that there is still no or absence of the ulnar styloid. Even thus, this for treatment. classifi cation method with high reproducibility. was a limited rating system: it did not consider To be considered good, a rating system KEY WORDS: Colles’ fracture. Radius fractures. factors like the extent of fragment displace- must be valid, reliable and reproducible. Classifi cation. Reproducibility of results. Valida- tion studies. ment, presence or absence of comminution Furthermore, an ideal rating system should and instability factors. standardize a trustworthy communication lan- In 1984, Melone published a rating sys- guage that provides guidelines for treatment, tem for distal radius joint fractures based on indicates the possibilities of complications, four parts: radius styloid, radius shaft, dorsal evaluates fracture stability and enables frac- fragment and palmar radius. This rating sys- ture prognosis. This ideal system should also Sao Paulo Med J. 2008;126(3):180-5. ORIGINAL ARTICLE 181 provide a mechanism that allows comparison an illustrated brochure showing descriptions of studies to evaluate intraobserver and interob- of the results obtained with treatments un- degrees and types of injury. server reliability and reproducibility. The dertaken on similar fractures in other centers, At the first evaluation (time T1), all the kappa agreement coefc fi ient provides a parallel reported at different times in the literature. x-rays were assessed in numerical sequence. rating of the agreement among the observers Variation in evaluators’ expertise may Three weeks later, at the second evaluation that is randomly correct. Kappa values range have influenced evaluations carried out on (time T2), the initial x-ray order was ran- from -1 to +1; values between -1 and 0 indicate intraobserver and interobserver agreement. domly changed to generate a new sequence. that the observed agreement was lower than Studies have shown that less experienced A further randomization of the sequence was what was randomly expected, 0 indicates the observers attain lower rates of intraobserver performed for the third evaluation (time T3), random agreement level, and +1 indicates total 12,16 17 agreement than do expert physicians. after six weeks. The x-rays were scanned and agreement. In general, kappa values of less However, in a comparison of one group in analyzed in computers. Data were collected than 0.5 are considered unsatisfactory; values which the observers were more experienced in on spreadsheets and the kappa (κ) coefficient between 0.5 and 0.75 are considered satisfac- rating assessments with another group whose was used to assess agreements. tory and appropriate, and values above 0.75 expertise was lower, no signic fi ant difference in k was applied using the method proposed are considered excellent. 12 17 interobserver agreement was found. It would by Fleiss et al. , and the random expected This project was approved by the Research also be expected that, as observers study and agreement calculation described by Scott Ethics Committee of Unifesp-EPM, under become accustomed to using a given rating and Cohen was also used. The latter two No. 1076-06, on August 4, 2006. system, the agreement between them, and methods enable calculation of agreements within their own observations, would increase. for multiple (more than two) observers with RESULTS Yet, it was observed that repeated application, regard to evaluations of nominal variances. Out of the initial 98 fractures, eight were i.e. at different moments in time, of the same They have therefore frequently been used in excluded: four presented poor quality x-rays rating system, had no impact on intraobserver and interobserver reproducibility. Table 1. Intraobserver kappa values between the three times (T1, T2 and T3) Considering the high prevalence of these Classification kinds of fracture and the need to properly Observer and reproducibly classify them, we developed Universal AO/ASIF Frykman Fernández the present study. Its aim was to evaluate the OHS 0.6568 0.6362 0.6375 0.7115 reproducibility of the four most widely used OT 0.6452 0.634 0.6632 0.5274 rating systems in our field. XRP 0.3513 0.3111 0.3099 0.2882 R3 0.7589 0.4835 0.549 0.7412 UG6 0.6406 0.3751 0.5829 0.6812 OBJECTIVE Mean kappa 0.61056 0.48798 0.5485 0.5899 This objective of this study was to evaluate R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand the intraobserver and interobserver agreement surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association of the Universal, AO/ASIF, Frykman and for the Study of Internal Fixation. Fernández rating systems for fractures with regard to displacement of the radius distal Table 2. Intraobserver kappa values between times T1 and T2 extremity. Classification Observer Universal AO/ASIF Frykman Fernández MATERIAL AND METHODS OHS 0.6914 0.6284 0.6121 0.738 This was a ratings reproducibility study OT 0.5784 0.623 0.6075 0.4933 using the kappa index. Ninety-eight displaced XRP 0.2478 0.2144 0.2971 0.2638 distal radius fractures in 96 patients over the R3 0.7089 0.4341 0.6076 0.7699 age of 40 years who had been treated at the UG6 0.5821 0.385 0.5036 0.626 Hand Institute of Universidade Federal de São Mean kappa 0.56172 0.45698 0.52558 0.5782 Paulo — Escola Paulista de Medicina (Unifesp- R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand EPM) were retrospectively evaluated from the surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association for the Study of Internal Fixation. radiographic archives. Five observers were involved: one third-year orthopedic resident Table 3. Intraobserver kappa values between times T2 and T3 (R3), one sixth-year undergraduate medical Classification student (UG6), one radiologist physician Observer Universal AO/ASIF Frykman Fernández (XRP), one orthopedic trauma specialist (OT) OHS 0.6597 0.7076 0.6896 0.7721 and one orthopedic hand surgery specialist OT 0.6905 0.6618 0.7112 0.4433 (OHS). These observers used four classic fi ation XRP 0.4523 0.4381 0.3116 0.3775 systems to label each case using simple x-rays R3 0.8909 0.6927 0.5397 0.8023 in two incidence planes (posteroanterior and UG6 0.6615 0.5504 0.7024 0.8117 lateral to the wrist). The classic fi ations used Mean kappa 0.67098 0.61012 0.5909 0.64138 were the Universal (Cooney), AO/ASIF, Fryk- R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand man and Fernández, and these were previously surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association presented and explained to the evaluators, with for the Study of Internal Fixation. Sao Paulo Med J. 2008;126(3):180-5. 182 and another four presented x-rays produced Table 4. Interobserver kappa values at each time (T1, T2, T3) with the forearm immobilized in plaster. Thus, Classification Time the sample size was reduced to 90 fractures. Universal AO/ASIF Frykman Fernández The highest mean intraobserver κ, tak- T1 0.3963 0.2702 0.2427 0.34 ing all three observation times, was from the T2 0.4004 0.2988 0.2589 0.4087 Universal classification ( κ = 0.61), followed T3 0.4118 0.3117 0.2608 0.4344 by Fernández (κ = 0.59), Frykman (κ = 0.55) AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/ Association for the Study of Internal Fixation. and AO/ASIF (κ = 0.49) (Table 1). Evaluation of the intraobserver k between Table 5. Analysis of interobserver kappa values for pairs at time 1 (T1) the times T1 and T2 showed that the highest mean was from the Fernández classifica- Classification Pairs tion (κ = 0.58), followed by the Universal Universal AO/ASIF Frykman Fernández (κ = 0.56), and the lowest mean was from the OHS versus OT 0.5429 0.4911 0.4007 0.3654 AO/ASIF (κ = 0.46) (Table 2). OHS versus XRP 0.2097 0.1635 0.1733 0.3036 Between times T2 and T3, the mean intrao- OHS versus R3 0.5465 0.4532 0.2999 0.6849 bserver κ was greater, ranging from κ = 0.59 for OHS versus UG6 0.3683 0.1644 0.0954 0.4304 the Frykman classic fi ation to κ = 0.67 for the OT versus XRP 0.3463 0.2158 0.2468 0.2186 Universal classic fi ation (Table 3). OT versus R3 0.594 0.4272 0.4173 0.3505 All the mean interobserver k values for the OT versus UG6 0.4453 0.2431 0.2225 0.3188 classic fi ations were higher at time 3, such that XRP versus R3 0.2689 0.1497 0.2513 0.1548 they were (in decreasing order) Fernández XRP versus UG6 0.1958 0.2158 0.1686 0.0584 κ = 0.44, Universal κ = 0.41, AO/ASIF R3 versus UG6 0.5726 0.2872 0.24 0.6029 κ = 0.31 and Frykman κ = 0.26 (Table 4). R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association Evaluation of the interobserver k by com- for the Study of Internal Fixation. paring pairs of observers at time 1 showed that the highest agreement was between the ob- servers R3 and UG6 (0.60) in the Fernández Table 6. Analysis of interobserver kappa values for pairs at time 2 (T2) classification. On the other hand, the lowest Classification Pairs agreement was between XRP and UG6 (0.06), Universal AO/ASIF Frykman Fernández in the same classification system (Table 5). OHS versus OT 0.5751 0.4889 0.399 0.4102 At time 2, the highest agreement was OHS versus XRP 0.2338 0.2191 0.1373 0.3286 obtained between OHS and R3 (0.77) in the OHS versus R3 0.4934 0.4602 0.308 0.7691 Fernández classification, while the lowest was OHS versus UG6 0.556 0.3695 0.1495 0.4989 between XRP and R3 (0.12) in the AO/ASIF OT versus XRP 0.2769 0.1808 0.2444 0.2737 system (Table 6). OT versus R3 0.5285 0.5285 0.4117 0.4193 At time 3, the highest κ was between OT versus UG6 0.5428 0.3342 0.2949 0.4153 OHS and R3 (0.6) in the Fernández classic fi a - XRP versus R3 0.2112 0.1174 0.2225 0.3397 tion, while the lowest was between XRP and XRP versus UG6 0.2187 0.1272 0.2748 0.205 UG6 (0.1) in the AO/ASIF system (Table 7). R3 versus UG6 0.4368 0.3646 0.2422 0.4926 R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association DISCUSSION for the Study of Internal Fixation. The four classification systems evaluated in the present study were chosen because they are the ones that are most widely studied Table 7. Analysis of interobserver kappa values for pairs at time 3 (T3) and used in our field to classify distal radius Classification fractures. Pairs Universal AO/ASIF Frykman Fernández In the Frykman classification, the general OHS versus OT 0.5441 0.5382 0.4512 0.4781 mean kappa value for intraobserver agreement OHS versus XRP 0.2692 0.2175 0.297 0.3492 was satisfactory (0.55), although the radiolo- OHS versus R3 0.532 0.4985 0.3277 0.6471 gist physician (XRP) presented an unsatisfac- OHS versus UG6 0.5011 0.4618 0.1265 0.5337 tory value (0.31) that was far from the other OT versus XRP 0.3481 0.222 0.4361 0.293 four observers. After recalculating the intrao- OT versus R3 0.5491 0.4306 0.2638 0.4564 bserver kappa without the medical student OT versus UG6 0.5142 0.3328 0.2312 0.4769 (UG6) and the orthopedic resident (R3), who XRP versus R3 0.2471 0.1551 0.1759 0.3381 were less experienced evaluators, the kappa XRP versus UG6 0.2244 0.1005 0.2337 0.3505 value decreased to 0.54. This showed that the R3 versus UG6 0.5118 0.4143 0.2105 0.534 professional’s expertise level had no signic fi ant R3 = third-year orthopedic resident; UG6 = sixth-year medical student; XRP = radiologist physician; OHS = orthopedic hand impact on the intraobserver agreement. Vari- surgery specialist; OT = orthopedic trauma specialist; AO/ASIF = Arbeitsgemeinschaft für Osteosynthesefragen/Association for the Study of Internal Fixation. ance analysis between the observation times Sao Paulo Med J. 2008;126(3):180-5. 183 showed that UG6 presented relatively high exclusively based on radiographic criteria: groups and subgroups (nine types) and the variance (0.51 to 0.70) that was 39% greater involvement or non-involvement of the radio- mean intraobserver value was unsatisfactory than among the other observers. This probably carpal joint, presence or absence of dislocation, (0.49). There was a significant difference be - resulted from the learning process required fracture reducibility and stability. The biggest tween the values for the more experienced to become accustomed to this classification difc fi ulty found in applying this classic fi ation observers (OHS κ = 0.64 and OT κ = 0.64) system. This assumption is reinforced by the was in assessing the degree of instability of and those for the less experienced ones observation that there was relatively lower the fracture. The literature did not demon- (R3 κ = 0.4835 and UG6 κ = 0.3751). This variance among the more experienced observ- strate any consensus regarding the best way to suggests that the expertise level had an influ - ers at the same times. This suggests that the predict specic fi instability criteria on the initial ence. Only the XRP observer presented a value observer’s conditioning and knowledge, spe- x-ray, and there are several studies with discor- at odds with what was expected (κ = 0.34) 23-26 cific to the Frykman system, had a significant dant results concerning such criteria. for the more experienced evaluators. When impact on the reproducibility obtained. It In the Universal classic fi ation, the average the intraobserver kappa was recalculated is important to make it clear that the profes- intraobserver index was satisfactory (0.61056). without the less experienced observers (R3 sional expertise level was different from the When the intraobserver kappa was recalcu- and UG6), there was an increase in κ to level of experience relating to the classic fi ation. lated without the less experienced observers 0.53, which reinforces the hypothesis that the The k-value for the intraobserver agreement (R3 and UG6), there was a reduction in kappa professional expertise level had a significant in the Frykman classification evaluated by to 0.5511. This demonstrated that the degree impact on the intraobserver agreement. The Andersen et al. in 1996 was 0.48. In 1998, of expertise did not influence the results, analysis of variation between the times T1 and Illarramendi et al. in 1998 found κ = 0.61, since an increased kappa would be expected T3 demonstrated that the UG6 observer (less and in 2003, Oliveira Filho et al. found κ = when excluding the less experienced evalu- experienced) had an increase in agreement 0.55. These coefficients reported in literature ators. On the other hand, analysis of how of 43%, XRP increased by 59.6%) and R3 were in line with the results from the present the agreement evolved from time T1 to M3 increased by 104.3%. This demonstrated that study (κ = 0.55). showed that UG6 presented an increase of conditioning to the classic fi ation had a signifi - With regard to the observer’s experience, 13%, which was lower than what was observed cant impact on intraobserver reproducibility, the study published by Oliveira Filho et al. for R3 (increase of 25.7%) and XRP (increase particularly among the individuals with less had similar conclusions to ours, thus demon- of 82.5%). The intraobserver agreement for expertise in using it. strating the positive effect of expertise on the the Universal classification was also satisfac - In the literature, we saw that κ ranged 16 10,12,16,22 agreement rate. tory in another study, which found κ = 0.54. from 0.37 to 0.60 in different studies, The interobserver agreement rate for However, that study demonstrated that the thus suggesting that the intraobserver repro- the Frykman classification was unsatisfac - observer’s experience was a factor that modi- ducibility of AO/ASIF should be close to 0.5. tory, albeit with a progressive increase from fied the agreement. In the present study, the mean κ was 0.48, T1 (0.2427) to T3 (0.2608). However, this The interobserver agreement for the with a range from 0.31 to 0.63. It was only in increase was relatively lower than what was ob- Universal classification was unsatisfactory, the study by Andersen et al. , that the profes- served from the other classification systems. but presented a progressive increase from T1 sional expertise level had no signic fi ant impact The analysis showed that, in comparison (0.3963) to T3 (0.4118). The XRP evaluator on intraobserver reproducibility. This could with the most experienced observers (OHS presented a lower agreement rate than what be explained by the presence of radiologist and OT), the XRP observer presented lower would be expected. However, we found that and orthopedist observers who were working agreement rates. This suggests that although this observer’s agreement rate increased in in similar fields and frequently applied the the XRP observer had professional experience relation to the OT and OHS evaluators. This AO/ASIF classification, in the same way as in 12,16,22 with radiographic evaluations, this observer suggests that conditioning to the Universal our study. In the other studies, expertise was not using these classification methods classification (i.e. the evolution from T1 to played a modifying role in relation to intrao- routinely. This demonstrates that professional T3) was a factor that acted positively on the bserver reproducibility. experience of radiographic evaluation is not, reproducibility. The interobserver agreement for the AO/ on its own, a determining factor for a higher The same difficulty described above for ASIF classic fi ation was also unsatisfactory, but agreement rate using these classification sys - the Universal classification was found in the presented progressive increase from T1 (0.27) tems. We also saw this when analyzing the AO/ASIF application, even considering that to T3 (0.31). The XRP evaluator presented other classification methods. in the latter, evaluation of the comminution increased agreement with OT and OHS, by In our study, the interobserver repro- location is extremely important for defining 0.8% and 3.0% respectively, and the UG6 ducibility of the Frykman classification was the groups. It is possible that this difficulty observer presented increased agreement with unsatisfactory (0.26 at T3), and the k value is the limiting factor for unsatisfactory agree- OT and OHS, by 36.9% and 180.9%. This was relatively lower than found in the studies ment rates that have been found in previous suggests that conditioning was a factor acting 10 22 10,16 by Andersen et al. and Illarramendi et al. , studies. positively towards interobserver reproduc- which presented k of 0.35 and 0.43 respec- Assuming that the presence and location ibility, particularly for the less experienced tively. Our unsatisfactory result from the of comminution are determining variables individuals. 10,12,16,22 Frykman classification probably results from with regard to fracture stability, thereby den fi i - In the literature, we saw that the the low agreement rate between XRP and the tively guiding the therapy, detailed investiga- interobserver κ ranged from 0.3 to 0.5. This other evaluators. tion of the reproducibility of these variables suggests that the AO/ASIF kappa is close to The Universal classic fi ation evaluates the on the radiograph becomes necessary. 0.4, implying unsatisfactory reproducibility. following variables of distal radius fractures, In the AO/ASIF classification, we used It also suggests that the professional expertise Sao Paulo Med J. 2008;126(3):180-5. 184 level had no impact on interobserver reproduc- there was a progressive increase in agreement still no classification method that is fully ibility in this classification, in the same way as between T1 (κ = 0.34) and T3 (κ = 0.44). reproducible. seen in our study. This was relatively greater than what was seen The best interobserver reproducibility rate In the Fernández classic fi ation, the mean in the other classifications. This suggests that was observed in the Fernández classification intraobserver κ was satisfactory (κ = 0.59). the conditioning in this classification had a (0.43) and the worst was in the Frykman When the intraobserver kappa was recalculated greater impact on reproducibility than did the classification (0.26). The intraobserver re - without the less experienced observers, there conditioning in other classifications. producibility was satisfactory in the Universal was a reduction in κ (0.51), thus demonstrat- It is important to mention that the present (0.61), Fernández (0.59) and Frykman (0.55) ing that professional expertise did not have any study was limited to evaluating the agree- classic fi ations, and it was unsatisfactory in the inu fl ence on intraobserver agreement. Likewise, ment between the observers’ opinions. The AO/ASIF classification (0.49). professional experience was not seen to have any study was unable to measure the accuracy of Implications for positive inu fl ence on interobserver agreement each observer’s opinion. To clarify the accuracy further research between the times T1 and T3. issue, studies in which clinical-radiographic Conditioning (through evolution from diagnoses made by each observer were com- There is a need to perform new studies T1 to T3) was seen to be a factor acting pared with an examination result or a standard aimed at clarifying which classification vari - positively on intraobserver reproducibility, procedure, i.e. one with high sensitivity and ables present the highest disagreement rates for the Fernández classification. There are no specificity, would be needed in order to prove between observers, with consequent limits equivalent studies on this classification in the the proposed diagnosis. to reproducibility. In the continuing search literature, which makes it impossible to make for an ideal classification, prospective studies comparisons with the present results. CONCLUSIONS to describe which variables can predict the Regarding the interobserver agreement The agreement rates observed in the instability factors in such fractures through for this classification, it could be seen that present study show that currently there is radiographic examination are necessary. REFERENCES 1. Pires PR. Fraturas do rádio distal. In: Sociedade Brasileira de 13. Cooney WP. Fractures of the distal radius. A modern 24. Lafontaine M, Hardy D, Delince P. Stability assessment of distal Ortopedia e Traumatologia. Traumatologia ortopédica. Rio de treatment-based classification. Orthop Clin North Am. radius fractures. Injury. 1989;20(4):208-10. Janeiro: Revinter; 2004. 1993;24(2):211-6. 25. Abbaszadegan H, Jonsson U, von Sivers K. Prediction 2. Falch JA. Epidemiology of fractures of the distal forearm in 14. Fernández DL. Fractures of the distal radius: operative treatment. of instability of Colles’ fractures. Acta Orthop Scand. Oslo, Norway. Acta Orthop Scand. 1983;54(2):291-5. Instr Course Lect. 1993;42:73-88. 1989;60(6):646-50. 3. Colles A. On the fracture of the carpal extremity of the radius. 15. Martin JS, Marsh JL. Current classification of fractures. Ratio - 26. Nesbitt KS, Failla JM, Les C. Assessment of instabil- Edinb Med Surg J. 1814;10:181. Clin Orthop Relat Res. nale and utility. Radiol Clin North Am. 1997;35(3):491-506. ity factors in adult distal radius fractures. J Hand Surg [Am]. 2006;445:5-7. 16. Oliveira Filho OM, Belangero WD, Teles JBM. Fraturas do 2004;29(6):1128-38. 4. Smith RW. A treatise on fractures in the vicinity of joints and on rádio distal: avaliação das classificações. Rev Assoc Med Bras 27. Lill CA, Goldhahn J, Albrecht A, Eckstein F, Gatska C, certain forms of accidental and congenital dislocations. Dublin: (1992). 2004;50(1):55-61. Schneider E. Impact of bone density on distal radius fracture Hodges & Smith; 1847. 17. Fleiss JL, Slakter MJ, Fischman SL, Park MH, Chilton patterns and comparison between five different fracture clas - 5. Pouteau C. Contenant quelques ree fl xions sur quelques fractures NW. Inter-examiner reliability in caries trials. J Dent Res. sifications. J Orthop Trauma. 2003;17(4):271-8. de l’avant-bras, sur le luxations incomplètes du poignet et sur 1979;58(2):604-9. le diastasis. In: Pouteau C. Oeuvres Posthumes de M. Pouteau. 18. Scott WA. Reliability of content analysis: the case of nominal Paris: Ph.D. Pierres; 1783. p. 251-66. scale coding. Public Opinion Quarterly. 1955;19:321-5. 6. Gartland JJ Jr, Werley CW. Evaluation of healed Colles’ fractures. Available from: http://poq.oxfordjournals.org/cgi/content/sum- J Bone Joint Surg Am. 1951;33-A(4):895-907. mary/19/3/321. Accessed in 2008 (Apr 7). 7. Lindstrom A. Fractures of the distal end of radius. A clinical 19. Cohen J. A coefficient of agreement for nominal scales. Educ and statistical study of end results. Acta Orthop Scand Suppl. Psychol Meas. 1960;20(3):37-46. Available from: http://www. 1959;41:1-118. garfield.library.upenn.edu/classics1986/A1986AXF2600001. 8. Frykman G. Fracture of distal radius including sequelae--shoul- pdf. Accessed in 2008 (Apr 7). der-hand-n fi ger syndrome, disturbance in the distal radio-ulnar 20. Martin JS, Marsh JL, Bonar SK, DeCoster TA, Found EM, joint and impairment of nerve function. A clinical and experi- Brandser EA. Assessment of the AO/ASIF fracture classic fi ation mental study. Acta Orthop Scand. 1967;Suppl 108:3+. for the distal tibia. J Orthop Trauma. 1997;11(7):477-83. 9. Melone CP Jr. Distal radius fractures: patterns of articular 21. Belloti JC, Santos JB, Atallah AN, Albertoni WM, Faloppa F. fragmentation. Orthop Clin North Am. 1993;24(2):239-53. Fractures of the distal radius (Colles’ fracture). Sao Paulo Med 10. Andersen DJ, Blair WF, Steyers CM Jr, Adams BD, el-Khouri J. 2007;125(3):132-8. GY, Brandser EA. Classification of distal radius fractures: an 22. Illarramendi A, González Della Valle A, Segal E, De Carli P, analysis of interobserver reliability and intraobserver reproduc- Maignon G, Gallucci G. Evaluation of simplified Frykman and ibility. J Hand Surg [Am]. 1996;21(4):574-82. AO classifications of fractures of the distal radius. Assessment 11. Burstein AH. Fracture classification systems: do they work and of interobserver and intraobserver agreement. Int Orthop. Sources of funding: None are they useful? J Bone Joint Surg Am. 1993;75(12):1743-4. 1998;22(2):111-5. Conflict of interest: None 12. Kreder HJ, Hanel DP, McKee M, Jupiter J, McGillivary G, 23. Handoll HH, Madhok R. Closed reduction methods for treating Date of first submission: October 15, 2007 Swiontkowski MF. Consistency of AO fracture classic fi ation for distal radial fractures in adults. Cochrane Database Syst Rev. Last received: April 17, 2008 the distal radius. J Bone Joint Surg Br. 1996;78(5):726-31. 2003;(1):CD003763. Accepted: April 18, 2008. Sao Paulo Med J. 2008;126(3):180-5. 185 AUTHOR INFORMATION RESUMO João Carlos Belloti, MD, MSc, PhD. Attending physician in As classificações das fraturas do rádio distal são reprodutíveis? Concordância intra e in - the Traumatology Sector, Department of Orthopedics and terobservadores Traumatology, Universidade Federal de São Paulo — Escola CONTEXTO E OBJETIVO: Para que as classificações das fraturas possam ser úteis, elas devem prover o Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil. prognóstico, apresentar concordância interobservador e reprodutibilidade intraobservador. O objetivo foi Marcel Jun Sugawara Tamaoki. Third-year resident in the avaliar a concordância intra e interobservadores das classificações das fraturas do rádio distal. Department of Orthopedics and Traumatology, Universi- dade Federal de São Paulo — Escola Paulista de Medicina TIPO DE ESTUDO E LOCAL: Estudo de validação (concordância intra e interobservadores), desenvolvido no (Unifesp-EPM), São Paulo, Brazil. Departamento de Ortopedia e Traumatologia da Universidade Federal de São Paulo — Escola Paulista de Carlos Eduardo da Silveira Franciozi. Third-year resident Medicina (Unifesp-EPM), São Paulo, Brasil. in the Department of Orthopedics and Traumatology, MÉTODO: Foram avaliadas 90 fraturas do rádio distal com desvio por meio de radiografias por cinco Universidade Federal de São Paulo — Escola Paulista de observadores (um médico residente de Ortopedia do terceiro ano, um graduando do sexto ano de medicina, Medicina (Unifesp-EPM), São Paulo, Brazil. um médico radiologista, um ortopedista especializado em trauma e um ortopedista especializado em João Baptista Gomes dos Santos, MD, PhD. Adjunct profes- cirurgia da mão) em três momentos diferentes, empregando as classic fi ações Universal (Cooney), AO/ASIF sor and head of Hand Surgery Clinic, Department of (Osteosynthesfragen/Association for the Study of Internal Fixation), Frykman e Fernández. Aplicou-se o Orthopedics and Traumatology, Universidade Federal de coeficiente de concordância kappa ( κ) para avaliação das classificações. São Paulo — Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil. RESULTADOS: O maior κ intraobservador médio, se considerarmos os três momentos, foi da classificação Universal (κ = 0,61), seguida da Fernández (κ = 0,59), Frykman (κ = 0,55) e AO/ASIF (κ = 0,49). Daniel Balbachevsky, MD. Attending physician in the Trauma- A concordância interobservador foi insatisfatória em todas as classic fi ações. A classic fi ação de Fernández tology Sector, Department of Orthopedics and Traumatol- ogy, Universidade Federal de São Paulo — Escola Paulista mostrou a melhor concordância (κ = 0,44) e a pior foi a de Frykman (κ = 0,26). de Medicina (Unifesp-EPM), São Paulo, Brazil. CONCLUSÃO: Os baixos níveis de concordância observados neste estudo sugerem que atualmente ainda Eduardo Chap Chap. Sixth-year undergraduate, Universidade não há um método de classificação plenamente reprodutível. Federal de São Paulo — Escola Paulista de Medicina PALAVRAS-CHAVE: Fratura de Colles. Fraturas do rádio. Classificação. Reprodutibilidade dos testes. (Unifesp-EPM), São Paulo, Brazil. Estudos de validação. Walter Manna Albertoni, MD, PhD. Titular professor of the Department of Orthopedics and Traumatology, Universi- dade Federal de São Paulo — Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil. Flávio Faloppa, MD, PhD. Titular professor and head of the Department of Orthopedics and Traumatology, Universi- dade Federal de São Paulo — Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil. Address for correspondence: João Carlos Belloti Rua Borges de Lagoa, 783 — 5 andar São Paulo (SP) — Brasil —CEP 04038-032 Tel./Fax (+55 11) 5571-6621 E-mail: [email protected] Copyright © 2008, Associação Paulista de Medicina Sao Paulo Med J. 2008;126(3):180-5.

Journal

Sao Paulo Medical Journal – Unpaywall

Published: May 1, 2008

There are no references for this article.