Reliability measurement and ICD-10 validation of ICPC-2 for coding/classification of diagnoses/health problems in an African primary care setting

Reliability measurement and ICD-10 validation of ICPC-2 for coding/classification of... Abstract Background The routine application of a primary care classification system to patients’ medical records in general practice/primary care is rare in the African region. Reliable data are crucial to understanding the domain of primary care in Nigeria, and this may be actualized through the use of a locally validated primary care classification system such as the International Classification of Primary Care, 2nd edition (ICPC-2). Although a few studies from Europe and Australia have reported that ICPC is a reliable and feasible tool for classifying data in primary care, the reliability and validity of the revised version (ICPC-2) is yet to be objectively determined particularly in Africa. Objectives (i) To determine the convergent validity of ICPC-2 diagnoses codes when correlated with International Statistical Classification of Diseases (ICD)-10 codes, (ii) to determine the inter-coder reliability among local and foreign ICPC-2 experts and (iii) to ascertain the level of accuracy when ICPC-2 is engaged by coders without previous training. Methods Psychometric analysis was carried out on ICPC-2 and ICD-10 coded data that were generated from physicians’ diagnoses, which were randomly selected from general outpatients’ clinic attendance registers, using a systematic sampling technique. Participants comprised two groups of coders (ICPC-2 coders and ICD-10 coders) who coded independently a total of 220 diagnoses/health problems with ICPC-2 and/or ICD-10, respectively. Results Two hundred and twenty diagnoses/health problems were considered and were found to cut across all 17 chapters of the ICPC-2. The dataset revealed a strong positive correlation between selected ICPC-2 codes and ICD-10 codes (r ≈ 0.7) at a sensitivity of 86.8%. Mean percentage agreement among the ICPC-2 coders was 97.9% at the chapter level and 95.6% at the rubric level. Similarly, Cohen’s kappa coefficients were very good (κ > 0.81) and were higher at chapter level (0.94–0.97) than rubric level (0.90–0.93) between sets of pairs of ICPC-2 coders. An accuracy of 74.5% was achieved by ICD-10 coders who had no previous experience or prior training on ICPC-2 usage. Conclusion Findings support the utility of ICPC-2 as a valid and reliable coding tool that may be adopted for routine data collection in the African primary care context. The level of accuracy achieved without training lends credence to the proposition that it is a simple-to-use classification and may be a useful starting point in a setting devoid of any primary care classification system for morbidity and mortality registration at such a critical level of public health importance. Classification, ICPC-2, ICD-10, primary care, reliability, validity Introduction Primary care has been defined as the provision of integrated accessible healthcare services by primary care practitioners, including family physicians and GPs, who are accountable for addressing the majority of personal healthcare needs, developing a sustained partnership with patients and practicing in the context of family and community (1). It is enhanced by longitudinal medical records that are collected over time, which can be a unique source of health information (2). Routine collection of primary care data, aggregated into large databases, can be useful for auditing purposes, quality improvement research (especially chronic disease management) and as a cost-effective source of data for epidemiological studies (2). Although most people are treated at the primary care level, there is little information about primary care morbidity in the African context. Practice-based data can provide a detailed picture of morbidity, and the absence of such information impedes the development of primary care. Information on morbidity can be used to set health priorities and to improve the processes of management and care (3). Epidemiological data obtained at this level are an invaluable source of health information for planning and policy formulation. Poor management of data collection systems contributes significantly to inaccurate data. In Nigeria, for instance, poor data management has been identified as one of the numerous challenges facing successful implementation of primary health care (4). Although the need for good quality, comparable data is clear, the practice of coding such data at the primary care level, either primarily by clinicians during consultation or secondarily by non-clinicians after consultation, is non-existent in the Nigerian primary care context. It is therefore important to consider several issues in the planning and successful implementation of a primary care data system, for the purpose of continuous data collection. One of the most important issues is to select an appropriate tool that can be used to standardize the coding of data for the specific context or setting where it is intended to be used. The International Classification of Primary Care (ICPC) is an evidenced-based coding system that is widely used in more than 45 countries as the standard for data classification in primary care. It is endorsed by the World Health Organization (WHO) in the WHO Family of International Classifications (WHO-FIC) (5,6). The ICPC was first published in 1987, and it has been used in primary care for three decades as the main ordering principle of that domain. It is designed to be simple to use, potentially codes the whole encounter (reason for encounter, the process of care, diagnoses) and focuses on problems that are frequently encountered in primary care (7). One of the coding reference classifications endorsed by the WHO-FIC is the International Statistical Classification of Diseases (ICD) and related health problems. As opposed to the ICPC, which was specially developed for primary care purposes, it supports the classification of diseases at a higher level of specificity, beyond what is required for primary care. Since its inception in 1977, several editions have evolved and the 10th edition (ICD-10) is the latest available version. With more than 14000 codes, it is extensive and caters to the labelling of health problems that are rarely seen at the primary care level. A revised version of the ICPC, the second edition of ICPC (ICPC-2), was published in 1998. Compared with the ICD-10, it has fewer codes (~1400), which are less specific, and more appropriate to classify data in a primary care setting (8). Moreover, it has been mapped to the ICD-10, which allows for additional labelling of health problems not often encountered at primary care level, and can be used if ever there is a need to classify at a higher level of granularity (9). The ICPC-2 has 17 chapters representing different body systems/problem areas, and it defines an encounter as the professional interchange between a patient and a primary care provider. An encounter is made up of three elements: reasons for encounter (RfE)/complaints, diagnoses/problems and interventions/processes/procedures. Its rubrics were developed in a European, an Australian and an American context (10), hence the need for its re-validation within the local context of intended use. The ICPC-2 has a bi-axial structure: 17 chapters on one axis, each with an alpha code, and seven identical components with rubrics bearing a two-digit numeric code as the second axis. The chapters are constructed in terms of organ systems, psychological and social problems, and one general chapter. ICPC has a significant mnemonic quality that facilitates its day-to-day use in primary care and simplifies the centralized manual coding of data recorded elsewhere. Each rubric has one alphabet and two-digit code number, and a title of limited length (10). For the purpose of this study, rubrics in component 1 (symptoms/complaints) and component 7 (diseases) were applied. The -29 and -99 codes are consistent across all chapters within the structure of ICPC-2 as residual rubrics, and they are designated for coding diagnoses for which there are no specific ICPC-2 codes. The original version was designed and field-tested for face validity and consensual validity by a group of experts (11). As the face and consensual validity are both based on the subjective opinions of experts, further objective measures of validation should be carried out in settings in which it is intended for use (12). Convergent validity is established when there is a strong relationship between the tool/measure under review and a previously validated tool that measures the same construct (13). It determines the extent to which a new tool agrees or correlates with an existing tool. It entails comparing with a ‘gold standard’ set by another instrument. The gold standard against which results of ICPC-2 coded data were validated in this study setting is ICD-10, which is the national coding standard at the secondary and tertiary healthcare levels. The mapping of ICPC-2 to ICD-10 facilitated this objective. Limited literature is available on primary healthcare data systems in Africa. Invariably, very little research on the use of a primary care classification system for primary care data management has been conducted (14–16). The routine application of ICPC-2 to patients’ medical records in primary care barely exists in the developing world (14–17). For instance, currently there is no classification system dedicated to routine data collection at a primary care level in Nigeria. In light of the importance of primary health care, its pivotal role in the health systems of most nations, particularly Africa, and the aforementioned relevance of primary care data, it is clear that there is a need to evaluate the reliability and validity of using ICPC-2 for coding/classifying diagnoses/health problems in this African primary care setting. Furthermore, applying an international classification system to local data affords the opportunity of generating data for international comparability which is important to global health (14). Aim The overall aim was to evaluate the reliability and validity of using ICPC-2 for coding and classifying diagnoses/health problems in an African primary care setting Objectives The objectives were to (i) determine the convergent validity of ICPC-2 diagnoses codes when correlated with ICD-10; (ii) determine the inter-coder reliability among local and foreign ICPC-2 experts and (iii) ascertain the level of accuracy when ICPC-2 is engaged by coders without previous training on ICPC-2 usage. Methods Study design The study design was a psychometric analysis of coded primary care health problems/diagnoses data. Study setting The study setting was the primary care section of a public secondary healthcare facility in Lagos State, Nigeria. The general outpatients’ clinic offers primary care services that are provided by primary care doctors to undifferentiated, self-referred adult patients. These services include first contact care, health promotion, disease prevention, care of uncomplicated chronic diseases and referrals to specialists and other healthcare providers. Selection of participants The participants were two groups of coders who were co-opted on the basis of experience/expertise with ICD-10 and ICPC-2 as described below. Each group was made up of four ICD-10 and four ICPC-2 coders. Selection criteria Four experienced local ICD-10 clinical coders were selected based on the following inclusion criteria: More than 5 years of experience as a clinical coder; Qualified as a clinical coder with an ordinary national diploma degree in health information management; Voluntariness to participate and Proficiency in English. Four ICPC-2 experts (a local and three foreign experts) were recruited to participate in the study based on the following criteria: Routine usage of ICPC-2 in their practice; Voluntariness to participate; Membership of the World Congress of Family Doctors’ (Wonca) International Classification Committee, which is the committee responsible for the ICPC-2 and Proficiency in English. Sample size and sampling technique Given a standard normal deviate of 1.96 at 5% type 1 error, proportion of agreement, Cohen’s kappa of 0.84 from a previous study conducted in France (18), a minimum sample size of 206 encounters (clinical consultations) was required to draw valid conclusions about the reliability and validity of ICPC-2 in the study setting (19). The researcher collected data in August 2016 from the 10 clinic attendance registers using a systematic sampling technique from entries for the previous 6 months. These registers were distributed across all the consulting rooms and they were completed simultaneously by primary care doctors, who used them routinely to record patients’ hospital number, age, gender and diagnoses in English. The number of patients in each register varied from 79 to 1620, and the contribution of each register to the total sample was stratified according to the number of entries. Patients from each register were then systematically sampled with an interval correlated with the number of patients required from the total (intervals ranged from every 5th to every 63rd entry). Data collection Two tools were used: the tabular list of ICD-10 and the two-page summary of ICPC-2 codes. The tabular list is volume 1 of ICD-10 and it contains about 14000 terms/codes on more than a thousand pages. The two-page summary of ICPC-2 is a colour-coded document that captures the structure of ICPC-2 with the chapters and individual rubrics (~1400 codes and their titles). Diagnoses were recorded verbatim from the clinic registers as described earlier and captured in an Excel spreadsheet. The list was distributed by hand to the local ICD-10 coders and sent via e-mail to local and foreign ICPC-2 coders for coding. To determine the convergent validity of ICPC-2, the ICPC-2 experts and the trained and experienced, routine ICD-10 coders independently coded the same sample of patients’ diagnoses with ICPC-2 and ICD-10, respectively. This was to determine the correlation between the assigned ICPC-2 diagnoses codes and assigned ICD-10 codes for the same diagnoses. The mapping of ICPC-2 to ICD-10 was used to facilitate this correlation (9). Due to the high granularity of ICD-10, several of its codes may be mapped to a single ICPC-2 code. The codes assigned to each diagnostic term from either of the two classifications were checked to determine if they were correlated on the mapping. In addition, ICD-10 coders who had no previous training or past experience with ICPC-2 were asked to code the same dataset with ICPC-2 for the purpose of determining the level of accuracy when ICPC-2 is used without previous training. Data analysis Coded data from each participant were entered into an Excel spreadsheet after which the data were checked and cleaned using the Excel filter tool. They were subsequently imported to the Statistical Package for Social Sciences version 23.0 (IBM Corp.2015, Armonk, NY) for statistical analysis. In determining the degree to which ICPC-2 coders agreed about ICPC-2 codes (inter-coder reliability), two statistical methods were used. Firstly, the average percentage agreement among ICPC-2 coders for all the diagnoses that emanated from each chapter and, secondly, computation of a kappa coefficient between coders by pairing each coder with all other coders sequentially. Inter-coder reliability was considered at two levels: agreement at chapter level (alpha codes) and agreement at rubric level (alphanumeric codes). Cohen’s kappa coefficient can range from –1 to 1. Zero indicates only chance agreement and 1 indicates perfect agreement. In rare situations, kappa can be negative. This is a sign that the two observers agreed less than would be expected by chance. Interpretations of agreement for different scores are as follows: <0.20, poor; 0.21–0.40, fair; 0.41–0.60 moderate; 0.61–0.80 good and 0.81–1.00, very good (20). The convergent validity of ICPC-2 codes when correlated with selected ICD-10 codes was analysed in the form of a correlation coefficient. Interpretations of the correlation for different scores are as follows: weak (0.1–0.3), moderate (0.4–0.6), strong (0.7–0.9) and perfect (1). Sensitivity analysis was done to determine the sensitivity of the correlation between the selected ICPC-2 codes and the corresponding ICD-10 codes. Accuracy for untrained ICPC-2 coders was analysed by checking the ICPC-2 codes selected by the non-routine users against the codes agreed upon by the ICPC-2 experts. Additional analysis such as the rate of usage of residual rubrics was analysed and used to deduce the content validity. Ethical considerations Ethical clearance to conduct this study was obtained from the HREC of Lagos State University Teaching Hospital (reference number: LREC.06/10/809) and Stellenbosch University (ethics reference number: S16/06/110). Permission was also obtained from the health research and statistics unit of the Lagos State Health Service Commission. The clinical coders gave written consent. Confidentiality and anonymity were guaranteed as the names and personal information of the patients were not disclosed at any point during the process of data collection, analysis and presentation of results. Results Two hundred and twenty diagnoses were included, and it was observed that they cut across all 17 chapters of the ICPC-2. Table 1 presents the distribution of diagnoses/health problems across the 17 chapters. Spearman’s correlation coefficient of 0.7 was obtained for the convergent validity of ICPC-2 compared to ICD-10 coding, and this was statistically significant (P < 0.01). The result suggests a strong positive correlation and high convergent validity of ICPC-2. Sensitivity analysis showed that the sensitivity of the correlation between the selected ICPC-2 codes and corresponding ICD-10 codes was 86.8%. Table 1. Distribution of diagnoses/health problems and average percentage agreement across chapters (n = 220) Chapters Distribution, n (%) Agreement at chapter level, median % (range) Agreement at rubric level, median % (range) K—Cardiovascular 43 (19.6) 100.0 (75.0–100.0) 100.0 (75.0–100.0) D—Digestive 30 (13.6) 100.0 (100.0–100.0) 100.0 (50.0–100.0) A—General and Unspecified 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) L—Musculoskeletal 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) X—Female Genital 15 (6.8) 100.0 (75.0–100.0) 100.0 (75.0–100.0) F—Eye 14 (6.4) 100.0 (100.0–100.0) 100.0 (75.0–100.0) T—Endocrine/Metabolic and Nutritional 13 (5.9) 100.0 (50.0–100.0) 100.0 (50.0–100.0) S—Skin 11 (5.0) 100.0 (75.0–100.0) 100.0 (75.0–100.0) R—Respiratory 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) U—Urological 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) N—Neurological 6 (2.7) 100.0 (75.0–100.0) 100.0 (50.0–100.0) H—Ear 5 (2.3) 100.0 (100.0–100.0) 100.0 (75.0–100.0) P—Psychological 4 (1.8) 100.0 (75.0–100.0) 100.0 (50.0–100.0) W—Pregnancy, Childbearing, Family Planning 3 (1.4) 100.0 (100.0–100.0) 100.0 (100.0–100.0) Y—Male Genital 2 (0.9) 87.5 (75.0–100.0) 87.5 (75.0–100.0) B—Blood, Blood Forming Organs and Immune Mechanism 1 (0.5) 100.0 (0) 100.0 (0) Z—Social Problems 1 (0.5) 75.0 (0) 75.0 (0) Chapters Distribution, n (%) Agreement at chapter level, median % (range) Agreement at rubric level, median % (range) K—Cardiovascular 43 (19.6) 100.0 (75.0–100.0) 100.0 (75.0–100.0) D—Digestive 30 (13.6) 100.0 (100.0–100.0) 100.0 (50.0–100.0) A—General and Unspecified 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) L—Musculoskeletal 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) X—Female Genital 15 (6.8) 100.0 (75.0–100.0) 100.0 (75.0–100.0) F—Eye 14 (6.4) 100.0 (100.0–100.0) 100.0 (75.0–100.0) T—Endocrine/Metabolic and Nutritional 13 (5.9) 100.0 (50.0–100.0) 100.0 (50.0–100.0) S—Skin 11 (5.0) 100.0 (75.0–100.0) 100.0 (75.0–100.0) R—Respiratory 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) U—Urological 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) N—Neurological 6 (2.7) 100.0 (75.0–100.0) 100.0 (50.0–100.0) H—Ear 5 (2.3) 100.0 (100.0–100.0) 100.0 (75.0–100.0) P—Psychological 4 (1.8) 100.0 (75.0–100.0) 100.0 (50.0–100.0) W—Pregnancy, Childbearing, Family Planning 3 (1.4) 100.0 (100.0–100.0) 100.0 (100.0–100.0) Y—Male Genital 2 (0.9) 87.5 (75.0–100.0) 87.5 (75.0–100.0) B—Blood, Blood Forming Organs and Immune Mechanism 1 (0.5) 100.0 (0) 100.0 (0) Z—Social Problems 1 (0.5) 75.0 (0) 75.0 (0) View Large Table 1. Distribution of diagnoses/health problems and average percentage agreement across chapters (n = 220) Chapters Distribution, n (%) Agreement at chapter level, median % (range) Agreement at rubric level, median % (range) K—Cardiovascular 43 (19.6) 100.0 (75.0–100.0) 100.0 (75.0–100.0) D—Digestive 30 (13.6) 100.0 (100.0–100.0) 100.0 (50.0–100.0) A—General and Unspecified 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) L—Musculoskeletal 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) X—Female Genital 15 (6.8) 100.0 (75.0–100.0) 100.0 (75.0–100.0) F—Eye 14 (6.4) 100.0 (100.0–100.0) 100.0 (75.0–100.0) T—Endocrine/Metabolic and Nutritional 13 (5.9) 100.0 (50.0–100.0) 100.0 (50.0–100.0) S—Skin 11 (5.0) 100.0 (75.0–100.0) 100.0 (75.0–100.0) R—Respiratory 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) U—Urological 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) N—Neurological 6 (2.7) 100.0 (75.0–100.0) 100.0 (50.0–100.0) H—Ear 5 (2.3) 100.0 (100.0–100.0) 100.0 (75.0–100.0) P—Psychological 4 (1.8) 100.0 (75.0–100.0) 100.0 (50.0–100.0) W—Pregnancy, Childbearing, Family Planning 3 (1.4) 100.0 (100.0–100.0) 100.0 (100.0–100.0) Y—Male Genital 2 (0.9) 87.5 (75.0–100.0) 87.5 (75.0–100.0) B—Blood, Blood Forming Organs and Immune Mechanism 1 (0.5) 100.0 (0) 100.0 (0) Z—Social Problems 1 (0.5) 75.0 (0) 75.0 (0) Chapters Distribution, n (%) Agreement at chapter level, median % (range) Agreement at rubric level, median % (range) K—Cardiovascular 43 (19.6) 100.0 (75.0–100.0) 100.0 (75.0–100.0) D—Digestive 30 (13.6) 100.0 (100.0–100.0) 100.0 (50.0–100.0) A—General and Unspecified 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) L—Musculoskeletal 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) X—Female Genital 15 (6.8) 100.0 (75.0–100.0) 100.0 (75.0–100.0) F—Eye 14 (6.4) 100.0 (100.0–100.0) 100.0 (75.0–100.0) T—Endocrine/Metabolic and Nutritional 13 (5.9) 100.0 (50.0–100.0) 100.0 (50.0–100.0) S—Skin 11 (5.0) 100.0 (75.0–100.0) 100.0 (75.0–100.0) R—Respiratory 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) U—Urological 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) N—Neurological 6 (2.7) 100.0 (75.0–100.0) 100.0 (50.0–100.0) H—Ear 5 (2.3) 100.0 (100.0–100.0) 100.0 (75.0–100.0) P—Psychological 4 (1.8) 100.0 (75.0–100.0) 100.0 (50.0–100.0) W—Pregnancy, Childbearing, Family Planning 3 (1.4) 100.0 (100.0–100.0) 100.0 (100.0–100.0) Y—Male Genital 2 (0.9) 87.5 (75.0–100.0) 87.5 (75.0–100.0) B—Blood, Blood Forming Organs and Immune Mechanism 1 (0.5) 100.0 (0) 100.0 (0) Z—Social Problems 1 (0.5) 75.0 (0) 75.0 (0) View Large Average percentage agreement between ICPC-2 coders was 97.9% at the chapter level and 95.6% at the rubric level. The range of percentage agreement was lower at rubric level than chapter level for the chapters Digestive, Eye, Respiratory, Urological, Neurological, Ear and Psychological Problems but were the same at both chapter and rubric levels for all other chapters. Inter-coder agreement was not uniform across chapters. The results for the kappa coefficient are presented in Table 2 and show that κ values were very good (>0.81) and were higher on chapter level (0.94–0.97) than rubric level (0.90–0.94). Pairwise agreements between local and foreign coders were >0.81 on both chapter and rubric levels, and this suggests very good inter-coder reliability for ICPC-2. The level of accuracy when ICPC-2 was used by coders without previous training was 74.5%. Table 2. Kappa measure of agreement (95% confidence interval) at chapter level Chapter level Coder 2 Coder 3 Coder 4 Coder 1 0.96 (0.93–0.99) 0.95 (0.92–0.98) 0.97 (0.95–0.99) Coder 2 0.94 (0.91–0.97) 0.95 (0.92–0.98) Coder 3 0.94 (0.91–0.97) Chapter level Coder 2 Coder 3 Coder 4 Coder 1 0.96 (0.93–0.99) 0.95 (0.92–0.98) 0.97 (0.95–0.99) Coder 2 0.94 (0.91–0.97) 0.95 (0.92–0.98) Coder 3 0.94 (0.91–0.97) View Large Table 2. Kappa measure of agreement (95% confidence interval) at chapter level Chapter level Coder 2 Coder 3 Coder 4 Coder 1 0.96 (0.93–0.99) 0.95 (0.92–0.98) 0.97 (0.95–0.99) Coder 2 0.94 (0.91–0.97) 0.95 (0.92–0.98) Coder 3 0.94 (0.91–0.97) Chapter level Coder 2 Coder 3 Coder 4 Coder 1 0.96 (0.93–0.99) 0.95 (0.92–0.98) 0.97 (0.95–0.99) Coder 2 0.94 (0.91–0.97) 0.95 (0.92–0.98) Coder 3 0.94 (0.91–0.97) View Large An additional finding was made with regard to the rate of utilization of the residual rubrics. A low utilization of residual rubrics of 3.8% indicates the availability of specific codes for most of the diagnoses. The residual rubrics were used in the following chapters: S – Skin (3), F – Eye (2), D – Digestive (1) and U – Urological (1). Discussion The study presents the first set of data measuring the convergent validity of the ICPC-2 diagnoses/health problem codes in relation to ICD-10 codes globally. A strong positive correlation (r ≈ 0.7) at a sensitivity of 86.8% denotes a high convergent validity of ICPC-2 and implies that it is a valid alternative coding system for primary care data in the study setting. The strong positive relationship suggests that ICPC-2 may be used at the primary care level where the routine classification of primary care data is lacking and where it may be inappropriate and cumbersome to successfully adapt the numerous codes of ICD-10. Unfortunately, comparison with other studies is difficult owing to the unavailability of literature that focussed on ICPC-2–ICD-10 combined datasets. An earlier study conducted in Australia in 1997, before the development of the mapping of ICPC-2 to ICD-10, reported satisfactory concurrent validity of the ICPC’s Reasons for Encounter (RfE) codes in comparison with the existing RfE codes’ utilization results from the Australian Morbidity and Treatment Survey 1990–1991 (12). Earlier researchers have worked extensively on the reliability of ICPC (1st edition) and they reported good reliability from Europe and Australia (18,21,22). Although the studies conducted in France and Argentina were based on ICPC, they focused on the reliability of coding for health problems rather than RfE, which was the focus of the study done in Australia (18,22). The study done in France reported κ measure of agreement at rubric level of 0.65 [95% confidence interval [CI]: 0.52–0.77], and for the chapters selected, 0.84 (95% CI: 0.78–0.91). Similarly, the study conducted in Argentina showed an inter-coder agreement of 95% (κ = 0.94; P < 0.0001) at chapter level and 82.3% (κ = 0.82; P < 0.0001) at rubric level (22). They both reported higher agreement at chapter level than rubric level, which is in concordance with the findings from this study. Regarding ICPC-2, a study of its reliability in a German general practice setting reported a high agreement at chapter level between two coders (mean κ = 0.80) with respect to managed health problems (23). The findings from the current study equally revealed a high level of agreement, more so at the chapter level (97.9%) than at the rubric level (95.6%). Kappa values were also higher at the chapter level, ranging from 0.94 to 0.97, than at the rubric level, from 0.90 to 0.94, for pairwise agreement between the four coders. It is difficult to compare inter-coder reliability for individual ICPC-2 chapters with other studies, because of the low frequency of diagnoses in some of the chapters in this study. This is due to the fact that the sample size was constructed for an overall average agreement for all chapters rather than agreement per individual chapters. However, this study improved upon the designs from some of the previous studies in mitigating the probable errors of interpretation that may have arisen when coders first interpreted clinical scenarios before proceeding to select codes that match their interpretation (21,23). The use of the same set of physicians’ diagnoses by all the coders, and the involvement of multiple experienced/expert coders are strengths of this study. Furthermore, coders who had no previous training in ICPC-2 usage coded health problems with an accuracy of 74.5%. This result is similar to the accuracy achieved by Portuguese family medicine residents who had no previous training but scored 74.8% for correctly coded health problems, whereas those with previous training scored 85.0% (24). The lack of excessive subclasses which characterizes the simplicity of a classification system played out in the level of accuracy that was achieved by non-routine users with no previous training in ICPC-2 usage. Health policy makers may find this a useful quality to consider in making a choice for practical purposes. Limitations of the study There is no other gold standard for comparison with ICPC-2, and as ICD-10 is the generally accepted standard of classification in Nigeria and elsewhere, it is applied to data from the primary care sections of both secondary and tertiary healthcare facilities in the absence of an alternative. It is, therefore, reasonable to compare ICPC-2 to ICD-10. The content validity of ICPC-2 for the Nigerian primary care was not directly assessed, but the low percentage of residual codes that were used suggests that a specific code was found in almost all cases. The primary care setting in this study is common in Nigeria, a general outpatient department in a secondary hospital, although primary care elsewhere usually implies a more community-based health centre or clinic context. Recommendations This study supports the use of ICPC-2 to code primary care diagnoses in the Nigerian setting as the system is less complex than ICD-10 and has sufficient validity and reliability. Validation studies focusing on the other elements of the encounter and experiences of coders regarding the relative ease of coding with respect to time taken to assign codes with ICPC-2 when compared with ICD-10 are recommended for future studies. The impact of training on accuracy may also be investigated. Conclusion The findings suggest that ICPC-2 is a valid and reliable alternative coding system than ICD-10 in primary care settings. The clear implication of the availability of specific codes for most of the frequently encountered health problems in the study setting is that it favours its application in the African primary care context, while also offering the possibility of optional hierarchical expansion from ICPC-2 to ICD-10 via the mapping. This allows relevant codes from the ICD-10 to be incorporated into state- or country-specific adaptations of the ICPC-2. Acknowledgements We appreciate the inputs of Dr. Abiodun Adewuya at the conceptualization stage and Dr. Daniel Pinto at the data collection stage. References 1. Bentzen N (ed.). Wonca Dictionary of General/Family Practice . Copenhagen : Maanedsskrift for PraktiskLaegegerning , 2003 . 2. de Lusignan S , van Weel C . The use of routinely collected computer data for research in primary care: opportunities and challenges . Fam Pract 2006 ; 23 : 253 – 63 . Google Scholar CrossRef Search ADS PubMed 3. de Andrade CT , Magedanz AM , Escobosa DM , et al. The importance of a database in the management of healthcare services . Einstein (Sao Paulo) 2012 ; 10 : 360 – 5 . Google Scholar CrossRef Search ADS PubMed 4. Adeshina Y. The Effect of Health Sector Reform on Health Care; The Revitalization Program for the Primary Health Care System . http://www.ihf-fih.org ( accessed on June 2013 ). 5. Britt H , Miller GC , Charles J , et al. (eds). A Decade of Australian General Practice Activity 2002–03 to 2011–12 . General Practice Series No. 32. Sydney : Sydney University Press , 2012 . 6. World Health Organization . Family of International Classifications . Geneva : WHO , 2004 . http://www.who.int/classifications/en/WHOFICFamily.pdf ( accessed on 21 September 2012 ). PubMed PubMed 7. Hofmans-Okkes IM , Lamberts H . The international classification of primary care (ICPC): new applications in research and computer-based patient records in family practice . Fam Pract 1996 ; 13 : 294 – 302 . Google Scholar CrossRef Search ADS PubMed 8. deLusignan S . The optimum granularity for coding diagnostic data in primary care: report of a workshop of the EFMI primary care informatics working group at MIE 2005 . Inform Prim Care 2006 ; 14 : 33 – 7 . 9. Becker HW , Oskam SK , Okkes IM , et al. ICPC2-ICD10 Thesaurus. A diagnostic terminology for semi-automatic double coding in electronic patient records . In: Okkes IM , Oskam SK , Lambert H (eds). ICPC in the Amsterdam Transition Project. CD-Rom . Amsterdam : Academic Medical Center/University of Amsterdam Department of Family Medicine , 2005 . 10. Classification Committee of the World Organization of Family doctors . ICPC-2: International Classification of Primary Care . 2nd edn . Oxford : Oxford University Press , 1998 , p 3 . 11. Lamberts H , Meads S , Wood M . Results of the international field trial with the reason for encounter classification . In: Cote RA , Protti AJ , Scherner JR (eds). Role of Informatics in Health Data Coding and Classification Systems . Amsterdam : Elsevier Scientific Publications , 1985 . Google Scholar CrossRef Search ADS 12. Britt H . A measure of the validity of the international classification of primary care in the classification of reasons for encounter . J Innov Health Inform 1997 ; 8 – 12 . 13. Anastasi A , Urbina S. Psychological Testing . 10th edn . New Delhi : Prentice-Hall , 2014 . 14. Olagundoye OA , van Boven K , van Weel C . International classification of primary care-2 coding of primary care data at the general out-patients’ clinic of General Hospital, Lagos, Nigeria . J Family Med Prim Care 2016 ; 5 : 291 – 7 . Google Scholar CrossRef Search ADS PubMed 15. Ayankogbe OO , Oyediran MA , Oke DA , et al. ICPC-2 defined pattern of illnesses in an urban region in West Africa . Afr J Prim Health Care Fam Med 2009 ; 1 : 103 – 6 . Google Scholar CrossRef Search ADS 16. Brueton V , Yogeswaran P , Chandia J , et al. Primary care morbidity in Eastern Cape Province . S Afr Med J 2010 ; 100 : 309 – 12 . Google Scholar CrossRef Search ADS PubMed 17. Rahman SM , Angeline RP , Cynthia S , et al. International classification of primary care: an Indian experience . J Family Med Prim Care 2014 ; 3 : 362 – 7 . Google Scholar PubMed 18. Letrilliart L , Guiguet M , Flahault A . Reliability of report coding of hospital referrals in primary care versus practice-based coding . Eur J Epidemiol 2000 ; 16 : 653 – 9 . Google Scholar CrossRef Search ADS PubMed 19. Yamane T. Statistics, An Introductory Analysis . 2nd edn . New York : Harper and Row , 1967 , pp. 159 – 60 . 20. Landis JR , Koch GG . The measurement of observer agreement for categorical data . Biometrics 1977 ; 33 : 159 – 74 . Google Scholar CrossRef Search ADS PubMed 21. Britt H . Reliability of central coding of patient reasons for encounter in general practice, using the international classification of primary care . J Innov Health Inform 1998 ; 7 ( 1 ): 210 . Google Scholar CrossRef Search ADS 22. Luna D , de Quirós FGB , Garfi L , et al. Reliability of secondary central coding of medical problems in primary care by non-medical coders, using the international classification of primary care (ICPC) . Medinfo 2001 ; 10 : 300 . 23. Frese T , Herrmann K , Bungert-Kahl P , et al. Inter-rater reliability of the ICPC-2 in a German general practice setting . Swiss Med Wkly 2012 . http://www.smw.ch/content/smw-2012–13621/ ( accessed on 22 August 2012 ) 24. Pinto D , Corte-Real S . Coding with the international classification of primary care by family medicine residents . Rev Port Clin Geral 2010 ; 26 : 370 – 82 . © The Author(s) 2018. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Family Practice Oxford University Press

Reliability measurement and ICD-10 validation of ICPC-2 for coding/classification of diagnoses/health problems in an African primary care setting

Family Practice , Volume Advance Article (4) – Jan 17, 2018

Loading next page...
 
/lp/ou_press/reliability-measurement-and-icd-10-validation-of-icpc-2-for-coding-Owug4PZLDT
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
ISSN
0263-2136
eISSN
1460-2229
D.O.I.
10.1093/fampra/cmx132
Publisher site
See Article on Publisher Site

Abstract

Abstract Background The routine application of a primary care classification system to patients’ medical records in general practice/primary care is rare in the African region. Reliable data are crucial to understanding the domain of primary care in Nigeria, and this may be actualized through the use of a locally validated primary care classification system such as the International Classification of Primary Care, 2nd edition (ICPC-2). Although a few studies from Europe and Australia have reported that ICPC is a reliable and feasible tool for classifying data in primary care, the reliability and validity of the revised version (ICPC-2) is yet to be objectively determined particularly in Africa. Objectives (i) To determine the convergent validity of ICPC-2 diagnoses codes when correlated with International Statistical Classification of Diseases (ICD)-10 codes, (ii) to determine the inter-coder reliability among local and foreign ICPC-2 experts and (iii) to ascertain the level of accuracy when ICPC-2 is engaged by coders without previous training. Methods Psychometric analysis was carried out on ICPC-2 and ICD-10 coded data that were generated from physicians’ diagnoses, which were randomly selected from general outpatients’ clinic attendance registers, using a systematic sampling technique. Participants comprised two groups of coders (ICPC-2 coders and ICD-10 coders) who coded independently a total of 220 diagnoses/health problems with ICPC-2 and/or ICD-10, respectively. Results Two hundred and twenty diagnoses/health problems were considered and were found to cut across all 17 chapters of the ICPC-2. The dataset revealed a strong positive correlation between selected ICPC-2 codes and ICD-10 codes (r ≈ 0.7) at a sensitivity of 86.8%. Mean percentage agreement among the ICPC-2 coders was 97.9% at the chapter level and 95.6% at the rubric level. Similarly, Cohen’s kappa coefficients were very good (κ > 0.81) and were higher at chapter level (0.94–0.97) than rubric level (0.90–0.93) between sets of pairs of ICPC-2 coders. An accuracy of 74.5% was achieved by ICD-10 coders who had no previous experience or prior training on ICPC-2 usage. Conclusion Findings support the utility of ICPC-2 as a valid and reliable coding tool that may be adopted for routine data collection in the African primary care context. The level of accuracy achieved without training lends credence to the proposition that it is a simple-to-use classification and may be a useful starting point in a setting devoid of any primary care classification system for morbidity and mortality registration at such a critical level of public health importance. Classification, ICPC-2, ICD-10, primary care, reliability, validity Introduction Primary care has been defined as the provision of integrated accessible healthcare services by primary care practitioners, including family physicians and GPs, who are accountable for addressing the majority of personal healthcare needs, developing a sustained partnership with patients and practicing in the context of family and community (1). It is enhanced by longitudinal medical records that are collected over time, which can be a unique source of health information (2). Routine collection of primary care data, aggregated into large databases, can be useful for auditing purposes, quality improvement research (especially chronic disease management) and as a cost-effective source of data for epidemiological studies (2). Although most people are treated at the primary care level, there is little information about primary care morbidity in the African context. Practice-based data can provide a detailed picture of morbidity, and the absence of such information impedes the development of primary care. Information on morbidity can be used to set health priorities and to improve the processes of management and care (3). Epidemiological data obtained at this level are an invaluable source of health information for planning and policy formulation. Poor management of data collection systems contributes significantly to inaccurate data. In Nigeria, for instance, poor data management has been identified as one of the numerous challenges facing successful implementation of primary health care (4). Although the need for good quality, comparable data is clear, the practice of coding such data at the primary care level, either primarily by clinicians during consultation or secondarily by non-clinicians after consultation, is non-existent in the Nigerian primary care context. It is therefore important to consider several issues in the planning and successful implementation of a primary care data system, for the purpose of continuous data collection. One of the most important issues is to select an appropriate tool that can be used to standardize the coding of data for the specific context or setting where it is intended to be used. The International Classification of Primary Care (ICPC) is an evidenced-based coding system that is widely used in more than 45 countries as the standard for data classification in primary care. It is endorsed by the World Health Organization (WHO) in the WHO Family of International Classifications (WHO-FIC) (5,6). The ICPC was first published in 1987, and it has been used in primary care for three decades as the main ordering principle of that domain. It is designed to be simple to use, potentially codes the whole encounter (reason for encounter, the process of care, diagnoses) and focuses on problems that are frequently encountered in primary care (7). One of the coding reference classifications endorsed by the WHO-FIC is the International Statistical Classification of Diseases (ICD) and related health problems. As opposed to the ICPC, which was specially developed for primary care purposes, it supports the classification of diseases at a higher level of specificity, beyond what is required for primary care. Since its inception in 1977, several editions have evolved and the 10th edition (ICD-10) is the latest available version. With more than 14000 codes, it is extensive and caters to the labelling of health problems that are rarely seen at the primary care level. A revised version of the ICPC, the second edition of ICPC (ICPC-2), was published in 1998. Compared with the ICD-10, it has fewer codes (~1400), which are less specific, and more appropriate to classify data in a primary care setting (8). Moreover, it has been mapped to the ICD-10, which allows for additional labelling of health problems not often encountered at primary care level, and can be used if ever there is a need to classify at a higher level of granularity (9). The ICPC-2 has 17 chapters representing different body systems/problem areas, and it defines an encounter as the professional interchange between a patient and a primary care provider. An encounter is made up of three elements: reasons for encounter (RfE)/complaints, diagnoses/problems and interventions/processes/procedures. Its rubrics were developed in a European, an Australian and an American context (10), hence the need for its re-validation within the local context of intended use. The ICPC-2 has a bi-axial structure: 17 chapters on one axis, each with an alpha code, and seven identical components with rubrics bearing a two-digit numeric code as the second axis. The chapters are constructed in terms of organ systems, psychological and social problems, and one general chapter. ICPC has a significant mnemonic quality that facilitates its day-to-day use in primary care and simplifies the centralized manual coding of data recorded elsewhere. Each rubric has one alphabet and two-digit code number, and a title of limited length (10). For the purpose of this study, rubrics in component 1 (symptoms/complaints) and component 7 (diseases) were applied. The -29 and -99 codes are consistent across all chapters within the structure of ICPC-2 as residual rubrics, and they are designated for coding diagnoses for which there are no specific ICPC-2 codes. The original version was designed and field-tested for face validity and consensual validity by a group of experts (11). As the face and consensual validity are both based on the subjective opinions of experts, further objective measures of validation should be carried out in settings in which it is intended for use (12). Convergent validity is established when there is a strong relationship between the tool/measure under review and a previously validated tool that measures the same construct (13). It determines the extent to which a new tool agrees or correlates with an existing tool. It entails comparing with a ‘gold standard’ set by another instrument. The gold standard against which results of ICPC-2 coded data were validated in this study setting is ICD-10, which is the national coding standard at the secondary and tertiary healthcare levels. The mapping of ICPC-2 to ICD-10 facilitated this objective. Limited literature is available on primary healthcare data systems in Africa. Invariably, very little research on the use of a primary care classification system for primary care data management has been conducted (14–16). The routine application of ICPC-2 to patients’ medical records in primary care barely exists in the developing world (14–17). For instance, currently there is no classification system dedicated to routine data collection at a primary care level in Nigeria. In light of the importance of primary health care, its pivotal role in the health systems of most nations, particularly Africa, and the aforementioned relevance of primary care data, it is clear that there is a need to evaluate the reliability and validity of using ICPC-2 for coding/classifying diagnoses/health problems in this African primary care setting. Furthermore, applying an international classification system to local data affords the opportunity of generating data for international comparability which is important to global health (14). Aim The overall aim was to evaluate the reliability and validity of using ICPC-2 for coding and classifying diagnoses/health problems in an African primary care setting Objectives The objectives were to (i) determine the convergent validity of ICPC-2 diagnoses codes when correlated with ICD-10; (ii) determine the inter-coder reliability among local and foreign ICPC-2 experts and (iii) ascertain the level of accuracy when ICPC-2 is engaged by coders without previous training on ICPC-2 usage. Methods Study design The study design was a psychometric analysis of coded primary care health problems/diagnoses data. Study setting The study setting was the primary care section of a public secondary healthcare facility in Lagos State, Nigeria. The general outpatients’ clinic offers primary care services that are provided by primary care doctors to undifferentiated, self-referred adult patients. These services include first contact care, health promotion, disease prevention, care of uncomplicated chronic diseases and referrals to specialists and other healthcare providers. Selection of participants The participants were two groups of coders who were co-opted on the basis of experience/expertise with ICD-10 and ICPC-2 as described below. Each group was made up of four ICD-10 and four ICPC-2 coders. Selection criteria Four experienced local ICD-10 clinical coders were selected based on the following inclusion criteria: More than 5 years of experience as a clinical coder; Qualified as a clinical coder with an ordinary national diploma degree in health information management; Voluntariness to participate and Proficiency in English. Four ICPC-2 experts (a local and three foreign experts) were recruited to participate in the study based on the following criteria: Routine usage of ICPC-2 in their practice; Voluntariness to participate; Membership of the World Congress of Family Doctors’ (Wonca) International Classification Committee, which is the committee responsible for the ICPC-2 and Proficiency in English. Sample size and sampling technique Given a standard normal deviate of 1.96 at 5% type 1 error, proportion of agreement, Cohen’s kappa of 0.84 from a previous study conducted in France (18), a minimum sample size of 206 encounters (clinical consultations) was required to draw valid conclusions about the reliability and validity of ICPC-2 in the study setting (19). The researcher collected data in August 2016 from the 10 clinic attendance registers using a systematic sampling technique from entries for the previous 6 months. These registers were distributed across all the consulting rooms and they were completed simultaneously by primary care doctors, who used them routinely to record patients’ hospital number, age, gender and diagnoses in English. The number of patients in each register varied from 79 to 1620, and the contribution of each register to the total sample was stratified according to the number of entries. Patients from each register were then systematically sampled with an interval correlated with the number of patients required from the total (intervals ranged from every 5th to every 63rd entry). Data collection Two tools were used: the tabular list of ICD-10 and the two-page summary of ICPC-2 codes. The tabular list is volume 1 of ICD-10 and it contains about 14000 terms/codes on more than a thousand pages. The two-page summary of ICPC-2 is a colour-coded document that captures the structure of ICPC-2 with the chapters and individual rubrics (~1400 codes and their titles). Diagnoses were recorded verbatim from the clinic registers as described earlier and captured in an Excel spreadsheet. The list was distributed by hand to the local ICD-10 coders and sent via e-mail to local and foreign ICPC-2 coders for coding. To determine the convergent validity of ICPC-2, the ICPC-2 experts and the trained and experienced, routine ICD-10 coders independently coded the same sample of patients’ diagnoses with ICPC-2 and ICD-10, respectively. This was to determine the correlation between the assigned ICPC-2 diagnoses codes and assigned ICD-10 codes for the same diagnoses. The mapping of ICPC-2 to ICD-10 was used to facilitate this correlation (9). Due to the high granularity of ICD-10, several of its codes may be mapped to a single ICPC-2 code. The codes assigned to each diagnostic term from either of the two classifications were checked to determine if they were correlated on the mapping. In addition, ICD-10 coders who had no previous training or past experience with ICPC-2 were asked to code the same dataset with ICPC-2 for the purpose of determining the level of accuracy when ICPC-2 is used without previous training. Data analysis Coded data from each participant were entered into an Excel spreadsheet after which the data were checked and cleaned using the Excel filter tool. They were subsequently imported to the Statistical Package for Social Sciences version 23.0 (IBM Corp.2015, Armonk, NY) for statistical analysis. In determining the degree to which ICPC-2 coders agreed about ICPC-2 codes (inter-coder reliability), two statistical methods were used. Firstly, the average percentage agreement among ICPC-2 coders for all the diagnoses that emanated from each chapter and, secondly, computation of a kappa coefficient between coders by pairing each coder with all other coders sequentially. Inter-coder reliability was considered at two levels: agreement at chapter level (alpha codes) and agreement at rubric level (alphanumeric codes). Cohen’s kappa coefficient can range from –1 to 1. Zero indicates only chance agreement and 1 indicates perfect agreement. In rare situations, kappa can be negative. This is a sign that the two observers agreed less than would be expected by chance. Interpretations of agreement for different scores are as follows: <0.20, poor; 0.21–0.40, fair; 0.41–0.60 moderate; 0.61–0.80 good and 0.81–1.00, very good (20). The convergent validity of ICPC-2 codes when correlated with selected ICD-10 codes was analysed in the form of a correlation coefficient. Interpretations of the correlation for different scores are as follows: weak (0.1–0.3), moderate (0.4–0.6), strong (0.7–0.9) and perfect (1). Sensitivity analysis was done to determine the sensitivity of the correlation between the selected ICPC-2 codes and the corresponding ICD-10 codes. Accuracy for untrained ICPC-2 coders was analysed by checking the ICPC-2 codes selected by the non-routine users against the codes agreed upon by the ICPC-2 experts. Additional analysis such as the rate of usage of residual rubrics was analysed and used to deduce the content validity. Ethical considerations Ethical clearance to conduct this study was obtained from the HREC of Lagos State University Teaching Hospital (reference number: LREC.06/10/809) and Stellenbosch University (ethics reference number: S16/06/110). Permission was also obtained from the health research and statistics unit of the Lagos State Health Service Commission. The clinical coders gave written consent. Confidentiality and anonymity were guaranteed as the names and personal information of the patients were not disclosed at any point during the process of data collection, analysis and presentation of results. Results Two hundred and twenty diagnoses were included, and it was observed that they cut across all 17 chapters of the ICPC-2. Table 1 presents the distribution of diagnoses/health problems across the 17 chapters. Spearman’s correlation coefficient of 0.7 was obtained for the convergent validity of ICPC-2 compared to ICD-10 coding, and this was statistically significant (P < 0.01). The result suggests a strong positive correlation and high convergent validity of ICPC-2. Sensitivity analysis showed that the sensitivity of the correlation between the selected ICPC-2 codes and corresponding ICD-10 codes was 86.8%. Table 1. Distribution of diagnoses/health problems and average percentage agreement across chapters (n = 220) Chapters Distribution, n (%) Agreement at chapter level, median % (range) Agreement at rubric level, median % (range) K—Cardiovascular 43 (19.6) 100.0 (75.0–100.0) 100.0 (75.0–100.0) D—Digestive 30 (13.6) 100.0 (100.0–100.0) 100.0 (50.0–100.0) A—General and Unspecified 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) L—Musculoskeletal 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) X—Female Genital 15 (6.8) 100.0 (75.0–100.0) 100.0 (75.0–100.0) F—Eye 14 (6.4) 100.0 (100.0–100.0) 100.0 (75.0–100.0) T—Endocrine/Metabolic and Nutritional 13 (5.9) 100.0 (50.0–100.0) 100.0 (50.0–100.0) S—Skin 11 (5.0) 100.0 (75.0–100.0) 100.0 (75.0–100.0) R—Respiratory 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) U—Urological 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) N—Neurological 6 (2.7) 100.0 (75.0–100.0) 100.0 (50.0–100.0) H—Ear 5 (2.3) 100.0 (100.0–100.0) 100.0 (75.0–100.0) P—Psychological 4 (1.8) 100.0 (75.0–100.0) 100.0 (50.0–100.0) W—Pregnancy, Childbearing, Family Planning 3 (1.4) 100.0 (100.0–100.0) 100.0 (100.0–100.0) Y—Male Genital 2 (0.9) 87.5 (75.0–100.0) 87.5 (75.0–100.0) B—Blood, Blood Forming Organs and Immune Mechanism 1 (0.5) 100.0 (0) 100.0 (0) Z—Social Problems 1 (0.5) 75.0 (0) 75.0 (0) Chapters Distribution, n (%) Agreement at chapter level, median % (range) Agreement at rubric level, median % (range) K—Cardiovascular 43 (19.6) 100.0 (75.0–100.0) 100.0 (75.0–100.0) D—Digestive 30 (13.6) 100.0 (100.0–100.0) 100.0 (50.0–100.0) A—General and Unspecified 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) L—Musculoskeletal 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) X—Female Genital 15 (6.8) 100.0 (75.0–100.0) 100.0 (75.0–100.0) F—Eye 14 (6.4) 100.0 (100.0–100.0) 100.0 (75.0–100.0) T—Endocrine/Metabolic and Nutritional 13 (5.9) 100.0 (50.0–100.0) 100.0 (50.0–100.0) S—Skin 11 (5.0) 100.0 (75.0–100.0) 100.0 (75.0–100.0) R—Respiratory 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) U—Urological 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) N—Neurological 6 (2.7) 100.0 (75.0–100.0) 100.0 (50.0–100.0) H—Ear 5 (2.3) 100.0 (100.0–100.0) 100.0 (75.0–100.0) P—Psychological 4 (1.8) 100.0 (75.0–100.0) 100.0 (50.0–100.0) W—Pregnancy, Childbearing, Family Planning 3 (1.4) 100.0 (100.0–100.0) 100.0 (100.0–100.0) Y—Male Genital 2 (0.9) 87.5 (75.0–100.0) 87.5 (75.0–100.0) B—Blood, Blood Forming Organs and Immune Mechanism 1 (0.5) 100.0 (0) 100.0 (0) Z—Social Problems 1 (0.5) 75.0 (0) 75.0 (0) View Large Table 1. Distribution of diagnoses/health problems and average percentage agreement across chapters (n = 220) Chapters Distribution, n (%) Agreement at chapter level, median % (range) Agreement at rubric level, median % (range) K—Cardiovascular 43 (19.6) 100.0 (75.0–100.0) 100.0 (75.0–100.0) D—Digestive 30 (13.6) 100.0 (100.0–100.0) 100.0 (50.0–100.0) A—General and Unspecified 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) L—Musculoskeletal 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) X—Female Genital 15 (6.8) 100.0 (75.0–100.0) 100.0 (75.0–100.0) F—Eye 14 (6.4) 100.0 (100.0–100.0) 100.0 (75.0–100.0) T—Endocrine/Metabolic and Nutritional 13 (5.9) 100.0 (50.0–100.0) 100.0 (50.0–100.0) S—Skin 11 (5.0) 100.0 (75.0–100.0) 100.0 (75.0–100.0) R—Respiratory 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) U—Urological 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) N—Neurological 6 (2.7) 100.0 (75.0–100.0) 100.0 (50.0–100.0) H—Ear 5 (2.3) 100.0 (100.0–100.0) 100.0 (75.0–100.0) P—Psychological 4 (1.8) 100.0 (75.0–100.0) 100.0 (50.0–100.0) W—Pregnancy, Childbearing, Family Planning 3 (1.4) 100.0 (100.0–100.0) 100.0 (100.0–100.0) Y—Male Genital 2 (0.9) 87.5 (75.0–100.0) 87.5 (75.0–100.0) B—Blood, Blood Forming Organs and Immune Mechanism 1 (0.5) 100.0 (0) 100.0 (0) Z—Social Problems 1 (0.5) 75.0 (0) 75.0 (0) Chapters Distribution, n (%) Agreement at chapter level, median % (range) Agreement at rubric level, median % (range) K—Cardiovascular 43 (19.6) 100.0 (75.0–100.0) 100.0 (75.0–100.0) D—Digestive 30 (13.6) 100.0 (100.0–100.0) 100.0 (50.0–100.0) A—General and Unspecified 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) L—Musculoskeletal 27 (12.3) 100.0 (50.0–100.0) 100.0 (50.0–100.0) X—Female Genital 15 (6.8) 100.0 (75.0–100.0) 100.0 (75.0–100.0) F—Eye 14 (6.4) 100.0 (100.0–100.0) 100.0 (75.0–100.0) T—Endocrine/Metabolic and Nutritional 13 (5.9) 100.0 (50.0–100.0) 100.0 (50.0–100.0) S—Skin 11 (5.0) 100.0 (75.0–100.0) 100.0 (75.0–100.0) R—Respiratory 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) U—Urological 9 (4.1) 100.0 (100.0–100.0) 100.0 (75.0–100.0) N—Neurological 6 (2.7) 100.0 (75.0–100.0) 100.0 (50.0–100.0) H—Ear 5 (2.3) 100.0 (100.0–100.0) 100.0 (75.0–100.0) P—Psychological 4 (1.8) 100.0 (75.0–100.0) 100.0 (50.0–100.0) W—Pregnancy, Childbearing, Family Planning 3 (1.4) 100.0 (100.0–100.0) 100.0 (100.0–100.0) Y—Male Genital 2 (0.9) 87.5 (75.0–100.0) 87.5 (75.0–100.0) B—Blood, Blood Forming Organs and Immune Mechanism 1 (0.5) 100.0 (0) 100.0 (0) Z—Social Problems 1 (0.5) 75.0 (0) 75.0 (0) View Large Average percentage agreement between ICPC-2 coders was 97.9% at the chapter level and 95.6% at the rubric level. The range of percentage agreement was lower at rubric level than chapter level for the chapters Digestive, Eye, Respiratory, Urological, Neurological, Ear and Psychological Problems but were the same at both chapter and rubric levels for all other chapters. Inter-coder agreement was not uniform across chapters. The results for the kappa coefficient are presented in Table 2 and show that κ values were very good (>0.81) and were higher on chapter level (0.94–0.97) than rubric level (0.90–0.94). Pairwise agreements between local and foreign coders were >0.81 on both chapter and rubric levels, and this suggests very good inter-coder reliability for ICPC-2. The level of accuracy when ICPC-2 was used by coders without previous training was 74.5%. Table 2. Kappa measure of agreement (95% confidence interval) at chapter level Chapter level Coder 2 Coder 3 Coder 4 Coder 1 0.96 (0.93–0.99) 0.95 (0.92–0.98) 0.97 (0.95–0.99) Coder 2 0.94 (0.91–0.97) 0.95 (0.92–0.98) Coder 3 0.94 (0.91–0.97) Chapter level Coder 2 Coder 3 Coder 4 Coder 1 0.96 (0.93–0.99) 0.95 (0.92–0.98) 0.97 (0.95–0.99) Coder 2 0.94 (0.91–0.97) 0.95 (0.92–0.98) Coder 3 0.94 (0.91–0.97) View Large Table 2. Kappa measure of agreement (95% confidence interval) at chapter level Chapter level Coder 2 Coder 3 Coder 4 Coder 1 0.96 (0.93–0.99) 0.95 (0.92–0.98) 0.97 (0.95–0.99) Coder 2 0.94 (0.91–0.97) 0.95 (0.92–0.98) Coder 3 0.94 (0.91–0.97) Chapter level Coder 2 Coder 3 Coder 4 Coder 1 0.96 (0.93–0.99) 0.95 (0.92–0.98) 0.97 (0.95–0.99) Coder 2 0.94 (0.91–0.97) 0.95 (0.92–0.98) Coder 3 0.94 (0.91–0.97) View Large An additional finding was made with regard to the rate of utilization of the residual rubrics. A low utilization of residual rubrics of 3.8% indicates the availability of specific codes for most of the diagnoses. The residual rubrics were used in the following chapters: S – Skin (3), F – Eye (2), D – Digestive (1) and U – Urological (1). Discussion The study presents the first set of data measuring the convergent validity of the ICPC-2 diagnoses/health problem codes in relation to ICD-10 codes globally. A strong positive correlation (r ≈ 0.7) at a sensitivity of 86.8% denotes a high convergent validity of ICPC-2 and implies that it is a valid alternative coding system for primary care data in the study setting. The strong positive relationship suggests that ICPC-2 may be used at the primary care level where the routine classification of primary care data is lacking and where it may be inappropriate and cumbersome to successfully adapt the numerous codes of ICD-10. Unfortunately, comparison with other studies is difficult owing to the unavailability of literature that focussed on ICPC-2–ICD-10 combined datasets. An earlier study conducted in Australia in 1997, before the development of the mapping of ICPC-2 to ICD-10, reported satisfactory concurrent validity of the ICPC’s Reasons for Encounter (RfE) codes in comparison with the existing RfE codes’ utilization results from the Australian Morbidity and Treatment Survey 1990–1991 (12). Earlier researchers have worked extensively on the reliability of ICPC (1st edition) and they reported good reliability from Europe and Australia (18,21,22). Although the studies conducted in France and Argentina were based on ICPC, they focused on the reliability of coding for health problems rather than RfE, which was the focus of the study done in Australia (18,22). The study done in France reported κ measure of agreement at rubric level of 0.65 [95% confidence interval [CI]: 0.52–0.77], and for the chapters selected, 0.84 (95% CI: 0.78–0.91). Similarly, the study conducted in Argentina showed an inter-coder agreement of 95% (κ = 0.94; P < 0.0001) at chapter level and 82.3% (κ = 0.82; P < 0.0001) at rubric level (22). They both reported higher agreement at chapter level than rubric level, which is in concordance with the findings from this study. Regarding ICPC-2, a study of its reliability in a German general practice setting reported a high agreement at chapter level between two coders (mean κ = 0.80) with respect to managed health problems (23). The findings from the current study equally revealed a high level of agreement, more so at the chapter level (97.9%) than at the rubric level (95.6%). Kappa values were also higher at the chapter level, ranging from 0.94 to 0.97, than at the rubric level, from 0.90 to 0.94, for pairwise agreement between the four coders. It is difficult to compare inter-coder reliability for individual ICPC-2 chapters with other studies, because of the low frequency of diagnoses in some of the chapters in this study. This is due to the fact that the sample size was constructed for an overall average agreement for all chapters rather than agreement per individual chapters. However, this study improved upon the designs from some of the previous studies in mitigating the probable errors of interpretation that may have arisen when coders first interpreted clinical scenarios before proceeding to select codes that match their interpretation (21,23). The use of the same set of physicians’ diagnoses by all the coders, and the involvement of multiple experienced/expert coders are strengths of this study. Furthermore, coders who had no previous training in ICPC-2 usage coded health problems with an accuracy of 74.5%. This result is similar to the accuracy achieved by Portuguese family medicine residents who had no previous training but scored 74.8% for correctly coded health problems, whereas those with previous training scored 85.0% (24). The lack of excessive subclasses which characterizes the simplicity of a classification system played out in the level of accuracy that was achieved by non-routine users with no previous training in ICPC-2 usage. Health policy makers may find this a useful quality to consider in making a choice for practical purposes. Limitations of the study There is no other gold standard for comparison with ICPC-2, and as ICD-10 is the generally accepted standard of classification in Nigeria and elsewhere, it is applied to data from the primary care sections of both secondary and tertiary healthcare facilities in the absence of an alternative. It is, therefore, reasonable to compare ICPC-2 to ICD-10. The content validity of ICPC-2 for the Nigerian primary care was not directly assessed, but the low percentage of residual codes that were used suggests that a specific code was found in almost all cases. The primary care setting in this study is common in Nigeria, a general outpatient department in a secondary hospital, although primary care elsewhere usually implies a more community-based health centre or clinic context. Recommendations This study supports the use of ICPC-2 to code primary care diagnoses in the Nigerian setting as the system is less complex than ICD-10 and has sufficient validity and reliability. Validation studies focusing on the other elements of the encounter and experiences of coders regarding the relative ease of coding with respect to time taken to assign codes with ICPC-2 when compared with ICD-10 are recommended for future studies. The impact of training on accuracy may also be investigated. Conclusion The findings suggest that ICPC-2 is a valid and reliable alternative coding system than ICD-10 in primary care settings. The clear implication of the availability of specific codes for most of the frequently encountered health problems in the study setting is that it favours its application in the African primary care context, while also offering the possibility of optional hierarchical expansion from ICPC-2 to ICD-10 via the mapping. This allows relevant codes from the ICD-10 to be incorporated into state- or country-specific adaptations of the ICPC-2. Acknowledgements We appreciate the inputs of Dr. Abiodun Adewuya at the conceptualization stage and Dr. Daniel Pinto at the data collection stage. References 1. Bentzen N (ed.). Wonca Dictionary of General/Family Practice . Copenhagen : Maanedsskrift for PraktiskLaegegerning , 2003 . 2. de Lusignan S , van Weel C . The use of routinely collected computer data for research in primary care: opportunities and challenges . Fam Pract 2006 ; 23 : 253 – 63 . Google Scholar CrossRef Search ADS PubMed 3. de Andrade CT , Magedanz AM , Escobosa DM , et al. The importance of a database in the management of healthcare services . Einstein (Sao Paulo) 2012 ; 10 : 360 – 5 . Google Scholar CrossRef Search ADS PubMed 4. Adeshina Y. The Effect of Health Sector Reform on Health Care; The Revitalization Program for the Primary Health Care System . http://www.ihf-fih.org ( accessed on June 2013 ). 5. Britt H , Miller GC , Charles J , et al. (eds). A Decade of Australian General Practice Activity 2002–03 to 2011–12 . General Practice Series No. 32. Sydney : Sydney University Press , 2012 . 6. World Health Organization . Family of International Classifications . Geneva : WHO , 2004 . http://www.who.int/classifications/en/WHOFICFamily.pdf ( accessed on 21 September 2012 ). PubMed PubMed 7. Hofmans-Okkes IM , Lamberts H . The international classification of primary care (ICPC): new applications in research and computer-based patient records in family practice . Fam Pract 1996 ; 13 : 294 – 302 . Google Scholar CrossRef Search ADS PubMed 8. deLusignan S . The optimum granularity for coding diagnostic data in primary care: report of a workshop of the EFMI primary care informatics working group at MIE 2005 . Inform Prim Care 2006 ; 14 : 33 – 7 . 9. Becker HW , Oskam SK , Okkes IM , et al. ICPC2-ICD10 Thesaurus. A diagnostic terminology for semi-automatic double coding in electronic patient records . In: Okkes IM , Oskam SK , Lambert H (eds). ICPC in the Amsterdam Transition Project. CD-Rom . Amsterdam : Academic Medical Center/University of Amsterdam Department of Family Medicine , 2005 . 10. Classification Committee of the World Organization of Family doctors . ICPC-2: International Classification of Primary Care . 2nd edn . Oxford : Oxford University Press , 1998 , p 3 . 11. Lamberts H , Meads S , Wood M . Results of the international field trial with the reason for encounter classification . In: Cote RA , Protti AJ , Scherner JR (eds). Role of Informatics in Health Data Coding and Classification Systems . Amsterdam : Elsevier Scientific Publications , 1985 . Google Scholar CrossRef Search ADS 12. Britt H . A measure of the validity of the international classification of primary care in the classification of reasons for encounter . J Innov Health Inform 1997 ; 8 – 12 . 13. Anastasi A , Urbina S. Psychological Testing . 10th edn . New Delhi : Prentice-Hall , 2014 . 14. Olagundoye OA , van Boven K , van Weel C . International classification of primary care-2 coding of primary care data at the general out-patients’ clinic of General Hospital, Lagos, Nigeria . J Family Med Prim Care 2016 ; 5 : 291 – 7 . Google Scholar CrossRef Search ADS PubMed 15. Ayankogbe OO , Oyediran MA , Oke DA , et al. ICPC-2 defined pattern of illnesses in an urban region in West Africa . Afr J Prim Health Care Fam Med 2009 ; 1 : 103 – 6 . Google Scholar CrossRef Search ADS 16. Brueton V , Yogeswaran P , Chandia J , et al. Primary care morbidity in Eastern Cape Province . S Afr Med J 2010 ; 100 : 309 – 12 . Google Scholar CrossRef Search ADS PubMed 17. Rahman SM , Angeline RP , Cynthia S , et al. International classification of primary care: an Indian experience . J Family Med Prim Care 2014 ; 3 : 362 – 7 . Google Scholar PubMed 18. Letrilliart L , Guiguet M , Flahault A . Reliability of report coding of hospital referrals in primary care versus practice-based coding . Eur J Epidemiol 2000 ; 16 : 653 – 9 . Google Scholar CrossRef Search ADS PubMed 19. Yamane T. Statistics, An Introductory Analysis . 2nd edn . New York : Harper and Row , 1967 , pp. 159 – 60 . 20. Landis JR , Koch GG . The measurement of observer agreement for categorical data . Biometrics 1977 ; 33 : 159 – 74 . Google Scholar CrossRef Search ADS PubMed 21. Britt H . Reliability of central coding of patient reasons for encounter in general practice, using the international classification of primary care . J Innov Health Inform 1998 ; 7 ( 1 ): 210 . Google Scholar CrossRef Search ADS 22. Luna D , de Quirós FGB , Garfi L , et al. Reliability of secondary central coding of medical problems in primary care by non-medical coders, using the international classification of primary care (ICPC) . Medinfo 2001 ; 10 : 300 . 23. Frese T , Herrmann K , Bungert-Kahl P , et al. Inter-rater reliability of the ICPC-2 in a German general practice setting . Swiss Med Wkly 2012 . http://www.smw.ch/content/smw-2012–13621/ ( accessed on 22 August 2012 ) 24. Pinto D , Corte-Real S . Coding with the international classification of primary care by family medicine residents . Rev Port Clin Geral 2010 ; 26 : 370 – 82 . © The Author(s) 2018. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Journal

Family PracticeOxford University Press

Published: Jan 17, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off