Implementation of the Autovalidation Algorithm for Clinical Chemistry Testing in the Laboratory Information System

Implementation of the Autovalidation Algorithm for Clinical Chemistry Testing in the Laboratory... Abstract Objective Autovalidation algorithm should be properly designed with clearly defined criteria and any data that do not meet the criteria, must be reviewed and manually validated. The aim was to define the rules for autovalidation in our laboratory information system (LIS), and validate the algorithm prior to its implementation in routine laboratory work. Methods Autovalidation was implemented for all routine serum biochemistry tests. The algorithm included analytical measurement ranges (AMR), delta check, critical values, serum indices and all preanalytical and analytical flags from the analyzer. Results In the validation process 9805 samples were included, and 78.3% (7677) of all samples were autovalidated. The highest percentage of non-validated samples (54.9%) refers to those with at least one result outside the method linearity ranges (AMR criteria) while critical values were observed to be the least frequent criterion for stopping autovalidation (1.8%). Also, 38 samples were manually validated as they failed to meet the autovalidation criteria. Conclusion Implementation of algorithm for autovalidation in our institution resulted in the redesign of the existing LIS. This model of the autovalidation algorithm significantly decreased the number of manually validated test results and can be used as a model for introducing autovalidation in other laboratory settings. autovalidation algorithm, laboratory information system, serum biochemistry tests, validation, postanalytical phase, laboratory optimization Autovalidation is a rule-based system for validation of laboratory test results based on a selection of predefined rules in the autovalidation algorithm.1-5 Autovalidation automatically releases test results without any manual intervention and additional effort from laboratory staff if those results meet the criteria of all defined rules in the algorithm. Introduction of an automated tool for validation of laboratory results of this kind is essential in the present-day automated laboratory environment, due to an increasing number of laboratory test requests, the possibility of errors in the case of manual reporting of test results, and the increasing need for reducing test-reporting turnaround time (TAT).6 An autovalidation algorithm can be implemented in the laboratory information system (LIS),1-3 can function as an independent validation program,6-9 or can be introduced in the middleware software that interconnects LIS and laboratory analyzers.4,10 Proper design of the autovalidation algorithm and definition of autovalidation rules are prerequisites for algorithm implementation.1-5 Because autovalidation ensures uniform and objective validation of laboratory test results based on predefined rules, all selected rules must be clearly and unambiguously defined.2 Autovalidation rules commonly include analytical measurement ranges (AMRs)4; interference indices for hemolysis, icterus, and lipemia11; preanalytical and analytical flags generated from the analyzer4,5,9; reference ranges and decision limits4,8; critical values4,8,9; delta check4-6,8,9,12; quality-control (QC) results4,5,8; moving averages; and lot checks.5 The sole technical implementation of autovalidation rules in the LIS can vary and can even be limited by the capabilities of the LIS software used (software upgrades are often required). Decisions regarding the extent of autovalidation rules and regarding the criteria that are to be implemented depend on the specificity of the laboratory setting and the complexity of medical care performed in the institution; thus, these decisions must not be generalized. The criteria for AMRs and interference indices are usually defined according to the criteria defined by the reagent manufacturer and/or data obtained during method validation. Reference ranges, decision limits, and criteria for critical values can be population-based, implemented according to relevant published guidelines and recommendations, and/or designed to accommodate current trends. The rules regarding analyzer and reagent performance (error flags, QC, lot checks) are analyzer-/reagent-specific and depend mainly on the characteristics of those instruments. The delta-check rule serves for comparison of the current test result with the previous one for the same patient in a defined time frame. Delta check can be defined in several forms, namely, as delta percent change (the most commonly used form), delta difference, rate percent change, and rate difference.13 Some authors12 recommend that delta check should be expressed as the reference change value (RCV) of each selected laboratory test. The RCV is calculated using the data from analytical control results and intraindividual biological coefficient of variation.12,14 After definition of autovalidation rules and before implementation of autovalidation in routine laboratory practice, according to Clinical and Laboratory Standards Institute (CLSI) guidelines5 and best laboratory practice, it is necessary to validate the autovalidation algorithm.2,4,10 Further, with every change of autovalidation rules and/or criteria, the autovalidation algorithm should be revalidated.2 In this way, potential pitfalls can be detected, and the release of erroneous laboratory results can be prevented. At our institution, the field of general clinical chemistry has been identified as the most suitable for the first implementation of the autovalidation algorithm, due to the large number of specimens being processed on a daily basis and the lack of an adequate amount of personnel. Although some published articles3,4,15,16 describe experiences in implementing autovalidation, such as in clinical chemistry, coagulation, and hematology, none of these articles focus on the detailed design of the autovalidation algorithm for clinical-chemistry tests in the LIS. For that reason, we established our own algorithm and herein, we present it as a raw model that can be applicable to any setting and modified according to the specific requirements of the institution. In this study, we aimed to define the rules of the autovalidation algorithm for serum biochemistry tests and subsequently to validate the algorithm before its implementation in routine laboratory work. Materials and Methods Study Setting The study was conducted at the Department of Laboratory Diagnostics, University Hospital Center, Zagreb, Croatia. This department is composed of 13 divisions and performs approximately 3.6 million tests per year, including highly specialized and differential diagnostic tests for inpatients and outpatients. The Division for General Clinical Chemistry processes approximately 550 specimens per day, which corresponds to 6500 laboratory test results per day. The clinical chemistry automation line, where all tests are performed, includes a Cobas Modular preanalytics system, followed by 3 Cobas c501 biochemistry analyzers (F. Hoffman-La Roche Ltd). After specimen analysis, the results and respective flags are transferred from the analyzer to the LIS (BioNET LIS, IN2 Health Sector). The Cobas c501 analyzers are interfaced to the LIS via a middleware Process System Manager (F. Hoffman-La Roche Ltd). Implementation of Autovalidation Autovalidation for all routine serum biochemistry tests was implemented in August 2014. An autovalidation algorithm that consists of autovalidation rules was elaborated in our LIS database as a new functionality. The technical characteristics of this new feature were developed by information technology (IT) specialists. However, laboratory specialists decided which specific items and rules would be included in the autovalidation algorithm, based on the requisites of the laboratory. After that, the chosen criteria for autovalidation rules for all included biochemistry tests were applied and tested appropriately, as described later herein. As a result, a background algorithm that combines all the chosen rules for autovalidation was developed by IT specialists. The rules included in our autovalidation algorithm were as follows: AMRs, delta check, critical values, serum indices (hemolysis, lipemia, icteria), and all preanalytical flags (eg, specimen short, specimen clot, specimen bubbles) and analytical flags (eg, prozone effect, reagent short, expired reagent, mechanical errors, autodilution) from the analyzer. Autovalidation rules for all analytes are listed in Table 1. Table 1. List of All Biochemistry Tests With Specific Criteria Included in Autovalidation Rules. Analytical Measurement Ranges (AMR) Critical Values Interference Indices (mg/dL) Delta Check (%) Test Name Unit Low Value High Value Low Critical High Critical Lipemia Icterus Hemolysis Alanine aminotransferase (ALT) U/L 5 700 NA 1000 150 60 700 60 Alkaline phosphatase U/L 5 1200 NA NA 2000 60 200 30 Amylase U/L 3 1500 NA NA 1500 60 500 100 Angiotensin converting enzyme (ACE) U/L 5 100 NA NA 100 10 30 NA Aspartate aminotransferase (AST) U/L 5 700 NA 1000 150 60 40 60 Bilirubin direct µmol/L 2 291 NA NA 750 0 20 30 Bilirubin total µmol/L 2 650 NA 257 300 0 50 50 C reactive protein (CRP) mg/L 0 350 NA NA 1000 60 1000 500 Calcium mmol/L 0.1 5.0 1.65 3.50 2000 60 1000 10 Chloride mmol/L 60 140 75 125 2000 60 100 10 Cholesterol mmol/L 0.1 20.7 NA NA 2000 16 700 35 Copper mg/L 12 25 NA NA 100 5 30 NA Creatine kinase (CK) U/L 7 2000 NA NA 1000 60 80 300 Creatine kinase MB (CK-MB) U/L 3 500 NA NA 200 20 30 300 Creatinine mmol/L 10 2226 NA NA 800 5 1000 40 Gamma glutamyl transferase (GGT) U/L 3 1200 NA NA 1500 20 200 60 Glucose mmol/L 0.1 41.6 2.5 27.8 1000 60 1000 50 HDL cholesterol mmol/L 0.08 3.12 NA NA 1800 60 1200 20 Iron µmol/L 0.9 179 NA NA 1500 60 200 25 Lactate dehydrogenase (LDH) U/L 0 1000 NA 500 1500 60 30 300 LDL cholesterol mmol/L 0.1 14.2 NA NA 200 60 1000 20 Lipase U/L 3 300 NA 700 2000 50 1000 100 Magnesium mmol/L 0.1 2.5 0.41 5.00 1700 60 30 30 Phosphate mmol/L 0.1 6.5 0.32 2.90 1250 40 300 50 Potassium mmol/L 1.5 10 2.8 6.2 2000 60 30 20 Protein (total) g/L 2 120 NA NA 2000 20 1000 25 Sodium mmol/L 80 180 120 160 2000 60 100 5 Triglyceride mmol/L 0,1 10 NA NA 2000 35 700 70 Unsaturated iron binding capacity (UIBC) µmol/L 3 125 NA NA 300 60 40 25 Urea mmol/L 0.5 40.0 NA 35.6 1000 60 1000 60 Uric acid µmol/L 11.9 1487 NA 773 1500 40 1000 25 Analytical Measurement Ranges (AMR) Critical Values Interference Indices (mg/dL) Delta Check (%) Test Name Unit Low Value High Value Low Critical High Critical Lipemia Icterus Hemolysis Alanine aminotransferase (ALT) U/L 5 700 NA 1000 150 60 700 60 Alkaline phosphatase U/L 5 1200 NA NA 2000 60 200 30 Amylase U/L 3 1500 NA NA 1500 60 500 100 Angiotensin converting enzyme (ACE) U/L 5 100 NA NA 100 10 30 NA Aspartate aminotransferase (AST) U/L 5 700 NA 1000 150 60 40 60 Bilirubin direct µmol/L 2 291 NA NA 750 0 20 30 Bilirubin total µmol/L 2 650 NA 257 300 0 50 50 C reactive protein (CRP) mg/L 0 350 NA NA 1000 60 1000 500 Calcium mmol/L 0.1 5.0 1.65 3.50 2000 60 1000 10 Chloride mmol/L 60 140 75 125 2000 60 100 10 Cholesterol mmol/L 0.1 20.7 NA NA 2000 16 700 35 Copper mg/L 12 25 NA NA 100 5 30 NA Creatine kinase (CK) U/L 7 2000 NA NA 1000 60 80 300 Creatine kinase MB (CK-MB) U/L 3 500 NA NA 200 20 30 300 Creatinine mmol/L 10 2226 NA NA 800 5 1000 40 Gamma glutamyl transferase (GGT) U/L 3 1200 NA NA 1500 20 200 60 Glucose mmol/L 0.1 41.6 2.5 27.8 1000 60 1000 50 HDL cholesterol mmol/L 0.08 3.12 NA NA 1800 60 1200 20 Iron µmol/L 0.9 179 NA NA 1500 60 200 25 Lactate dehydrogenase (LDH) U/L 0 1000 NA 500 1500 60 30 300 LDL cholesterol mmol/L 0.1 14.2 NA NA 200 60 1000 20 Lipase U/L 3 300 NA 700 2000 50 1000 100 Magnesium mmol/L 0.1 2.5 0.41 5.00 1700 60 30 30 Phosphate mmol/L 0.1 6.5 0.32 2.90 1250 40 300 50 Potassium mmol/L 1.5 10 2.8 6.2 2000 60 30 20 Protein (total) g/L 2 120 NA NA 2000 20 1000 25 Sodium mmol/L 80 180 120 160 2000 60 100 5 Triglyceride mmol/L 0,1 10 NA NA 2000 35 700 70 Unsaturated iron binding capacity (UIBC) µmol/L 3 125 NA NA 300 60 40 25 Urea mmol/L 0.5 40.0 NA 35.6 1000 60 1000 60 Uric acid µmol/L 11.9 1487 NA 773 1500 40 1000 25 NA, not available View Large Table 1. List of All Biochemistry Tests With Specific Criteria Included in Autovalidation Rules. Analytical Measurement Ranges (AMR) Critical Values Interference Indices (mg/dL) Delta Check (%) Test Name Unit Low Value High Value Low Critical High Critical Lipemia Icterus Hemolysis Alanine aminotransferase (ALT) U/L 5 700 NA 1000 150 60 700 60 Alkaline phosphatase U/L 5 1200 NA NA 2000 60 200 30 Amylase U/L 3 1500 NA NA 1500 60 500 100 Angiotensin converting enzyme (ACE) U/L 5 100 NA NA 100 10 30 NA Aspartate aminotransferase (AST) U/L 5 700 NA 1000 150 60 40 60 Bilirubin direct µmol/L 2 291 NA NA 750 0 20 30 Bilirubin total µmol/L 2 650 NA 257 300 0 50 50 C reactive protein (CRP) mg/L 0 350 NA NA 1000 60 1000 500 Calcium mmol/L 0.1 5.0 1.65 3.50 2000 60 1000 10 Chloride mmol/L 60 140 75 125 2000 60 100 10 Cholesterol mmol/L 0.1 20.7 NA NA 2000 16 700 35 Copper mg/L 12 25 NA NA 100 5 30 NA Creatine kinase (CK) U/L 7 2000 NA NA 1000 60 80 300 Creatine kinase MB (CK-MB) U/L 3 500 NA NA 200 20 30 300 Creatinine mmol/L 10 2226 NA NA 800 5 1000 40 Gamma glutamyl transferase (GGT) U/L 3 1200 NA NA 1500 20 200 60 Glucose mmol/L 0.1 41.6 2.5 27.8 1000 60 1000 50 HDL cholesterol mmol/L 0.08 3.12 NA NA 1800 60 1200 20 Iron µmol/L 0.9 179 NA NA 1500 60 200 25 Lactate dehydrogenase (LDH) U/L 0 1000 NA 500 1500 60 30 300 LDL cholesterol mmol/L 0.1 14.2 NA NA 200 60 1000 20 Lipase U/L 3 300 NA 700 2000 50 1000 100 Magnesium mmol/L 0.1 2.5 0.41 5.00 1700 60 30 30 Phosphate mmol/L 0.1 6.5 0.32 2.90 1250 40 300 50 Potassium mmol/L 1.5 10 2.8 6.2 2000 60 30 20 Protein (total) g/L 2 120 NA NA 2000 20 1000 25 Sodium mmol/L 80 180 120 160 2000 60 100 5 Triglyceride mmol/L 0,1 10 NA NA 2000 35 700 70 Unsaturated iron binding capacity (UIBC) µmol/L 3 125 NA NA 300 60 40 25 Urea mmol/L 0.5 40.0 NA 35.6 1000 60 1000 60 Uric acid µmol/L 11.9 1487 NA 773 1500 40 1000 25 Analytical Measurement Ranges (AMR) Critical Values Interference Indices (mg/dL) Delta Check (%) Test Name Unit Low Value High Value Low Critical High Critical Lipemia Icterus Hemolysis Alanine aminotransferase (ALT) U/L 5 700 NA 1000 150 60 700 60 Alkaline phosphatase U/L 5 1200 NA NA 2000 60 200 30 Amylase U/L 3 1500 NA NA 1500 60 500 100 Angiotensin converting enzyme (ACE) U/L 5 100 NA NA 100 10 30 NA Aspartate aminotransferase (AST) U/L 5 700 NA 1000 150 60 40 60 Bilirubin direct µmol/L 2 291 NA NA 750 0 20 30 Bilirubin total µmol/L 2 650 NA 257 300 0 50 50 C reactive protein (CRP) mg/L 0 350 NA NA 1000 60 1000 500 Calcium mmol/L 0.1 5.0 1.65 3.50 2000 60 1000 10 Chloride mmol/L 60 140 75 125 2000 60 100 10 Cholesterol mmol/L 0.1 20.7 NA NA 2000 16 700 35 Copper mg/L 12 25 NA NA 100 5 30 NA Creatine kinase (CK) U/L 7 2000 NA NA 1000 60 80 300 Creatine kinase MB (CK-MB) U/L 3 500 NA NA 200 20 30 300 Creatinine mmol/L 10 2226 NA NA 800 5 1000 40 Gamma glutamyl transferase (GGT) U/L 3 1200 NA NA 1500 20 200 60 Glucose mmol/L 0.1 41.6 2.5 27.8 1000 60 1000 50 HDL cholesterol mmol/L 0.08 3.12 NA NA 1800 60 1200 20 Iron µmol/L 0.9 179 NA NA 1500 60 200 25 Lactate dehydrogenase (LDH) U/L 0 1000 NA 500 1500 60 30 300 LDL cholesterol mmol/L 0.1 14.2 NA NA 200 60 1000 20 Lipase U/L 3 300 NA 700 2000 50 1000 100 Magnesium mmol/L 0.1 2.5 0.41 5.00 1700 60 30 30 Phosphate mmol/L 0.1 6.5 0.32 2.90 1250 40 300 50 Potassium mmol/L 1.5 10 2.8 6.2 2000 60 30 20 Protein (total) g/L 2 120 NA NA 2000 20 1000 25 Sodium mmol/L 80 180 120 160 2000 60 100 5 Triglyceride mmol/L 0,1 10 NA NA 2000 35 700 70 Unsaturated iron binding capacity (UIBC) µmol/L 3 125 NA NA 300 60 40 25 Urea mmol/L 0.5 40.0 NA 35.6 1000 60 1000 60 Uric acid µmol/L 11.9 1487 NA 773 1500 40 1000 25 NA, not available View Large AMR criteria were set according to the method linearity ranges defined by the manufacturer and verified in the laboratory through assay validation, according to International Organization for Standardization (ISO) 15189.17 Also, we used serum indices for all tests, as taken from the insert package sheets of each reagent. Critical values were introduced in concordance with International Federation of Clinical Chemistry (IFCC) published recommendations.18 Those values were equal to the critical values established by the Croatian Chamber of Medical Biochemists.19 Delta-check values were determined according to the respective RCV value for each test, and the time frame was set as 5 days for all tests. The 5-day time frame was chosen to include various patient populations (pediatric and adult intensive care patients, hematology and oncology patients, etc) in our hospital and also the repeated measurements for those patients. RCV values were calculated according to the following formula: RCV=(2)1/2*Zp*(CV2A+CV2I)1/2 where Zp is the standard deviation for the appropriate probability of error (95% Zp = 1.96), CVA is the analytical coefficient of variation, and CVI is the intraindividual biological coefficient of variation. Futher, delta-check values were slightly modified, taking in consideration the patient population in our institution and in agreement with laboratory specialists involved in the design of the autovalidation algorithm. Autovalidation of completed tests can be achieved only when all the specimen results are transferred from the analyzer to the LIS. Autovalidation must be triggered by selecting the autovalidation icon. After this step, all test results are checked through all the defined rules from the algorithm. All rules have the same level of importance—for instance, if 1 or more tests do not meet the defined rules for any specimen, the results are not verified automatically. All nonautoverified test results (with various flags) remain for review and manual validation by a laboratory specialist, according to standard operational procedures usually used in routine practice (Figure 1). Figure 1 View largeDownload slide Autovalidation flowchart. AMR indicates analytical measurement range. Figure 1 View largeDownload slide Autovalidation flowchart. AMR indicates analytical measurement range. Validation of Autovalidation Algorithm Before the introduction of autovalidation in routine practice, validation of the autovalidation algorithm was performed according to CLSI guideline AUTO10-A.5 In the validation process, the results from chosen biochemistry specimens were assessed to check the validity of autovalidation rules. Validation was performed within the Unit for General Clinical Chemistry discontinuously on working days during a 3-month period (May 2014–July 2014). Rules were tested in a test version of our LIS and therefore served solely for validation purposes and not for reporting of laboratory results to health care professionals. Completed test results, which at that time were all manually validated by laboratory specialists, were downloaded on a daily basis from the LIS to its test version, and manually reviewed by laboratory specialists according to the defined autovalidation rules. Specimens that did not meet all the criteria for autovalidation and had to be manually validated were noted in the records. Afterward, autovalidation was triggered, and we verified whether all the remaining specimens for manual validation were in concordance with the results of the manual review. Statistical Analysis The results from our study are presented as absolute numbers and as percentage of autovalidated test results in the total number of tests performed. We archived and processed data using Microsoft Excel, version 2010 (Microsoft Corporation). Results In the validation process, a total of 9805 specimens were included, of which 78.3% (7677) of all specimens were autovalidated according to the established algorithm. The highest percentage of nonvalidated specimens (54.9%) was made up of those with at least 1 result outside the method linearity ranges (AMR criteria). In contrast, critical values were observed to be the least frequent criterion for stopping autovalidation (1.8%) (Figure 2). Figure 2 View largeDownload slide The rate of autovalidation stopping criteria. Figure 2 View largeDownload slide The rate of autovalidation stopping criteria. Specifically, AMR criteria caused the most frequently autovalidation stoppage for sodium considering the low AMR limit, and for potassium and blood urea nitrogen (BUN) regarding the high AMR limit. The second most common stopping criteria referred to delta check, which mostly affected potassium and creatinine results. Moreover, hemolysis, as the most frequent criteria among serum indices, caused autovalidation failure for lactate dehydrogenase (LDH) and aspartate aminostransferase (AST). Comparison between manual validation and autovalidation showed consistent results for 99.5% of specimens. In this process, 38 specimens were manually validated because they failed to meet the autovalidation criteria. The obtained difference was due to the existence of written remarks next to test results. These comments were not defined as criteria for stopping autovalidation but were laboratory-specific remarks (ie, predefined comments about the analytical methods used, as well as short notes to healthcare professionals). However, these comments were recognized as such and caused stopping of autovalidation because they were placed in the same field as flags from the analyzer. When noticed during validation, this pitfall was immediately corrected. For this reason, only preanalytical and analytical flags from the analyzer in the fields for comments should be recognized as autovalidation-stopping criteria. Discussion In this study, we defined an autovalidation algorithm for our LIS and applied it to the automated biochemistry Roche system, which showed almost 100% concordance with manual validation. The autovalidation algorithm consisting of predefined rules was elaborated and introduced in the LIS, and validation of the algorithm was performed before its implementation in routine practice. This was the first experience regarding implementation of autovalidation not only in our laboratory but throughout Croatia. Our incentive to implement autovalidation resulted in the development of new functionality in the existing LIS. Close cooperation between laboratory and IT specialists was essential in designing the autovalidation functionality.4 Being the first group to test the operation of the autovalidation feature, we decided to validate the autovalidation algorithm on a large number of specimens because, to our knowledge, no guidelines define the optimal number of specimens that should be included in the validation process. This decision was made after thorough discussion among laboratory specialists in our laboratory. The rate of autovalidated results in our study was 78.3%. Published data show great variations in the rates of autovalidated results, which depend on the setting and implemented autovalidation rules. Krasowski et al4 demonstrated in their study that initial implementation of autovalidation rules in middleware yielded 95% autovalidated results, followed by 99% and 99.5% rates in the years 2010 and 2013, respectively. Autovalidation rules in this study included manual-review limits, critical values, delta check, interferences indices, AMR, autoextended range, instrument error flags, and QC. Oosterhuis et al6 report that the total percentage of test results validated by an automated validation system in their laboratory was 86.6%. Further, some studies display the percentage of the autovalidation passing rate in the daily laboratory routine (eg, ranging from 92% to 95% in Shih et al10). Their algorithm included limit check, delta check, critical values, and consistency-check rules. A potential explanation for the lower rate of autovalidated results observed in our study is tightened stopping rules as a result of the first-ever implementation of autovalidation algorithm in our laboratory. However, these rules will surely be modified over time according to accumulated experience, resulting in a higher percentage of autovalidated results. Beside studies that showed higher percentages, there are also studies with lower percentages of autovalidation passing rate. Torke et al1 show in their study results that 62% of their chemistry test panels and 73% of single- analyte assays were autovalidated. The largest number of specimens for which processing was stopped by autovalidation and remained for manual revision in our study are the ones that did not meet the AMR criteria, with results below or above the limits of the determined range. However, the number of specimens stopped by the AMR rule could be decreased by extending the AMR range. For introduction of such modified AMR ranges, it is necessary to perform additional assay validation, to test the linearity of all assays outside the recommended linearity range by the manufacturer and, based on the obtained results, to modify these criteria accordingly. The second most frequent stopping criterium was the delta-check rule, which covers one quarter of all specimens for which processing was stopped. It is important to emphasize that the delta-check rule was automatically satisfied if no previous data existed for comparison (ie, in cases of the first visit of a patient to the laboratory). Delta check is colloquially considered to be a useful tool, not only in the context of identifying clinically significant changes in laboratory parameters but also in recognizing cases of patient misidentification and specimen mix-ups. Nowadays, there is controversy regarding the usefulness of delta checking. Although it is a classic laboratory QC tool, there are limited published data regarding the usefulness of delta checking for detecting problems other than specimen-mislabeling errors. However, available data show that delta checking is mostly used as part of autovalidation procedures.20 We are aware that including this rule may cause additional stoppage of autovalidation and may generate more manual work. However, we did not hesitate to include it because it proved to be important in detecting not only patient misidentification but also deterioration of laboratory results that should be immediately communicated to health care professionals, so that patients can receive immediate medical attention. Our data showed that testing for potassium and creatinine are usually stopped in the autovalidation process, probably due to a large number of patient undergoing dialysis, in whom fluctuations of those 2 analytes are expected. Among serum indices, hemolysis was the most frequent, whereas lipemia was less frequent in triggering the rule for blocking autovalidation. These data are consistent with a large number of articles/published data that show that hemolysis is still a challenging preanalytical issue.21 Finally, critical values and flags from the analyzer criteria caused stopping of autovalidation for the least number of nonautovalidated specimens. We also considered it important to review and evaluate differences between the number of manually validated specimens without additional interventions performed by laboratory specialists and the number of autovalidated specimens in the preimplementation period. A similar validation approach was applied for evaluation of an automated validation system.6 In our study the disagreement in testing-phase results was observed in 38 specimens that were manually validated but according to the implemented rules should be autovalidated. An observed pitfall in the algorithm was corrected before implementation of autovalidation in the routine work. This observation clearly confirms the importance of verifying the autovalidation algorithm before its use in routine work. This apparently small number of specimens indicated pitfalls in the algorithm that were subsequently corrected. It is important to emphasize that misidentification of laboratory-specific remarks as autovalidation-stopping rules was the only pitfall of the designed algorithm observed in the validation process. This observation clearly confirmed the importance of validating the autovalidation algorithm before its use in routine work. Still, it can be considered a minor fallacy because it prevented autovalidation, rather than inappropriately causing autovalidation and subsequent release of potentially erroneous results needing manual review. At our institution, we believe that the first introduction of autovalidation should be implemented with caution because, as mentioned earlier herein, this was also the first attempt to implement autovalidation in our LIS. For that reason, we believe that autovalidation that should be initiated by selecting an icon, rather that real-time autovalidation, was a reasonable approach with which to start. However, further steps should include an upgrade to real-time autovalidation, which inevitably means the introduction of additional criteria, primarily, inclusion of QC rules in the autovalidation algorithm.2 This study has some limitations. First, the stopping rules for delta check and critical values were not divided by sex and age. Second, the aforementioned rules were not adapted to the patient population of different hospital wards but, rather, were used uniformly. These limitations can cause a lower percentage of autovalidated results and they can be avoided with adjustment of these criteria to the patient population of a specific ward (eg, intensive care unit, pediatric care). During the implementation of the autovalidation process, we reviewed only data from the LIS without simulation of specimens with, for example, high serum indices. However, because our study verified almost 10,000 specimens from various hospital patients, we consider that implemented rules are reliable. Nevertheless, we believe that our study results present a valuable improvement in the postanalytical phase in our laboratory, contributing to a more efficient and objective reporting of test results. Also, they present a systematic approach for introduction of an autovalidation algorithm through the LIS with general requirements, which can be useful as a model for its implementation in any laboratory setting and any analytical system used. However, adjustment of the algorithm rules to the specific laboratory setting is a prerequisite for successful implementation. Autovalidation has numerous benefits, including shortening of the TAT; reduction in the number of results for manual revision and, consequently, a greater focus on potentially problematic patient specimens; and uniform test reporting that is less subject to subjective variations.2 However, autovalidation is still only a computerized tool that works according to defined rules, which surely enhances laboratory efficiency but cannot completely replace laboratory specialist work in the process of reporting laboratory results and decision making. Conclusion Our initiative to implement autovalidation resulted in the redesign of the existing LIS. The model of the autovalidation algorithm described herein proved to decrease the number of manually validated test results and can be applied to different laboratory settings. Abbreviations TAT turnaround time LIS laboratory information system AMRs analytical measurement ranges QC quality-control RCV reference change value CLSI Clinical and Laboratory Standards Institute IT information technology ISO International Organization for Standardization IFCC International Federation of Clinical Chemistry BUN blood urea nitrogen LDH lactate dehydrogenase AST aspartate aminostransferase NA not applicable ALP alkaline phosphatase ALT alanine aminotransferase NA not available ALP alkaline phosphatase ACE angiotensin-converting enzyme CRP C-reactive protein CK creatine kinase CK-MB creatine kinase GGT gamma-glutamyl transferase HDL high-density lipoprotein LDH lactate dehydrogenase LDL low-density lipoprotein UIBC unsaturated iron-binding capacity. References 1. Torke N , Boral L , Nguyen T , Perri A , Chakrin A . Process improvement and operational efficiency through test result autoverification . Clin Chem . 2005 ; 51 ( 12 ): 2406 – 2408 . 2. Duca DJ . Autoverification in a laboratory information system . Lab Med . 2002 ; 33 : 21 – 25 . 3. Onelöv L , Gustafsson E , Grönlund E , et al. Autoverification of routine coagulation assays in a multi-center laboratory . Scand J Clin Lab Invest . 2016 ; 76 ( 6 ): 500 – 502 . 4. Krasowski MD , Davis SR , Drees D , et al. Autoverification in a core clinical chemistry laboratory at an academic medical center . J Pathol Inform . 2014 ; 5 ( 1 ): 13 . 5. Clinical and Laboratory Standard Institute (CLSI) . AUTO 10-A . Wayne, PA : CLSI ; 2006 . 6. Oosterhuis WP , Ulenkate HJ , Goldschmidt HM . Evaluation of LabRespond, a new automated validation system for clinical laboratory test results . Clin Chem . 2000 ; 46 ( 11 ): 1811 – 1817 . 7. Kuijsters JP . Lab respond: a tool for autoverification . Accred Qual Assur . 2002 ; 7 : 477 – 479 . 8. Prost L , Rogari P . How autoverification through the expert system VALAB can make your laboratory more efficient . Accred Qual Assur . 2002 ; 7 : 480 – 487 . 9. Guidi GC , Poli G , Bassi A , Giobelli L , Benetollo PP , Lippi G . Development and implementation of an automatic system for verification, validation and delivery of laboratory test results . Clin Chem Lab Med . 2009 ; 47 ( 11 ): 1355 – 1360 . 10. Shih MC , Chang HM , Tien N , et al. Building and validation an autoverification system in the clinical chemistry laboratory . Lab Med . 2011 ; 42 : 668 – 673 . 11. Vermeer HJ , Thomassen E , de Jonge N . Automated processing of serum indices used for interference detection by the laboratory information system . Clin Chem . 2005 ; 51 ( 1 ): 244 – 247 . 12. Fraser CG , Stevenson HP , Kennedy IMG . Biological variation data are necessary prerequisites for objective autoverification of clinical laboratory data . Accred Qual Assur . 2002 ; 7 : 455 – 460 . 13. Park SH , Kim SY , Lee W , Chun S , Min WK . New decision criteria for selecting delta check methods based on the ratio of the delta difference to the width of the reference range can be generally applicable for each clinical chemistry test item . Ann Lab Med . 2012 ; 32 ( 5 ): 345 – 354 . 14. Ricós C , Cava F , García-Lario JV , et al. The reference change value: a proposal to interpret laboratory reports in serial testing based on biological variation . Scand J Clin Lab Invest . 2004 ; 64 ( 3 ): 175 – 184 . 15. Froom P , Saffuri-Elias E , Barak M . Autovalidation rates in an outpatient coagulation laboratory . Int J Lab Hematol . 2015 ; 37 ( 5 ): 680 – 685 . 16. Froom P , Barak M . Auto-validation of complete blood counts in an outpatient’s regional laboratory . Clin Chem Lab Med . 2015 ; 53 ( 2 ): 275 – 279 . 17. International Organization for Standardization . ISO 15189:2012. Medical laboratories: Particular requirements for quality and competence. 18. Thomas L . Crtical limits of laboratory results for urgent clinical notification . eJIFCC . 2003 ; 14(1) . http://www.ifcc.org/ejifcc/vol14no1/140103200303.pdf. Accessed December 26, 2017. 19. Critical values of laboratory results and reporting of critical values- 2nd edition . http://www.hkmb.hr/dokumenti/povjerenstva/HKMB%20PPSP%208.pdf. Accessed December 26, 2017 . 20. Schifman RB , Talbert M , Souers RJ . Delta check practices and outcomes: a Q-probes study involving 49 health care facilities and 6541 delta check alerts . Arch Pathol Lab Med . 2017 ; 141 ( 6 ): 813 – 823 . 21. Plebani M , Sciacovelli L , Aita A , Chiozza ML . Harmonization of pre-analytical quality indicators . Biochem Med (Zagreb) . 2014 ; 24 ( 1 ): 105 – 113 . © American Society for Clinical Pathology 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Laboratory Medicine Oxford University Press

Implementation of the Autovalidation Algorithm for Clinical Chemistry Testing in the Laboratory Information System

Loading next page...
 
/lp/ou_press/implementation-of-the-autovalidation-algorithm-for-clinical-chemistry-Ht02bLQ0BU
Publisher
Oxford University Press
Copyright
© American Society for Clinical Pathology 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com
ISSN
0007-5027
eISSN
1943-7730
D.O.I.
10.1093/labmed/lmx089
Publisher site
See Article on Publisher Site

Abstract

Abstract Objective Autovalidation algorithm should be properly designed with clearly defined criteria and any data that do not meet the criteria, must be reviewed and manually validated. The aim was to define the rules for autovalidation in our laboratory information system (LIS), and validate the algorithm prior to its implementation in routine laboratory work. Methods Autovalidation was implemented for all routine serum biochemistry tests. The algorithm included analytical measurement ranges (AMR), delta check, critical values, serum indices and all preanalytical and analytical flags from the analyzer. Results In the validation process 9805 samples were included, and 78.3% (7677) of all samples were autovalidated. The highest percentage of non-validated samples (54.9%) refers to those with at least one result outside the method linearity ranges (AMR criteria) while critical values were observed to be the least frequent criterion for stopping autovalidation (1.8%). Also, 38 samples were manually validated as they failed to meet the autovalidation criteria. Conclusion Implementation of algorithm for autovalidation in our institution resulted in the redesign of the existing LIS. This model of the autovalidation algorithm significantly decreased the number of manually validated test results and can be used as a model for introducing autovalidation in other laboratory settings. autovalidation algorithm, laboratory information system, serum biochemistry tests, validation, postanalytical phase, laboratory optimization Autovalidation is a rule-based system for validation of laboratory test results based on a selection of predefined rules in the autovalidation algorithm.1-5 Autovalidation automatically releases test results without any manual intervention and additional effort from laboratory staff if those results meet the criteria of all defined rules in the algorithm. Introduction of an automated tool for validation of laboratory results of this kind is essential in the present-day automated laboratory environment, due to an increasing number of laboratory test requests, the possibility of errors in the case of manual reporting of test results, and the increasing need for reducing test-reporting turnaround time (TAT).6 An autovalidation algorithm can be implemented in the laboratory information system (LIS),1-3 can function as an independent validation program,6-9 or can be introduced in the middleware software that interconnects LIS and laboratory analyzers.4,10 Proper design of the autovalidation algorithm and definition of autovalidation rules are prerequisites for algorithm implementation.1-5 Because autovalidation ensures uniform and objective validation of laboratory test results based on predefined rules, all selected rules must be clearly and unambiguously defined.2 Autovalidation rules commonly include analytical measurement ranges (AMRs)4; interference indices for hemolysis, icterus, and lipemia11; preanalytical and analytical flags generated from the analyzer4,5,9; reference ranges and decision limits4,8; critical values4,8,9; delta check4-6,8,9,12; quality-control (QC) results4,5,8; moving averages; and lot checks.5 The sole technical implementation of autovalidation rules in the LIS can vary and can even be limited by the capabilities of the LIS software used (software upgrades are often required). Decisions regarding the extent of autovalidation rules and regarding the criteria that are to be implemented depend on the specificity of the laboratory setting and the complexity of medical care performed in the institution; thus, these decisions must not be generalized. The criteria for AMRs and interference indices are usually defined according to the criteria defined by the reagent manufacturer and/or data obtained during method validation. Reference ranges, decision limits, and criteria for critical values can be population-based, implemented according to relevant published guidelines and recommendations, and/or designed to accommodate current trends. The rules regarding analyzer and reagent performance (error flags, QC, lot checks) are analyzer-/reagent-specific and depend mainly on the characteristics of those instruments. The delta-check rule serves for comparison of the current test result with the previous one for the same patient in a defined time frame. Delta check can be defined in several forms, namely, as delta percent change (the most commonly used form), delta difference, rate percent change, and rate difference.13 Some authors12 recommend that delta check should be expressed as the reference change value (RCV) of each selected laboratory test. The RCV is calculated using the data from analytical control results and intraindividual biological coefficient of variation.12,14 After definition of autovalidation rules and before implementation of autovalidation in routine laboratory practice, according to Clinical and Laboratory Standards Institute (CLSI) guidelines5 and best laboratory practice, it is necessary to validate the autovalidation algorithm.2,4,10 Further, with every change of autovalidation rules and/or criteria, the autovalidation algorithm should be revalidated.2 In this way, potential pitfalls can be detected, and the release of erroneous laboratory results can be prevented. At our institution, the field of general clinical chemistry has been identified as the most suitable for the first implementation of the autovalidation algorithm, due to the large number of specimens being processed on a daily basis and the lack of an adequate amount of personnel. Although some published articles3,4,15,16 describe experiences in implementing autovalidation, such as in clinical chemistry, coagulation, and hematology, none of these articles focus on the detailed design of the autovalidation algorithm for clinical-chemistry tests in the LIS. For that reason, we established our own algorithm and herein, we present it as a raw model that can be applicable to any setting and modified according to the specific requirements of the institution. In this study, we aimed to define the rules of the autovalidation algorithm for serum biochemistry tests and subsequently to validate the algorithm before its implementation in routine laboratory work. Materials and Methods Study Setting The study was conducted at the Department of Laboratory Diagnostics, University Hospital Center, Zagreb, Croatia. This department is composed of 13 divisions and performs approximately 3.6 million tests per year, including highly specialized and differential diagnostic tests for inpatients and outpatients. The Division for General Clinical Chemistry processes approximately 550 specimens per day, which corresponds to 6500 laboratory test results per day. The clinical chemistry automation line, where all tests are performed, includes a Cobas Modular preanalytics system, followed by 3 Cobas c501 biochemistry analyzers (F. Hoffman-La Roche Ltd). After specimen analysis, the results and respective flags are transferred from the analyzer to the LIS (BioNET LIS, IN2 Health Sector). The Cobas c501 analyzers are interfaced to the LIS via a middleware Process System Manager (F. Hoffman-La Roche Ltd). Implementation of Autovalidation Autovalidation for all routine serum biochemistry tests was implemented in August 2014. An autovalidation algorithm that consists of autovalidation rules was elaborated in our LIS database as a new functionality. The technical characteristics of this new feature were developed by information technology (IT) specialists. However, laboratory specialists decided which specific items and rules would be included in the autovalidation algorithm, based on the requisites of the laboratory. After that, the chosen criteria for autovalidation rules for all included biochemistry tests were applied and tested appropriately, as described later herein. As a result, a background algorithm that combines all the chosen rules for autovalidation was developed by IT specialists. The rules included in our autovalidation algorithm were as follows: AMRs, delta check, critical values, serum indices (hemolysis, lipemia, icteria), and all preanalytical flags (eg, specimen short, specimen clot, specimen bubbles) and analytical flags (eg, prozone effect, reagent short, expired reagent, mechanical errors, autodilution) from the analyzer. Autovalidation rules for all analytes are listed in Table 1. Table 1. List of All Biochemistry Tests With Specific Criteria Included in Autovalidation Rules. Analytical Measurement Ranges (AMR) Critical Values Interference Indices (mg/dL) Delta Check (%) Test Name Unit Low Value High Value Low Critical High Critical Lipemia Icterus Hemolysis Alanine aminotransferase (ALT) U/L 5 700 NA 1000 150 60 700 60 Alkaline phosphatase U/L 5 1200 NA NA 2000 60 200 30 Amylase U/L 3 1500 NA NA 1500 60 500 100 Angiotensin converting enzyme (ACE) U/L 5 100 NA NA 100 10 30 NA Aspartate aminotransferase (AST) U/L 5 700 NA 1000 150 60 40 60 Bilirubin direct µmol/L 2 291 NA NA 750 0 20 30 Bilirubin total µmol/L 2 650 NA 257 300 0 50 50 C reactive protein (CRP) mg/L 0 350 NA NA 1000 60 1000 500 Calcium mmol/L 0.1 5.0 1.65 3.50 2000 60 1000 10 Chloride mmol/L 60 140 75 125 2000 60 100 10 Cholesterol mmol/L 0.1 20.7 NA NA 2000 16 700 35 Copper mg/L 12 25 NA NA 100 5 30 NA Creatine kinase (CK) U/L 7 2000 NA NA 1000 60 80 300 Creatine kinase MB (CK-MB) U/L 3 500 NA NA 200 20 30 300 Creatinine mmol/L 10 2226 NA NA 800 5 1000 40 Gamma glutamyl transferase (GGT) U/L 3 1200 NA NA 1500 20 200 60 Glucose mmol/L 0.1 41.6 2.5 27.8 1000 60 1000 50 HDL cholesterol mmol/L 0.08 3.12 NA NA 1800 60 1200 20 Iron µmol/L 0.9 179 NA NA 1500 60 200 25 Lactate dehydrogenase (LDH) U/L 0 1000 NA 500 1500 60 30 300 LDL cholesterol mmol/L 0.1 14.2 NA NA 200 60 1000 20 Lipase U/L 3 300 NA 700 2000 50 1000 100 Magnesium mmol/L 0.1 2.5 0.41 5.00 1700 60 30 30 Phosphate mmol/L 0.1 6.5 0.32 2.90 1250 40 300 50 Potassium mmol/L 1.5 10 2.8 6.2 2000 60 30 20 Protein (total) g/L 2 120 NA NA 2000 20 1000 25 Sodium mmol/L 80 180 120 160 2000 60 100 5 Triglyceride mmol/L 0,1 10 NA NA 2000 35 700 70 Unsaturated iron binding capacity (UIBC) µmol/L 3 125 NA NA 300 60 40 25 Urea mmol/L 0.5 40.0 NA 35.6 1000 60 1000 60 Uric acid µmol/L 11.9 1487 NA 773 1500 40 1000 25 Analytical Measurement Ranges (AMR) Critical Values Interference Indices (mg/dL) Delta Check (%) Test Name Unit Low Value High Value Low Critical High Critical Lipemia Icterus Hemolysis Alanine aminotransferase (ALT) U/L 5 700 NA 1000 150 60 700 60 Alkaline phosphatase U/L 5 1200 NA NA 2000 60 200 30 Amylase U/L 3 1500 NA NA 1500 60 500 100 Angiotensin converting enzyme (ACE) U/L 5 100 NA NA 100 10 30 NA Aspartate aminotransferase (AST) U/L 5 700 NA 1000 150 60 40 60 Bilirubin direct µmol/L 2 291 NA NA 750 0 20 30 Bilirubin total µmol/L 2 650 NA 257 300 0 50 50 C reactive protein (CRP) mg/L 0 350 NA NA 1000 60 1000 500 Calcium mmol/L 0.1 5.0 1.65 3.50 2000 60 1000 10 Chloride mmol/L 60 140 75 125 2000 60 100 10 Cholesterol mmol/L 0.1 20.7 NA NA 2000 16 700 35 Copper mg/L 12 25 NA NA 100 5 30 NA Creatine kinase (CK) U/L 7 2000 NA NA 1000 60 80 300 Creatine kinase MB (CK-MB) U/L 3 500 NA NA 200 20 30 300 Creatinine mmol/L 10 2226 NA NA 800 5 1000 40 Gamma glutamyl transferase (GGT) U/L 3 1200 NA NA 1500 20 200 60 Glucose mmol/L 0.1 41.6 2.5 27.8 1000 60 1000 50 HDL cholesterol mmol/L 0.08 3.12 NA NA 1800 60 1200 20 Iron µmol/L 0.9 179 NA NA 1500 60 200 25 Lactate dehydrogenase (LDH) U/L 0 1000 NA 500 1500 60 30 300 LDL cholesterol mmol/L 0.1 14.2 NA NA 200 60 1000 20 Lipase U/L 3 300 NA 700 2000 50 1000 100 Magnesium mmol/L 0.1 2.5 0.41 5.00 1700 60 30 30 Phosphate mmol/L 0.1 6.5 0.32 2.90 1250 40 300 50 Potassium mmol/L 1.5 10 2.8 6.2 2000 60 30 20 Protein (total) g/L 2 120 NA NA 2000 20 1000 25 Sodium mmol/L 80 180 120 160 2000 60 100 5 Triglyceride mmol/L 0,1 10 NA NA 2000 35 700 70 Unsaturated iron binding capacity (UIBC) µmol/L 3 125 NA NA 300 60 40 25 Urea mmol/L 0.5 40.0 NA 35.6 1000 60 1000 60 Uric acid µmol/L 11.9 1487 NA 773 1500 40 1000 25 NA, not available View Large Table 1. List of All Biochemistry Tests With Specific Criteria Included in Autovalidation Rules. Analytical Measurement Ranges (AMR) Critical Values Interference Indices (mg/dL) Delta Check (%) Test Name Unit Low Value High Value Low Critical High Critical Lipemia Icterus Hemolysis Alanine aminotransferase (ALT) U/L 5 700 NA 1000 150 60 700 60 Alkaline phosphatase U/L 5 1200 NA NA 2000 60 200 30 Amylase U/L 3 1500 NA NA 1500 60 500 100 Angiotensin converting enzyme (ACE) U/L 5 100 NA NA 100 10 30 NA Aspartate aminotransferase (AST) U/L 5 700 NA 1000 150 60 40 60 Bilirubin direct µmol/L 2 291 NA NA 750 0 20 30 Bilirubin total µmol/L 2 650 NA 257 300 0 50 50 C reactive protein (CRP) mg/L 0 350 NA NA 1000 60 1000 500 Calcium mmol/L 0.1 5.0 1.65 3.50 2000 60 1000 10 Chloride mmol/L 60 140 75 125 2000 60 100 10 Cholesterol mmol/L 0.1 20.7 NA NA 2000 16 700 35 Copper mg/L 12 25 NA NA 100 5 30 NA Creatine kinase (CK) U/L 7 2000 NA NA 1000 60 80 300 Creatine kinase MB (CK-MB) U/L 3 500 NA NA 200 20 30 300 Creatinine mmol/L 10 2226 NA NA 800 5 1000 40 Gamma glutamyl transferase (GGT) U/L 3 1200 NA NA 1500 20 200 60 Glucose mmol/L 0.1 41.6 2.5 27.8 1000 60 1000 50 HDL cholesterol mmol/L 0.08 3.12 NA NA 1800 60 1200 20 Iron µmol/L 0.9 179 NA NA 1500 60 200 25 Lactate dehydrogenase (LDH) U/L 0 1000 NA 500 1500 60 30 300 LDL cholesterol mmol/L 0.1 14.2 NA NA 200 60 1000 20 Lipase U/L 3 300 NA 700 2000 50 1000 100 Magnesium mmol/L 0.1 2.5 0.41 5.00 1700 60 30 30 Phosphate mmol/L 0.1 6.5 0.32 2.90 1250 40 300 50 Potassium mmol/L 1.5 10 2.8 6.2 2000 60 30 20 Protein (total) g/L 2 120 NA NA 2000 20 1000 25 Sodium mmol/L 80 180 120 160 2000 60 100 5 Triglyceride mmol/L 0,1 10 NA NA 2000 35 700 70 Unsaturated iron binding capacity (UIBC) µmol/L 3 125 NA NA 300 60 40 25 Urea mmol/L 0.5 40.0 NA 35.6 1000 60 1000 60 Uric acid µmol/L 11.9 1487 NA 773 1500 40 1000 25 Analytical Measurement Ranges (AMR) Critical Values Interference Indices (mg/dL) Delta Check (%) Test Name Unit Low Value High Value Low Critical High Critical Lipemia Icterus Hemolysis Alanine aminotransferase (ALT) U/L 5 700 NA 1000 150 60 700 60 Alkaline phosphatase U/L 5 1200 NA NA 2000 60 200 30 Amylase U/L 3 1500 NA NA 1500 60 500 100 Angiotensin converting enzyme (ACE) U/L 5 100 NA NA 100 10 30 NA Aspartate aminotransferase (AST) U/L 5 700 NA 1000 150 60 40 60 Bilirubin direct µmol/L 2 291 NA NA 750 0 20 30 Bilirubin total µmol/L 2 650 NA 257 300 0 50 50 C reactive protein (CRP) mg/L 0 350 NA NA 1000 60 1000 500 Calcium mmol/L 0.1 5.0 1.65 3.50 2000 60 1000 10 Chloride mmol/L 60 140 75 125 2000 60 100 10 Cholesterol mmol/L 0.1 20.7 NA NA 2000 16 700 35 Copper mg/L 12 25 NA NA 100 5 30 NA Creatine kinase (CK) U/L 7 2000 NA NA 1000 60 80 300 Creatine kinase MB (CK-MB) U/L 3 500 NA NA 200 20 30 300 Creatinine mmol/L 10 2226 NA NA 800 5 1000 40 Gamma glutamyl transferase (GGT) U/L 3 1200 NA NA 1500 20 200 60 Glucose mmol/L 0.1 41.6 2.5 27.8 1000 60 1000 50 HDL cholesterol mmol/L 0.08 3.12 NA NA 1800 60 1200 20 Iron µmol/L 0.9 179 NA NA 1500 60 200 25 Lactate dehydrogenase (LDH) U/L 0 1000 NA 500 1500 60 30 300 LDL cholesterol mmol/L 0.1 14.2 NA NA 200 60 1000 20 Lipase U/L 3 300 NA 700 2000 50 1000 100 Magnesium mmol/L 0.1 2.5 0.41 5.00 1700 60 30 30 Phosphate mmol/L 0.1 6.5 0.32 2.90 1250 40 300 50 Potassium mmol/L 1.5 10 2.8 6.2 2000 60 30 20 Protein (total) g/L 2 120 NA NA 2000 20 1000 25 Sodium mmol/L 80 180 120 160 2000 60 100 5 Triglyceride mmol/L 0,1 10 NA NA 2000 35 700 70 Unsaturated iron binding capacity (UIBC) µmol/L 3 125 NA NA 300 60 40 25 Urea mmol/L 0.5 40.0 NA 35.6 1000 60 1000 60 Uric acid µmol/L 11.9 1487 NA 773 1500 40 1000 25 NA, not available View Large AMR criteria were set according to the method linearity ranges defined by the manufacturer and verified in the laboratory through assay validation, according to International Organization for Standardization (ISO) 15189.17 Also, we used serum indices for all tests, as taken from the insert package sheets of each reagent. Critical values were introduced in concordance with International Federation of Clinical Chemistry (IFCC) published recommendations.18 Those values were equal to the critical values established by the Croatian Chamber of Medical Biochemists.19 Delta-check values were determined according to the respective RCV value for each test, and the time frame was set as 5 days for all tests. The 5-day time frame was chosen to include various patient populations (pediatric and adult intensive care patients, hematology and oncology patients, etc) in our hospital and also the repeated measurements for those patients. RCV values were calculated according to the following formula: RCV=(2)1/2*Zp*(CV2A+CV2I)1/2 where Zp is the standard deviation for the appropriate probability of error (95% Zp = 1.96), CVA is the analytical coefficient of variation, and CVI is the intraindividual biological coefficient of variation. Futher, delta-check values were slightly modified, taking in consideration the patient population in our institution and in agreement with laboratory specialists involved in the design of the autovalidation algorithm. Autovalidation of completed tests can be achieved only when all the specimen results are transferred from the analyzer to the LIS. Autovalidation must be triggered by selecting the autovalidation icon. After this step, all test results are checked through all the defined rules from the algorithm. All rules have the same level of importance—for instance, if 1 or more tests do not meet the defined rules for any specimen, the results are not verified automatically. All nonautoverified test results (with various flags) remain for review and manual validation by a laboratory specialist, according to standard operational procedures usually used in routine practice (Figure 1). Figure 1 View largeDownload slide Autovalidation flowchart. AMR indicates analytical measurement range. Figure 1 View largeDownload slide Autovalidation flowchart. AMR indicates analytical measurement range. Validation of Autovalidation Algorithm Before the introduction of autovalidation in routine practice, validation of the autovalidation algorithm was performed according to CLSI guideline AUTO10-A.5 In the validation process, the results from chosen biochemistry specimens were assessed to check the validity of autovalidation rules. Validation was performed within the Unit for General Clinical Chemistry discontinuously on working days during a 3-month period (May 2014–July 2014). Rules were tested in a test version of our LIS and therefore served solely for validation purposes and not for reporting of laboratory results to health care professionals. Completed test results, which at that time were all manually validated by laboratory specialists, were downloaded on a daily basis from the LIS to its test version, and manually reviewed by laboratory specialists according to the defined autovalidation rules. Specimens that did not meet all the criteria for autovalidation and had to be manually validated were noted in the records. Afterward, autovalidation was triggered, and we verified whether all the remaining specimens for manual validation were in concordance with the results of the manual review. Statistical Analysis The results from our study are presented as absolute numbers and as percentage of autovalidated test results in the total number of tests performed. We archived and processed data using Microsoft Excel, version 2010 (Microsoft Corporation). Results In the validation process, a total of 9805 specimens were included, of which 78.3% (7677) of all specimens were autovalidated according to the established algorithm. The highest percentage of nonvalidated specimens (54.9%) was made up of those with at least 1 result outside the method linearity ranges (AMR criteria). In contrast, critical values were observed to be the least frequent criterion for stopping autovalidation (1.8%) (Figure 2). Figure 2 View largeDownload slide The rate of autovalidation stopping criteria. Figure 2 View largeDownload slide The rate of autovalidation stopping criteria. Specifically, AMR criteria caused the most frequently autovalidation stoppage for sodium considering the low AMR limit, and for potassium and blood urea nitrogen (BUN) regarding the high AMR limit. The second most common stopping criteria referred to delta check, which mostly affected potassium and creatinine results. Moreover, hemolysis, as the most frequent criteria among serum indices, caused autovalidation failure for lactate dehydrogenase (LDH) and aspartate aminostransferase (AST). Comparison between manual validation and autovalidation showed consistent results for 99.5% of specimens. In this process, 38 specimens were manually validated because they failed to meet the autovalidation criteria. The obtained difference was due to the existence of written remarks next to test results. These comments were not defined as criteria for stopping autovalidation but were laboratory-specific remarks (ie, predefined comments about the analytical methods used, as well as short notes to healthcare professionals). However, these comments were recognized as such and caused stopping of autovalidation because they were placed in the same field as flags from the analyzer. When noticed during validation, this pitfall was immediately corrected. For this reason, only preanalytical and analytical flags from the analyzer in the fields for comments should be recognized as autovalidation-stopping criteria. Discussion In this study, we defined an autovalidation algorithm for our LIS and applied it to the automated biochemistry Roche system, which showed almost 100% concordance with manual validation. The autovalidation algorithm consisting of predefined rules was elaborated and introduced in the LIS, and validation of the algorithm was performed before its implementation in routine practice. This was the first experience regarding implementation of autovalidation not only in our laboratory but throughout Croatia. Our incentive to implement autovalidation resulted in the development of new functionality in the existing LIS. Close cooperation between laboratory and IT specialists was essential in designing the autovalidation functionality.4 Being the first group to test the operation of the autovalidation feature, we decided to validate the autovalidation algorithm on a large number of specimens because, to our knowledge, no guidelines define the optimal number of specimens that should be included in the validation process. This decision was made after thorough discussion among laboratory specialists in our laboratory. The rate of autovalidated results in our study was 78.3%. Published data show great variations in the rates of autovalidated results, which depend on the setting and implemented autovalidation rules. Krasowski et al4 demonstrated in their study that initial implementation of autovalidation rules in middleware yielded 95% autovalidated results, followed by 99% and 99.5% rates in the years 2010 and 2013, respectively. Autovalidation rules in this study included manual-review limits, critical values, delta check, interferences indices, AMR, autoextended range, instrument error flags, and QC. Oosterhuis et al6 report that the total percentage of test results validated by an automated validation system in their laboratory was 86.6%. Further, some studies display the percentage of the autovalidation passing rate in the daily laboratory routine (eg, ranging from 92% to 95% in Shih et al10). Their algorithm included limit check, delta check, critical values, and consistency-check rules. A potential explanation for the lower rate of autovalidated results observed in our study is tightened stopping rules as a result of the first-ever implementation of autovalidation algorithm in our laboratory. However, these rules will surely be modified over time according to accumulated experience, resulting in a higher percentage of autovalidated results. Beside studies that showed higher percentages, there are also studies with lower percentages of autovalidation passing rate. Torke et al1 show in their study results that 62% of their chemistry test panels and 73% of single- analyte assays were autovalidated. The largest number of specimens for which processing was stopped by autovalidation and remained for manual revision in our study are the ones that did not meet the AMR criteria, with results below or above the limits of the determined range. However, the number of specimens stopped by the AMR rule could be decreased by extending the AMR range. For introduction of such modified AMR ranges, it is necessary to perform additional assay validation, to test the linearity of all assays outside the recommended linearity range by the manufacturer and, based on the obtained results, to modify these criteria accordingly. The second most frequent stopping criterium was the delta-check rule, which covers one quarter of all specimens for which processing was stopped. It is important to emphasize that the delta-check rule was automatically satisfied if no previous data existed for comparison (ie, in cases of the first visit of a patient to the laboratory). Delta check is colloquially considered to be a useful tool, not only in the context of identifying clinically significant changes in laboratory parameters but also in recognizing cases of patient misidentification and specimen mix-ups. Nowadays, there is controversy regarding the usefulness of delta checking. Although it is a classic laboratory QC tool, there are limited published data regarding the usefulness of delta checking for detecting problems other than specimen-mislabeling errors. However, available data show that delta checking is mostly used as part of autovalidation procedures.20 We are aware that including this rule may cause additional stoppage of autovalidation and may generate more manual work. However, we did not hesitate to include it because it proved to be important in detecting not only patient misidentification but also deterioration of laboratory results that should be immediately communicated to health care professionals, so that patients can receive immediate medical attention. Our data showed that testing for potassium and creatinine are usually stopped in the autovalidation process, probably due to a large number of patient undergoing dialysis, in whom fluctuations of those 2 analytes are expected. Among serum indices, hemolysis was the most frequent, whereas lipemia was less frequent in triggering the rule for blocking autovalidation. These data are consistent with a large number of articles/published data that show that hemolysis is still a challenging preanalytical issue.21 Finally, critical values and flags from the analyzer criteria caused stopping of autovalidation for the least number of nonautovalidated specimens. We also considered it important to review and evaluate differences between the number of manually validated specimens without additional interventions performed by laboratory specialists and the number of autovalidated specimens in the preimplementation period. A similar validation approach was applied for evaluation of an automated validation system.6 In our study the disagreement in testing-phase results was observed in 38 specimens that were manually validated but according to the implemented rules should be autovalidated. An observed pitfall in the algorithm was corrected before implementation of autovalidation in the routine work. This observation clearly confirms the importance of verifying the autovalidation algorithm before its use in routine work. This apparently small number of specimens indicated pitfalls in the algorithm that were subsequently corrected. It is important to emphasize that misidentification of laboratory-specific remarks as autovalidation-stopping rules was the only pitfall of the designed algorithm observed in the validation process. This observation clearly confirmed the importance of validating the autovalidation algorithm before its use in routine work. Still, it can be considered a minor fallacy because it prevented autovalidation, rather than inappropriately causing autovalidation and subsequent release of potentially erroneous results needing manual review. At our institution, we believe that the first introduction of autovalidation should be implemented with caution because, as mentioned earlier herein, this was also the first attempt to implement autovalidation in our LIS. For that reason, we believe that autovalidation that should be initiated by selecting an icon, rather that real-time autovalidation, was a reasonable approach with which to start. However, further steps should include an upgrade to real-time autovalidation, which inevitably means the introduction of additional criteria, primarily, inclusion of QC rules in the autovalidation algorithm.2 This study has some limitations. First, the stopping rules for delta check and critical values were not divided by sex and age. Second, the aforementioned rules were not adapted to the patient population of different hospital wards but, rather, were used uniformly. These limitations can cause a lower percentage of autovalidated results and they can be avoided with adjustment of these criteria to the patient population of a specific ward (eg, intensive care unit, pediatric care). During the implementation of the autovalidation process, we reviewed only data from the LIS without simulation of specimens with, for example, high serum indices. However, because our study verified almost 10,000 specimens from various hospital patients, we consider that implemented rules are reliable. Nevertheless, we believe that our study results present a valuable improvement in the postanalytical phase in our laboratory, contributing to a more efficient and objective reporting of test results. Also, they present a systematic approach for introduction of an autovalidation algorithm through the LIS with general requirements, which can be useful as a model for its implementation in any laboratory setting and any analytical system used. However, adjustment of the algorithm rules to the specific laboratory setting is a prerequisite for successful implementation. Autovalidation has numerous benefits, including shortening of the TAT; reduction in the number of results for manual revision and, consequently, a greater focus on potentially problematic patient specimens; and uniform test reporting that is less subject to subjective variations.2 However, autovalidation is still only a computerized tool that works according to defined rules, which surely enhances laboratory efficiency but cannot completely replace laboratory specialist work in the process of reporting laboratory results and decision making. Conclusion Our initiative to implement autovalidation resulted in the redesign of the existing LIS. The model of the autovalidation algorithm described herein proved to decrease the number of manually validated test results and can be applied to different laboratory settings. Abbreviations TAT turnaround time LIS laboratory information system AMRs analytical measurement ranges QC quality-control RCV reference change value CLSI Clinical and Laboratory Standards Institute IT information technology ISO International Organization for Standardization IFCC International Federation of Clinical Chemistry BUN blood urea nitrogen LDH lactate dehydrogenase AST aspartate aminostransferase NA not applicable ALP alkaline phosphatase ALT alanine aminotransferase NA not available ALP alkaline phosphatase ACE angiotensin-converting enzyme CRP C-reactive protein CK creatine kinase CK-MB creatine kinase GGT gamma-glutamyl transferase HDL high-density lipoprotein LDH lactate dehydrogenase LDL low-density lipoprotein UIBC unsaturated iron-binding capacity. References 1. Torke N , Boral L , Nguyen T , Perri A , Chakrin A . Process improvement and operational efficiency through test result autoverification . Clin Chem . 2005 ; 51 ( 12 ): 2406 – 2408 . 2. Duca DJ . Autoverification in a laboratory information system . Lab Med . 2002 ; 33 : 21 – 25 . 3. Onelöv L , Gustafsson E , Grönlund E , et al. Autoverification of routine coagulation assays in a multi-center laboratory . Scand J Clin Lab Invest . 2016 ; 76 ( 6 ): 500 – 502 . 4. Krasowski MD , Davis SR , Drees D , et al. Autoverification in a core clinical chemistry laboratory at an academic medical center . J Pathol Inform . 2014 ; 5 ( 1 ): 13 . 5. Clinical and Laboratory Standard Institute (CLSI) . AUTO 10-A . Wayne, PA : CLSI ; 2006 . 6. Oosterhuis WP , Ulenkate HJ , Goldschmidt HM . Evaluation of LabRespond, a new automated validation system for clinical laboratory test results . Clin Chem . 2000 ; 46 ( 11 ): 1811 – 1817 . 7. Kuijsters JP . Lab respond: a tool for autoverification . Accred Qual Assur . 2002 ; 7 : 477 – 479 . 8. Prost L , Rogari P . How autoverification through the expert system VALAB can make your laboratory more efficient . Accred Qual Assur . 2002 ; 7 : 480 – 487 . 9. Guidi GC , Poli G , Bassi A , Giobelli L , Benetollo PP , Lippi G . Development and implementation of an automatic system for verification, validation and delivery of laboratory test results . Clin Chem Lab Med . 2009 ; 47 ( 11 ): 1355 – 1360 . 10. Shih MC , Chang HM , Tien N , et al. Building and validation an autoverification system in the clinical chemistry laboratory . Lab Med . 2011 ; 42 : 668 – 673 . 11. Vermeer HJ , Thomassen E , de Jonge N . Automated processing of serum indices used for interference detection by the laboratory information system . Clin Chem . 2005 ; 51 ( 1 ): 244 – 247 . 12. Fraser CG , Stevenson HP , Kennedy IMG . Biological variation data are necessary prerequisites for objective autoverification of clinical laboratory data . Accred Qual Assur . 2002 ; 7 : 455 – 460 . 13. Park SH , Kim SY , Lee W , Chun S , Min WK . New decision criteria for selecting delta check methods based on the ratio of the delta difference to the width of the reference range can be generally applicable for each clinical chemistry test item . Ann Lab Med . 2012 ; 32 ( 5 ): 345 – 354 . 14. Ricós C , Cava F , García-Lario JV , et al. The reference change value: a proposal to interpret laboratory reports in serial testing based on biological variation . Scand J Clin Lab Invest . 2004 ; 64 ( 3 ): 175 – 184 . 15. Froom P , Saffuri-Elias E , Barak M . Autovalidation rates in an outpatient coagulation laboratory . Int J Lab Hematol . 2015 ; 37 ( 5 ): 680 – 685 . 16. Froom P , Barak M . Auto-validation of complete blood counts in an outpatient’s regional laboratory . Clin Chem Lab Med . 2015 ; 53 ( 2 ): 275 – 279 . 17. International Organization for Standardization . ISO 15189:2012. Medical laboratories: Particular requirements for quality and competence. 18. Thomas L . Crtical limits of laboratory results for urgent clinical notification . eJIFCC . 2003 ; 14(1) . http://www.ifcc.org/ejifcc/vol14no1/140103200303.pdf. Accessed December 26, 2017. 19. Critical values of laboratory results and reporting of critical values- 2nd edition . http://www.hkmb.hr/dokumenti/povjerenstva/HKMB%20PPSP%208.pdf. Accessed December 26, 2017 . 20. Schifman RB , Talbert M , Souers RJ . Delta check practices and outcomes: a Q-probes study involving 49 health care facilities and 6541 delta check alerts . Arch Pathol Lab Med . 2017 ; 141 ( 6 ): 813 – 823 . 21. Plebani M , Sciacovelli L , Aita A , Chiozza ML . Harmonization of pre-analytical quality indicators . Biochem Med (Zagreb) . 2014 ; 24 ( 1 ): 105 – 113 . © American Society for Clinical Pathology 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Journal

Laboratory MedicineOxford University Press

Published: Feb 8, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off