TY - JOUR AU1 - Cheng, Christine, M. AU2 - Fu,, Carrie AU3 - Guglielmo, B., Joseph AU4 - Auerbach, Andrew, D. AB - Abstract Purpose Inconsistencies in boxed warnings between drug information resources and the manufacturer’s prescribing information (PI) were evaluated. Methods This study was a cross-sectional evaluation of boxed warnings in Black-BoxRx, DrugDex, Facts and Comparisons, Epocrates, Lexicomp, and PDR.net conducted in June 2010. New molecular entities with boxed warnings and PI that conformed to current Food and Drug Administration labeling requirements were included. Each resource was reviewed for warnings that appeared verbatim with the full boxed warning in the PI. Two drug information pharmacists independently reviewed the remaining nonverbatim boxed warnings to determine concordance with the boxed warning summary in the “highlights” section of the PI. Tests of proportions were used to examine differences among resources in the proportion of warnings concordant with the PI. Interrater reliability was assessed with the kappa statistic. Results A total of 71 drugs with unique boxed warnings were included in the evaluation. Resources revealed varying degrees of discordances with the boxed warning in the PI. The resource with the lowest number of verbatim warnings contained a significantly higher percentage of warnings with discordant information when compared with all other resources (p < 0.0001 for all paired comparisons). Interrater reliability was excellent (kappa = 0.86). Conclusion Boxed warning information presented in major drug information resources may be missing key elements of the official boxed warning in the current PI. The current PI may be the most reliable approach to accessing the complete, up-to-date boxed warning for a given drug. Drug information, Food and Drug Administration (U.S.), Industry, pharmaceutical, Labeling, Regulations, Standards, Toxicity A boxed warning, or black-box warning, is the most serious warning the Food and Drug Administration (FDA) can issue for a marketed prescription drug.1 The purpose of the boxed warning is to alert clinicians of a drug’s association with a potentially life-threatening adverse reaction.2 Boxed warnings may include a description of the adverse reaction, provide strategies for reducing the risk or minimizing the severity of the adverse reaction, or specify whether a restricted distribution program is in place for a particular drug.3 Due to their critical nature, boxed warnings can profoundly affect the use of drugs that previously did not carry such a warning; in some instances, the boxed warning is the last FDA risk communication before withdrawing a drug from the market.4–6 The full boxed warning is presented as text surrounded by a border in the prescribing information (PI) and may vary in detail, language, and length. PI that conforms to current FDA labeling requirements has a “highlights of the PI” section, which provides a succinct summary of key points from the full boxed warning within 20 lines of text.7 In addition to the PI, the boxed warning also appears in all written promotional materials for the drug. Despite their importance, there is no official repository of boxed warnings or list of drugs that carry these warnings.8 Consequently, health care providers must rely on drug information resources for boxed warning information. Awareness of these warnings can help ensure safe, appropriate drug use and compliance with regulatory requirements.9 The ability to identify drugs with boxed warnings using drug information resources has been shown to vary widely.8 However, it is not known whether the content in boxed warnings in these resources is accurate or complete. In this study, we conducted a qualitative evaluation of boxed warning information from major drug information resources and compared it with the boxed warning in the PI. The goal of this study was to determine whether resource representation of the boxed warning captured key elements of the official boxed warning from the current PI. Methods Resources A cross-sectional comparative evaluation of the boxed warning in the manufacturer’s PI and the boxed warning content in six online drug information resources was conducted. The selected resources were BlackBoxRx (Formulary Productions, Memphis, TN), Epocrates Online (Epocrates, San Mateo, CA), Facts and Comparisons (Wolter Kluwer Health, Philadelphia, PA), Lexi-Drugs (Lexi-comp, Hudson, OH), Micromedex DrugDex (Thomson Healthcare Series, New York, NY) and PDR.net Full Labels A-Z (PDR Network, Montvale, NJ). These resources were selected because of their reputability among medical librarians, students, faculty, drug information centers, and patient safety organizations, as well as their inclusion of boxed warning information in a dedicated section of their drug monographs.10–12 Drug selection We compiled a list of currently marketed prescription drugs with boxed warnings through June 2010 using a previously described methodology.8 We obtained the manufacturer’s PI from the FDA Center for Drug Evaluation and Research Drugs@FDA database,13 the National Library of Medicine DailyMed website,14 or the manufacturer’s website. We selected PI with a “highlights” section so that we could compare resources’ warnings with content from both the full boxed warning and the PI highlights of the boxed warning. Since our objective was to conduct a qualitative evaluation of boxed warning content in the various resources, we included only drugs with unique boxed warnings in our study sample. We excluded boxed warning entries for combination drugs if the boxed warnings for the individual components were included in our evaluation. For drugs within the same drug class that carried identical warnings (e.g., angiotensin-converting enzyme inhibitors, angiotensin receptor blockers, low-molecular-weight heparins, nonsteroidal antiinflammatory drugs), we chose to evaluate the boxed warning of the drug with warning information present in the most drug information resources possible. Data extraction In June 2010, we searched our key resources for boxed warning information for our selected drugs. We copied and pasted information designated as the boxed warning in each resource into a Microsoft Access 2007 (Microsoft Corporation, Redmond, WA) database. Content evaluation We evaluated each resource’s boxed warning content in three steps. First, we classified each resource’s boxed warnings as either a verbatim or nonverbatim report of the full boxed warning in the PI. One author identified the verbatim warnings by comparing the full boxed warning in the PI with the corresponding warning in each resource and determined whether the two were identical, excepting abbreviations, brand and generic name substitutions, and rearrangements of full sentences. Second, two pharmacist-authors with drug information experience evaluated the warnings that were nonverbatim with the PI boxed warning. We chose to use the PI highlights of the boxed warning (PI highlights) as the reference standard for this evaluation to minimize interpretive differences between the two reviewers. We had previously conducted a pilot evaluation using the full boxed warning as the standard and found a high frequency of disagreement between the two reviewers. For this reason, we turned to the PI highlights as our standard. Since the PI highlights represent a summary of key points from the full boxed warning, we decided that resources’ summaries should, at minimum, contain all information in the PI highlights for a given drug. We designated each sentence or bulleted phrase in the PI highlights as a line of content. Each pharmacist independently reviewed each nonverbatim warning and compared the information in the resource with each line of content in the corresponding PI highlights. Concordance was defined as resource inclusion of all information from all lines of content in the PI highlights. Discordance was defined as resource omission of information from any line of content. The pharmacists compared their evaluations and discussed differences until consensus was reached. Third, the same two pharmacists categorized the discordances and classified them as major or minor. A major discordance was one in which the resource omitted (1) mention of a serious adverse reaction, (2) discussion of steps to reduce the risk of, minimize the severity of, or manage an adverse reaction, or (3) information about restricted distribution. Other discordances in the incidence, symptoms, expected outcome (e.g., death, hospitalization), or data source (e.g., postmarketing reports, clinical trials) for an adverse reaction were considered minor. Data analysis We used tests of proportions with an a priori significance level set at 0.05 to examine differences between resources in the percentage of warnings concordant with the PI highlights. All verbatim warnings were considered to be concordant with the PI highlights. We calculated a kappa statistic to assess interrater reliability between the two reviewers. We considered a kappa value of <0.20 as poor, 0.21–0.40 as fair, 0.41–0.60 as good, 0.61–0.80 as substantial, and >0.81 as excellent agreement.15 Results Seventy-one drugs, each with a unique boxed warning, were included in our study sample. Each PI highlights contained a mean of 4.8 lines of content (range, 1–12 lines). The percentage of boxed warnings in each resource that was concordant with the PI highlights varied (Table 1). The resource with the greatest number of warnings verbatim with the PI (DrugDex) was also the resource with the highest percentage of warnings concordant with the PI highlights. Conversely, the resource with the lowest number of warnings verbatim with the PI (LexiDrugs) was also the resource with the lowest percentage of warnings concordant with the PI (61%, p < 0.0001 for all paired comparisons). Table 1 Concordance of Boxed Warnings Between Drug Information Resources and Prescribing Information (PI), by Number of Drugs Resource No. (%)Drugs With Information Present (n = 71) No. (% Drugs With Information Present) Boxed Warnings Concordant With PI Boxed Warnings Verbatim With PI BlackBoxRx 70 (99) 61/70 (87) 44/70 (63) DrugDex 71 (100) 69/71 (98) 61/71 (86) Epocrates 67 (94) 57/67 (85) 2/67 (3) Facts and Comparisons 68 (96) 61/68 (90) 51/68 (75) Lexi-Drugs 70 (99) 43/70 (61) 0 PDR.net 52 (73) 47/52 (90) 46/52 (88) Resource No. (%)Drugs With Information Present (n = 71) No. (% Drugs With Information Present) Boxed Warnings Concordant With PI Boxed Warnings Verbatim With PI BlackBoxRx 70 (99) 61/70 (87) 44/70 (63) DrugDex 71 (100) 69/71 (98) 61/71 (86) Epocrates 67 (94) 57/67 (85) 2/67 (3) Facts and Comparisons 68 (96) 61/68 (90) 51/68 (75) Lexi-Drugs 70 (99) 43/70 (61) 0 PDR.net 52 (73) 47/52 (90) 46/52 (88) Open in new tab Table 1 Concordance of Boxed Warnings Between Drug Information Resources and Prescribing Information (PI), by Number of Drugs Resource No. (%)Drugs With Information Present (n = 71) No. (% Drugs With Information Present) Boxed Warnings Concordant With PI Boxed Warnings Verbatim With PI BlackBoxRx 70 (99) 61/70 (87) 44/70 (63) DrugDex 71 (100) 69/71 (98) 61/71 (86) Epocrates 67 (94) 57/67 (85) 2/67 (3) Facts and Comparisons 68 (96) 61/68 (90) 51/68 (75) Lexi-Drugs 70 (99) 43/70 (61) 0 PDR.net 52 (73) 47/52 (90) 46/52 (88) Resource No. (%)Drugs With Information Present (n = 71) No. (% Drugs With Information Present) Boxed Warnings Concordant With PI Boxed Warnings Verbatim With PI BlackBoxRx 70 (99) 61/70 (87) 44/70 (63) DrugDex 71 (100) 69/71 (98) 61/71 (86) Epocrates 67 (94) 57/67 (85) 2/67 (3) Facts and Comparisons 68 (96) 61/68 (90) 51/68 (75) Lexi-Drugs 70 (99) 43/70 (61) 0 PDR.net 52 (73) 47/52 (90) 46/52 (88) Open in new tab There were a total of 106 discordances found among all resources, 52 of which were considered to be major (Table 2). In general, resources with more warnings that were nonverbatim with the PI were associated with a higher number of discordances. Most major discordances involved omitted information for reducing risk or minimizing severity of a serious adverse reaction (e.g., monitoring before, during, or after therapy) (Table 3). There were several drugs with discordant boxed warnings in at least three of the resources; these were the tumor necrosis factor blocker adalimumab (Humira, Genentech, South San Francisco, CA), long-acting β-2 agonist formoterol fumarate (Perforomist, Dey Pharma, Basking Ridge, NJ), epidermal growth factor receptor antagonist panitumumab (Vectibix, Amgen, Thousand Oaks, CA), and uric-acid-lowering agent rasburicase (Elitek, Sanofi-Aventis, Bridgewater, NJ). The level of agreement between the two reviewers in identifying Table 3 Examples of Major and Minor Discordances Between Boxed Warning Information in the Prescribing Information Highlights and at Least One Drug Information Resource Generic Name (Brand Name) Drug Type Line(s) of Content Missinga Major Discordances  Adalimumab (Humira) Tumor necrosis factor (TNF) blocker “HUMIRA should be discontinued if a patient develops a serious infection or sepsis during treatment.” “Lymphoma and other malignancies, some fatal, have been reported in children and adolescent patients treated with TNF blockers, of which HUMIRA is a member.”  Aglucosidase alfa (Lumizyme) Enzyme-replacement therapy for Pompe disease “Because of the potential risk of rapid disease progression in Pompe disease patients less than 8 years of age, LUMIZYME is available only through a restricted distribution program called the LUMIZYME ACE Program. Only prescribers and healthcare facilities enrolled in the program may prescribe, dispense or administer LUMIZYME. LUMIZYME may be administered only to patients who are enrolled in and meet all the conditions of the LUMIZYME ACE Program. To enroll in the LUMIZYME ACE Program call 1-800-745-4447.”  Ambrisentan (Letairis) Vasodilator “Discontinue if aminotransferases are > 5 X ULN or if elevations are accompanied by bilirubin > 2 x ULN or by signs or symptoms of liver dysfunction.”  Bevacizumab (Avastin) Monoclonal antibody “Discontinue at least 28 days prior to elective surgery. Do not initiate Avastin for at least 28 days after surgery and until the surgical wound is fully healed.”  Dalteparin (Fragmin) Low-molecular-weight heparin “Factors that can increase the risk of developing epidural or spinal hematoma . . . in patients include . . . a history of spinal deformity or spinal surgery.”  Docetaxel (Taxotere) Antineoplastic “Contraindicated if history of severe hypersensitivity reactions to TAXOTERE or to drugs formulated with polysorbate 80.”  Fentanyl transmucosal lozenge (Actiq) Opioid analgesic “Use with strong and moderate CYP450 3A4 inhibitors may result in potentially fatal respiratory depression.”  Levofloxacin (Levaquin) Fluoroquinolone “This risk [of tendinitis and tendon rupture] is further increased in older patients usually over 60 years of age, in patients taking corticosteroid drugs, and in patients with kidney, heart or lung transplants.”  Ritonavir (Norvir) Antiretroviral “Co-administration of NORVIR with sedative hypnotics, antiarrhythmics, or ergot alkaloid preparations may result in potentially serious and/or life-threatening adverse events due to possible effects of NORVIR on the hepatic metabolism of certain drugs.”  Varenicline (Chantix) Smoking-cessation aid “Advise patients and caregivers that the patient should stop taking CHANTIX and contact a healthcare provider immediately if agitation, hostility, depressed mood, or changes in behavior or thinking that are not typical for the patient are observed, or if the patient develops suicidal ideation or suicidal behavior while taking CHANTIX or shortly after discontinuing CHANTIX.” Minor Discordances  Candesartan (Atacand) Cardiovascular agent “When used in pregnancy during the second and third trimesters, drugs that act directly on the renin-angiotensin system can cause injury and even death to the developing fetus.”  Formoterol inhalation solution (Perforomist) Long-acting β-2 agonist “Safety and efficacy of Perforomist in patients with asthma have not been established.”  Iopromide (Ultravist) Radiographic contrast agent “Inadvertent intrathecal administration may cause death, convulsions, cerebral hemorrhage, coma, paralysis, arachnoiditis, acute renal failure, cardiac arrest, seizures, rhabdomyolysis, hyperthermia, and brain edema.”  Metoclopramide Prokinetic, gastroesophageal reflux agent “There is no known treatment for tardive dyskinesia. In some patients, symptoms may lessen or resolve after metoclopramide therapy is stopped.”  Prasugrel (Effient) Antiplatelet agent “Stopping Effient, particularly in the first few weeks after acute coronary syndrome, increases the risk of subsequent cardiovascular events.”  Rosiglitazone (Avandia) Thiazolidinedione antidiabetic agent “A meta-analysis of 42 clinical studies (mean duration 6 months; 14,237 total patients), most of which compared AVANDIA to placebo, showed AVANDIA to be associated with an increased risk of myocardial ischemic events such as angina or myocardial infarction. Three other studies (mean duration 41 months; 14,067 total patients), comparing AVANDIA to some other approved oral antidiabetic agents or placebo, have not confirmed or excluded this risk. In their entirety, the available data on the risk of myocardial ischemia are inconclusive.” Generic Name (Brand Name) Drug Type Line(s) of Content Missinga Major Discordances  Adalimumab (Humira) Tumor necrosis factor (TNF) blocker “HUMIRA should be discontinued if a patient develops a serious infection or sepsis during treatment.” “Lymphoma and other malignancies, some fatal, have been reported in children and adolescent patients treated with TNF blockers, of which HUMIRA is a member.”  Aglucosidase alfa (Lumizyme) Enzyme-replacement therapy for Pompe disease “Because of the potential risk of rapid disease progression in Pompe disease patients less than 8 years of age, LUMIZYME is available only through a restricted distribution program called the LUMIZYME ACE Program. Only prescribers and healthcare facilities enrolled in the program may prescribe, dispense or administer LUMIZYME. LUMIZYME may be administered only to patients who are enrolled in and meet all the conditions of the LUMIZYME ACE Program. To enroll in the LUMIZYME ACE Program call 1-800-745-4447.”  Ambrisentan (Letairis) Vasodilator “Discontinue if aminotransferases are > 5 X ULN or if elevations are accompanied by bilirubin > 2 x ULN or by signs or symptoms of liver dysfunction.”  Bevacizumab (Avastin) Monoclonal antibody “Discontinue at least 28 days prior to elective surgery. Do not initiate Avastin for at least 28 days after surgery and until the surgical wound is fully healed.”  Dalteparin (Fragmin) Low-molecular-weight heparin “Factors that can increase the risk of developing epidural or spinal hematoma . . . in patients include . . . a history of spinal deformity or spinal surgery.”  Docetaxel (Taxotere) Antineoplastic “Contraindicated if history of severe hypersensitivity reactions to TAXOTERE or to drugs formulated with polysorbate 80.”  Fentanyl transmucosal lozenge (Actiq) Opioid analgesic “Use with strong and moderate CYP450 3A4 inhibitors may result in potentially fatal respiratory depression.”  Levofloxacin (Levaquin) Fluoroquinolone “This risk [of tendinitis and tendon rupture] is further increased in older patients usually over 60 years of age, in patients taking corticosteroid drugs, and in patients with kidney, heart or lung transplants.”  Ritonavir (Norvir) Antiretroviral “Co-administration of NORVIR with sedative hypnotics, antiarrhythmics, or ergot alkaloid preparations may result in potentially serious and/or life-threatening adverse events due to possible effects of NORVIR on the hepatic metabolism of certain drugs.”  Varenicline (Chantix) Smoking-cessation aid “Advise patients and caregivers that the patient should stop taking CHANTIX and contact a healthcare provider immediately if agitation, hostility, depressed mood, or changes in behavior or thinking that are not typical for the patient are observed, or if the patient develops suicidal ideation or suicidal behavior while taking CHANTIX or shortly after discontinuing CHANTIX.” Minor Discordances  Candesartan (Atacand) Cardiovascular agent “When used in pregnancy during the second and third trimesters, drugs that act directly on the renin-angiotensin system can cause injury and even death to the developing fetus.”  Formoterol inhalation solution (Perforomist) Long-acting β-2 agonist “Safety and efficacy of Perforomist in patients with asthma have not been established.”  Iopromide (Ultravist) Radiographic contrast agent “Inadvertent intrathecal administration may cause death, convulsions, cerebral hemorrhage, coma, paralysis, arachnoiditis, acute renal failure, cardiac arrest, seizures, rhabdomyolysis, hyperthermia, and brain edema.”  Metoclopramide Prokinetic, gastroesophageal reflux agent “There is no known treatment for tardive dyskinesia. In some patients, symptoms may lessen or resolve after metoclopramide therapy is stopped.”  Prasugrel (Effient) Antiplatelet agent “Stopping Effient, particularly in the first few weeks after acute coronary syndrome, increases the risk of subsequent cardiovascular events.”  Rosiglitazone (Avandia) Thiazolidinedione antidiabetic agent “A meta-analysis of 42 clinical studies (mean duration 6 months; 14,237 total patients), most of which compared AVANDIA to placebo, showed AVANDIA to be associated with an increased risk of myocardial ischemic events such as angina or myocardial infarction. Three other studies (mean duration 41 months; 14,067 total patients), comparing AVANDIA to some other approved oral antidiabetic agents or placebo, have not confirmed or excluded this risk. In their entirety, the available data on the risk of myocardial ischemia are inconclusive.” a For entries with underlined text, only the underlined text was missing from the resource. discordances was excellent (kappa = 0.86). Open in new tab Table 3 Examples of Major and Minor Discordances Between Boxed Warning Information in the Prescribing Information Highlights and at Least One Drug Information Resource Generic Name (Brand Name) Drug Type Line(s) of Content Missinga Major Discordances  Adalimumab (Humira) Tumor necrosis factor (TNF) blocker “HUMIRA should be discontinued if a patient develops a serious infection or sepsis during treatment.” “Lymphoma and other malignancies, some fatal, have been reported in children and adolescent patients treated with TNF blockers, of which HUMIRA is a member.”  Aglucosidase alfa (Lumizyme) Enzyme-replacement therapy for Pompe disease “Because of the potential risk of rapid disease progression in Pompe disease patients less than 8 years of age, LUMIZYME is available only through a restricted distribution program called the LUMIZYME ACE Program. Only prescribers and healthcare facilities enrolled in the program may prescribe, dispense or administer LUMIZYME. LUMIZYME may be administered only to patients who are enrolled in and meet all the conditions of the LUMIZYME ACE Program. To enroll in the LUMIZYME ACE Program call 1-800-745-4447.”  Ambrisentan (Letairis) Vasodilator “Discontinue if aminotransferases are > 5 X ULN or if elevations are accompanied by bilirubin > 2 x ULN or by signs or symptoms of liver dysfunction.”  Bevacizumab (Avastin) Monoclonal antibody “Discontinue at least 28 days prior to elective surgery. Do not initiate Avastin for at least 28 days after surgery and until the surgical wound is fully healed.”  Dalteparin (Fragmin) Low-molecular-weight heparin “Factors that can increase the risk of developing epidural or spinal hematoma . . . in patients include . . . a history of spinal deformity or spinal surgery.”  Docetaxel (Taxotere) Antineoplastic “Contraindicated if history of severe hypersensitivity reactions to TAXOTERE or to drugs formulated with polysorbate 80.”  Fentanyl transmucosal lozenge (Actiq) Opioid analgesic “Use with strong and moderate CYP450 3A4 inhibitors may result in potentially fatal respiratory depression.”  Levofloxacin (Levaquin) Fluoroquinolone “This risk [of tendinitis and tendon rupture] is further increased in older patients usually over 60 years of age, in patients taking corticosteroid drugs, and in patients with kidney, heart or lung transplants.”  Ritonavir (Norvir) Antiretroviral “Co-administration of NORVIR with sedative hypnotics, antiarrhythmics, or ergot alkaloid preparations may result in potentially serious and/or life-threatening adverse events due to possible effects of NORVIR on the hepatic metabolism of certain drugs.”  Varenicline (Chantix) Smoking-cessation aid “Advise patients and caregivers that the patient should stop taking CHANTIX and contact a healthcare provider immediately if agitation, hostility, depressed mood, or changes in behavior or thinking that are not typical for the patient are observed, or if the patient develops suicidal ideation or suicidal behavior while taking CHANTIX or shortly after discontinuing CHANTIX.” Minor Discordances  Candesartan (Atacand) Cardiovascular agent “When used in pregnancy during the second and third trimesters, drugs that act directly on the renin-angiotensin system can cause injury and even death to the developing fetus.”  Formoterol inhalation solution (Perforomist) Long-acting β-2 agonist “Safety and efficacy of Perforomist in patients with asthma have not been established.”  Iopromide (Ultravist) Radiographic contrast agent “Inadvertent intrathecal administration may cause death, convulsions, cerebral hemorrhage, coma, paralysis, arachnoiditis, acute renal failure, cardiac arrest, seizures, rhabdomyolysis, hyperthermia, and brain edema.”  Metoclopramide Prokinetic, gastroesophageal reflux agent “There is no known treatment for tardive dyskinesia. In some patients, symptoms may lessen or resolve after metoclopramide therapy is stopped.”  Prasugrel (Effient) Antiplatelet agent “Stopping Effient, particularly in the first few weeks after acute coronary syndrome, increases the risk of subsequent cardiovascular events.”  Rosiglitazone (Avandia) Thiazolidinedione antidiabetic agent “A meta-analysis of 42 clinical studies (mean duration 6 months; 14,237 total patients), most of which compared AVANDIA to placebo, showed AVANDIA to be associated with an increased risk of myocardial ischemic events such as angina or myocardial infarction. Three other studies (mean duration 41 months; 14,067 total patients), comparing AVANDIA to some other approved oral antidiabetic agents or placebo, have not confirmed or excluded this risk. In their entirety, the available data on the risk of myocardial ischemia are inconclusive.” Generic Name (Brand Name) Drug Type Line(s) of Content Missinga Major Discordances  Adalimumab (Humira) Tumor necrosis factor (TNF) blocker “HUMIRA should be discontinued if a patient develops a serious infection or sepsis during treatment.” “Lymphoma and other malignancies, some fatal, have been reported in children and adolescent patients treated with TNF blockers, of which HUMIRA is a member.”  Aglucosidase alfa (Lumizyme) Enzyme-replacement therapy for Pompe disease “Because of the potential risk of rapid disease progression in Pompe disease patients less than 8 years of age, LUMIZYME is available only through a restricted distribution program called the LUMIZYME ACE Program. Only prescribers and healthcare facilities enrolled in the program may prescribe, dispense or administer LUMIZYME. LUMIZYME may be administered only to patients who are enrolled in and meet all the conditions of the LUMIZYME ACE Program. To enroll in the LUMIZYME ACE Program call 1-800-745-4447.”  Ambrisentan (Letairis) Vasodilator “Discontinue if aminotransferases are > 5 X ULN or if elevations are accompanied by bilirubin > 2 x ULN or by signs or symptoms of liver dysfunction.”  Bevacizumab (Avastin) Monoclonal antibody “Discontinue at least 28 days prior to elective surgery. Do not initiate Avastin for at least 28 days after surgery and until the surgical wound is fully healed.”  Dalteparin (Fragmin) Low-molecular-weight heparin “Factors that can increase the risk of developing epidural or spinal hematoma . . . in patients include . . . a history of spinal deformity or spinal surgery.”  Docetaxel (Taxotere) Antineoplastic “Contraindicated if history of severe hypersensitivity reactions to TAXOTERE or to drugs formulated with polysorbate 80.”  Fentanyl transmucosal lozenge (Actiq) Opioid analgesic “Use with strong and moderate CYP450 3A4 inhibitors may result in potentially fatal respiratory depression.”  Levofloxacin (Levaquin) Fluoroquinolone “This risk [of tendinitis and tendon rupture] is further increased in older patients usually over 60 years of age, in patients taking corticosteroid drugs, and in patients with kidney, heart or lung transplants.”  Ritonavir (Norvir) Antiretroviral “Co-administration of NORVIR with sedative hypnotics, antiarrhythmics, or ergot alkaloid preparations may result in potentially serious and/or life-threatening adverse events due to possible effects of NORVIR on the hepatic metabolism of certain drugs.”  Varenicline (Chantix) Smoking-cessation aid “Advise patients and caregivers that the patient should stop taking CHANTIX and contact a healthcare provider immediately if agitation, hostility, depressed mood, or changes in behavior or thinking that are not typical for the patient are observed, or if the patient develops suicidal ideation or suicidal behavior while taking CHANTIX or shortly after discontinuing CHANTIX.” Minor Discordances  Candesartan (Atacand) Cardiovascular agent “When used in pregnancy during the second and third trimesters, drugs that act directly on the renin-angiotensin system can cause injury and even death to the developing fetus.”  Formoterol inhalation solution (Perforomist) Long-acting β-2 agonist “Safety and efficacy of Perforomist in patients with asthma have not been established.”  Iopromide (Ultravist) Radiographic contrast agent “Inadvertent intrathecal administration may cause death, convulsions, cerebral hemorrhage, coma, paralysis, arachnoiditis, acute renal failure, cardiac arrest, seizures, rhabdomyolysis, hyperthermia, and brain edema.”  Metoclopramide Prokinetic, gastroesophageal reflux agent “There is no known treatment for tardive dyskinesia. In some patients, symptoms may lessen or resolve after metoclopramide therapy is stopped.”  Prasugrel (Effient) Antiplatelet agent “Stopping Effient, particularly in the first few weeks after acute coronary syndrome, increases the risk of subsequent cardiovascular events.”  Rosiglitazone (Avandia) Thiazolidinedione antidiabetic agent “A meta-analysis of 42 clinical studies (mean duration 6 months; 14,237 total patients), most of which compared AVANDIA to placebo, showed AVANDIA to be associated with an increased risk of myocardial ischemic events such as angina or myocardial infarction. Three other studies (mean duration 41 months; 14,067 total patients), comparing AVANDIA to some other approved oral antidiabetic agents or placebo, have not confirmed or excluded this risk. In their entirety, the available data on the risk of myocardial ischemia are inconclusive.” a For entries with underlined text, only the underlined text was missing from the resource. discordances was excellent (kappa = 0.86). Open in new tab Table 2 Concordance of Boxed Warnings Between Drug Information Resources and Prescribing Information (PI), by Lines of Contenta Resource No. (%) Total Lines of Content From PI Highlights (n = 338) No. (% Total Lines of Content) Discordancesb Major Discordancesc Minor Discordancesd BlackBoxRx 337 (99) 16 (5) 9 (3) 7 (2) DrugDex 338 (100) 3 (1) 3 (1) 0 Epocrates 313 (93) 16 (5) 6 (2) 10 (3) Facts and Comparisons 323 (96) 10 (3) 4 (1) 6 (2) Lexi-Drugs 332 (98) 55 (17) 27 (8) 28 (8) PDR.net 256 (76) 6 (2) 3 (1) 3 (1) Resource No. (%) Total Lines of Content From PI Highlights (n = 338) No. (% Total Lines of Content) Discordancesb Major Discordancesc Minor Discordancesd BlackBoxRx 337 (99) 16 (5) 9 (3) 7 (2) DrugDex 338 (100) 3 (1) 3 (1) 0 Epocrates 313 (93) 16 (5) 6 (2) 10 (3) Facts and Comparisons 323 (96) 10 (3) 4 (1) 6 (2) Lexi-Drugs 332 (98) 55 (17) 27 (8) 28 (8) PDR.net 256 (76) 6 (2) 3 (1) 3 (1) a A line of content is a sentence or bulleted phrase in the boxed warning section of the PI highlights for a drug with a boxed warning. b A discordance is any discrepancy with a line of content in the boxed warning section of the PI highlight. c A major discordance is resource exclusion of (1) a mention of a serious adverse reaction, (2) measures to reduce the risk of, minimize the severity of, or manage an adverse reaction, or (3) information about restricted distribution. d A minor discordance is resource exclusion of symptoms, expected outcomes, or a data source for a serious adverse drug reaction. Open in new tab Table 2 Concordance of Boxed Warnings Between Drug Information Resources and Prescribing Information (PI), by Lines of Contenta Resource No. (%) Total Lines of Content From PI Highlights (n = 338) No. (% Total Lines of Content) Discordancesb Major Discordancesc Minor Discordancesd BlackBoxRx 337 (99) 16 (5) 9 (3) 7 (2) DrugDex 338 (100) 3 (1) 3 (1) 0 Epocrates 313 (93) 16 (5) 6 (2) 10 (3) Facts and Comparisons 323 (96) 10 (3) 4 (1) 6 (2) Lexi-Drugs 332 (98) 55 (17) 27 (8) 28 (8) PDR.net 256 (76) 6 (2) 3 (1) 3 (1) Resource No. (%) Total Lines of Content From PI Highlights (n = 338) No. (% Total Lines of Content) Discordancesb Major Discordancesc Minor Discordancesd BlackBoxRx 337 (99) 16 (5) 9 (3) 7 (2) DrugDex 338 (100) 3 (1) 3 (1) 0 Epocrates 313 (93) 16 (5) 6 (2) 10 (3) Facts and Comparisons 323 (96) 10 (3) 4 (1) 6 (2) Lexi-Drugs 332 (98) 55 (17) 27 (8) 28 (8) PDR.net 256 (76) 6 (2) 3 (1) 3 (1) a A line of content is a sentence or bulleted phrase in the boxed warning section of the PI highlights for a drug with a boxed warning. b A discordance is any discrepancy with a line of content in the boxed warning section of the PI highlight. c A major discordance is resource exclusion of (1) a mention of a serious adverse reaction, (2) measures to reduce the risk of, minimize the severity of, or manage an adverse reaction, or (3) information about restricted distribution. d A minor discordance is resource exclusion of symptoms, expected outcomes, or a data source for a serious adverse drug reaction. Open in new tab Discussion This study was a qualitative evaluation of boxed warning content in major drug information resources in comparison with the boxed warning in the current PI. Our findings demonstrate that some boxed warning content in these resources may be incomplete. Not surprisingly, those resources that tended to summarize, paraphrase, or condense the boxed warning were more likely to contain discordances. While variability in resource content is not a novel finding, inconsistencies or omission of information specific to the boxed warning is concerning because of the critical nature of this information.2,16,17 Although our evaluation designated certain types of information to be more important than others, it may be argued that all elements of the boxed warning are essential for providers to consider in their decision to use a specific drug therapy. Maintaining current boxed warning information requires continuous surveillance of several data sources, including (1) FDA MedWatch notifications of new or revised boxed warnings, (2) safety-related labeling changes that reveal the actual final text of the boxed warning, and (3) new drug approvals for drugs with a boxed warning at the time of approval.8,18,19 Some of the discrepancies, particularly those detected in resources that primarily presented warnings verbatim with the PI, simply reflected information that was outdated. Drug information resources collect, reconcile, and streamline information from published reviews, clinical trials, and expert opinion in an effort to deliver evidence-based, clinically relevant, up-to-date information. Publishers may supplement or replace information that they determine to be vague, lengthy, or difficult to interpret with more explicit, concise content that prescribers can more readily use at the point of care. Differences in editorial approaches to presenting drug information in these resources can affect the presentation of the boxed warning information. As shown by our results, resources that contained greater numbers of warnings that were nonverbatim with the PI were more likely to omit information from the actual boxed warning. Indeed, omission of monitoring recommendations intended to prevent a serious adverse reaction or exclusion of a major recognized risk may lead prescribers to make uninformed or misinformed decisions about the use of a drug. Absence of information on the nature of the evidence supporting the association of a drug with a serious adverse reaction may also affect an individual prescriber’s perception of the risks and benefits of drug use for a particular patient. For example, the boxed warning for rosiglitazone (Avandia, GlaxoSmith-Kline, Research Triangle Park, NC) states that results of a meta-analysis link the drug to an increased risk of myocardial ischemic events; however, the warning further states that “three other studies . . . comparing Avandia to some other approved oral antidiabetic agent or placebo, have not confirmed or excluded this risk. In their entirety, the available data on the risk of myocardial ischemia are inconclusive.” Resource omission of the latter two statements was considered a minor discordance in our study; however, it is possible that this information, when considered with the rest of the warning, could change a prescriber’s view of the drug’s risks. While it is not known if or how viewing the boxed warning in its original form, versus a truncated or modified version, may affect a prescriber’s attitude toward or willingness to use a drug, clinicians should be aware that resources differ in the design, editing, maintenance, and delivery of their content. Although the wording of some boxed warnings has been criticized as being vague and unhelpful, errors and interpretive differences may arise from resources’ decisions to modify, include, or exclude certain components of drug information from the PI.20–22 Our study had several limitations. First, we focused only on the section of each resource’s drug monograph that was designated as the boxed warning; it is possible that information missing from the boxed warning section appeared elsewhere in the resource. Second, we did not account for extraneous information in the resources’ boxed warnings (i.e., information present in the resource but not in the PI boxed warning). We also did not conduct evaluations for drugs with identical boxed warnings; some resources may have attributed the same classwide warning to all drugs within the same class while others may have not. Thus, our results apply only to resource omission of boxed warning information for drugs in our sample and cannot be used to draw conclusions about the overall boxed warning accuracy rate among the resources. Third, while we conscientiously maintained rigorous, objective, and consistent evaluation methods to identify discordances in boxed warning information, it is possible that other clinicians would interpret the information differently and disagree with our evaluations. Fourth, we did not analyze the design, editing, or maintenance processes for the databases to determine reasons for the discordances identified. We also did not assess how the discordances may affect prescriber attitudes or behavior toward a particular drug; further studies are needed to address these issues. Finally, our findings are limited to the warnings reviewed in the selected resources in June 2010. Conclusion Boxed warning information presented in major drug information resources may be missing key elements of the official boxed warning in the current PI. The current PI may be the most reliable approach to accessing the complete, up-to-date boxed warning for a given drug. Footnotes The authors have declared no potential conflicts of interest. Funded by a research training grant from the American Society of Health-System Pharmacy (ASHP) Research and Education Foundation. The ASHP Research and Education Foundation had no role in the study design, data collection, data analysis or interpretation, or writing or approval of the manuscript. References 1 21 C.F.R.1.21. http://edocket.access.gpo.gov/cfr_2001/aprqtr/pdf/21cfr201.57.pdf (accessed 2010 Nov 16). 2 Food and Drug Administration . Guidance for industry. Warnings and precautions, contraindications, and boxed warning sections of labeling for human prescription drug and biological products—content and format. Draft guidance . www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm075096.pdf (accessed 2010 Nov 15). 3 O’Connor NR . FDA boxed warnings: how to prescribe drugs safely . Am Fam Physician. 2010 ; 81 : 298 – 303 . Google Scholar PubMed WorldCat 4 Ceilley R Eisenthal A . The unintended effects of a boxed warning . J Clin Aesthet Dermatol. 2009 ; 2 (9) : 33 – 9 . Google Scholar PubMed WorldCat 5 Libby AM Orton HD Valuck RJ . Persisting decline in depression treatment after FDA warnings . Arch Gen Psychiatry. 2009 ; 66 : 633 – 9 . Google Scholar Crossref Search ADS PubMed WorldCat 6 Lasser KE Allen PD Woolhandler SJ et al. . Timing of new black box warnings and withdrawals for prescription medicines . JAMA. 2002 ; 287 : 2215 – 20 . Google Scholar Crossref Search ADS PubMed WorldCat 7 Lal R Kremzer M . Introduction to the new prescription drug labeling by the Food and Drug Administration . Am J Health-Syst Pharm. 2007 ; 64 : 2488 – 94 . Google Scholar Crossref Search ADS PubMed WorldCat 8 Cheng CM Guglielmo BJ Maselli J et al. . Coverage of FDA medication boxed warnings in commonly used drug information resources . Arch Intern Med. 2010 ; 170 : 831 – 3 . Google Scholar Crossref Search ADS PubMed WorldCat 9 Thompson CA . Hospital inspectors eye black-box warnings . Am J Health-Syst Pharm. 2008 ; 65 : 890 – 4 . News. Google Scholar Crossref Search ADS WorldCat 10 Rosenberg JM Koumis T Nathan JP et al. . Current status of pharmacist-operated drug information centers in the United States . Am J Health-Syst Pharm. 2004 ; 61 : 2023 – 32 . Google Scholar Crossref Search ADS PubMed WorldCat 11 Kupferberg N Jones HL . Evaluation of five full-text drug databases by pharmacy students, faculty and librarians: do the groups agree? J Med Libr Assoc. 2004 ; 92 : 66 – 71 . Google Scholar PubMed WorldCat 12 Institute for Safe Medication Practices . Medication safety tools and resources: black box warnings . www.ismp.org/Tools/default.asp (accessed 2010 Dec 10). 13 Food and Drug Administration . Center for Drug Evaluation and Research Drugs@FDA database . www.accessdata.fda.gov/scripts/cder/drugsatfda (accessed 2010 Nov 10). 14 National Library of Medicine . DailyMed Current Medication Information page . http://dailymed.nlm.nih.gov/dailymed (accessed 2010 Nov 16). 15 Landis JR Koch GG . The measurement of observer agreement for categorical data . Biometrics. 1977 ; 33 : 159 – 74 . Google Scholar Crossref Search ADS PubMed WorldCat 16 Clauson KA Marsh WA . Clinical decision support tools: analysis of online drug information databases . BMC Med Inform Decis Mak. 2007 ; 7 : 7 . Google Scholar Crossref Search ADS PubMed WorldCat 17 Polen HH Zapantis A Clauson KA et al. . Ability of online drug databases to assist in clinical decision-making with infectious disease therapies . BMC Infect Dis. 2008 ; 8 : 153 . Google Scholar Crossref Search ADS PubMed WorldCat 18 MedWatch . Safety alerts for human medical products webpage . www.fda.gov/Safety/MedWatch/SafetyInformation/SafetyAlertsforHumanMedicalProducts/default.htm (accessed 2010 Dec 10). 19 MedWatch . Drug safety labeling changes webpage . www.fda.gov/Safety/MedWatch/SafetyInformation/Safety-RelatedDrugLabelingChanges/default.htm (accessed 2010 Dec 10). 20 Lasser KE Seger DL Yu T et al. . Adherence to black box warnings for prescription medications in outpatients . Arch Intern Med. 2006 ; 166 : 338 – 44 . Google Scholar Crossref Search ADS PubMed WorldCat 21 Roden DM Shuldiner AR . Responding to the clopidogrel warning by the US Food and Drug Administration: real life is complicated . Circulation. 2010 ; 122 : 445 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 22 Lineberry TW Bostwick JM Beebe TJ et al. . Impact of the FDA black box warning on physician antidepressant prescribing and practice patterns: opening Pandora’s suicide box . Mayo Clin Proc. 2007 ; 82 : 518 – 20 . Google Scholar Crossref Search ADS PubMed WorldCat Copyright © 2011, American Society of Health-System Pharmacists, Inc. All rights reserved. TI - Boxed warning inconsistencies between drug information resources and the prescribing information JF - American Journal of Health-System Pharmacy DO - 10.2146/ajhp110025 DA - 2011-09-01 UR - https://www.deepdyve.com/lp/oxford-university-press/boxed-warning-inconsistencies-between-drug-information-resources-and-PNzwJ2SinS SP - 1626 VL - 68 IS - 17 DP - DeepDyve ER -