American Journal of Health-System Pharmacy

Traynor,, Kate

doi: 10.1093/ajhp/zxz049pmid: 30856247

View largeDownload slide Kathleen S. Pawlicki View largeDownload slide Kathleen S. Pawlicki At Beaumont Health in Michigan, managing drug product shortages is not a 1-person job—the task requires the equivalent of 6 full-time employees, or 12,000 staff hours each year, said Chief Pharmacist and ASHP President-Elect Kathleen S. Pawlicki. “We have about 75 [shortages] that we are actively monitoring. There are more out there, but those are the ones that affect us on a daily basis,” she said. She said there is often no advance notice of when a shortage will occur, and the health system does not learn about a shortage until it attempts to purchase the product. Pawlicki was one of the speakers at a February 7 briefing in Washington, D.C., on drug product shortages and drug costs. The event was sponsored by ASHP and the American Hospital Association (AHA) and featured highlights from the January 15 report “Recent Trends in Hospital Drug Spending and Manufacturer Shortages” from AHA, ASHP, and the Federation of American Hospitals. According to the report, essentially all hospitals and group purchasing organizations that responded to a survey indicated that managing drug product shortages is at least somewhat challenging. Nearly 80% reported that managing shortages is extremely challenging. The report also found that drug shortages place a financial strain on hospitals. Nearly 80% of survey respondents said drug shortages had at least a moderate effect on overall drug spending during fiscal years 2015–2017. Pawlicki said hospitals respond to drug shortages by finding alternative products and revising, sometimes repeatedly, the electronic medical record and other components of the medication use process. But she cautioned that those responses may come with a cost. “The medication use process has multiple steps within a healthcare setting . . . to assure that we deliver safe care to our patients. Frequent disruptions to this process and increased variability within the process create an opportunity or chance for error,” she said. Pawlicki also explained how rising drug costs are affecting her hospital. From 2015 to 2018, she said, Beaumont’s spending on drugs rose by $85 million due to increases in the prices of the hospital’s top 10 drugs, purchases of high-cost new drug products, skyrocketing costs for the generic drugs nitroprusside and isoproterenol, and restrictions from manufacturers on products available from wholesalers. Molly Smith, vice president of coverage and state issues for AHA, said that starting around 2013, AHA’s member hospitals began reporting that patients were coming to the emergency room because they couldn’t afford their medications. Hospitals were also concerned about multiple price hikes for medications throughout the course of each year. Those and related concerns led to the development of the January 15 report. “The results were not surprising to us,” Smith said. According to the report, hospitals’ spending on drugs increased by 18.5%, on average, from fiscal year 2015 to fiscal year 2017. Spending on inpatient medications rose by 9.6% per admission, on average, and spending on outpatient medications grew by 28.7% per encounter. The lack of generic competition is one factor that contributes to high drug prices, said Jack Hoadley, research professor emeritus at Georgetown University in Washington, D.C. AHA’s Smith urged attendees to foster generic competition by supporting the Creating and Restoring Equal Access to Equivalent Samples (CREATES) Act. The act would prevent manufacturers of brand-name drugs from withholding samples of those products from drug makers that want to produce generic versions of the medications. ASHP also supports the CREATES Act. Martin VanTrieste, President of Civica Rx, described how his not-for-profit organization is working to establish fair pricing for medications and prevent drug shortages. The organization represents about 800 U.S. hospitals that are collaborating to produce critical medications under a transparent pricing model sustained by long-term purchasing agreements. VanTrieste said Civica expects to begin manufacturing 14 drug products this year. © American Society of Health-System Pharmacists 2019. All rights reserved. For permissions, please e-mail: [email protected]. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Traynor,, Kate

doi: 10.1093/ajhp/zxz048pmid: 30852600

Pharmacists at the Department of Veterans Affairs (VA) who specialize in mental health disorders say their greatest rewards come from knowing they have helped patients improve their lives. “I have the best job in the world,” said Cynthia A. Gutierrez, clinical pharmacy specialist (CPS) and associate chief of clinical pharmacy programs at the South Texas Veterans Health Care System in San Antonio. Gutierrez practices at an outpatient specialty clinic for veterans with serious mental illnesses, including psychotic disorders, bipolar disorder, severe posttraumatic stress disorder (PTSD), and major depression. “I like working with this population, just because I think they are so underserved and misunderstood,” Gutierrez said. “It’s just very, very rewarding.” She recalled one patient she met soon after she started working at the clinic in 2003—a veteran with major depressive disorder with psychotic features. She said he came to his appointments but was “essentially mute” during the sessions. “I would see him every month, every 6 weeks. I tried to get him into psychotherapy and of course he didn’t really want to talk. Psychotherapy doesn’t work if you’re not willing to talk,” she said. Her biggest worry was that the veteran might harm himself. Over time, she said, he started to open up and was found to have PTSD, for which he received treatment. He became an active participant in his care and even researched ways to manage his nightmares and cognitive distortions. “Now he’s much more able to function. He’s taking on projects, working on his car. And he’s happier. He’ll come in and he’ll talk, and he’ll tell stories,” Gutierrez said. “It’s hard to believe that he’s the same person.” One of the most memorable patients for Elayne D. Ansara, CPS for the behavioral health interdisciplinary program at Richard L. Roudebush VA Medical Center in Indianapolis, Indiana, is a veteran she’s never met. “He probably lives about an hour and a half from the facility, and he actually sees his nurse practitioner via telehealth. So I’ve only talked to him on the phone,” Ansara said. She said the veteran suffers from PTSD and had nightmares that didn’t respond to standard medications. Ansara suggested a different medication but warned the patient that the drug came with an increased risk of adverse reactions. “He said, ‘OK.’ And I called him back a month later and both he and his wife could not believe the difference that it made. His nightmares, although they’re still there, are far less frequent. And he’s able to engage in day-to-day life, whereas he was not able to do that before,” Ansara said. As of January, 407 CPSs at 119 VA facilities operated under an advanced scope of practice in mental health, said Julie A. Groppi, pharmacy benefits management program manager for clinical pharmacy practice, policy, and standards at VA’s Clinical Pharmacy Practice Office. “There’s a great opportunity for pharmacists in mental health—especially in substance use disorders, helping to treat patients with addiction,” Groppi added. “I continue to see this as a growth area for our pharmacists.” That is the case at the William S. Middleton Memorial Veterans Hospital in Madison, Wisconsin, said CPS Theresa Frey. “We have pharmacists that are in advanced roles in mental health and substance abuse,” she said. “We serve on interdisciplinary teams with other providers and in collaborative practice settings in order to treat patients and meet patients’ needs.” Frey said she works with about 12 veterans per day at the clinic, where she manages medication therapy, schedules and reviews laboratory tests, and makes follow-up calls when needed. “But a big part of what I do is motivational interviewing. That’s a way to meet the patients where they’re at and help them reach goals related to things that matter to them. It can be very helpful for behavior changes,” she said. Frey said patients tell her they appreciate having someone to listen to them and help them turn their lives around. “It’s really rewarding,” she said. Christopher Thomas, inpatient mental health CPS at the Chillicothe VA Medical Center in Ohio, estimated that about a third of the veterans admitted to the 28-bed unit are withdrawing from various substances. Thomas said it is often hard for those veterans to change the behaviors that led to their admission. “A lot of what happens is, those patients go back to where they came from. And many times, in those environments, . . . they end up relapsing,” Thomas said. “For substance use disorders, it’s just kind of the nature of the beast.” In addition to patients undergoing withdrawal, Thomas said, the unit typically houses an equal mix of patients involuntarily admitted for schizophrenia and veterans receiving care for bipolar disorder or major depression. The unit’s treatment team consists of a psychiatrist, social worker, nurse, nurse practitioner, and Thomas, who counsels, consults, orders and reviews laboratory tests, and manages medications for a variety of conditions that affect the patients. “It could be anything,” he said. “Diabetes, hypertension, of course, psychiatry. Because unfortunately in psychiatry, a lot of these guys have unmet medical needs that often go neglected. So when they come to the unit, it’s a good time for us to do a checkup on them.” Thomas said he and the pharmacy and psychiatry residents who rotate through the unit do a lot of “deprescribing,” and patients are often discharged on fewer medications than when they were admitted. “The patients really do appreciate the fact that we try to be concise and really hone in on their medications,” he said. Thomas said perhaps the greatest challenge his patients face is adherence to their medication regimen after discharge from the unit. “Once they get out, they start feeling better. Then they oftentimes don’t take their medications,” he said. Thomas said he’s fortunate to serve these patients in the VA system with its “robust scope of practice.” Groppi said that in addition to expanding VA’s CPS workforce in mental health, the agency continues to add and train social workers, psychologists, chaplains, and others to support veterans and make mental health services more accessible. © American Society of Health-System Pharmacists 2019. All rights reserved. For permissions, please e-mail: [email protected]. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

doi: 10.1093/ajhp/zxz066pmid: 31361858

ASHP Past President Paul W. Bush and ASHP Chief Operating Officer Kasey K. Thompson delivered the presentation “Value of an Optimized Pharmacy Enterprise: Quality, Safety and Cost” on March 6 at the American College of Healthcare Executives Congress on Healthcare Leadership in Chicago, Illinois. Patrick Bridgeman, Pharm.D., BCPS, clinical assistant professor at Rutgers University’s Ernest Mario School of Pharmacy in Piscataway, New Jersey, served as an expert participant at a March 12 forum on medication-assisted treatment for opioid use disorder held in Washington, DC. The forum was hosted by the National Quality Forum and the Blue Cross Blue Shield Association. © American Society of Health-System Pharmacists 2019. All rights reserved. For permissions, please e-mail: [email protected]. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Dunn,, Louis;Anderson,, Jared

doi: 10.1093/ajhp/zxz039pmid: 31361857

barcodes, cost savings, hospital, informatics, pharmacy, pharmacy service, safety Barcode medication administration (BCMA) is endorsed by the American Society of Health-System Pharmacists (ASHP) for use in hospitals to improve safety during medication administration.1–3 In inpatient settings, BCMA verifies that patient, drug, dose, route, and time are correct by scanning patient and medication barcodes against a pharmacist-verified drug order.2,3 Since 2014, use of BCMA within the operating room (OR) has been recommended by the Anesthesia Patient Safety Foundation.4 The safety benefit, unique from inpatient application, is detection of incorrect drug administration when medications are not picked, delivered, and verified by pharmacy.4,5 For medication use processes handled exclusively by OR staff, there is no pharmacist double-check of drug product. Scanning medications during administration is a means of providing this double-check. Within our community hospital’s ORs, procedural medications are ordered, dispensed, and administered in this way, with no pharmacist order verification or visual checks of dispensed products. Additionally, ORs are not equipped with automated dispensing cabinets or automated anesthesia carts, which provide additional means of checking. These technologies have not been implemented due to cost barriers. However, medication barcode scanners are already available in OR suites and are exclusively used to scan blood product administration. BCMA systems can also improve documentation through National Drug Code (NDC) and expiration date capture. This information is required for compliance with the 340B Drug Discount Program. The 340B Program lowers drug costs for hospitals treating high percentages of impoverished and indigent populations by heavily discounting NDC-specific drugs.6 On average, 340B prices are 51% lower than wholesale acquisition cost, 39% lower than manufacturer acquisition price, and 15% lower than group purchasing organization price.7 So, given availability of BCMA scanners and potential benefits of use, it was decided to implement BCMA workflows for OR medication administration. Anesthesia, OR, and pharmacy stakeholders agreed to a goal of ≥90% BCMA compliance, which would allow for increased 340B medication purchases and cost savings for the hospital. BCMA functionality and workflow were tested, and staff training provided prior to implementation. Initial end-user feedback described an inability to scan induction medications, as current practice involved drawing up doses and discarding medication vials prior to procedures. In response, a laminated list of induction medication barcodes was distributed to OR staff to ensure scanning could occur without vials. Each medication was listed by name, vial size, and NDC code, so users could quickly and accurately find the correct code. This accommodation is not best practice as OR medication supply is subject to frequent change and laminated sheets can become outdated. Compliance rates were substantially below goal, with a peak of 28% and trough of 10%. These values represent the percentage of medications scanned divided by total medications administered in OR areas over in a 1-week period. Implementation of BCMA was unsuccessful for several reasons. First, the change in workflow did not result in any physical change to the work environment as scanners were already present on medication carts. Staff feedback cited difficulty in simply remembering to complete the BCMA process for this reason, despite education on the workflow. BCMA workflow was also perceived as time-adding rather than time-saving. Workflow required 4 physical actions: pick up a scanner, press button to scan medication, reholster scanner, and type a dose into the EHR. If an increased focus on human factors had been adopted prior to implementation, more deliberate efforts to maintain fast and patient-centered BCMA workflows could have improved compliance. Human factors engineering assesses an activity by its component parts through evaluation of physical demands, skill demands, mental workload, team dynamics, and other environmental aspects such as noise, lighting, distractions, and physical layout.8 In a future iteration of this project, a “hands-free” scanner could be used, so staff could complete BCMA workflow faster. A final problem was the rotating nature of OR staff. This hindered ability of staff members to fully embrace and consistently use the workflow. To complicate matters, implementation was only at a single site, and workflows remained unchanged at other hospitals staffed by the same employees. BCMA can improve safety in operating room settings and provide NDC documentation for 340B Program compliance purposes, facilitating significant cost savings for a hospital. Implementation at a single site was unsuccessful due to flaws in workflow design. Future improvements can be made to improve BCMA workflow within the OR environment. Acknowledgments The authors acknowledge the assistance of Brandon Ordway, Pharm.D., M.S.; Ashlee Anderson, Pharm.D., BCPS; Kathryn Kline, DNAP, CRNA; and the HealthEast Residency Oversight Committee. Disclosures The authors have declared no potential conflicts of interest. The Letters column is a forum for rapid exchange of ideas among readers of AJHP. Liberal criteria are applied in the review of submissions to encourage contributions to this column. The Letters column includes the following types of contributions: (1) comments, addenda, and minor updates on previously published work, (2) alerts on potential problems in practice, (3) observations or comments on trends in drug use, (4) opinions on apparent trends or controversies in drug therapy or clinical research, (5) opinions on public health issues of interest to pharmacists in health systems, (6) comments on ASHP activities, and (7) human interest items about life as a pharmacist. Reports of adverse drug reactions must present a reasonably clear description of causality. Short papers on practice innovations and other original work are included in the Notes section rather than in Letters. Letters commenting on an AJHP article must be received within 3 months of the article’s publication. Letters should be submitted electronically through http://ajhp.msubmit.net. The following conditions must be adhered to: (1) the body of the letter must be no longer than 2 typewritten pages, (2) the use of references and tables should be minimized, and (3) the entire letter (including references, tables, and authors’ names) must be typed double-spaced. After acceptance of a letter, the authors are required to sign an exclusive publication statement and a copyright transferal form. All letters are subject to revision by the editors. References 1. ASHP Section of Pharmacy Informatics and Technology. ASHP statement on bar-code-enabled medication administration technology . Am J Health-Syst Pharm. 2009 ; 66 : 588 - 90 . Crossref Search ADS PubMed 2. Nolen AL , Rodes WD 2nd . Bar-code medication administration system for anesthetics: effects on documentation and billing . Am J Health-Syst Pharm. 2008 ; 65 : 655 - 9 . Google Scholar Crossref Search ADS PubMed 3. Shah K , Lo C , Babich M et al. barcode medication administration technology: a systematic review of impact on patient safety when used with computerized prescriber order entry and automated dispensing devices . Can J Hosp Pharm. 2016 ; 69 : 394 - 402 . Google Scholar PubMed 4. Bree Brown L . Medication administration in the operating room: new standards and recommendations . AANA. 2014 ; 8 : 465 - 9 . 5. Litman RS . How to prevent medication errors in the operating room? Take away the human factor . Br J Anaesth. 2018 ; 120 : 438 - 40 . Google Scholar Crossref Search ADS PubMed 6. Apexus. 340B University: Grapevine edition (2016 Jan 25) . https://docs.340bpvp.com/documents/public/resourcecenter/340B_University/01.25.2016/Janaury_2016_340B_University_Slide_Deck_for_App.pdf (accessed 2018 Apr ). 7. PYA. Journey to 340B compliance (2017 Nov 27). https://www.pyapc.com/insights/pya-experts-discuss-340b-compliance-alternative-payment-models-and-healthcare-accounting-and-reporting-at-tscpa-conference/ (accessed 2018 Jun ). 8. Agency for Healthcare Research and Quality . Human factors engineering (updated 2017 Jun) . https://psnet.ahrq.gov/primers/primer/23/Medication-Errors-and-Adverse-Drug-Events (accessed 2018 Jun ). © American Society of Health-System Pharmacists 2019. All rights reserved. For permissions, please e-mail: [email protected]. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Pabst,, Damon;Ogden, Richard, K

doi: 10.1093/ajhp/zxz041pmid: 30887018

communication, high reliability, resilience, safety Communication within a health-system pharmacy is challenging due to the large number of personnel that must be communicated with, decentralized pharmacies within the system, and the preponderance of emails that are sent on a daily basis. Dissemination of information via email has historically been a primary mode of communication; however, employees find it difficult to perform their required tasks as well as finding time to read their emails. As hospitals strive to become high reliability organizations (HROs) it is vital that effective communication is present. One of the principles of HROs is a commitment to resilience. Resilience is the capability of an organization to continue to function or to quickly return to a functional state following an error.1 An HRO has developed tools to not only identify the error but also contain and prevent the error from happening again.2,3 The organization must be able to continue functioning after an error occurs. This requires effective communication when vital process changes are required in a short space of time. Standard process when errors occur is to perform a deep dive to better understand contributing and causal factors leading to the event. From this process, we are better able to identify trends which, if fixed, are most likely to prevent recurrence of errors. With communication a common factor in patient safety events, we focused on increasing reliability by developing a tool to disseminate information quickly and effectively when risk is identified. We have since developed a tiered communication system within our pharmacy department consisting of “red flag alerts” and “safety blitzes.” When a serious adverse drug event or a near miss that could have resulted in a serious safety event occurs, a red flag alert is initiated by a member of the pharmacy management team. This alert consists of a red flag page to all pharmacy management, notifying them of a time to meet. The event is then discussed to determine if a red flag alert email is needed. Process changes that can be implemented in the short term to mitigate any future occurrence of the event are determined. A red flag alert email is composed in SBAR (situation, background, assessment, recommendation) form with Red Flag Alert in the subject line. This email is sent to the pharmacy department, printed, and displayed in each satellite pharmacy for 1 week. In addition, the managers discuss the event and solutions with each employee on the same day that the alert is sent. The red flag alert is discussed for 1 week at the twice-daily pharmacy operational huddles. Any suggestions or questions regarding the red flag alert that are discussed at the huddle are presented to the management team. Our pharmacy department uses a charge pharmacist report that is emailed to the department twice daily. The red flag alert is outlined in this report. When the night shift arrives, a manager will call the night charge pharmacist and discuss the red flag alert. This pharmacist ensures that the alert is communicated orally to each overnight pharmacy employee. When the issue has been resolved, a red flag resolution email is sent to all pharmacy staff with the actions taken to correct the issue. To ensure a consistent approach to the red flag alert communication strategy, a standardized checklist is completed. A safety blitz is used whenever a theme of events appears in the event-reporting system or is identified by a member of the pharmacy management team. These events do not require an urgent response. A manager will discuss the safety blitz at the weekly pharmacy leadership meeting. If the topic is approved to become a safety blitz, then a safety blitz email in SBAR form is sent to the pharmacy department; Safety Blitz is placed in the subject line. In addition, the pharmacy managers will round to influence on the topic. This allows for a discussion among employees for additional ideas to mitigate the issue and communication of best practices. The safety blitz is printed and hung in each pharmacy area and is discussed daily for 1 week at the twice-daily operational huddles of the pharmacy department. Both the red flag alert and safety blitz are used with discretion to ensure the emails do not become routine. We have found this communication system to be an effective way to share important information with the department. The authors have declared no potential conflicts of interest. References 1. Hollnagel E . Resilience: the challenge of the unstable . In: Hollnagel E , Woods DD , Leveson N , eds. Resilience engineering:concepts and precepts . Burlington, VT : Ashgate ; 2006 : 16 . 2. Wildavsky A. Searching for safety . New Brunswick, NJ : Transaction ; 1991 . 3. Weick KE , Sutcliffe KM. Managing the unexpected . San Francisco, CA : Jossey-Bass ; 2007 : 14 . © American Society of Health-System Pharmacists 2019. All rights reserved. For permissions, please e-mail: [email protected]. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Kisch, Glenn, L

doi: 10.1093/ajhp/zxz040pmid: 30887016

competency, pharmacist, shortage Beginning in 2015, the Franciscan Alliance—Central Indiana Region department of pharmacy adopted a policy detailing the selection and measurement of pharmacist competencies.1 Each fall, pharmacists are selected and interviewed with open-ended questions designed to determine areas of practice that are new, changed, high risk, or problematic. The interview responses are tabulated, and activities that are frequently mentioned are selected for competency measurement. Since our competency needs assessment was performed in the fall of 2017, the critical shortages of 0.9% sodium chloride injection were frequently identified as a high-risk and problematic aspect of the job. However, since the shortages were caused by unanticipated, catastrophic events—Hurricanes Irma and Maria—there seemed to be little we could provide to staff in the way of education for future unanticipated events. In other words, our challenge was how to formulate a competency measurement for an event that we never anticipated happening. The Wright method allows for discussion–reflection groups as a way of measuring competency. Other common examples of using discussion–reflection groups to measure competency include a debriefing session after a cardiac arrest, debriefing session after a mock event or disaster, a discussion group using a hypothetical situation, and a discussion group to analyze a sentinel event. Between April 30 and June 14 of 2018, we scheduled 11 discussion–reflection meetings. Present at each meeting were a facilitator (pharmacist educator), the pharmacy director, the pharmacy inventory supervisor, and 2 to 5 clinical staff pharmacists. Our goals were to evaluate the critical thinking skills of participants, encourage group cohesiveness, and promote preparedness (emotionally and psychologically) for future shortages. These goals were reviewed at the beginning of each meeting, and the reasons why we were using a novel way of measuring competency were reviewed. The response time was divided into 3 sections: reaction, analysis, and application. Each pharmacist had an opportunity to discuss 1 question from each of the 3 sections. For the reaction section, the questions were as follows: “What went well during the shortage crisis?” “What helped you cope with the additional stress?” For the analysis section, the questions were as follows: “What were the main issues that you had to deal with?” “What tools would have been helpful during the shortage situation?” “Evaluate this statement: ‘The pharmacy needs to make sure that shortages are invisible to physicians, nurses and patients.’” “How were you best able to assist end users such as physicians and nurses?” “What opportunities to lead or manage technicians presented themselves?” For the application section, the questions were as follows: “What are 1 or 2 ‘take-aways’ that will help you in the future?” “Are there any outstanding issues we need to discuss before we close?” The facilitator wrote down the answers to the questions as they were stated. Each participant comment was then classified into 1 of several categories and then categorized as either appreciative or critical. Enough time was provided so that pharmacists could affirm or otherwise discuss each participant’s statement. The pharmacy director and inventory supervisor were also encouraged to respond if appropriate, in order to address a particular question or issue. A frequently expressed suggestion for the future was for the department to keep a repository of updated shortage information. This already existed, but it was modified to be more descriptive of shortage status and severity. At the end of each meeting, there was a consistent expression of confidence. Most members agreed that it was a difficult trial and that while we had been tested, we had survived and were a better organization for it. To prepare for future shortages, we were able to creatively document competency using the Wright model. Team members were able to witness other team members performing critical tasks in the areas of purchasing, inventory management, informatics, education, and management. The meetings promoted appreciation for the struggles experienced by other team members and allowed our manager multiple opportunities to point out areas in which team members exceeded expectations and helped the department persevere through the critical shortage. In addition, pharmacists were given the opportunity to express individual struggles and coping mechanisms. Finally, several suggestions that were made during the meetings will be used in the future to communicate shortage information and direct alternative therapies. The net result is a more cohesive and self-aware team that will be able to face future catastrophic events with greater confidence. The author has declared no potential conflicts of interest. References 1. Wright D. The ultimate guide to competency assessment in health care . 3rd ed . Minneapolis : Creative Health Care Management, Inc ; 2005 . © American Society of Health-System Pharmacists 2019. All rights reserved. For permissions, please e-mail: [email protected]. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

McCarthy, Bryan, C;Dickerson, David, M;Knoebel, Randall, W

doi: 10.1093/ajhp/zxz038pmid: 30916312

controlled substances, opioid crisis, public health How often have patients expressed interest in returning unused, unwanted, or expired controlled substances to your patient-facing pharmacy? Controlled substance collection receptacles installed in pharmacy lobbies or other patient care areas are a means to lawfully provide patients this opportunity, and they also create many benefits for pharmacy leaders to strongly consider. The purpose of this article is to share the experience and lessons learned before, during, and after controlled substance collection receptacle installation by the department of pharmacy leadership at the University of Chicago Medicine (UCM). Opioid abuse is a forefront public health challenge in the United States.1 Over 33,000 deaths occurred due to opioid overdose in 2015, which represents a 400% increase since 1999.2 Prescription opioids are considered a significant contributing factor to the opioid epidemic. Opioid abuse prevention and treatment efforts must largely include health system pharmacist and multidisciplinary stakeholders. Physician prescribing-behavior intervention is a significant opportunity to reduce the opioid supply entering the community.3 Unused prescribed opioids may be subject to sharing, selling, and diversion. Nonmedical opioid users often obtain supplies from friends or relatives for free.4 A majority of adult Americans reported in a recent survey that it is “somewhat easy” to access pain killers prescribed to others.5 Historically, opportunities for patients to dispose of their unused opioids have been limited to using authorized law enforcement agency take-back programs, flushing them down the sink or toilet at home, or discarding them in household trash.6 The United States Drug Enforcement Administration (DEA) issued regulations to expand options for patient disposal of unused, unwanted, or expired controlled substances in 2014. The goals are to (1) promote secure, convenient, and responsible controlled substance collection and destruction, (2) decrease controlled substance introduction into the environment, and (3) reduce the supply of unused controlled substances in the home and thus reduce risk of diversion or harm.6 Specific public and private entities, including health systems with an onsite pharmacy, are eligible to participate in controlled substance take-back events and mail-back programs and implement collection receptacles on a voluntary basis. Health systems with an onsite pharmacy interested in implementing a controlled substance collection receptacle must comply with detailed DEA requirements. A written request to modify existing DEA controlled substance registration must be submitted to become an authorized collector.7 The collection receptacle must be securely fastened to a permanent structure in a location that is regularly monitored by employees and not in proximity of emergency or urgent care.8 Collected controlled substances must be destroyed and rendered “non-retrievable”; however, no specific method is endorsed to accomplish this.9 Robust collection receptacle inner-liner requirements include size and serial number markings and immediate sealing after removal from the collection receptacle by a minimum of 2 employees.10 The UCM Pain Stewardship Committee is composed of multidisciplinary membership including pharmacists and physicians specialized in pain medicine, anesthesiology, and implementation medicine. The value of UCM seeking controlled substance collector authorization was discussed by this committee in 2016. An online search of authorized controlled substance collector locations within a 20-mile radius of the main UCM campus revealed a significant shortage of local patient options. Assisted removal of unused, unwanted, or expired controlled substances from the community also aligned with the “Patient Experience” and “Quality and Safety” pillars of the organization’s annual operating plan. Shortly after discussion, a recommendation was provided to the UCM department of pharmacy leadership to pursue registration and implementation of a controlled substance collection receptacle. The first challenge identified by the department of pharmacy was internal capabilities to perform “nonretrievable” destruction of controlled substances. Internal interpretation of processes required to permanently alter the physical or chemical states of the controlled substances included incineration, mechanical crushing, and other means that were not readily available. A request for third-party proposals was subsequently initiated in collaboration with the supply chain department to source a collection receptacle vendor. This vendor would serve as an authorized entity to transfer collected controlled substances for destruction offsite and also provide the collection receptacle and inner liners. Because the financial expense related to a collection receptacle was not budgeted for, discretionary capital was approved and allocated for this. The second challenge identified was coordination of the multiple internal multidisciplinary subject matter experts required to create a detailed organization policy and procedure to achieve and maintain DEA regulation compliance. Representatives from Risk Management provided supplementary DEA regulation interpretation and recommendations to achieve quality assurance with collection receptacle inner-liner documentation requirements. Representatives from Security evaluated the proposed location of the collection receptacle, including accessible security camera view angles. Representatives from Waste Management cross-referenced the new controlled substance disposal process with existing organization pharmaceutical waste disposal processes. Significant concerns from the various subject matter experts were discussed and addressed. The destruction processes of the collection receptacle vendor were audited for both DEA and United States Environmental Protection Agency regulation compliance. Specific items were prohibited from the collection receptacle using patient-facing signage, including Schedule I controlled substances, syringes and needles, and chemotherapy waste. UCM staff members were prohibited from accepting and placing controlled substances into the collection receptacle on behalf of patients. The pharmacist in charge and a security representative were assigned responsibility to document, install, replace, store, and ship collection receptacle inner liners. The selected vendor collection receptacle featured 2 unique locks and keys required at the same time for inner-liner access. To minimize risk of diversion, the department of pharmacy and the security department were each assigned a separate, unique key. A controlled substance collection receptacle was installed in the UCM onsite retail pharmacy lobby in 2017, and a communication campaign commenced. The chief medical officer and department of pharmacy leadership coauthored an email communication to all physicians describing the collection receptacle location and patient counseling points. The onsite retail pharmacy system vendor was engaged to customize an auxiliary label that automatically populates on all dispensed Schedule II–V controlled substances prescription vials. The label reads “Bring Meds Back.” Infographics were developed and included in relevant patient care orientation materials and physician offices and front desk areas and were prominently displayed on the health system’s digital media monitors. Finally, an organization newsletter article was published to promote all-staff awareness. One year later, controlled substance collection receptacle implementation at UCM has been largely successful and celebrated internally. Senior organization leadership, physicians, and patients have recognized and endorsed the value of this service and the removal of unused, unwanted, or expired controlled substances from the community. Frequency of inner-liner change and disposal due to a full collection receptacle increased from implementation from the vendor-recommended 1 change every 3 months (for the size of our institution) to 1 change every other week. The department of pharmacy is currently exploring installation of a second controlled substance collection receptacle. Several lessons may be of value to organizations pursuing implementation of a controlled substance collection receptacle. Multidisciplinary leadership is critical to gauge organization value in seeking DEA controlled substance collector authorization. If working with a third-party collection receptacle vendor, carefully use primary resources to determine respective DEA regulation compliance. Lastly, consider the appropriate time and resources needed for collection receptacle implementation. UCM implementation spanned approximately 6 months from concept to completion. Allow for internal multidisciplinary subject matter experts’ input and ownership in the collection-receptacle policies and procedures to achieve and maintain DEA regulation compliance. Prior to writing this article, national presentations of this patient safety initiative by the authors created several formal opportunities to assist peer institutions in implementing a similar controlled substance receptacle at their health system. These collaborations increase the public health impact of a single institution’s quality improvement work and reflect a need for ongoing dissemination and discussion of successes in health care delivery science. The authors have declared no potential conflicts of interest. References 1. President’s Commission on Combating Drug Addiction and the Opioid Crisis . Interim report , 2017. https://www.whitehouse.gov/ondcp/presidents-commission (accessed 2018 Jun 25 ). 2. Centers for Disease Control and Prevention . Wide-ranging online data for epidemiologic research (WONDER) (updated 2016 April 5). https://healthdata.gov/dataset/wide-ranging-online-data-epidemiologic-research-wonder (accessed 2019 Mar 4). 3. Hill MV , Stucke RS , McMahon ML et al. An educational intervention decreases opioid prescribing after general surgical operations . Ann Surg. 2018 ; 267 : 468 -72 . Google Scholar Crossref Search ADS PubMed 4. Jones CM , Paulozzi LJ , Mack KA . Sources of prescription opioid pain relievers by frequency of past-year nonmedical use united states, 2008–2011 . JAMA Intern Med. 2014 ; 174 : 802 - 3 . Google Scholar Crossref Search ADS PubMed 5. Fry E . Here’s who Americans blame most for the opioid epidemic (Jun 21, 2017 ). http://fortune.com/2017/06/21/opioid-epidemic-blame-doctors/ (accessed 2018 Jun 25 ). 6. Secure and Responsible Drug Disposal Act of 2010 , 156 Cong Rec H7316. 7. 21 C.F.R. 1317.40. 8. 21 C.F.R. 1317.75. 9. 21 C.F.R. 1300.05. 10. 21 C.F.R. 1300.60. © American Society of Health-System Pharmacists 2019. All rights reserved. For permissions, please e-mail: [email protected]. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Farzadeh,, Shireen

doi: 10.1093/ajhp/zxz037pmid: 30934059

clinical pharmacist, communication, community service, student pharmacist, volunteer Many pharmacy students have developed a passion for the pharmacy profession by participating in patient care projects and outreach events in their communities. Volunteering allows pharmacy students to give back to their communities, develop skills needed for practice in an ambulatory care or community setting, gain insight into the lives of the people who are underserved, and learn about people of different cultures and backgrounds. Why should pharmacy students volunteer? To develop empathy. Empathy involves acknowledging patients’ concerns, incorporating patients’ perspectives into care, and demonstrating appropriate verbal and nonverbal language. Empathy is essential in developing successful relationships between pharmacists and patients and optimizing clinical outcomes. Educating the underserved can help pharmacy students develop empathy for their patients. Examples of opportunities to educate the underserved include presenting about pediatric immunizations at a young mothers’ shelter or providing diabetes education at a homeless shelter. For example, when presenting on diabetes at a homeless shelter, pharmacy students must be sensitive to the meals accessible to the audience, as well as the stress and financial struggles faced by that the homeless population, and should tailor the presentation with those factors in mind. To develop communication skills for various age groups. Each age group of patients requires pharmacists to incorporate different strategies to prevent drug-related problems and improve patient adherence. Pharmacy students have the opportunity to improve their communication skills with people of all ages at outreach events. Some examples of this might be talking to students at middle schools about the dangers of abusing prescription drugs or by helping elderly patients at senior centers select optimal Medicare Part D plans. With young patients, pharmacists may ask how they would like their medications flavored or advise them not to share their medications. With elderly patients, pharmacists may need to take into account how cognitive impairment can affect patients’ abilities to adhere to therapy. Pharmacists can help elderly patients by supplying medications in easy-open bottles or printing large font educational materials. Having patience and understanding for the various behaviors and conditions affecting patients of all ages is crucial in creating patient-centered relationships. To gain cultural competence. Providing blood pressure readings, diabetes risk assessments, and vision screenings at health fairs in parks and hospitals allows pharmacy students to practice their communication skills with, and develop understanding of, people of different cultures and backgrounds. Providing direct patient care at community service events and developing knowledge of foreign languages may help pharmacy students learn how to effectively communicate with patients. Pharmacists must ensure patients understand them or, if this is not possible, at the minimum give patients printed materials in the their native language. Pharmacists must be realistic when suggesting the types of healthy meal options for patients of different cultural backgrounds with hypertension or diabetes, as people are more likely to eat healthy meals suitable for their cultures. Pharmacists must also be aware of the various alternative medicines that patients of different cultures may use, such as homeopathy. If pharmacists need to discourage use of any form of alternative medicines because they are harmful to the patients’ drug regimens, they must do so respectfully. Community service is an opportunity to gain cultural competence, which is essential in developing strong relationships with patients and optimizing patients’ adherence to therapy. To find a faculty mentor. Besides teaching pharmacy students in classrooms, faculty members often practice and precept at sites in the community. By learning from and observing faculty members who may attend community service events, pharmacy students can develop mentor–mentee relationships. That connection can turn into an opportunity to conduct research or receive a glowing letter of recommendation. To further develop drug information and patient counseling skills. One of the best ways to practice drug information skills is by reviewing patient medications at “brown bag” events. These events encourage patients to bring a list of their medications for pharmacy students and preceptors to review. Pharmacy students can gain experience with finding drug interactions, assessing patient adherence, and counseling on medications. At flu clinics, students can gain experience in counseling patients on the importance of staying up-to-date with vaccinations. To develop new friendships. Volunteering with other pharmacy students makes the experience more enjoyable, as pharmacy students can take a break from studying to come together to speak to patients, make a difference in people’s lives, and perhaps go out together for a treat afterwards. Pharmacy students who gather with the same passion for community service may develop strong bonds and friendships that can help them grow academically and emotionally—whether it be through partnering on research projects, sharing study notes, or supporting each other through the rigors of pharmacy school. Community service opportunities outside of pharmacy. Pharmacy students should be aware of the plethora of volunteer opportunities available in their own communities. Volunteering within the medical sector is an outstanding opportunity and accomplishment, but it is not feasible for everyone. Volunteering at non–pharmacy-related community service events is just as effective in fostering personal growth. While delivering sandwiches to the homeless or cooking meals for families at Ronald McDonald House, pharmacy students can learn more about the challenges of the underserved population. Volunteering at community service events will help pharmacy students become better pharmacists and better people. Challenges of community service. A common reason pharmacy students hesitate to volunteer is that the time commitment takes away from the time for studying. However, the time spent volunteering will be helpful in preparing students to become successful practicing pharmacists. Another challenge is the fear or apprehension that goes along with entering unfamiliar environments. Many pharmacy students are uneasy about interacting with the homeless and underserved. Also, language barriers could potentially limit pharmacy students from providing care to certain patient populations of different cultures and backgrounds. Working with preceptors or others with experience in these community service areas can help pharmacy students increase their comfort level and gain skills in adapting patient care to the population served (e.g., developing treatment plans based on patient circumstances, use of translator when needed). There are many ways to overcome the challenges that go along with community service events, and pharmacy students should not be discouraged from volunteering their time in providing patient care and education to their communities. How to find community service opportunities. There are multiple ways to become involved in community service. As a first-year (P1) pharmacy student, the easiest way to find community service opportunities is to join pharmacy organizations (e.g., Student Society of Health-System Pharmacists, American Pharmacists Association—Academy of Student Pharmacists, Student College of Clinical Pharmacy), or non-pharmacy service organizations (e.g., Circle K International, Habitat for Humanity). P2 and P3 pharmacy students should be responsible for organizing community service events, as they have developed relationships with faculty members or have executive board positions requiring them to be in charge of finding opportunities. Pharmacy students can reach out to their local high schools, middle schools, elementary schools, churches, senior centers, and libraries to ask if they would like to have pharmacy students present a pertinent health topic or host a health fair. Pharmacy students can also look for volunteering opportunities online through organizations such as the American Diabetes Association or American Heart Association. Students should also consult faculty members when looking for volunteer opportunities. Faculty members play a role in assisting pharmacy students in setting up community service events, as faculty members often host health fairs at their clinical or community sites or volunteer regularly at local community centers. Pharmacy students may also have opportunities to attend community service events with their preceptors on rotations and may want to consider that when selecting rotations. It is essential that pharmacy students network with the people who host the community service events, such as the community center coordinator or library director, and make long-lasting good impressions. Closing notes. The passion that pharmacy students gain from volunteering for patient-care projects and outreach events truly shows during residency interviews; organizing community service events provides pharmacy students with leadership experience. By volunteering, pharmacy students are able to network within their school and within their communities. Community service events are tremendous opportunities for pharmacy students to communicate with people from a variety of backgrounds, cultures, living standards, and ages. Practicing direct and indirect patient care in the community prepares pharmacy students to become more competent and empathetic pharmacists in the future. The author has declared no potential conflicts of interest. The Pharmacy Student Forum column features articles that address the unique needs of students throughout school as they transition into new practitioners. Authors include students and others with expertise in a topic of interest to students. AJHP readers are invited to submit topics or articles for this column to the Pharmacy Student Forum, c/o Diana Dabdub, 4500 East-West Highway, Suite 900, Bethesda, MD 20814 (301-664-8854 or [email protected]). Published by Oxford University Press on behalf of the American Society of Health-System Pharmacists 2019. This work is written by (a) US Government employee(s) and is in the public domain in the US.

Naidjate, Safiya, S;Zullo, Andrew, R;Dapaah-Afriyie,, Ruth;Hersey, Michelle, L;Marshall, Brandon D, L;Winkler, Richelle, Manalang;Berard-Collins,, Christine

doi: 10.1093/ajhp/zxz034pmid: 30873537

Abstract Purpose The optimal health care delivery models for providing services to patients with infections caused by hepatitis C virus (HCV) remain unknown. Pharmacist involvement may be a key component of optimal HCV care delivery. We examined the comparative effectiveness of a pharmacist-managed HCV clinic versus a pharmacist-assisted HCV clinic. Methods This retrospective cohort study used electronic health record data on patients ≥18 years old initiating HCV treatment at a pharmacist-managed clinic or a pharmacist-assisted clinic within a single health-system between January 2015 and June 2017. Outcomes included treatment completion, sustained virologic response 12 weeks following treatment completion (SVR-12), and dispensation of direct-acting antiviral agents at the institution-based specialty pharmacy. Inverse probability of treatment-weighted (IPTW) logistic regression models were used to compare outcomes between the 2 clinic models. Results A total of 127 patients initiated HCV treatment therapy: 64 patients from the pharmacist-managed clinic and 63 patients from the pharmacist-assisted clinic. The cohort had a mean age of 55 years, was 51% male, and 68% white. In IPTW analyses, there was no difference in treatment completion (odds ratio [OR], 1.1; 95% confidence interval [CI], 0.1–13.8; p = 0.93), achievement of sustained virologic response at 12 months (SVR-12) (OR, 1.0; 95% CI, 0.2–4.5; p = 0.62), or use of institution-based specialty pharmacy (OR, 0.6; 95% CI, 0.2–1.7; p = 0.33) between pharmacist-managed and pharmacist-assisted clinics. Conclusion There were no significant differences in outcomes between patients receiving care at the pharmacist-managed HCV clinic and the pharmacist-assisted clinic. Given the frequency of SVR-12 achieved in both groups, both pharmacist-managed and pharmacist-assisted clinic models may be reasonable alternatives for providing outpatient HCV care. ambulatory care, comparative effectiveness research, delivery of health care, drug therapy, hepatitis C, pharmacists KEY POINTS The optimal care delivery models for providing hepatitis C virus (HCV) services, including the roles of pharmacists, remain unknown, and there are few data available to suggest one model is more effective than another. This study examined the comparative effectiveness of pharmacist-managed versus pharmacist-assisted HCV clinic care delivery models for treatment completion, sustained viral response, and other outcomes. There was no difference in treatment completion and sustained viral response between the two clinic models, suggesting that both are appropriate options for providing outpatient HCV care. Chronic hepatitis C virus (HCV) is the most prevalent bloodborne chronic infection in the United States, with an estimated 2.7 to 3.9 million people infected with chronic HCV nationwide.1,2 According to the Centers for Disease Control and Prevention, both the prevalence and incidence of HCV-associated mortality increased between 2003 and 2013, surpassing that of HIV, pneumococcal diseases, and tuberculosis combined.3,4 Data from the Chronic Hepatitis Cohort Study and the National Health and Nutrition Examination Survey suggest that only 6% to 13% of HCV-infected patients receive antiviral treatment, despite the introduction of oral direct-acting antiviral agents.5,6 These drugs are very effective at suppressing viral load, but their cost remains high and has led to a considerable financial burden on third-party payers.7 Despite the individual and public health benefits of expanding access to HCV treatment,8 the best health care delivery model for HCV remains unknown. The role of pharmacists in patient care has continued to expand over the years, and it has been documented that pharmacist-managed care is associated with improved treatment outcomes for HCV using interferon-based therapies, as well as conditions other than HCV.9-13 The role of pharmacists in pharmacist-assisted clinic models may consist of screening for drug interactions, assistance with medication access, and patient education and counseling at a particular point in time while the patient receives direct care from a non-pharmacist prescriber. In addition to the services offered in pharmacist-assisted clinics, pharmacists practicing in a pharmacist-managed care delivery model assume direct care for patients throughout the course of treatment, are involved in decision making, spend more time with patients, and provide consistent follow-up. A pharmacist-managed clinic may therefore provide an effective care delivery model for HCV in the modern era of oral-based therapies. However, due to the rapid evolution of HCV treatment, little is known about the role of pharmacist-managed HCV using direct-acting antiviral (DAA) HCV therapy and pharmacy practice. Furthermore, the ideal level of pharmacist involvement in HCV care delivery remains unclear. While some evidence exists to suggest that greater pharmacist involvement can improve quality of care, the effects of pharmacist involvement on health and economic outcomes remain unknown. The 3 objectives of this study were therefore to (1) describe the distribution of characteristics of the population of patients receiving care at outpatient HCV clinics, (2) identify predictors of enrollment in a pharmacist-managed HCV clinic versus a pharmacist-assisted HCV clinic, and (3) examine the comparative effectiveness of a pharmacist-managed HCV clinic versus a pharmacist-assisted clinic with less pharmacist involvement. Our hypothesis for the second study objective was that lower household income, male sex, and history of illicit drug use (intravenous or intranasal) would be significant predictors of enrollment in the pharmacist-managed clinic compared to the pharmacist-assisted clinic. Our hypothesis for the third objective was that the pharmacist-managed clinic would improve outcomes compared to the pharmacist-assisted clinic. Methods Study design and data. This was a retrospective cohort study that used existing electronic medical record data from HCV patients ≥18 years of age enrolled in 1 of 2 clinics within a single health system: a pharmacist-managed HCV clinic and a pharmacist-assisted clinic. The study was approved by the Institutional Review Board. The study cohort included all patients who initiated DAA HCV therapy at either the pharmacist-managed clinic or the pharmacist-assisted clinic between January 1, 2015, and June 30, 2017, a study period that reflects current HCV treatment practices. Treatment-experienced patients were included in the study population only if their prior experience was outside the health system in which our study was conducted. All individuals in the study population were therefore “new users” of HCV clinic services within the health system. This “new-user” design facilitates confounding control, prevents conditioning on post-exposure variables (e.g., mediators of the effects of clinical enrollment), provides a natural “time zero” for each patient, and allows for ascertainment of covariates during the pre-exposure (i.e., pre-clinic) period.14 Models of HCV care delivery (exposure) and causal contrast of interest. The pharmacist-managed HCV clinic model consists of 3 phases—pre-treatment visit, medication teach visit, and subsequent monthly follow-up visits—and is operated in accordance with a collaborative drug therapy management protocol (Table 1). A pre-treatment visit is first conducted to determine treatment eligibility, select appropriate therapy, screen for relevant drug interactions, and initiate prior authorization for third-party payment. Patients who are deemed eligible for HCV therapy are then seen by a pharmacist for up to 1 hour for a medication teach visit, where comprehensive medication counseling regarding administration, adherence, adverse events, and monitoring is conducted, and the first dose of medication is administered. Follow-up visits to assess adherence and tolerance to medication therapy, screen for drug interactions, reinforce prior medication-related teaching, and monitor viral load, among other pertinent laboratory parameters, are conducted every 4 weeks until completion of therapy. Physicians and nurses are involved in the clinic as consultants with whom the pharmacist can discuss the treatment plan and other clinical decisions. No prescribers practiced at both clinics. Table 1. Characteristics of HCV Clinic Modelsa Characteristic Pharmacist-Managed Clinic Pharmacist-Assisted Clinic Pharmacy service offered  Pre-treatment evaluation Yes No  Medication teach visit Yes Yes  Monthly follow-up visits Yes No  End-of-treatment visit Yes No Number of pharmacist hours in the clinic per month 48 16 Number of visits with the pharmacist per patient 5–10 1 Characteristic Pharmacist-Managed Clinic Pharmacist-Assisted Clinic Pharmacy service offered  Pre-treatment evaluation Yes No  Medication teach visit Yes Yes  Monthly follow-up visits Yes No  End-of-treatment visit Yes No Number of pharmacist hours in the clinic per month 48 16 Number of visits with the pharmacist per patient 5–10 1 aHCV = hepatitis C virus. View Large Table 1. Characteristics of HCV Clinic Modelsa Characteristic Pharmacist-Managed Clinic Pharmacist-Assisted Clinic Pharmacy service offered  Pre-treatment evaluation Yes No  Medication teach visit Yes Yes  Monthly follow-up visits Yes No  End-of-treatment visit Yes No Number of pharmacist hours in the clinic per month 48 16 Number of visits with the pharmacist per patient 5–10 1 Characteristic Pharmacist-Managed Clinic Pharmacist-Assisted Clinic Pharmacy service offered  Pre-treatment evaluation Yes No  Medication teach visit Yes Yes  Monthly follow-up visits Yes No  End-of-treatment visit Yes No Number of pharmacist hours in the clinic per month 48 16 Number of visits with the pharmacist per patient 5–10 1 aHCV = hepatitis C virus. View Large The pharmacist-assisted HCV clinic is managed primarily by physicians and nurse practitioners. The pharmacist’s role in this clinic is limited to 1 visit, where medication counseling is provided prior to initiation of HCV therapy. Pharmacists at both sites maintained a dataset of all patients seen within their respective clinics. The primary care practices that referred patients to a gastroenterology or hepatology clinic all had the opportunity to form relationships with both the pharmacist-managed and pharmacist-assisted clinics, though it was impossible to measure the strength of the relationships between the primary care practices and the clinics. It is possible that the primary care practice referring patients to a gastroenterology or hepatology specialist may have had a preference for one clinic or clinician over another and, thus, had an influence on the HCV clinic that the patient was enrolled in and treated at, but patients were ultimately free to choose which clinic to attend. The causal contrast of interest was defined as the effect of initiating care in one clinic versus another, regardless of early subsequent discontinuation of care or switching from one clinic to another (although there were no switches between clinics). This is the observational study analog of the intention-to-treat analyses in randomized controlled trials, because patients are analyzed according to their initial exposure. Outcomes. When studying the predictors of clinic enrollment, the outcome of interest was enrollment in the pharmacist-managed clinic versus the pharmacist-assisted clinic. This analysis aimed to help us understand the reasons why patients chose to enroll in the pharmacist-managed clinic versus the pharmacist-assisted clinic. When studying the comparative effectiveness of the clinics, the primary outcomes were completion of HCV treatment, the achievement of sustained virologic response at 12 months (SVR-12), and use of the institution-based specialty pharmacy. We also examined 2 composite outcomes to examine the net effects of enrolling in one clinic over the other. The first composite included failure to complete treatment, failure to achieve SVR-12, or not using the institution-based specialty pharmacy. The second composite included failure to complete treatment or failure to achieve SVR-12. Patients were considered to have completed treatment if there was documentation of completion in the electronic medical record. Patients achieving SVR-12 had an undetectable hepatitis C viral load at least 12 weeks after therapy completion. Non-use of the institution-based specialty pharmacy was excluded from the second composite because clinicians do not universally agree that use of the pharmacy is a positive patient outcome, though many believe that it improves coordination of care, patient education, and access to care. Confounding by baseline covariates. Potential common causes (i.e., reasons why patients chose one clinic over the other) of enrollment in the pharmacist-managed versus pharmacist-assisted clinic and the outcomes of interest were prespecified and measured on the day of clinic enrollment and up to 1 year before. The covariates representing potential common causes were selected based on our conceptual framework, which was informed by related previously published studies and the authors’ clinical practice experience. Covariates included gender, age, HCV genotype, presence of cirrhosis, and history of exposure to known risk factors for HCV, including history of illicit injection drug use, illicit intranasal drug use, blood transfusion, organ transplantation, high-risk sexual activity (i.e., male-to-male sexual contact or multiple sexual partners), history of incarceration, and tattoos or piercings. Geographic location was used to approximate annual household income, using patients’ individual zip codes obtained from the electronic medical record. Median estimates of household income were provided by the United States Census Bureau, and we then constructed a categorical variable for each patient’s income level, coded as low, medium, or high income.15 Enrollment in a government-sponsored, need-based insurance plan was also used as a proxy for socioeconomic status based on documentation of Medicaid or affiliated third-party payer plans in the electronic medical record. Statistical analysis. Descriptive statistics were used to analyze characteristics of HCV patients seen at both the pharmacist-managed clinic and the pharmacist-assisted clinic. Predictors of enrollment at either clinic were first assessed using bivariable logistic regressions. A multivariable logistic regression model was then used to assess independent predictors of pharmacist-managed HCV clinic enrollment. Predictors of a priori interest were included, as well as those found to be significant during the bivariable logistic regression analyses. Variables of particular interest included annual household income and type of medical insurance. Since we were interested in estimating the causal effect of enrolling in one clinic versus the other on outcomes and needed to adjust for potential confounding, we estimated propensity scores for use in the analyses.16 The propensity score (PS) is the probability that someone would be treated in the pharmacist-managed clinic versus the comparator clinic conditional on their characteristics (i.e., covariates or potential predictors of patients choosing one clinic over the other) prior to or at the time of clinic enrollment.17-19 We used a flexible logistic regression model containing all potential predictors of clinic enrollment to estimate the probability (i.e., PS) for each patient in the study population.20 Informally, these weights were calculated as the inverse of the estimated probability of enrolling in the clinic that each person was actually enrolled in conditional on potential confounding variables (i.e., the denominator is equal to the estimated PS for pharmacist-managed clinic patients and 1 minus the PS for pharmacist-assisted clinic patients).21-23 The distribution of the inverse probability of treatment weights (IPTW) was examined using histograms and descriptive statistics. We used standardized mean differences after IPTW weighting to assess covariate balance across clinic groups. Using logistic regression models weighted by the constructed IPTW, we compared treatment completion, SVR-12, and use of the institution-based specialty pharmacy to fill medications for the pharmacist-managed clinic versus the pharmacist-assisted clinic. All IPTW analyses used the robust (Huber-White) estimator of the sampling variance.20, 22 Consistent with prior guidance, we avoided the use of a p-value cutoff when interpreting estimates of the comparative effects of the two clinic models.24 However, when assessing potential predictors of enrolling in the pharmacist-managed versus the pharmacist-assisted clinic, we calculated p-values to help guide our interpretation of the findings. All analyses were conducted using SAS software (Version 9.4; SAS Institute Inc., Cary, NC). Results Study cohort. During the study period, a total of 127 patients were started on HCV treatment therapy with a DAA agent: 64 patients at the pharmacist-managed clinic and 63 patients at the pharmacist-assisted clinic (Table 2). The overall mean ± S.D. age was 55 ± 9.7 years. The overall population was 51% male and 68% white. One-third of the study population lived in an area with an average household income of $49,622 or more. Table 2. Predictors of Enrollment in the Pharmacist-Managed Clinic Versus Pharmacist-Assisted Clinica Overall (n = 127), no. (%) Pharmacist-Managed Clinic (n = 64), no. (%) Pharmacist-Assisted Clinic (n = 63), no. (%) Bivariable OR (95% CI) Multivariableb OR (95% CI) Male 62 (49) 46 (71) 19 (29) 5.9 (2.8–12.7) 15.4 (4.4–54.3) Age (years)  Tertile 1 (28–52) 41 (32) 20 (31) 21 (33) Reference Reference  Tertile 2 (53–60) 46 (36) 24 (38) 22 (35) 1.2 (0.5–2.7) 0.6 (0.2–2.3)  Tertile 3 (61–85) 40 (32) 20 (31) 20 (32) 1.1 (0.4–2.5) 1.1 (0.3–4.5) Annual household income  Tertile 1 ($29,108–$35,953) 46 (36) 22 (34) 24 (38) Reference Reference  Tertile 2 ($36,420–$48,672) 39 (31) 26 (41) 13 (21) 2.2 (0.9–5.3) 3.5 (0.7–18.5)  Tertile 3 ($49,622–$108,776) 42 (33) 16 (25) 26 (41) 0.7 (0.3–1.6) 0.9 (0.2–4.0) Medicaid 103 (81) 57 (89) 46 (73) 3.0 (1.2–7.9) 2.1 (0.5–9.9) Race  White 86 (68) 39 (61) 47 (75) Reference Reference  Black or African American 24 (19) 17 (27) 7 (11) 2.9 (1.1–7.8) 6.8 (1.5–31.5)  Other 17 (13) 8 (13) 9 (14) 1.1 (0.4–3.0) 0.7 (0.1–6.0) Hispanic or Latino 23 (18) 13 (20) 10 (16) 1.4 (0.5–3.4) 0.7 (0.1–4.8) Genotype 1A 101 (80) 49 (77) 52 (83) 0.7 (0.3–1.7) 0.4 (0.1–2.1) Treatment-experienced 33 (26) 15 (23) 18 (29) 0.8 (0.3–1.7) 0.4 (0.1–2.8) 12-week length of treatment 97 (76) 47 (73) 50 (79) 1.4 (0.6–3.2) 1.1 (0.1–8.3) Any cirrhosis 75 (59) 44 (69) 31 (49) 2.2 (1.1–4.7) 2.8 (0.8–9.6) Decompensated cirrhosis 23 (18) 17 (27) 6 (10) 3.4 (1.3–9.4) 3.2 (0.7–15.5) Intravenous illicit drug use 71 (56) 39 (61) 32 (51) 1.5 (0.7–3.1) 0.3 (0.1–1.2) Intranasal illicit drug use 58 (46) 37 (58) 21 (33) 2.7 (1.3–5.6) 3.1 (0.9–10.1) Treatment with ledipasvir—sofosbuvir 104 (82) 51 (80) 53 (84) 0.7 (0.3–1.8) 0.9 (0.2–4.6) Tobacco use, current 73 (57) 35 (55) 38 (60) 0.8 (0.4–1.6) 0.4 (0.1–1.3) Marijuana use, current 31 (24) 13 (20) 18 (29) 0.6 (0.3–1.4) 0.3 (0.1–1.2) History of blood transfusion 36 (28) 19 (30) 17 (27) 1.1 (0.5–2.5) 1.1 (0.3–3.9) History of high-risk sexual activity 6 (5) 5 (8) 1 (2) 5.3 (0.6–46.3) 6.1 (0.3–124.5) Accidental or occupational exposure 8 (6) 3 (5) 5 (8) 0.6 (0.1–2.5) 0.6 (0.1–5.2) HCV-infected family member or partner 23 (18) 7 (11) 16 (25) 0.4 (0.1–1.0) 0.4 (0.1–1.8) Incarceration 35 (28) 27 (42) 8 (13) 5.0 (2.1–12.2) 6.1 (1.4–26.7) Overall (n = 127), no. (%) Pharmacist-Managed Clinic (n = 64), no. (%) Pharmacist-Assisted Clinic (n = 63), no. (%) Bivariable OR (95% CI) Multivariableb OR (95% CI) Male 62 (49) 46 (71) 19 (29) 5.9 (2.8–12.7) 15.4 (4.4–54.3) Age (years)  Tertile 1 (28–52) 41 (32) 20 (31) 21 (33) Reference Reference  Tertile 2 (53–60) 46 (36) 24 (38) 22 (35) 1.2 (0.5–2.7) 0.6 (0.2–2.3)  Tertile 3 (61–85) 40 (32) 20 (31) 20 (32) 1.1 (0.4–2.5) 1.1 (0.3–4.5) Annual household income  Tertile 1 ($29,108–$35,953) 46 (36) 22 (34) 24 (38) Reference Reference  Tertile 2 ($36,420–$48,672) 39 (31) 26 (41) 13 (21) 2.2 (0.9–5.3) 3.5 (0.7–18.5)  Tertile 3 ($49,622–$108,776) 42 (33) 16 (25) 26 (41) 0.7 (0.3–1.6) 0.9 (0.2–4.0) Medicaid 103 (81) 57 (89) 46 (73) 3.0 (1.2–7.9) 2.1 (0.5–9.9) Race  White 86 (68) 39 (61) 47 (75) Reference Reference  Black or African American 24 (19) 17 (27) 7 (11) 2.9 (1.1–7.8) 6.8 (1.5–31.5)  Other 17 (13) 8 (13) 9 (14) 1.1 (0.4–3.0) 0.7 (0.1–6.0) Hispanic or Latino 23 (18) 13 (20) 10 (16) 1.4 (0.5–3.4) 0.7 (0.1–4.8) Genotype 1A 101 (80) 49 (77) 52 (83) 0.7 (0.3–1.7) 0.4 (0.1–2.1) Treatment-experienced 33 (26) 15 (23) 18 (29) 0.8 (0.3–1.7) 0.4 (0.1–2.8) 12-week length of treatment 97 (76) 47 (73) 50 (79) 1.4 (0.6–3.2) 1.1 (0.1–8.3) Any cirrhosis 75 (59) 44 (69) 31 (49) 2.2 (1.1–4.7) 2.8 (0.8–9.6) Decompensated cirrhosis 23 (18) 17 (27) 6 (10) 3.4 (1.3–9.4) 3.2 (0.7–15.5) Intravenous illicit drug use 71 (56) 39 (61) 32 (51) 1.5 (0.7–3.1) 0.3 (0.1–1.2) Intranasal illicit drug use 58 (46) 37 (58) 21 (33) 2.7 (1.3–5.6) 3.1 (0.9–10.1) Treatment with ledipasvir—sofosbuvir 104 (82) 51 (80) 53 (84) 0.7 (0.3–1.8) 0.9 (0.2–4.6) Tobacco use, current 73 (57) 35 (55) 38 (60) 0.8 (0.4–1.6) 0.4 (0.1–1.3) Marijuana use, current 31 (24) 13 (20) 18 (29) 0.6 (0.3–1.4) 0.3 (0.1–1.2) History of blood transfusion 36 (28) 19 (30) 17 (27) 1.1 (0.5–2.5) 1.1 (0.3–3.9) History of high-risk sexual activity 6 (5) 5 (8) 1 (2) 5.3 (0.6–46.3) 6.1 (0.3–124.5) Accidental or occupational exposure 8 (6) 3 (5) 5 (8) 0.6 (0.1–2.5) 0.6 (0.1–5.2) HCV-infected family member or partner 23 (18) 7 (11) 16 (25) 0.4 (0.1–1.0) 0.4 (0.1–1.8) Incarceration 35 (28) 27 (42) 8 (13) 5.0 (2.1–12.2) 6.1 (1.4–26.7) aOR = odds ratio; CI = confidence interval; HCV = hepatitis C virus. bEach variable adjusted for all other variables shown in the table. View Large Table 2. Predictors of Enrollment in the Pharmacist-Managed Clinic Versus Pharmacist-Assisted Clinica Overall (n = 127), no. (%) Pharmacist-Managed Clinic (n = 64), no. (%) Pharmacist-Assisted Clinic (n = 63), no. (%) Bivariable OR (95% CI) Multivariableb OR (95% CI) Male 62 (49) 46 (71) 19 (29) 5.9 (2.8–12.7) 15.4 (4.4–54.3) Age (years)  Tertile 1 (28–52) 41 (32) 20 (31) 21 (33) Reference Reference  Tertile 2 (53–60) 46 (36) 24 (38) 22 (35) 1.2 (0.5–2.7) 0.6 (0.2–2.3)  Tertile 3 (61–85) 40 (32) 20 (31) 20 (32) 1.1 (0.4–2.5) 1.1 (0.3–4.5) Annual household income  Tertile 1 ($29,108–$35,953) 46 (36) 22 (34) 24 (38) Reference Reference  Tertile 2 ($36,420–$48,672) 39 (31) 26 (41) 13 (21) 2.2 (0.9–5.3) 3.5 (0.7–18.5)  Tertile 3 ($49,622–$108,776) 42 (33) 16 (25) 26 (41) 0.7 (0.3–1.6) 0.9 (0.2–4.0) Medicaid 103 (81) 57 (89) 46 (73) 3.0 (1.2–7.9) 2.1 (0.5–9.9) Race  White 86 (68) 39 (61) 47 (75) Reference Reference  Black or African American 24 (19) 17 (27) 7 (11) 2.9 (1.1–7.8) 6.8 (1.5–31.5)  Other 17 (13) 8 (13) 9 (14) 1.1 (0.4–3.0) 0.7 (0.1–6.0) Hispanic or Latino 23 (18) 13 (20) 10 (16) 1.4 (0.5–3.4) 0.7 (0.1–4.8) Genotype 1A 101 (80) 49 (77) 52 (83) 0.7 (0.3–1.7) 0.4 (0.1–2.1) Treatment-experienced 33 (26) 15 (23) 18 (29) 0.8 (0.3–1.7) 0.4 (0.1–2.8) 12-week length of treatment 97 (76) 47 (73) 50 (79) 1.4 (0.6–3.2) 1.1 (0.1–8.3) Any cirrhosis 75 (59) 44 (69) 31 (49) 2.2 (1.1–4.7) 2.8 (0.8–9.6) Decompensated cirrhosis 23 (18) 17 (27) 6 (10) 3.4 (1.3–9.4) 3.2 (0.7–15.5) Intravenous illicit drug use 71 (56) 39 (61) 32 (51) 1.5 (0.7–3.1) 0.3 (0.1–1.2) Intranasal illicit drug use 58 (46) 37 (58) 21 (33) 2.7 (1.3–5.6) 3.1 (0.9–10.1) Treatment with ledipasvir—sofosbuvir 104 (82) 51 (80) 53 (84) 0.7 (0.3–1.8) 0.9 (0.2–4.6) Tobacco use, current 73 (57) 35 (55) 38 (60) 0.8 (0.4–1.6) 0.4 (0.1–1.3) Marijuana use, current 31 (24) 13 (20) 18 (29) 0.6 (0.3–1.4) 0.3 (0.1–1.2) History of blood transfusion 36 (28) 19 (30) 17 (27) 1.1 (0.5–2.5) 1.1 (0.3–3.9) History of high-risk sexual activity 6 (5) 5 (8) 1 (2) 5.3 (0.6–46.3) 6.1 (0.3–124.5) Accidental or occupational exposure 8 (6) 3 (5) 5 (8) 0.6 (0.1–2.5) 0.6 (0.1–5.2) HCV-infected family member or partner 23 (18) 7 (11) 16 (25) 0.4 (0.1–1.0) 0.4 (0.1–1.8) Incarceration 35 (28) 27 (42) 8 (13) 5.0 (2.1–12.2) 6.1 (1.4–26.7) Overall (n = 127), no. (%) Pharmacist-Managed Clinic (n = 64), no. (%) Pharmacist-Assisted Clinic (n = 63), no. (%) Bivariable OR (95% CI) Multivariableb OR (95% CI) Male 62 (49) 46 (71) 19 (29) 5.9 (2.8–12.7) 15.4 (4.4–54.3) Age (years)  Tertile 1 (28–52) 41 (32) 20 (31) 21 (33) Reference Reference  Tertile 2 (53–60) 46 (36) 24 (38) 22 (35) 1.2 (0.5–2.7) 0.6 (0.2–2.3)  Tertile 3 (61–85) 40 (32) 20 (31) 20 (32) 1.1 (0.4–2.5) 1.1 (0.3–4.5) Annual household income  Tertile 1 ($29,108–$35,953) 46 (36) 22 (34) 24 (38) Reference Reference  Tertile 2 ($36,420–$48,672) 39 (31) 26 (41) 13 (21) 2.2 (0.9–5.3) 3.5 (0.7–18.5)  Tertile 3 ($49,622–$108,776) 42 (33) 16 (25) 26 (41) 0.7 (0.3–1.6) 0.9 (0.2–4.0) Medicaid 103 (81) 57 (89) 46 (73) 3.0 (1.2–7.9) 2.1 (0.5–9.9) Race  White 86 (68) 39 (61) 47 (75) Reference Reference  Black or African American 24 (19) 17 (27) 7 (11) 2.9 (1.1–7.8) 6.8 (1.5–31.5)  Other 17 (13) 8 (13) 9 (14) 1.1 (0.4–3.0) 0.7 (0.1–6.0) Hispanic or Latino 23 (18) 13 (20) 10 (16) 1.4 (0.5–3.4) 0.7 (0.1–4.8) Genotype 1A 101 (80) 49 (77) 52 (83) 0.7 (0.3–1.7) 0.4 (0.1–2.1) Treatment-experienced 33 (26) 15 (23) 18 (29) 0.8 (0.3–1.7) 0.4 (0.1–2.8) 12-week length of treatment 97 (76) 47 (73) 50 (79) 1.4 (0.6–3.2) 1.1 (0.1–8.3) Any cirrhosis 75 (59) 44 (69) 31 (49) 2.2 (1.1–4.7) 2.8 (0.8–9.6) Decompensated cirrhosis 23 (18) 17 (27) 6 (10) 3.4 (1.3–9.4) 3.2 (0.7–15.5) Intravenous illicit drug use 71 (56) 39 (61) 32 (51) 1.5 (0.7–3.1) 0.3 (0.1–1.2) Intranasal illicit drug use 58 (46) 37 (58) 21 (33) 2.7 (1.3–5.6) 3.1 (0.9–10.1) Treatment with ledipasvir—sofosbuvir 104 (82) 51 (80) 53 (84) 0.7 (0.3–1.8) 0.9 (0.2–4.6) Tobacco use, current 73 (57) 35 (55) 38 (60) 0.8 (0.4–1.6) 0.4 (0.1–1.3) Marijuana use, current 31 (24) 13 (20) 18 (29) 0.6 (0.3–1.4) 0.3 (0.1–1.2) History of blood transfusion 36 (28) 19 (30) 17 (27) 1.1 (0.5–2.5) 1.1 (0.3–3.9) History of high-risk sexual activity 6 (5) 5 (8) 1 (2) 5.3 (0.6–46.3) 6.1 (0.3–124.5) Accidental or occupational exposure 8 (6) 3 (5) 5 (8) 0.6 (0.1–2.5) 0.6 (0.1–5.2) HCV-infected family member or partner 23 (18) 7 (11) 16 (25) 0.4 (0.1–1.0) 0.4 (0.1–1.8) Incarceration 35 (28) 27 (42) 8 (13) 5.0 (2.1–12.2) 6.1 (1.4–26.7) aOR = odds ratio; CI = confidence interval; HCV = hepatitis C virus. bEach variable adjusted for all other variables shown in the table. View Large Predictors of clinic enrollment. In bivariable analyses (Table 2), notable predictors of enrollment in the pharmacist-managed clinic versus pharmacist-assisted clinic included male sex (odds ratio [OR], 5.9; 95% confidence interval [CI], 2.8–12.7; p < 0.001), black or African American race (OR, 2.9; 95% CI,1.1–7.8; p = 0.03), use of Medicaid (OR, 3.0; 95% CI,1.2–7.9; p = 0.02), presence of any cirrhosis (OR, 2.3; 95% CI, 1.1–4.7; p = 0.02), presence of decompensated cirrhosis (OR, 3.4; 95% CI, 1.3–9.4; p = 0.02), history of illicit intranasal drug use (OR, 2.7; 95% CI, 1.3–5.6; p < 0.01), and history of incarceration (OR, 5.0; 95% CI, 2.1–12.2; p < 0.001). Predictors of enrollment that were notable in multivariable analyses (Table 2) included: male sex (OR, 15.4; 95% CI, 4.4–54.4; p < 0.001), black or African American race (OR, 6.8; 95% CI, 1. 5–31. 5; p = 0.01), history of incarceration (OR, 6.1; 95% CI, 1.4–26. 7; p = 0.01), presence of cirrhosis (OR, 2.8; 95% CI, 0.8–9.6; p = 0.09), and history of illicit intranasal drug use (OR, 3.1; 95% CI, 0.9–10.1; p = 0.07). IPTW and baseline covariate balance. The estimated IPTW had a mean ± S.D. of 1.01 ± 1.29 and ranged from 0.50 to 6.98, indicating that the weight estimation model was likely to have been specified correctly.21 After weighting using IPTW, all measured covariates had absolute standardized mean differences less than 0.10, which is consistent with excellent covariate balance between the pharmacist-managed clinic and pharmacist-assisted clinic groups.25 Comparative effectiveness of care delivery. In unweighted analyses, no differences in outcomes were observed between the clinic groups (Table 3). In IPTW analyses, there was no difference in treatment completion status (OR, 1.1; 95% CI, 0.1–13.8; p = 0.93), SVR-12 achievement (OR, 1.0; 95% CI, 0.2–4.5; p = 0.62), or use of the institution-affiliated specialty pharmacy to dispense HCV medications (OR, 0.6; 95% CI, 0.2–1.7; p = 0.33) between the pharmacist-managed clinic and the pharmacist-assisted clinic (Table 3). There was also no difference observed between clinics for the composite of failure to complete treatment, failure to achieve SVR-12, and not using the institution-based specialty pharmacy (OR, 1.1; 95% CI, 0.4–3.7; p = 0.81) and the composite of failure to complete treatment and failure to achieve SVR-12 (OR, 1.4; 95% CI, 0.3–6.4; p = 0.68). Table 3. Effect of the Pharmacist-Managed Clinic (n = 64) Versus Pharmacist-Assisted Clinic (n = 63) on Patient Outcomes Among 127 Patients Initiating DAA Therapya Variable Total Events, no. (%) Pharmacist- Managed Clinic Events, no. (%) Pharmacist-Assisted Clinic Events, no. (%) Unweighted OR (95% CI) IPTW OR (95% CI) Treatment completed 122 (96) 62 (97) 60 (95) 0.6 (0.1–4.0) 1.1 (0.1–13.8) SVR-12 achieved 104 (82) 51 (80) 53 (84) 0.8 (0.2–3.1) 1.0 (0.2–4.5) Institution-based DAA agent fills 57 (45) 31 (48) 26 (41) 1.3 (0.6–2.7) 0.6 (0.2–1.7) Composite of failure to complete treatment, failure to achieve SVR-12, and not using the institution-based specialty pharmacy 45 (35) 25 (39) 20 (32) 0.7 (0.4–1.5) 1.1 (0.4–3.7) Composite of failure to complete treatment and failure to achieve SVR-12 102 (80) 49 (77) 53 (84) 1.6 (0.7–4.0) 1.4 (0.3–6.4) Variable Total Events, no. (%) Pharmacist- Managed Clinic Events, no. (%) Pharmacist-Assisted Clinic Events, no. (%) Unweighted OR (95% CI) IPTW OR (95% CI) Treatment completed 122 (96) 62 (97) 60 (95) 0.6 (0.1–4.0) 1.1 (0.1–13.8) SVR-12 achieved 104 (82) 51 (80) 53 (84) 0.8 (0.2–3.1) 1.0 (0.2–4.5) Institution-based DAA agent fills 57 (45) 31 (48) 26 (41) 1.3 (0.6–2.7) 0.6 (0.2–1.7) Composite of failure to complete treatment, failure to achieve SVR-12, and not using the institution-based specialty pharmacy 45 (35) 25 (39) 20 (32) 0.7 (0.4–1.5) 1.1 (0.4–3.7) Composite of failure to complete treatment and failure to achieve SVR-12 102 (80) 49 (77) 53 (84) 1.6 (0.7–4.0) 1.4 (0.3–6.4) aOR = odds ratio; CI = confidence interval; IPTW = inverse probability of treatment-weighted; SVR-12 = sustained virologic response 12 weeks following treatment completion; DAA = direct-acting antiviral. View Large Table 3. Effect of the Pharmacist-Managed Clinic (n = 64) Versus Pharmacist-Assisted Clinic (n = 63) on Patient Outcomes Among 127 Patients Initiating DAA Therapya Variable Total Events, no. (%) Pharmacist- Managed Clinic Events, no. (%) Pharmacist-Assisted Clinic Events, no. (%) Unweighted OR (95% CI) IPTW OR (95% CI) Treatment completed 122 (96) 62 (97) 60 (95) 0.6 (0.1–4.0) 1.1 (0.1–13.8) SVR-12 achieved 104 (82) 51 (80) 53 (84) 0.8 (0.2–3.1) 1.0 (0.2–4.5) Institution-based DAA agent fills 57 (45) 31 (48) 26 (41) 1.3 (0.6–2.7) 0.6 (0.2–1.7) Composite of failure to complete treatment, failure to achieve SVR-12, and not using the institution-based specialty pharmacy 45 (35) 25 (39) 20 (32) 0.7 (0.4–1.5) 1.1 (0.4–3.7) Composite of failure to complete treatment and failure to achieve SVR-12 102 (80) 49 (77) 53 (84) 1.6 (0.7–4.0) 1.4 (0.3–6.4) Variable Total Events, no. (%) Pharmacist- Managed Clinic Events, no. (%) Pharmacist-Assisted Clinic Events, no. (%) Unweighted OR (95% CI) IPTW OR (95% CI) Treatment completed 122 (96) 62 (97) 60 (95) 0.6 (0.1–4.0) 1.1 (0.1–13.8) SVR-12 achieved 104 (82) 51 (80) 53 (84) 0.8 (0.2–3.1) 1.0 (0.2–4.5) Institution-based DAA agent fills 57 (45) 31 (48) 26 (41) 1.3 (0.6–2.7) 0.6 (0.2–1.7) Composite of failure to complete treatment, failure to achieve SVR-12, and not using the institution-based specialty pharmacy 45 (35) 25 (39) 20 (32) 0.7 (0.4–1.5) 1.1 (0.4–3.7) Composite of failure to complete treatment and failure to achieve SVR-12 102 (80) 49 (77) 53 (84) 1.6 (0.7–4.0) 1.4 (0.3–6.4) aOR = odds ratio; CI = confidence interval; IPTW = inverse probability of treatment-weighted; SVR-12 = sustained virologic response 12 weeks following treatment completion; DAA = direct-acting antiviral. View Large Discussion In this cohort study of HCV patients from a single health system, we observed no difference in the effectiveness of pharmacist-managed versus pharmacist-assisted HCV clinic models for treatment completion, achieving SVR-12, prescriptions filled at the institution-based specialty pharmacy, and composite outcomes. Predictors of enrollment in the pharmacist-managed clinic instead of the pharmacist-assisted clinic included male sex, black or African American race, history of incarceration, history of intranasal drug use, and presence of cirrhosis. The greater prevalence of incarceration, drug use, and cirrhosis among the pharmacist-managed clinic patients suggests that the pharmacist-managed clinic may have treated a more complex patient population. If the differences in severity of hepatic disease and clinical complexity between the populations of the two clinics were not fully addressed by our measured covariates, then residual confounding may be one of the reasons for the absence of a difference in outcomes between the clinic models. Another potential explanation for the absence of a difference between the clinics is the ease of administration of current oral DAA medications as compared with older interferon-based injectable regimens, which required more counseling and monitoring. Therefore, a clinic model with fewer pharmacist-led visits and less contact time may still be able to produce comparable outcomes versus a more intensive pharmacist-managed clinic model. Although our findings need to be confirmed in a larger study, they suggest that pharmacist-managed clinics may consider reducing the number of clinic hours per patient in order to allow a greater number of patients to receive HCV treatment. Given the ongoing national trend of Medicaid programs removing fibrosis restrictions for patients in need of HCV treatment, it is likely that a greater number of patients will need access to care and pharmacist-involved clinics could aid in efficiently expanding access. To the best of our knowledge, prior studies have not directly compared pharmacist care delivery models for the modern-day treatment of HCV. Most prior studies simply described a single HCV care delivery model in isolation. Nonetheless, our findings were generally consistent with these descriptive studies suggesting that varying degrees of pharmacist involvement in health care delivery may improve the quality of care. A recently published descriptive study using 2013–2015 data assessed the impact of a pharmacist in managing drug–drug interactions among patients receiving HCV therapy in a pharmacist-assisted care delivery model in Colorado.26 During the 20-month study period, 664 patients receiving treatment were identified as having at least 1 drug–drug interaction that could have impacted the likelihood of HCV treatment success. This resulted in discontinuation of at least 1 medication among 29% of treated patients. The authors of that study extrapolated the findings to conclude that the inclusion of pharmacists in HCV management may optimize patient care, even when involved in a limited capacity.26 In a retrospective cohort study of patients receiving HCV care at a Veteran Affairs (VA) hospital between 2002 and 2004, HCV treatment was managed primarily by a clinical pharmacist.10 The VA hospital clinic was similar to the pharmacist-managed clinic in our study. Enrolled patients with genotypes 1–4 received treatment with pegylated interferon combined with ribavirin. SVR-12 was achieved in 63% of patients, a result comparable to the established standard of care in the era of interferon-based therapy.10,27,28 Although this study did not compare the pharmacist-managed cohort to any other group and was descriptive in nature, the authors concluded that the pharmacist-managed clinic achieved similar outcomes to earlier models of HCV care delivery in the VA. A second VA retrospective cohort study without a comparator group was conducted using 2014–2015 data.29 The study was similar to the original VA study, but focused more on describing the economic outcomes of a pharmacist-managed HCV clinic in the era of DAA treatment. The authors found that 94.1% of veterans cared for by the clinic achieved an SVR-12, and the overall cost ratio of total drug spend to cure rate was $40,135.22. The conclusions of the VA studies are consistent with our own and the results of our study.10,29 There are several limitations to our study. The first limitation is the small sample size, which limited the statistical power of the study. In the absence of multi-site cohort studies capable of enrolling larger sample sizes, our study may be the best available evidence despite the imprecision of the estimates. Ideally, we would have examined whether prior treatment experience modified the effects of one clinic model versus another, but the sample size limited us from also conducting this analysis. A second limitation is the lack of a comparator HCV clinic without pharmacist involvement to permit a causal contrast between pharmacist-managed and pharmacist-uninvolved HCV clinic models. Therefore, the results of our study only permit inferences about the comparative effectiveness of more versus less pharmacist involvement in HCV care. The absence of a pharmacist-uninvolved comparator is also a strength of the study, since such comparisons are often subject to a greater risk of confounding bias. Third, our study was unable to examine economic or cost-related outcomes (e.g., cost savings). It is important to consider the high cost associated with DAA therapy, and future studies comparing HCV clinic models should strive to examine economic outcomes. Fourth, as previously stated, this is an observational study and residual or unmeasured confounding may be present, despite our use of advanced causal inference methods to help address measured confounding. Prior literature suggests that the data in our study captured most of the factors driving enrollment in the pharmacist-managed clinic versus the pharmacist-assisted clinic, but others could still exist.30,31 Other potentially important covariates that we considered included proximity of the patients’ residence to the clinic, transportation availability, and physician referral, none of which were measured in the available data. Both clinics were adjacent to primary care clinics; though we did not include where patients received primary care as a covariate in the analysis, it is likely that referral patterns and ultimate HCV clinic assignment were influenced by location of the patient’s primary care clinic due to familiarity and convenience for both the patient as well as the primary care provider. Fifth, due to the retrospective nature of this study, baseline characteristics among patients enrolled at either clinic were assessed by manual review of the electronic medical record. Because some of the characteristics were subjective and documented by different providers at the two clinics, there may be some inconsistency in data collection (i.e., differential measurement error in covariate ascertainment) that could result in bias. Finally, household income was ascertained using zip code and is only a proxy, so patients’ actual incomes may be misclassified. The pharmacist-managed clinic was located in an underserved community and may have had greater access to social workers and community resources. It is possible that patients earning an income less than the median of their respective zip code preferred the pharmacist-managed clinic due to the additional financial resources available. Despite these limitations, our study provides some of the only empirical information about the comparative effectiveness of pharmacist-involved care delivery models for HCV. Conclusion In this retrospective cohort study employing causal inference methods, no difference was observed between the pharmacist-managed and pharmacist-assisted clinics for treatment completion, achievement of SVR-12, and prescriptions filled at the institution-based specialty pharmacy. Additionally, no difference was observed in the composite of failure to complete treatment, failure to achieve SVR-12, or not using the institution-based specialty pharmacy, as well as the composite of failure to complete treatment or failure to achieve SVR-12. Given the frequency of SVR-12 achievement in both the pharmacist-managed and pharmacist-assisted clinic model groups, the two models appear to be reasonable alternatives for providing outpatient HCV care. Disclosures This study was funded by a research grant from the ASHP Research and Education Foundation. Dr. Zullo is funded in part by an award from the Agency for Healthcare Research and Quality (5K12HS022998) and a U.S. Department of Veterans Affairs Office of Academic Affiliations Advanced Fellowship in Health Services Research and Development. The authors have declared no other potential conflicts of interest. References 1. Denniston MM , Jiles RB , Drobeniuc J et al. Chronic hepatitis C virus infection in the United States, National Health and Nutrition Examination Survey 2003 to 2010 . Ann Intern Med. 2014 ; 160 : 293 - 300 . Google Scholar Crossref Search ADS PubMed 2. Rosenberg ES , Hall EW , Sullivan PS et al. Estimation of state-level prevalence of hepatitis C virus infection, US states and District of Columbia, 2010 . Clin Infect Dis. 2017 ; 64 : 1573 - 81 . Google Scholar Crossref Search ADS PubMed 3. Centers for Disease Control and Prevention . CDC newsroom releases: Hepatitis C kills more Americans than any other infectious disease (May 2016). https://www.cdc.gov/media/releases/2016/p0504-hepc-mortality.html ( accessed 2017 Jun 23 ). 4. Ly KN , Hughes EM , Jiles RB , Holmberg SD . Rising mortality associated with hepatitis C virus in the United States, 2003-2013 . Clin Infect Dis. 2016 ; 62 : 1287 - 8 . Google Scholar Crossref Search ADS PubMed 5. Spradling PR , Rupp L , Moorman AC et al. Chronic Hepatitis Cohort Study Investigators . Hepatitis B and C virus infection among 1.2 million persons with access to care: factors associated with testing and infection prevalence . Clin Infect Dis. 2012 ; 55 : 1047 - 55 . Google Scholar Crossref Search ADS PubMed 6. Moorman AC , Gordon SC , Rupp LB et al. Chronic Hepatitis Cohort Study Investigators . Baseline characteristics and mortality among people in care for chronic viral hepatitis: the chronic hepatitis cohort study . Clin Infect Dis. 2013 ; 56 : 40 - 50 . Google Scholar Crossref Search ADS PubMed 7. Chhatwal J , Kanwal F , Roberts MS , Dunn MA . Cost-effectiveness and budget impact of hepatitis C virus treatment with sofosbuvir and ledipasvir in the United States . Ann Intern Med. 2015 ; 162 : 397 - 406 . Google Scholar Crossref Search ADS PubMed 8. Kabiri M , Jazwinski AB , Roberts MS et al. The changing burden of hepatitis C virus infection in the United States: model-based predictions . Ann Intern Med. 2014 ; 161 : 170 - 80 . Google Scholar Crossref Search ADS PubMed 9. Mikolas LA , Jacques K , Huq M et al. Utilizing clinical pharmacist specialist to manage hepatitis C virus patients on direct-acting antiviral therapy . J Pharm Pract. Epub ahead of print. (DOI: 10.1177/0897190018777345). 10. Smith JP , Dong MH , Kaunitz JD . Evaluation of a pharmacist-managed hepatitis C care clinic . Am J Health-Syst Pharm. 2007 ; 64 : 632 - 6 . Google Scholar Crossref Search ADS PubMed 11. Hale GM , Hassan SL , Hummel SL et al. Impact of a pharmacist-managed heart failure postdischarge (bridge) clinic for veterans . Ann Pharmacother. 2017 ; 51 : 555 - 62 . Google Scholar Crossref Search ADS PubMed 12. Lam MS , Cheung N . Impact of oncology pharmacist-managed oral anticancer therapy in patients with chronic myelogenous leukemia . J Oncol Pharm Pract. 2016 ; 22 : 741 - 8 . Google Scholar Crossref Search ADS PubMed 13. Schultz J , Horner K , McDanel D et al. Comparing clinical outcomes of a pharmacist-managed diabetes clinic to usual physician-based care . J Pharm Pract. 2018 ; 3 : 268 - 71 . Google Scholar Crossref Search ADS 14. Ray WA . Evaluating medication effects outside of clinical trials: new-user designs . Am J Epidemiol. 2003 ; 158 : 915 - 20 . Google Scholar Crossref Search ADS PubMed 15. U.S. Census Bureau ( 2016 ). Median household income. https://factfinder.census.gov/ ( accessed 2017 Jan 10 ). 16. Rubin D . Estimating causal effects of treatments in randomized and nonrandomized studies . J Educ Psychol. 1974 ; 66 : 688 - 701 . Google Scholar Crossref Search ADS 17. Rosenbaum P , Rubin D . The central role of the propensity score in observational studies for causal effects . Biometrika . 1983 ; 70 : 41 - 55 . Google Scholar Crossref Search ADS 18. Rubin D. Matched sampling for causal effects . New York : Cambridge University Press ; 2006 ; 170 - 184 . 19. Rubin DB . Estimating causal effects from large data sets using propensity scores . Ann Intern Med. 1997 ; 127 : 757 - 63 . Google Scholar Crossref Search ADS PubMed 20. Kurth T , Walker AM , Glynn RJ et al. Results of multivariable logistic regression, propensity matching, propensity adjustment, and propensity-based weighting under conditions of nonuniform effect . Am J Epidemiol. 2006 ; 163 : 262 - 70 . Google Scholar Crossref Search ADS PubMed 21. Cole SR , Hernán MA . Constructing inverse probability weights for marginal structural models . Am J Epidemiol. 2008 ; 168 : 656 - 64 . Google Scholar Crossref Search ADS PubMed 22. Robins JM , Hernán MA , Brumback B . Marginal structural models and causal inference in epidemiology . Epidemiology. 2000 ; 11 : 550 - 60 . Google Scholar Crossref Search ADS PubMed 23. Holland P . Statistics and causal inference . J Am Stat Assoc. 1986 ; 81 : 945 - 960 . Google Scholar Crossref Search ADS 24. McCarren M , Hampp C , Gerhard T , Mehta S . Recommendations on the use and nonuse of the p value in biomedical research . Am J Health-Syst Pharm. 2017 ; 74 : 1262 - 6 . Google Scholar Crossref Search ADS PubMed 25. Stuart EA . Matching methods for causal inference: a review and a look forward . Stat Sci. 2010 ; 25 : 1 - 21 . Google Scholar Crossref Search ADS PubMed 26. Langness JA , Nguyen M , Wieland A et al. Optimizing hepatitis C virus treatment through pharmacist interventions: identification and management of drug-drug interactions . World J Gastroenterol. 2017 ; 23 : 1618 - 26 . Google Scholar Crossref Search ADS PubMed 27. McHutchison JG , Manns M , Patel K et al. ; International Hepatitis Interventional Therapy Group . Adherence to combination therapy enhances sustained response in genotype-1-infected patients with chronic hepatitis C . Gastroenterology. 2002 ; 123 : 1061 - 9 . Google Scholar Crossref Search ADS PubMed 28. Fried MW , Shiffman ML , Reddy KR et al. Peginterferon alfa-2a plus ribavirin for chronic hepatitis C virus infection . N Engl J Med. 2002 ; 347 : 975 - 82 . Google Scholar Crossref Search ADS PubMed 29. Yang S , Britt RB , Hashem MG , Brown JN . Outcomes of pharmacy-led hepatitis C direct-acting antiviral utilization management at a Veterans Affairs medical center . J Manag Care Spec Pharm. 2017 ; 23 : 364 - 9 . Google Scholar PubMed 30. Patel M , Rab S , Kalapila AG et al. Highly successful hepatitis C virus (HCV) treatment outcomes in human immunodeficiency virus/HCV-coinfected patients at a large, urban, Ryan White clinic . Open Forum Infect Dis. 2017 ; 4 : ofx062 . Google Scholar Crossref Search ADS PubMed 31. Shah M , Tilton J , Kim S . Factors influencing enrollment in the medication therapy management clinic at an academic ambulatory care clinic . J Pharm Pract. 2016 ; 29 : 106 - 9 . Google Scholar Crossref Search ADS PubMed © American Society of Health-System Pharmacists 2019. All rights reserved. For permissions, please e-mail: [email protected]. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Jeon,, Nakyung;Staley,, Ben;Henriksen,, Carl;Lipori, Gloria, Pflugfelder;Winterstein, Almut, G

doi: 10.1093/ajhp/zxz043pmid: 31361856

Abstract Purpose Using information from institutional electronic health records, we aimed to develop dynamic predictive models to identify patients at high risk of acute kidney injury (AKI) among those who received a nephrotoxic medication during their hospital stay. Methods Candidate predictors were measured for each of the first 5 hospital days where a patient received a nephrotoxic medication (risk model days) to predict an AKI, using logistic regression with reduced backward variables elimination in 100 bootstrap samples. An AKI event was defined as an increase of serum creatinine ≥ 200% of a baseline SCr within 5 days after a risk model day. Final models were internally validated by replication in 100 bootstrap samples and a risk score for each patient was calculated from the validated model. As performance measures, the area under the receiver operation characteristic curves (AUC) and the number of AKI events among patients who had high risk scores were estimated. Results The study population included 62,561 admissions followed by 1,212 AKI events (1.9 events/100 admissions). We constructed 5 risk models corresponding to the first 5 hospital days where patients were exposed to at least one nephrotoxic medication. Validated AUCs of the 5 models ranged from 0.78 to 0.81. Depending on risk model day, admissions ranked in the 90th percentile of the risk score captured between 43% to 49% of all AKI events. Conclusion A dynamic prediction model was built successfully for inpatient AKI with excellent discriminative validity and good calibration, allowing clinicians to focus on a select high-risk population that captures the majority of AKI events. acute kidney injury, electronic health records, nephrotoxic medications, prediction model, risk score KEY POINTS Using clinical information from fully-integrated electronic health records, this study successfully built dynamic prediction models for inpatient AKI allowing full automation of risk models for daily priority ranking of patients for clinical intervention. Model AUCs ranged 0.79 – 0.81, demonstrating excellent model discriminative performances. Admissions ranked in the upper 90% percentile of the prediction models captured between 43 % and 49% of all AKI events. If only admissions in the upper 90th percentile of the risk score were targeted, providers would need to intervene on 10 - 17 admissions to prevent one AKI event. Albuminemia, liver injury, obesity, stage 1 AKI, systemic inflammatory reaction, and concomitant use of piperacillin/tazobactam (TZP) and vancomycin were strongest risk factors across all models. Acute kidney injury (AKI) is a prevalent adverse event among hospitalized patients. According to a systematic review, the average rate of AKI is 21.6% (95% confidence interval [CI]: 19.3%–24.1%) in adults during hospitalization.1 AKI risk can be as high as 50% in critically ill patients with underlying risk conditions, such as advanced age, sepsis, severe dehydration, and chronic kidney disease.2 Irrespective of its etiology, AKI appears independently associated with increased length of hospital stay and higher in-hospital mortality.3-6 Exposure to nephrotoxic medications poses additional AKI risk in hospitalized patients. Drugs are the third to fifth leading cause of hospital-acquired AKI.7 More than 40% of hospitalized patients are treated with at least one nephrotoxic medication, which is often not properly adjusted or used based on renal function.8 Certain drugs can alter intraglomerular hemodynamics or cause renal tubular inflammation, renal obstruction, and interstitial nephritis, leading to AKI.9 Factors that determine patient susceptibility to drug-associated AKI include dehydration, certain demographic characteristics, acute and chronic comorbidities, and drugs themselves. The interaction between predisposing risk factors and the type and extent of drug exposure plays a key role in AKI risk. Cumulative effects of those variables throughout the hospital stay can also lead to renal deterioration. Thus, a continuous evaluation of risk factors should be considered to assess and avoid the risk of drug-associated AKI during hospitalization. Several studies have developed algorithms predicting AKI risk in inpatient settings.10-18 These algorithms have used data in the electronic health records (EHR) but lack a comprehensive evaluation of drug-related AKI risk factors. Risk models to date have been static and do not incorporate changes in clinical risk factors or treatment throughout the hospital stay. Finally, the available AKI risk algorithms are generally limited to narrow populations, such as exposure to radiographic contrast agents, percutaneous coronary insertion, or open cardiac surgery, with limited consideration of drug-related causes. Prediction models specific to inpatients who receive nephrotoxic medications are virtually nonexistent. Considering the need to promote prevention strategies of AKI in all hospitalized patients, an automated risk algorithm may enhance providers’ recognition of emerging drug-associated AKI risk, even in patient populations that do not undergo high-risk procedures. This is particularly relevant in the hospital setting, where risk factors can change expeditiously, including rapid changes in treatment plans to reduce exposure to high-risk medications.19 In this study, we aimed to construct dynamic risk models for AKI specific to hospitalized patients who received nephrotoxic medications, using comprehensive information from hospital EHRs. Methods Study population. We constructed a retrospective cohort of patients aged 18 years or older who were admitted and discharged from UF Health Shands (January 2012 through December 2013) or UF Health Jacksonville (March 2013 through December 2013). Included patients were those who received a nephrotoxic medication for systemic use at least once during the first 5 hospital days. The list of nephrotoxic medications assembled was based on guidelines and landmark studies and includes 101 unique active ingredients (see Appendix A).20-23 We excluded patients who were discharged from the hospitals on the same day of admission, were dialysis-dependent, or had an operation on the urinary system (kidney, ureter, urinary bladder, urethra, or urinary tract) within up to 365 days prior to admission. The study was approved by the institutional review and privacy boards of the University of Florida. Study data. Patients’ characteristics, such as age, sex, admission and discharge information, medication administration records, laboratory test results, physician orders, diagnoses, and procedures, were retrospectively obtained from the hospital EHR system. Medication information including drug name, route, dose, and date and time of administration was obtained from inpatient electronic medication administration records. Similarly, laboratory test type and results, and date and time of the results were obtained. Patients who received radiologic tests (i.e., computerized tomography scans, angiography, arteriography, cardiac catheterization, and venography) were flagged based on discrete records in physician order data in the EHR. Diseases and surgical procedures for each patient were obtained from the International Classification of Diseases version 9 (ICD-9-CM) and Current Procedural Terminology (CPT) codes, respectively. If available, we obtained diagnoses, procedures, and laboratory values from outpatient and inpatient encounters in the EHR data up to 365 days prior to the current hospital admission in order to capture patient medical history. Concept of the prediction model. We developed 5 individual daily risk models (Day 1–Day 5) corresponding to the first 5 hospital days, referred to as “risk model days”. Because the majority of patients are discharged within 5 days of admission, resulting in sample size constraints thereafter, we did not attempt to construct risk models for later hospital days. Each of the 5 models included risk factor information aggregated up to the risk model day. Thus, later risk models had richer information as more test results became available and patients progressed through their admission. For example, while the Day 1 risk model could only consider nephrotoxic medication history and exposure on the admission day, later day risk models could consider cumulative exposure during the hospital stay. Each risk factor was assigned a duration to capture proximity to development of AKI. For example, low blood pressure was considered a risk factor for 1 day, while contrast agent exposure was considered a risk factor for 4 days. Patients were considered for prediction modeling for a particular risk day if they received any nephrotoxic medication. For example, a patient who received a nephrotoxic medication on hospital day 1 and 2 and discontinued thereafter would be considered for the Day 1 and Day 2 risk models only. Outcome measures. Because AKI manifestation may take several days after a renal insult, we followed a patient up to 5 days after a risk model day to ascertain onset. Following the Kidney Disease International Global Organization (KDIGO) stage 2 AKI criteria, AKI was defined as the first day with an increase of serum creatinine (SCr) ≥ 200% of a baseline SCr within 1–5 days.19 Baseline SCr was defined as the most recent SCr within 2 days prior to a given risk model day. We chose stage 2 AKI in order to focus on events that would be appropriate targets for preventive action while retaining adequate sample size (number of events) for prediction modeling. We made an assumption that patients would be discharged only if they were healthy enough not to develop AKI shortly after leaving the hospital, because we did not have follow-up data beyond discharge. Candidate risk factors and operational definitions. We defined more than 40 candidate risk factors based on clinical expert input and a comprehensive literature search on studies of AKI risk factors in hospitalized patients.19,24-33 We conducted a series of univariate analyses to examine associations between each of the risk factors and AKI and removed risk factors whose prevalence estimates or examined associations suggested measurement error. Subsequently, to avoid overfitting, collinearity between risk factors was examined via cluster analysis, which was set to identify groups of highly correlated variables until 75% of total variation was accounted for. Among each identified group, we chose the risk factor with the strongest predictive property for developing an AKI, measured by the Akaike Information Criterion. In some cases, we collapsed 2 or more risk factors into a composite variable for increased statistical efficiency if the statistically correlated risk factors also captured similar clinical scenarios. Finally, we arrived at 27 drug- or nondrug-associated risk factors considered for prediction model development. Drug-associated risk factors included the number of (high-risk) nephrotoxic medications, use of angiotensin-converting enzyme inhibitors (ACEIs) or angiotensin-receptor blockers (ARBs) plus diuretics plus nonsteroidal antiinflammatory drugs (NSAIDs), and the use of vancomycin plus piperacillin-tazobactam. Nondrug-associated risk factors included surgery-related (i.e., cardiac surgery, other high-risk surgery, prolonged surgery, and procedures involving contrast use) and kidney function-related susceptibilities (i.e., mild AKI, oliguria, increasing trend in SCr, chronic kidney disease, and rhabdomyolysis), and others, such as female, advanced age, critical illness, and nonkidney function-related comorbidities (i.e., acidosis, albuminemia, anemia, dehydration, diabetes, heart failure, hypotension, liver injury, obesity, and systemic inflammatory response). Missing data for a risk factor were accounted for either by a missing data indicator (if missing information had its own predictivity), or a simple imputation replacing the missing with an extrapolated value, such as mean.34 For example, sicker or older patients may have more laboratory assessments; thus, missingness may indicate lower risk for AKI. Table 1 shows the definition, measurement period, and the use of missing indicators and imputation of the 27 risk factors. Table 1. Definition, Use of Missing Indicators/Imputations, and Measurement Period of Candidate Variablesa Variable Variable Definition Measurement Period Missing Indicator/Imputation Drug use-related risk factors  No. of nephrotoxic drugs Total number of administered nephrotoxic medications On risk day …b  No. of high nephrotoxic drugs Total number of administered high nephrotoxic medications at modeling day On risk day …  Use of ACEIs/ARBs + NSAIDs + diuretics Administration of diuretics within 24 hours of ARBs or ACEI, and within 24 hours of NSAIDs; all drugs must be within 24 hours of each other, and at least 1 drug must be administered on risk day At or up to 1 day before risk day …  Piperacillin and tazobactam + vancomycin Administration of vancomycin within 24 hours of piperacillin and tazobactam (or vice versa); at least one of the 2 medications must be administered on risk day; the other may be administered risk day or previous day At or up to 1 day before risk day … Surgery/procedure-related susceptibilities  Cardiac surgery Presence of CPT codes for cardiac surgery At or up to 4 days before risk day …  Prolonged surgery Use start and end surgery time to determine if surgery time ≥ 10 hr At or up to 4 days before risk day No surgeries as a missing indicator  Use of contrast Presence of physician order for contrast associated procedures (angiography, arteriography, cardiac catheterization, venogram, or computed tomography with contrast) At or up to 4 days before risk day …  High-risk surgery Presence of CPT codes for thoracic (except cardiac), abdominal, or vascular surgery, or transplant At or up to 4 days before risk day … Kidney function-related susceptibilities  Stage 1 AKI Presence of an increase in SCr ≥ 0.3 mg/dL within 48 hr; or an SCr increase 100% to < 200% of a baseline SCr within 7 days At or up to 4 days before risk day No SCr measured as a missing indicator  Oliguria The total daily volume of recorded urine output < 500 mL At or up to 1 day before risk day No urine output measured as a missing indicator  Increasing trend in SCRs LSRL of SCr points is positive (consider the most recent 4–10 SCr values at or upto 4 days prior to risk day to calculate the slope) At or up to 4 days before risk day If fewer than 4 records of SCr values available, missing indicator was used  Chronic kidney disease Presence of ICD9 585.X; or the most recent CrCl < 60 estimated by Cockcroft-Gault formula For ICD-9-CM code: at this or previous admissions within the past year for CrCl: at or up to 30 days before risk day If CrCl is missing, impute it to 90  Rhabdomyolysis Serum creatinine kinase > 5,000 IU/L At or up to 4 days before risk day If serum creatinine kinase is missing, impute it to 1,000 IU/L Other susceptibilities  Sex Male On admission …  Acidosis The most recent pH < 7.3 and the most recent CO2 < 20 meq/L; or the most recent plasma lactate concentration > 4 meq/L At or up to 4 days before risk day Missing indicator, if no measured pH, CO2, and plasma lactate concentration  Advanced age Age 65 years or older On the day of admission …  Albuminemia The most recent serum albumin < 3.4 g/dL At or up to 4 days before risk day Use missing indicator if no measured serum albumin level  Anemia The lowest value of HCT < 31.0% and HGB < 10.0 g/dL At or up to 4 days before risk day If no HCT or Hgb value, impute HCT to 40% and Hgb to 13.0 g/dL  Hypertension Presence of ICD-9-CM codes 401.xx–405.xx At this or previous admissions within the past year …  Critically ill Patient who stayed in the ICU On risk day …  Dehydration The ratio of BUN to SCr > 20 At or up to 1 day before risk day Use a missing indicator if BUN or SCr value is missing  Diabetes Presence of ICD-9-CM code (250.xx); or the most recent HbA1c > 7% For ICD-9-CM code: at this or previous admission within the past year; for HbA1c: within up to 30 days preadmission If HbA1c is missing impute it to 5%  Heart failure Presence of ICD-9-CM code (428.x); or the lowest LVEF < 60; or the highest BNP > 400 pg/mL; or the highest NT-proBNP > 2,000 pg/mL For ICD-9-CM code: at this or previous admission within the past year; for LVEF, BNP, or NT-proBNP: at or up to 30 days before risk day If LVEF, BNP, or NT-proBNP is missing imput LVEF to 65%, BNP to 100 pg/mL, and NT-proBNP to 400 pg/mL  Hypotension The lowest value for SBP < 80 mm Hg or MAP < 65 mm Hg for MAP; use MAP = (2 DBP + SBP)/3 At or up to 1 day before risk day If SBP or MAP is missing, impute SBP to 120 mm Hg and MAP to 75  Liver injury Presence of ICD-9-CM (070.0, 070.2x, 070.4x, 070.6x, 070.71, 456.0, 456.1, 456.2, 571.xx, 572.2, 572.3, 572.4, 572.8, 573.1x, 573.2x, 573.3x, 573.4x, 573.5x, 570.xx, 573.8x, 573.9x); or AST/ALT greater than 3 times of the ULN (35 = ULN for both); or bilirubin > 2.0 mg/dL; or elevated INR > 1.5 while not on warfarin and warfarin did not appear in the PTA medication list For ICD-9-CM code: at this or previous admission within the past year; for AST/ALT/bilirubin: any day prior to risk day but up to 30 preadmission days; for INR and warfarin: any day prior to risk day during admission If AST, ALT, bilirubin, or INR is missing, impute AST/ALT to as 35, bilirubin as 0.9 mg/dL, and INR to 1.0  Obesity The most recent BMI > 35; BMI = Weight (kg)/height2 (m) At or up to 1 month before risk day If weight or height is missing, impute BMI to 27  Systemic inflammatory response Presence of 3 or more of the following criteria (at least 3 must manifest on a single day): body temperature < 36°C or > 38°C, heart rate > 90 beats per minute, respiratory rate > 20 breaths per min; WBC count < 4,000 cells/mm3 or > 12,000 cells/mm3 or the presence of > 10% INP At or up to 4 days before risk day If body temperature, heart rate, respiratory rate, WBC, or INP is missing, impute body temperature to 36.8°C, heart rate to 80 beats per minute, respiratory rate to 17 breaths per min, WBC count to 8,000 cells/mm3, and INP to 7% Variable Variable Definition Measurement Period Missing Indicator/Imputation Drug use-related risk factors  No. of nephrotoxic drugs Total number of administered nephrotoxic medications On risk day …b  No. of high nephrotoxic drugs Total number of administered high nephrotoxic medications at modeling day On risk day …  Use of ACEIs/ARBs + NSAIDs + diuretics Administration of diuretics within 24 hours of ARBs or ACEI, and within 24 hours of NSAIDs; all drugs must be within 24 hours of each other, and at least 1 drug must be administered on risk day At or up to 1 day before risk day …  Piperacillin and tazobactam + vancomycin Administration of vancomycin within 24 hours of piperacillin and tazobactam (or vice versa); at least one of the 2 medications must be administered on risk day; the other may be administered risk day or previous day At or up to 1 day before risk day … Surgery/procedure-related susceptibilities  Cardiac surgery Presence of CPT codes for cardiac surgery At or up to 4 days before risk day …  Prolonged surgery Use start and end surgery time to determine if surgery time ≥ 10 hr At or up to 4 days before risk day No surgeries as a missing indicator  Use of contrast Presence of physician order for contrast associated procedures (angiography, arteriography, cardiac catheterization, venogram, or computed tomography with contrast) At or up to 4 days before risk day …  High-risk surgery Presence of CPT codes for thoracic (except cardiac), abdominal, or vascular surgery, or transplant At or up to 4 days before risk day … Kidney function-related susceptibilities  Stage 1 AKI Presence of an increase in SCr ≥ 0.3 mg/dL within 48 hr; or an SCr increase 100% to < 200% of a baseline SCr within 7 days At or up to 4 days before risk day No SCr measured as a missing indicator  Oliguria The total daily volume of recorded urine output < 500 mL At or up to 1 day before risk day No urine output measured as a missing indicator  Increasing trend in SCRs LSRL of SCr points is positive (consider the most recent 4–10 SCr values at or upto 4 days prior to risk day to calculate the slope) At or up to 4 days before risk day If fewer than 4 records of SCr values available, missing indicator was used  Chronic kidney disease Presence of ICD9 585.X; or the most recent CrCl < 60 estimated by Cockcroft-Gault formula For ICD-9-CM code: at this or previous admissions within the past year for CrCl: at or up to 30 days before risk day If CrCl is missing, impute it to 90  Rhabdomyolysis Serum creatinine kinase > 5,000 IU/L At or up to 4 days before risk day If serum creatinine kinase is missing, impute it to 1,000 IU/L Other susceptibilities  Sex Male On admission …  Acidosis The most recent pH < 7.3 and the most recent CO2 < 20 meq/L; or the most recent plasma lactate concentration > 4 meq/L At or up to 4 days before risk day Missing indicator, if no measured pH, CO2, and plasma lactate concentration  Advanced age Age 65 years or older On the day of admission …  Albuminemia The most recent serum albumin < 3.4 g/dL At or up to 4 days before risk day Use missing indicator if no measured serum albumin level  Anemia The lowest value of HCT < 31.0% and HGB < 10.0 g/dL At or up to 4 days before risk day If no HCT or Hgb value, impute HCT to 40% and Hgb to 13.0 g/dL  Hypertension Presence of ICD-9-CM codes 401.xx–405.xx At this or previous admissions within the past year …  Critically ill Patient who stayed in the ICU On risk day …  Dehydration The ratio of BUN to SCr > 20 At or up to 1 day before risk day Use a missing indicator if BUN or SCr value is missing  Diabetes Presence of ICD-9-CM code (250.xx); or the most recent HbA1c > 7% For ICD-9-CM code: at this or previous admission within the past year; for HbA1c: within up to 30 days preadmission If HbA1c is missing impute it to 5%  Heart failure Presence of ICD-9-CM code (428.x); or the lowest LVEF < 60; or the highest BNP > 400 pg/mL; or the highest NT-proBNP > 2,000 pg/mL For ICD-9-CM code: at this or previous admission within the past year; for LVEF, BNP, or NT-proBNP: at or up to 30 days before risk day If LVEF, BNP, or NT-proBNP is missing imput LVEF to 65%, BNP to 100 pg/mL, and NT-proBNP to 400 pg/mL  Hypotension The lowest value for SBP < 80 mm Hg or MAP < 65 mm Hg for MAP; use MAP = (2 DBP + SBP)/3 At or up to 1 day before risk day If SBP or MAP is missing, impute SBP to 120 mm Hg and MAP to 75  Liver injury Presence of ICD-9-CM (070.0, 070.2x, 070.4x, 070.6x, 070.71, 456.0, 456.1, 456.2, 571.xx, 572.2, 572.3, 572.4, 572.8, 573.1x, 573.2x, 573.3x, 573.4x, 573.5x, 570.xx, 573.8x, 573.9x); or AST/ALT greater than 3 times of the ULN (35 = ULN for both); or bilirubin > 2.0 mg/dL; or elevated INR > 1.5 while not on warfarin and warfarin did not appear in the PTA medication list For ICD-9-CM code: at this or previous admission within the past year; for AST/ALT/bilirubin: any day prior to risk day but up to 30 preadmission days; for INR and warfarin: any day prior to risk day during admission If AST, ALT, bilirubin, or INR is missing, impute AST/ALT to as 35, bilirubin as 0.9 mg/dL, and INR to 1.0  Obesity The most recent BMI > 35; BMI = Weight (kg)/height2 (m) At or up to 1 month before risk day If weight or height is missing, impute BMI to 27  Systemic inflammatory response Presence of 3 or more of the following criteria (at least 3 must manifest on a single day): body temperature < 36°C or > 38°C, heart rate > 90 beats per minute, respiratory rate > 20 breaths per min; WBC count < 4,000 cells/mm3 or > 12,000 cells/mm3 or the presence of > 10% INP At or up to 4 days before risk day If body temperature, heart rate, respiratory rate, WBC, or INP is missing, impute body temperature to 36.8°C, heart rate to 80 beats per minute, respiratory rate to 17 breaths per min, WBC count to 8,000 cells/mm3, and INP to 7% aACEI = angiotensin-converting enzyme inhibitors, AKI = acute kidney injury, ALT = alanine transaminase, ARB = angiotensin-receptor blockers, AST = aspartate transaminase, BMI = body mass index, BNP = brain natriuretic peptide, BUN = blood urea nitrogen, CPT = current procedural terminology, CrCl = creatinine clearance, DBP = diastolic blood pressure, Hgb = hemoglobin, HbA1C = glycosylated hemoglobin, HCT = hematocrit, ICD = International Classification of Diseases, INP = immature neutrophils percent, INR = international normalized ratio, LSRL = slope of the least square regression line, LVEF = left ventricular ejection fraction, MAP = mean arterial pressure, NSAID = nonsteroidal antiinflammatory drug, PTA = prior to admission, SBP = systolic blood pressure, SCr = serum creatinine, ULN = upper limit of normal, WBC = white blood cell. bNot applicable. View Large Table 1. Definition, Use of Missing Indicators/Imputations, and Measurement Period of Candidate Variablesa Variable Variable Definition Measurement Period Missing Indicator/Imputation Drug use-related risk factors  No. of nephrotoxic drugs Total number of administered nephrotoxic medications On risk day …b  No. of high nephrotoxic drugs Total number of administered high nephrotoxic medications at modeling day On risk day …  Use of ACEIs/ARBs + NSAIDs + diuretics Administration of diuretics within 24 hours of ARBs or ACEI, and within 24 hours of NSAIDs; all drugs must be within 24 hours of each other, and at least 1 drug must be administered on risk day At or up to 1 day before risk day …  Piperacillin and tazobactam + vancomycin Administration of vancomycin within 24 hours of piperacillin and tazobactam (or vice versa); at least one of the 2 medications must be administered on risk day; the other may be administered risk day or previous day At or up to 1 day before risk day … Surgery/procedure-related susceptibilities  Cardiac surgery Presence of CPT codes for cardiac surgery At or up to 4 days before risk day …  Prolonged surgery Use start and end surgery time to determine if surgery time ≥ 10 hr At or up to 4 days before risk day No surgeries as a missing indicator  Use of contrast Presence of physician order for contrast associated procedures (angiography, arteriography, cardiac catheterization, venogram, or computed tomography with contrast) At or up to 4 days before risk day …  High-risk surgery Presence of CPT codes for thoracic (except cardiac), abdominal, or vascular surgery, or transplant At or up to 4 days before risk day … Kidney function-related susceptibilities  Stage 1 AKI Presence of an increase in SCr ≥ 0.3 mg/dL within 48 hr; or an SCr increase 100% to < 200% of a baseline SCr within 7 days At or up to 4 days before risk day No SCr measured as a missing indicator  Oliguria The total daily volume of recorded urine output < 500 mL At or up to 1 day before risk day No urine output measured as a missing indicator  Increasing trend in SCRs LSRL of SCr points is positive (consider the most recent 4–10 SCr values at or upto 4 days prior to risk day to calculate the slope) At or up to 4 days before risk day If fewer than 4 records of SCr values available, missing indicator was used  Chronic kidney disease Presence of ICD9 585.X; or the most recent CrCl < 60 estimated by Cockcroft-Gault formula For ICD-9-CM code: at this or previous admissions within the past year for CrCl: at or up to 30 days before risk day If CrCl is missing, impute it to 90  Rhabdomyolysis Serum creatinine kinase > 5,000 IU/L At or up to 4 days before risk day If serum creatinine kinase is missing, impute it to 1,000 IU/L Other susceptibilities  Sex Male On admission …  Acidosis The most recent pH < 7.3 and the most recent CO2 < 20 meq/L; or the most recent plasma lactate concentration > 4 meq/L At or up to 4 days before risk day Missing indicator, if no measured pH, CO2, and plasma lactate concentration  Advanced age Age 65 years or older On the day of admission …  Albuminemia The most recent serum albumin < 3.4 g/dL At or up to 4 days before risk day Use missing indicator if no measured serum albumin level  Anemia The lowest value of HCT < 31.0% and HGB < 10.0 g/dL At or up to 4 days before risk day If no HCT or Hgb value, impute HCT to 40% and Hgb to 13.0 g/dL  Hypertension Presence of ICD-9-CM codes 401.xx–405.xx At this or previous admissions within the past year …  Critically ill Patient who stayed in the ICU On risk day …  Dehydration The ratio of BUN to SCr > 20 At or up to 1 day before risk day Use a missing indicator if BUN or SCr value is missing  Diabetes Presence of ICD-9-CM code (250.xx); or the most recent HbA1c > 7% For ICD-9-CM code: at this or previous admission within the past year; for HbA1c: within up to 30 days preadmission If HbA1c is missing impute it to 5%  Heart failure Presence of ICD-9-CM code (428.x); or the lowest LVEF < 60; or the highest BNP > 400 pg/mL; or the highest NT-proBNP > 2,000 pg/mL For ICD-9-CM code: at this or previous admission within the past year; for LVEF, BNP, or NT-proBNP: at or up to 30 days before risk day If LVEF, BNP, or NT-proBNP is missing imput LVEF to 65%, BNP to 100 pg/mL, and NT-proBNP to 400 pg/mL  Hypotension The lowest value for SBP < 80 mm Hg or MAP < 65 mm Hg for MAP; use MAP = (2 DBP + SBP)/3 At or up to 1 day before risk day If SBP or MAP is missing, impute SBP to 120 mm Hg and MAP to 75  Liver injury Presence of ICD-9-CM (070.0, 070.2x, 070.4x, 070.6x, 070.71, 456.0, 456.1, 456.2, 571.xx, 572.2, 572.3, 572.4, 572.8, 573.1x, 573.2x, 573.3x, 573.4x, 573.5x, 570.xx, 573.8x, 573.9x); or AST/ALT greater than 3 times of the ULN (35 = ULN for both); or bilirubin > 2.0 mg/dL; or elevated INR > 1.5 while not on warfarin and warfarin did not appear in the PTA medication list For ICD-9-CM code: at this or previous admission within the past year; for AST/ALT/bilirubin: any day prior to risk day but up to 30 preadmission days; for INR and warfarin: any day prior to risk day during admission If AST, ALT, bilirubin, or INR is missing, impute AST/ALT to as 35, bilirubin as 0.9 mg/dL, and INR to 1.0  Obesity The most recent BMI > 35; BMI = Weight (kg)/height2 (m) At or up to 1 month before risk day If weight or height is missing, impute BMI to 27  Systemic inflammatory response Presence of 3 or more of the following criteria (at least 3 must manifest on a single day): body temperature < 36°C or > 38°C, heart rate > 90 beats per minute, respiratory rate > 20 breaths per min; WBC count < 4,000 cells/mm3 or > 12,000 cells/mm3 or the presence of > 10% INP At or up to 4 days before risk day If body temperature, heart rate, respiratory rate, WBC, or INP is missing, impute body temperature to 36.8°C, heart rate to 80 beats per minute, respiratory rate to 17 breaths per min, WBC count to 8,000 cells/mm3, and INP to 7% Variable Variable Definition Measurement Period Missing Indicator/Imputation Drug use-related risk factors  No. of nephrotoxic drugs Total number of administered nephrotoxic medications On risk day …b  No. of high nephrotoxic drugs Total number of administered high nephrotoxic medications at modeling day On risk day …  Use of ACEIs/ARBs + NSAIDs + diuretics Administration of diuretics within 24 hours of ARBs or ACEI, and within 24 hours of NSAIDs; all drugs must be within 24 hours of each other, and at least 1 drug must be administered on risk day At or up to 1 day before risk day …  Piperacillin and tazobactam + vancomycin Administration of vancomycin within 24 hours of piperacillin and tazobactam (or vice versa); at least one of the 2 medications must be administered on risk day; the other may be administered risk day or previous day At or up to 1 day before risk day … Surgery/procedure-related susceptibilities  Cardiac surgery Presence of CPT codes for cardiac surgery At or up to 4 days before risk day …  Prolonged surgery Use start and end surgery time to determine if surgery time ≥ 10 hr At or up to 4 days before risk day No surgeries as a missing indicator  Use of contrast Presence of physician order for contrast associated procedures (angiography, arteriography, cardiac catheterization, venogram, or computed tomography with contrast) At or up to 4 days before risk day …  High-risk surgery Presence of CPT codes for thoracic (except cardiac), abdominal, or vascular surgery, or transplant At or up to 4 days before risk day … Kidney function-related susceptibilities  Stage 1 AKI Presence of an increase in SCr ≥ 0.3 mg/dL within 48 hr; or an SCr increase 100% to < 200% of a baseline SCr within 7 days At or up to 4 days before risk day No SCr measured as a missing indicator  Oliguria The total daily volume of recorded urine output < 500 mL At or up to 1 day before risk day No urine output measured as a missing indicator  Increasing trend in SCRs LSRL of SCr points is positive (consider the most recent 4–10 SCr values at or upto 4 days prior to risk day to calculate the slope) At or up to 4 days before risk day If fewer than 4 records of SCr values available, missing indicator was used  Chronic kidney disease Presence of ICD9 585.X; or the most recent CrCl < 60 estimated by Cockcroft-Gault formula For ICD-9-CM code: at this or previous admissions within the past year for CrCl: at or up to 30 days before risk day If CrCl is missing, impute it to 90  Rhabdomyolysis Serum creatinine kinase > 5,000 IU/L At or up to 4 days before risk day If serum creatinine kinase is missing, impute it to 1,000 IU/L Other susceptibilities  Sex Male On admission …  Acidosis The most recent pH < 7.3 and the most recent CO2 < 20 meq/L; or the most recent plasma lactate concentration > 4 meq/L At or up to 4 days before risk day Missing indicator, if no measured pH, CO2, and plasma lactate concentration  Advanced age Age 65 years or older On the day of admission …  Albuminemia The most recent serum albumin < 3.4 g/dL At or up to 4 days before risk day Use missing indicator if no measured serum albumin level  Anemia The lowest value of HCT < 31.0% and HGB < 10.0 g/dL At or up to 4 days before risk day If no HCT or Hgb value, impute HCT to 40% and Hgb to 13.0 g/dL  Hypertension Presence of ICD-9-CM codes 401.xx–405.xx At this or previous admissions within the past year …  Critically ill Patient who stayed in the ICU On risk day …  Dehydration The ratio of BUN to SCr > 20 At or up to 1 day before risk day Use a missing indicator if BUN or SCr value is missing  Diabetes Presence of ICD-9-CM code (250.xx); or the most recent HbA1c > 7% For ICD-9-CM code: at this or previous admission within the past year; for HbA1c: within up to 30 days preadmission If HbA1c is missing impute it to 5%  Heart failure Presence of ICD-9-CM code (428.x); or the lowest LVEF < 60; or the highest BNP > 400 pg/mL; or the highest NT-proBNP > 2,000 pg/mL For ICD-9-CM code: at this or previous admission within the past year; for LVEF, BNP, or NT-proBNP: at or up to 30 days before risk day If LVEF, BNP, or NT-proBNP is missing imput LVEF to 65%, BNP to 100 pg/mL, and NT-proBNP to 400 pg/mL  Hypotension The lowest value for SBP < 80 mm Hg or MAP < 65 mm Hg for MAP; use MAP = (2 DBP + SBP)/3 At or up to 1 day before risk day If SBP or MAP is missing, impute SBP to 120 mm Hg and MAP to 75  Liver injury Presence of ICD-9-CM (070.0, 070.2x, 070.4x, 070.6x, 070.71, 456.0, 456.1, 456.2, 571.xx, 572.2, 572.3, 572.4, 572.8, 573.1x, 573.2x, 573.3x, 573.4x, 573.5x, 570.xx, 573.8x, 573.9x); or AST/ALT greater than 3 times of the ULN (35 = ULN for both); or bilirubin > 2.0 mg/dL; or elevated INR > 1.5 while not on warfarin and warfarin did not appear in the PTA medication list For ICD-9-CM code: at this or previous admission within the past year; for AST/ALT/bilirubin: any day prior to risk day but up to 30 preadmission days; for INR and warfarin: any day prior to risk day during admission If AST, ALT, bilirubin, or INR is missing, impute AST/ALT to as 35, bilirubin as 0.9 mg/dL, and INR to 1.0  Obesity The most recent BMI > 35; BMI = Weight (kg)/height2 (m) At or up to 1 month before risk day If weight or height is missing, impute BMI to 27  Systemic inflammatory response Presence of 3 or more of the following criteria (at least 3 must manifest on a single day): body temperature < 36°C or > 38°C, heart rate > 90 beats per minute, respiratory rate > 20 breaths per min; WBC count < 4,000 cells/mm3 or > 12,000 cells/mm3 or the presence of > 10% INP At or up to 4 days before risk day If body temperature, heart rate, respiratory rate, WBC, or INP is missing, impute body temperature to 36.8°C, heart rate to 80 beats per minute, respiratory rate to 17 breaths per min, WBC count to 8,000 cells/mm3, and INP to 7% aACEI = angiotensin-converting enzyme inhibitors, AKI = acute kidney injury, ALT = alanine transaminase, ARB = angiotensin-receptor blockers, AST = aspartate transaminase, BMI = body mass index, BNP = brain natriuretic peptide, BUN = blood urea nitrogen, CPT = current procedural terminology, CrCl = creatinine clearance, DBP = diastolic blood pressure, Hgb = hemoglobin, HbA1C = glycosylated hemoglobin, HCT = hematocrit, ICD = International Classification of Diseases, INP = immature neutrophils percent, INR = international normalized ratio, LSRL = slope of the least square regression line, LVEF = left ventricular ejection fraction, MAP = mean arterial pressure, NSAID = nonsteroidal antiinflammatory drug, PTA = prior to admission, SBP = systolic blood pressure, SCr = serum creatinine, ULN = upper limit of normal, WBC = white blood cell. bNot applicable. View Large Model development. We used multivariate logistic regression to model the relationship between risk factors compiled from a given risk model day (or preceding days) and AKI that occurred at the following 1–5 days. To optimize model fit, we built 4 different models: (1) a full model including a full set of risk factors; (2) a parsimonious model developed via fast backward elimination technique that retained all variables with a P value < 0.15; (3) a reduced model with variables that were retained at least 40 out of 100 backward elimination processes, which were repeated by bootstrap resampling; and (4) an expert model which included variables that were selected a priori by our work group. Discrimination and predictive performance of models were assessed using the C-statistic. The Hosmer-Lemeshow test was performed to examine model calibration.35 Model validation. Because model performance can be biased due to overfitting, we conducted an internal validation to ensure that predictions are valid for subjects that were not included in model development. To derive optimism-corrected (unbiased) C-statistics, we refitted model parameters from 100 bootstrap resamples to the original data set. We accepted a model if the unbiased C-statistic fell into the 95% confidence interval of the original C-statistic. Finally, we used β regression coefficient values from the validated model to calculate a risk score for each admission and examined the proportion of patients with AKI captured across risk score percentiles. All statistical analyses were conducted using SAS version 9.4 (SAS Inc., Cary, NC). Results Among 83,787 admissions to the UF hospitals during the study period, 62,561 (74.67%) received a nephrotoxic medication at least once during the first 5 hospital days. Admissions received about 1.5 nephrotoxic medications, including aspirin, lisinopril, and furosemide as the most commonly administered nephrotoxic medications (Table 2). About one third of the admissions was older than 65 years and 10% had chronic kidney disease on admission. Table 2. Characteristics of Study Populationa Characteristics Population = 62,561 Age > 65 years 20,257 (32.38%) Mean age ± S.D. 54.36 ± 18.90 Sex, Male 27,812 (44.46%) Race  White 40,976 (65.50%)  Black 16,912 (27.03%)  Others 4,673 (0.07%) ICU on admission 6,381 (10.20%) Existing renal complications and function presented on admission  AKI (ICD-9-CM codes: 584.X) 5,793 (9.26%)  Chronic kidney disease (ICD-9-CM codes: 585.X) 6,656 (10.64%) Other common comorbid conditions  Hypertension 39,783 (63.59%)  Diabetes 19,288 (30.83%)  Heart Failure 14,789 (23.64%)  Anemia 11,749 (18.78%)  Liver Disease 10,222 (16.34%) Median days of hospital stay (IQR) 4 days (3–7) AKI during hospitalization (KDIGO Stage 2) 1,212 (1.94%) Number of nephrotoxic medication per hospital day, mean ± S.D. 1.56 ± 0.89 10 most frequently administered nephrotoxic medications  Aspirin 22,530 (36.01%)  Lisinopril 16,338 (26.12%)  Furosemide 14,094 (22.53%)  Vancomycin 13,568 (21.69%)  Ibuprofen 11,746 (18.78%)  Cefazolin 8,511 (13.60%)  Cefepime 7,745 (12.38%)  Ceftriaxone 6,282 (10.04%)  Piperacillin-tazobactam 6,055 (9.68%)  Ketorolac 6,028 (9.64%) Characteristics Population = 62,561 Age > 65 years 20,257 (32.38%) Mean age ± S.D. 54.36 ± 18.90 Sex, Male 27,812 (44.46%) Race  White 40,976 (65.50%)  Black 16,912 (27.03%)  Others 4,673 (0.07%) ICU on admission 6,381 (10.20%) Existing renal complications and function presented on admission  AKI (ICD-9-CM codes: 584.X) 5,793 (9.26%)  Chronic kidney disease (ICD-9-CM codes: 585.X) 6,656 (10.64%) Other common comorbid conditions  Hypertension 39,783 (63.59%)  Diabetes 19,288 (30.83%)  Heart Failure 14,789 (23.64%)  Anemia 11,749 (18.78%)  Liver Disease 10,222 (16.34%) Median days of hospital stay (IQR) 4 days (3–7) AKI during hospitalization (KDIGO Stage 2) 1,212 (1.94%) Number of nephrotoxic medication per hospital day, mean ± S.D. 1.56 ± 0.89 10 most frequently administered nephrotoxic medications  Aspirin 22,530 (36.01%)  Lisinopril 16,338 (26.12%)  Furosemide 14,094 (22.53%)  Vancomycin 13,568 (21.69%)  Ibuprofen 11,746 (18.78%)  Cefazolin 8,511 (13.60%)  Cefepime 7,745 (12.38%)  Ceftriaxone 6,282 (10.04%)  Piperacillin-tazobactam 6,055 (9.68%)  Ketorolac 6,028 (9.64%) aAKI = acute kidney injury, ICD = International Classification of Diseases, ICU = intensive care unit, IQR = interquartile range, KDIGO = Kidney Disease International Global Organization. View Large Table 2. Characteristics of Study Populationa Characteristics Population = 62,561 Age > 65 years 20,257 (32.38%) Mean age ± S.D. 54.36 ± 18.90 Sex, Male 27,812 (44.46%) Race  White 40,976 (65.50%)  Black 16,912 (27.03%)  Others 4,673 (0.07%) ICU on admission 6,381 (10.20%) Existing renal complications and function presented on admission  AKI (ICD-9-CM codes: 584.X) 5,793 (9.26%)  Chronic kidney disease (ICD-9-CM codes: 585.X) 6,656 (10.64%) Other common comorbid conditions  Hypertension 39,783 (63.59%)  Diabetes 19,288 (30.83%)  Heart Failure 14,789 (23.64%)  Anemia 11,749 (18.78%)  Liver Disease 10,222 (16.34%) Median days of hospital stay (IQR) 4 days (3–7) AKI during hospitalization (KDIGO Stage 2) 1,212 (1.94%) Number of nephrotoxic medication per hospital day, mean ± S.D. 1.56 ± 0.89 10 most frequently administered nephrotoxic medications  Aspirin 22,530 (36.01%)  Lisinopril 16,338 (26.12%)  Furosemide 14,094 (22.53%)  Vancomycin 13,568 (21.69%)  Ibuprofen 11,746 (18.78%)  Cefazolin 8,511 (13.60%)  Cefepime 7,745 (12.38%)  Ceftriaxone 6,282 (10.04%)  Piperacillin-tazobactam 6,055 (9.68%)  Ketorolac 6,028 (9.64%) Characteristics Population = 62,561 Age > 65 years 20,257 (32.38%) Mean age ± S.D. 54.36 ± 18.90 Sex, Male 27,812 (44.46%) Race  White 40,976 (65.50%)  Black 16,912 (27.03%)  Others 4,673 (0.07%) ICU on admission 6,381 (10.20%) Existing renal complications and function presented on admission  AKI (ICD-9-CM codes: 584.X) 5,793 (9.26%)  Chronic kidney disease (ICD-9-CM codes: 585.X) 6,656 (10.64%) Other common comorbid conditions  Hypertension 39,783 (63.59%)  Diabetes 19,288 (30.83%)  Heart Failure 14,789 (23.64%)  Anemia 11,749 (18.78%)  Liver Disease 10,222 (16.34%) Median days of hospital stay (IQR) 4 days (3–7) AKI during hospitalization (KDIGO Stage 2) 1,212 (1.94%) Number of nephrotoxic medication per hospital day, mean ± S.D. 1.56 ± 0.89 10 most frequently administered nephrotoxic medications  Aspirin 22,530 (36.01%)  Lisinopril 16,338 (26.12%)  Furosemide 14,094 (22.53%)  Vancomycin 13,568 (21.69%)  Ibuprofen 11,746 (18.78%)  Cefazolin 8,511 (13.60%)  Cefepime 7,745 (12.38%)  Ceftriaxone 6,282 (10.04%)  Piperacillin-tazobactam 6,055 (9.68%)  Ketorolac 6,028 (9.64%) aAKI = acute kidney injury, ICD = International Classification of Diseases, ICU = intensive care unit, IQR = interquartile range, KDIGO = Kidney Disease International Global Organization. View Large A total of 1,212 unique AKI events occurred with a rate of 2 cases of stage 2 AKI per 100 admissions and 7 cases per 1,000 risk model days. AKI rates increased throughout risk model days (Table 3). Table 3. AKI Rates by Risk Daysa Risk Day (Follow-Up Days) Patients at Risk AKI Events During Follow-Up Rate, % Day 1 (2–6) 45,235 630 1.4 Day 2 (3–7) 47,071 752 1.6 Day 3 (4–8) 35,249 663 1.9 Day 4 (5–9) 25,908 550 2.1 Day 5 (6–10) 20,227 458 2.3 Risk Day (Follow-Up Days) Patients at Risk AKI Events During Follow-Up Rate, % Day 1 (2–6) 45,235 630 1.4 Day 2 (3–7) 47,071 752 1.6 Day 3 (4–8) 35,249 663 1.9 Day 4 (5–9) 25,908 550 2.1 Day 5 (6–10) 20,227 458 2.3 aAKI = acute kidney injury. View Large Table 3. AKI Rates by Risk Daysa Risk Day (Follow-Up Days) Patients at Risk AKI Events During Follow-Up Rate, % Day 1 (2–6) 45,235 630 1.4 Day 2 (3–7) 47,071 752 1.6 Day 3 (4–8) 35,249 663 1.9 Day 4 (5–9) 25,908 550 2.1 Day 5 (6–10) 20,227 458 2.3 Risk Day (Follow-Up Days) Patients at Risk AKI Events During Follow-Up Rate, % Day 1 (2–6) 45,235 630 1.4 Day 2 (3–7) 47,071 752 1.6 Day 3 (4–8) 35,249 663 1.9 Day 4 (5–9) 25,908 550 2.1 Day 5 (6–10) 20,227 458 2.3 aAKI = acute kidney injury. View Large For each risk model day, the reduced backward elimination process produced the best-fitting risk model. A total of 17–22 variables were included in the final risk models (Table 4). Advanced age, high-risk surgery, and rhabdomyolysis were removed in all 5 models during the backward elimination process due to lack of an association with AKI. Albuminemia, liver injury, obesity, stage 1 AKI, systemic inflammatory reaction, and concomitant use of piperacillin/tazobactam and vancomycin were retained in all models with statistically significant odds ratios (OR). Stage 1 AKI was the strongest predictor across all models (ORs: 2.91–3.98), followed by concomitant use of piperacillin/tazobactam and vancomycin (ORs: 1.85–2.46). Effects of surgery tapered off during later hospital days, likely because surgeries were scheduled early during admission and patients recovered from the renal insult during their hospital stay. Table 4. Final Variables and Their ORs by Modela OR (95% CI) by Risk Models Predictors Day 1 Day 2 Day 3 Day 4 Day 5 Acidosis 1.06 (0.77–1.47) 1.61 (1.10–2.15)b 1.27 (0.92–1.76) 1.44 (1.01–2.04)b 1.25 (0.83–1.87) Advanced age …c … … … … Albuminemia 1.67 (1.34–2.09)b 1.35 (1.11–1.63)b 1.26 (1.03–1.54)b 1.43 (1.14–1.80)b 1.52 (1.18–1.96)b Anemia 1.26 (1.05–1.53)b … … … 0.86 (0.70–1.07) Cardiac surgery 1.93 (1.21–3.10)b 2.09 (1.44–3.03)b 1.65 (1.11–2.46)b 1.59 (1.07–2.36)b 1.43 (0.96–2.16) Chronic hypertension 1.25 (1.02–1.54)b 1.12 (0.93–1.35) … … … Chronic kidney disease 0.71 (0.59–0.85)b … … … … Critically ill 1.41 (1.11–1.78)b 1.39 (1.13–1.71)b 1.39 (1.12–1.73)b 1.24 (0.99–1.55) 1.40 (1.10–1.79)b Dehydration 1.13 (0.95–1.35) 1.21 (1.04–1.41)b 1.19 (1.01–1.41)b 1.25 (1.04–1.51)b … Diabetes 1.26 (1.05–1.50)b 1.13 (0.96–1.33) … 1.09 (0.91–1.31) … Sex 1.14 (0.96–1.35) 1.25 (1.08–1.47)b 1.22 (1.03–1.22)b 1.18 (0.98–1.41) … Heart failure 1.54 (1.28–1.84)b 1.44 (1.21–1.71)b 1.16 (0.97–1.40) 1.16 (0.96–1.40) 1.20 (0.98–1.48) High-risk surgery … … … … … Hypotension … 1.13 (0.96–1.32) 1.19 (1.01–1.41)b 1.41 (1.17–1.70)b 1.54 (1.26–1.89)b Increasing trend in SCrs 1.96 (1.32–2.92)b 1.71 (0.92–3.19) 1.56 (1.29–1.89)b … 1.82 (1.48–2.25)b Liver injury 1.44 (1.19–1.73)b 1.44 (1.22–1.71)b 1.24 (1.04–1.49)b 1.24 (1.02–1.51)b 1.24 (1.00–1.52)b Obesity 1.38 (1.14–1.68)b 1.61 (1.29–2.01)b 1.23 (1.01–1.50)b 1.28 (1.03–1.59)b 1.34 (1.06–1.69)b Oliguria 1.27 (1.03–1.56)b 1.13 (0.96–1.33) 1.43 (1.17–1.75)b 1.59 (1.27–1.98)b 1.99 (1.57–2.53)b Prolonged surgery 2.11 (1.05–4.25)b 1.73 (1.05–2.85)b 1.43 (1.17–1.75)b … … Rhabdomyolysis … … … … … Stage 1 AKI 3.75 (2.82–4.98)b 3.60 (2.94–4.40)b 3.98 (3.30–4.81)b 3.33 (2.72–4.08)b 2.91 (2.35–3.61)b Systemic inflammatory response 1.57 (1.30–1.89)b 1.41 (1.19–1.68)b 1.36 (1.13–1.63)b 1.39 (1.13–1.71)b 1.27 (1.01–1.59)b The number of highly nephrotoxic drugs … 1.20 (1.02–1.29)b 1.31 (1.10–1.57)b 1.40 (1.16–1.70)b 1.29 (1.05–1.59)b The number of nephrotoxic drugs 1.15 (1.06–1.25)b 1.19 (1.11–1.29)b 1.19 (1.09–1.29)b 1.06 (0.97–1.16) 1.23 (1.11–1.35)b Use of ACEIs/ARBs + NSAIDs + diuretics 1.29 (0.91–1.81) … … … … Use of contrast 1.16 (0.93–1.45) … 1.16 (0.97–1.39) 1.16 (0.96–1.40) 1.13 (0.92–1.38) Use of TZP+ vancomycin 1.85 (1.43–2.39)b 2.20 (1.76–2.74)b 2.29 (1.83–2.87)b 2.46 (1.94–3.12)b 2.32 (1.78–3.03)b OR (95% CI) by Risk Models Predictors Day 1 Day 2 Day 3 Day 4 Day 5 Acidosis 1.06 (0.77–1.47) 1.61 (1.10–2.15)b 1.27 (0.92–1.76) 1.44 (1.01–2.04)b 1.25 (0.83–1.87) Advanced age …c … … … … Albuminemia 1.67 (1.34–2.09)b 1.35 (1.11–1.63)b 1.26 (1.03–1.54)b 1.43 (1.14–1.80)b 1.52 (1.18–1.96)b Anemia 1.26 (1.05–1.53)b … … … 0.86 (0.70–1.07) Cardiac surgery 1.93 (1.21–3.10)b 2.09 (1.44–3.03)b 1.65 (1.11–2.46)b 1.59 (1.07–2.36)b 1.43 (0.96–2.16) Chronic hypertension 1.25 (1.02–1.54)b 1.12 (0.93–1.35) … … … Chronic kidney disease 0.71 (0.59–0.85)b … … … … Critically ill 1.41 (1.11–1.78)b 1.39 (1.13–1.71)b 1.39 (1.12–1.73)b 1.24 (0.99–1.55) 1.40 (1.10–1.79)b Dehydration 1.13 (0.95–1.35) 1.21 (1.04–1.41)b 1.19 (1.01–1.41)b 1.25 (1.04–1.51)b … Diabetes 1.26 (1.05–1.50)b 1.13 (0.96–1.33) … 1.09 (0.91–1.31) … Sex 1.14 (0.96–1.35) 1.25 (1.08–1.47)b 1.22 (1.03–1.22)b 1.18 (0.98–1.41) … Heart failure 1.54 (1.28–1.84)b 1.44 (1.21–1.71)b 1.16 (0.97–1.40) 1.16 (0.96–1.40) 1.20 (0.98–1.48) High-risk surgery … … … … … Hypotension … 1.13 (0.96–1.32) 1.19 (1.01–1.41)b 1.41 (1.17–1.70)b 1.54 (1.26–1.89)b Increasing trend in SCrs 1.96 (1.32–2.92)b 1.71 (0.92–3.19) 1.56 (1.29–1.89)b … 1.82 (1.48–2.25)b Liver injury 1.44 (1.19–1.73)b 1.44 (1.22–1.71)b 1.24 (1.04–1.49)b 1.24 (1.02–1.51)b 1.24 (1.00–1.52)b Obesity 1.38 (1.14–1.68)b 1.61 (1.29–2.01)b 1.23 (1.01–1.50)b 1.28 (1.03–1.59)b 1.34 (1.06–1.69)b Oliguria 1.27 (1.03–1.56)b 1.13 (0.96–1.33) 1.43 (1.17–1.75)b 1.59 (1.27–1.98)b 1.99 (1.57–2.53)b Prolonged surgery 2.11 (1.05–4.25)b 1.73 (1.05–2.85)b 1.43 (1.17–1.75)b … … Rhabdomyolysis … … … … … Stage 1 AKI 3.75 (2.82–4.98)b 3.60 (2.94–4.40)b 3.98 (3.30–4.81)b 3.33 (2.72–4.08)b 2.91 (2.35–3.61)b Systemic inflammatory response 1.57 (1.30–1.89)b 1.41 (1.19–1.68)b 1.36 (1.13–1.63)b 1.39 (1.13–1.71)b 1.27 (1.01–1.59)b The number of highly nephrotoxic drugs … 1.20 (1.02–1.29)b 1.31 (1.10–1.57)b 1.40 (1.16–1.70)b 1.29 (1.05–1.59)b The number of nephrotoxic drugs 1.15 (1.06–1.25)b 1.19 (1.11–1.29)b 1.19 (1.09–1.29)b 1.06 (0.97–1.16) 1.23 (1.11–1.35)b Use of ACEIs/ARBs + NSAIDs + diuretics 1.29 (0.91–1.81) … … … … Use of contrast 1.16 (0.93–1.45) … 1.16 (0.97–1.39) 1.16 (0.96–1.40) 1.13 (0.92–1.38) Use of TZP+ vancomycin 1.85 (1.43–2.39)b 2.20 (1.76–2.74)b 2.29 (1.83–2.87)b 2.46 (1.94–3.12)b 2.32 (1.78–3.03)b aACEI = angiotensin-converting enzyme inhibitors, AKI = acute kidney injury, ARB = angiotensin-receptor blockers, CI = confidence interval, NSAID = nonsteroidal antiinflammatory drug, OR = odds ratio, SCr = serum creatinine, TZP = piperacillin/tazobactam. bStatistically significant at 0.05 significance level. cNot included in the final models. View Large Table 4. Final Variables and Their ORs by Modela OR (95% CI) by Risk Models Predictors Day 1 Day 2 Day 3 Day 4 Day 5 Acidosis 1.06 (0.77–1.47) 1.61 (1.10–2.15)b 1.27 (0.92–1.76) 1.44 (1.01–2.04)b 1.25 (0.83–1.87) Advanced age …c … … … … Albuminemia 1.67 (1.34–2.09)b 1.35 (1.11–1.63)b 1.26 (1.03–1.54)b 1.43 (1.14–1.80)b 1.52 (1.18–1.96)b Anemia 1.26 (1.05–1.53)b … … … 0.86 (0.70–1.07) Cardiac surgery 1.93 (1.21–3.10)b 2.09 (1.44–3.03)b 1.65 (1.11–2.46)b 1.59 (1.07–2.36)b 1.43 (0.96–2.16) Chronic hypertension 1.25 (1.02–1.54)b 1.12 (0.93–1.35) … … … Chronic kidney disease 0.71 (0.59–0.85)b … … … … Critically ill 1.41 (1.11–1.78)b 1.39 (1.13–1.71)b 1.39 (1.12–1.73)b 1.24 (0.99–1.55) 1.40 (1.10–1.79)b Dehydration 1.13 (0.95–1.35) 1.21 (1.04–1.41)b 1.19 (1.01–1.41)b 1.25 (1.04–1.51)b … Diabetes 1.26 (1.05–1.50)b 1.13 (0.96–1.33) … 1.09 (0.91–1.31) … Sex 1.14 (0.96–1.35) 1.25 (1.08–1.47)b 1.22 (1.03–1.22)b 1.18 (0.98–1.41) … Heart failure 1.54 (1.28–1.84)b 1.44 (1.21–1.71)b 1.16 (0.97–1.40) 1.16 (0.96–1.40) 1.20 (0.98–1.48) High-risk surgery … … … … … Hypotension … 1.13 (0.96–1.32) 1.19 (1.01–1.41)b 1.41 (1.17–1.70)b 1.54 (1.26–1.89)b Increasing trend in SCrs 1.96 (1.32–2.92)b 1.71 (0.92–3.19) 1.56 (1.29–1.89)b … 1.82 (1.48–2.25)b Liver injury 1.44 (1.19–1.73)b 1.44 (1.22–1.71)b 1.24 (1.04–1.49)b 1.24 (1.02–1.51)b 1.24 (1.00–1.52)b Obesity 1.38 (1.14–1.68)b 1.61 (1.29–2.01)b 1.23 (1.01–1.50)b 1.28 (1.03–1.59)b 1.34 (1.06–1.69)b Oliguria 1.27 (1.03–1.56)b 1.13 (0.96–1.33) 1.43 (1.17–1.75)b 1.59 (1.27–1.98)b 1.99 (1.57–2.53)b Prolonged surgery 2.11 (1.05–4.25)b 1.73 (1.05–2.85)b 1.43 (1.17–1.75)b … … Rhabdomyolysis … … … … … Stage 1 AKI 3.75 (2.82–4.98)b 3.60 (2.94–4.40)b 3.98 (3.30–4.81)b 3.33 (2.72–4.08)b 2.91 (2.35–3.61)b Systemic inflammatory response 1.57 (1.30–1.89)b 1.41 (1.19–1.68)b 1.36 (1.13–1.63)b 1.39 (1.13–1.71)b 1.27 (1.01–1.59)b The number of highly nephrotoxic drugs … 1.20 (1.02–1.29)b 1.31 (1.10–1.57)b 1.40 (1.16–1.70)b 1.29 (1.05–1.59)b The number of nephrotoxic drugs 1.15 (1.06–1.25)b 1.19 (1.11–1.29)b 1.19 (1.09–1.29)b 1.06 (0.97–1.16) 1.23 (1.11–1.35)b Use of ACEIs/ARBs + NSAIDs + diuretics 1.29 (0.91–1.81) … … … … Use of contrast 1.16 (0.93–1.45) … 1.16 (0.97–1.39) 1.16 (0.96–1.40) 1.13 (0.92–1.38) Use of TZP+ vancomycin 1.85 (1.43–2.39)b 2.20 (1.76–2.74)b 2.29 (1.83–2.87)b 2.46 (1.94–3.12)b 2.32 (1.78–3.03)b OR (95% CI) by Risk Models Predictors Day 1 Day 2 Day 3 Day 4 Day 5 Acidosis 1.06 (0.77–1.47) 1.61 (1.10–2.15)b 1.27 (0.92–1.76) 1.44 (1.01–2.04)b 1.25 (0.83–1.87) Advanced age …c … … … … Albuminemia 1.67 (1.34–2.09)b 1.35 (1.11–1.63)b 1.26 (1.03–1.54)b 1.43 (1.14–1.80)b 1.52 (1.18–1.96)b Anemia 1.26 (1.05–1.53)b … … … 0.86 (0.70–1.07) Cardiac surgery 1.93 (1.21–3.10)b 2.09 (1.44–3.03)b 1.65 (1.11–2.46)b 1.59 (1.07–2.36)b 1.43 (0.96–2.16) Chronic hypertension 1.25 (1.02–1.54)b 1.12 (0.93–1.35) … … … Chronic kidney disease 0.71 (0.59–0.85)b … … … … Critically ill 1.41 (1.11–1.78)b 1.39 (1.13–1.71)b 1.39 (1.12–1.73)b 1.24 (0.99–1.55) 1.40 (1.10–1.79)b Dehydration 1.13 (0.95–1.35) 1.21 (1.04–1.41)b 1.19 (1.01–1.41)b 1.25 (1.04–1.51)b … Diabetes 1.26 (1.05–1.50)b 1.13 (0.96–1.33) … 1.09 (0.91–1.31) … Sex 1.14 (0.96–1.35) 1.25 (1.08–1.47)b 1.22 (1.03–1.22)b 1.18 (0.98–1.41) … Heart failure 1.54 (1.28–1.84)b 1.44 (1.21–1.71)b 1.16 (0.97–1.40) 1.16 (0.96–1.40) 1.20 (0.98–1.48) High-risk surgery … … … … … Hypotension … 1.13 (0.96–1.32) 1.19 (1.01–1.41)b 1.41 (1.17–1.70)b 1.54 (1.26–1.89)b Increasing trend in SCrs 1.96 (1.32–2.92)b 1.71 (0.92–3.19) 1.56 (1.29–1.89)b … 1.82 (1.48–2.25)b Liver injury 1.44 (1.19–1.73)b 1.44 (1.22–1.71)b 1.24 (1.04–1.49)b 1.24 (1.02–1.51)b 1.24 (1.00–1.52)b Obesity 1.38 (1.14–1.68)b 1.61 (1.29–2.01)b 1.23 (1.01–1.50)b 1.28 (1.03–1.59)b 1.34 (1.06–1.69)b Oliguria 1.27 (1.03–1.56)b 1.13 (0.96–1.33) 1.43 (1.17–1.75)b 1.59 (1.27–1.98)b 1.99 (1.57–2.53)b Prolonged surgery 2.11 (1.05–4.25)b 1.73 (1.05–2.85)b 1.43 (1.17–1.75)b … … Rhabdomyolysis … … … … … Stage 1 AKI 3.75 (2.82–4.98)b 3.60 (2.94–4.40)b 3.98 (3.30–4.81)b 3.33 (2.72–4.08)b 2.91 (2.35–3.61)b Systemic inflammatory response 1.57 (1.30–1.89)b 1.41 (1.19–1.68)b 1.36 (1.13–1.63)b 1.39 (1.13–1.71)b 1.27 (1.01–1.59)b The number of highly nephrotoxic drugs … 1.20 (1.02–1.29)b 1.31 (1.10–1.57)b 1.40 (1.16–1.70)b 1.29 (1.05–1.59)b The number of nephrotoxic drugs 1.15 (1.06–1.25)b 1.19 (1.11–1.29)b 1.19 (1.09–1.29)b 1.06 (0.97–1.16) 1.23 (1.11–1.35)b Use of ACEIs/ARBs + NSAIDs + diuretics 1.29 (0.91–1.81) … … … … Use of contrast 1.16 (0.93–1.45) … 1.16 (0.97–1.39) 1.16 (0.96–1.40) 1.13 (0.92–1.38) Use of TZP+ vancomycin 1.85 (1.43–2.39)b 2.20 (1.76–2.74)b 2.29 (1.83–2.87)b 2.46 (1.94–3.12)b 2.32 (1.78–3.03)b aACEI = angiotensin-converting enzyme inhibitors, AKI = acute kidney injury, ARB = angiotensin-receptor blockers, CI = confidence interval, NSAID = nonsteroidal antiinflammatory drug, OR = odds ratio, SCr = serum creatinine, TZP = piperacillin/tazobactam. bStatistically significant at 0.05 significance level. cNot included in the final models. View Large The number of nephrotoxic medications and number of high-risk nephrotoxic medications were significant predictors in all models except Day 4 and Day 1, respectively. Note that an OR of 1.2 implies that each additional drug increases the odds by 20%. Figures 1-5 show the receiver operating characteristic (ROC) curves with both development and validation C-statistics by daily risk models. Across all models, the C-statistics ranged from 0.79 to 0.81 in the development data set and from 0.78 to 0.81 in the 1,000 bootstrap datasets. Admissions ranked in the upper half of the risk score accounted for more than 85% of all AKI events and patients in the 90th percentile of the risk score accounted for more than 40% of AKI events. High-risk patients were successfully concentrated in the upper percentiles of the risk score: in the 90th percentile, the number of patient at risk per 1 AKI event was 17, 13, 11, 11, and 10 patients in the Day 1–Day 5 models, respectively. Figure 1. View largeDownload slide Ordered risk scores and ROC performance for 45,235 hospital admissions—Day 1 model. AKI = acute kidney injury, AUC = area under the curve, ROC = receiver operating characteristic. Figure 1. View largeDownload slide Ordered risk scores and ROC performance for 45,235 hospital admissions—Day 1 model. AKI = acute kidney injury, AUC = area under the curve, ROC = receiver operating characteristic. Figure 2. View largeDownload slide Ordered risk scores and ROC performance for 47,071 hospital admissions—Day 2 model. AKI = acute kidney injury, AUC = area under the curve, ROC = receiver operating characteristic. Figure 2. View largeDownload slide Ordered risk scores and ROC performance for 47,071 hospital admissions—Day 2 model. AKI = acute kidney injury, AUC = area under the curve, ROC = receiver operating characteristic. Figure 3. View largeDownload slide Ordered risk scores and ROC performance for 35,249 hospital admissions—day 3 model. AKI = acute kidney injury, AUC = area under the curve, ROC = receiver operating characteristic. Figure 3. View largeDownload slide Ordered risk scores and ROC performance for 35,249 hospital admissions—day 3 model. AKI = acute kidney injury, AUC = area under the curve, ROC = receiver operating characteristic. Figure 4. View largeDownload slide Ordered risk scores and ROC performance for 25,908 hospital admissions—Day 4 model. AKI = acute kidney injury, AUC = area under the curve, ROC = receiver operating characteristic. Figure 4. View largeDownload slide Ordered risk scores and ROC performance for 25,908 hospital admissions—Day 4 model. AKI = acute kidney injury, AUC = area under the curve, ROC = receiver operating characteristic. Figure 5. View largeDownload slide Ordered Risk scores and ROC performance for 20,227 hospital admissions—Day 5 model. AKI = acute kidney injury, AUC = area under the curve, ROC = receiver operating characteristic. Figure 5. View largeDownload slide Ordered Risk scores and ROC performance for 20,227 hospital admissions—Day 5 model. AKI = acute kidney injury, AUC = area under the curve, ROC = receiver operating characteristic. Discussion We successfully developed and validated AKI prediction models for hospitalized patients who received nephrotoxic medications, using comprehensive patient information obtained from EHRs. The risk models showed good discrimination in both the development and validation datasets. Our model performance was similar to previous AKI prediction models developed for patients undergoing select procedures. Recent systematic reviews reported the average C-statistics were 0.77 (range: 0.57–0.95) among 35 AKI prediction models in radiographic contrast recipients,36 0.85 (range: 0.79–0.90) in 6 studies of patients undergoing noncardiac surgery,37 and 0.79 (range: 0.72–0.84) in 7 studies with a focus on cardiac surgery populations.38 No model has focused on all hospitalized patients who are receiving nephrotoxic drugs. Because of the good discriminative properties of our models, the upper percentiles of the risk scores showed high rates of AKI, making these patients excellent targets for preventive intervention. For example, among patients in the 90th percentile of the risk score at hospital day 1, we found an obese patient with chronic hypertension, diabetes, and heart failure who was experiencing dehydration, hypotension, and stage 1 AKI on the first day of admission. The patient was on 7 different nephrotoxic medications in addition to receiving contrast-enhanced radiography. The prediction models were able to aggregate these risk factors, ranking this patient at the top risk for stage 2 AKI. By flagging each individual risk factor along with its quantitative contribution to risk, the AKI models can furthermore guide preventive interventions. We developed separate models for the first 5 hospital days given dynamic changes in risk factors and new clinical information as test results become available. As expected, the number and composition of risk factors varied across the 5 models, even though the overall predictive performance of the models remained similar. For example, while the Day 5 model included 17 risk factors, the Day 1 model included 22. Risk factors considered in Day 1 but omitted in the Day 5 model include chronic hypertension, chronic kidney disease, dehydration, the use of ACEIs/ARBs + NSAIDs + Diuretics, diabetes, sex, prolonged surgery, anemia, cardiac surgery, and heart failure. Some of these differences may be explained by changes in the prevalence of risk factors across hospital days. For example, surgeries typically occur early during the hospital stay and may lose their impact on AKI risk later during admission. Likewise, acute conditions such as dehydration and acute heart failure are likely attenuated soon after admission. Stage 1 AKI and concomitant use of tazobactam/piperacillin and vancomycin were the two strongest predictors in all 5 models. Other significant risk factors retained in all models were albuminemia, liver injury, obesity, and systemic inflammatory reaction. Hypotension played no role in the Day 1 model but became a significant risk factor in later days. Each additional nephrotoxic medication increased the AKI risk by about 10%–20% and each additional high-risk nephrotoxic medication increased the AKI risk by 20%–40%. The number of high-risk nephrotoxic medication was excluded in the Day 1 model due to small sample size. Surprisingly, chronic kidney disease was protective in the Day 1 model and lost significance thereafter. This reverse association could be related to increased vigilance, greater avoidance of additional nephrotoxic exposure, and use of additional preventive interventions. The only marginal contribution of contrast use may also be explained by this notion. Our study has several strengths. First, to our knowledge, this is the first risk model predicting drug-associated AKI. In order to capture AKI cases that are at least in part attributable to nephrotoxic medication exposure, we confined each risk model day to admissions with nephrotoxic medication use. It is important to note that the etiology of AKI is still multifactorial, even though we required nephrotoxic drug exposure prior to an event. Second, we considered and tested a broad range of risk factors and used a rigorous selection process to optimize risk model performance (i.e., cluster analysis, univariate analysis, and backward elimination). Unlike previous models that did not address missingness, we used missing indicators to optimize the predictive validity of our models, when certain laboratory values were not available. The risk factors incorporated in the final risk score are readily available in EHR systems and allow automated generation of the risk score. It should be noted that risk factors for preexisting conditions (diabetes, CKD, heart failure, liver injury) used a combination of laboratory values and diagnosis codes from previous encounters as well as the index encounter. Because diagnosis codes are typically assigned after hospital discharge, these definitions would need to be updated by using diagnoses on problem lists of the current encounter. Third, our outcome definition derived from KDIGO stage 2 AKI allowed a focus on relatively severe yet common adverse events in the inpatient settings. Stage 2 AKI is known to be associated with increased in-hospital mortality and need for dialysis. Therefore, our model can improve patient safety by flagging patients at risk for a clinically relevant adverse event.39 Several study limitations should be noted when interpreting the results. In defining AKI, we relied on SCr values only and omitted the KDIGO-specified requirements for decreases in urine output; this was because we found urine output to be unreliably captured across the entire risk pool of patients who received nephrotoxic medications. This limitation may be attenuated by the fact that drug-associated AKI is oftentimes nonoliguric in earlier stages.7 Furthermore, we excluded dialysis patients and patients with previous urinary system surgeries in order to focus the study on acute kidney injury and to exclude other causes of SCr fluctuations. Because we did not have patient data beyond discharge, our outcome assessment was limited to patients who stayed in the hospital until at least the day after the risk model day. We assumed patients were discharged because they were healthy and had minimal risk to develop an AKI immediately after discharge. The observation that rates of AKI increased from 1.4% in the Day 1 model to 2.3% in the Day 5 model may reflect retention of sicker patients during later hospital days, prolonged exposure nephrotoxic medications resulting in cumulative effects, and induction periods in the development of kidney injury. Regretfully, we had to abandon a variable indicating nephrotoxic medication use at high dose although nephrotoxicity is known to be dose-dependent. This was due to a lack of consensus and great complexity in defining high-dose for different drugs, especially for drugs that are dose adjusted based on baseline renal function, age, and weight and/or body surface area. Examination of individual drugs and dose ranges was not feasible due to sample size constraints and should be the focus of future larger studies. Finally, the estimates calculated in our models reflect the variables “as present” in two particular hospitals. In other words, local practice created unique meaning for some of the variables, which depends on specific protocols, formularies, and staffing. Likewise, the volume and accuracy of risk factors measured prior to admission depends on the size and interconnectivity of the health system. As a consequence, the high specificity of our models runs in parallel with the specific processes and data availability in both institutions. Implementation of these models in other institutions will require a calibration of their risk factors definitions and their predictive association with AKI to maintain the accuracy and discriminatory power of our risk score. Conclusion A dynamic prediction model was built successfully for inpatient AKI with excellent discriminative validity and good calibration, allowing clinicians to focus on a select high-risk population that captures the majority of AKI events. Disclosures This study was funded in part by the ASHP Foundation. Dr. Jeon is currently an assistant professor at the University of Utah. This research was conducted and completed while while she was a graduate student at the University of Florida. Acknowledgments The authors would like to thank the following individuals for their service as technical expert panel members on this study: Alison Apple, D.Ph., M.S.; Mary Blegen, Ph.D., R.N.; Dale Bratzler, D.O., M.P.H.; Michael Brownlee, Pharm.D., M.S.; Kyle Campbell, PharmD., M.S.; Mikael Cohen, B.S.Pharm., M.S., Sc.D.; Fran Cunningham, Pharm.D.; Debby Cowan, Pharm.D.; Bob Feroli, Pharm.D.; Allen Flynn, Pharm.D.; Tejal Gandhi, M.D., M.P.H.; Bruce Gordon, Pharm.D.; Shekhar Mehta, Pharm.D., M.S.; Richard Montgomery, B.S.Pharm., M.B.A.; Steve Pickette, B.S.Pharm.; Rebecca Prevost, Pharm.D.; Paul Szumita, Pharm.D.; Billy Woodward, B.S.Pharm.; William Zellmer, B.S.Pharm., M.P.H.; and Lisa Zumberg, Pharm.D. References 1. Susantitaphong P , Cruz DN , Cerda J et al. ; Acute Kidney Injury Advisory Group of the American Society of Nephrology . World incidence of AKI: a meta-analysis . Clin J Am Soc Nephrol. 2013 ; 8 : 1482 - 93 . Google Scholar Crossref Search ADS PubMed 2. Bentley ML , Corwin HL , Dasta J . Drug-induced acute kidney injury in the critically ill adult: recognition and prevention strategies . Crit Care Med. 2010 ; 38 ( 6 suppl ): S169 - 74 . Google Scholar Crossref Search ADS PubMed 3. Coca SG , Garg AX , Swaminathan M et al. ; TRIBE-AKI Consortium . Preoperative angiotensin-converting enzyme inhibitors and angiotensin receptor blocker use and acute kidney injury in patients undergoing cardiac surgery . Nephrol Dial Transplant. 2013 ; 28 : 2787 - 99 . Google Scholar Crossref Search ADS PubMed 4. Chertow GM , Burdick E , Honour M et al. Acute kidney injury, mortality, length of stay, and costs in hospitalized patients . J Am Soc Nephrol. 2005 ; 16 : 3365 - 70 . Google Scholar Crossref Search ADS PubMed 5. Ruiz-Criado J , Ramos-Barron MA , Fernandez-Fresnedo G et al. Long-term mortality among hospitalized non-ICU patients with acute kidney injury referred to nephrology . Nephron. 2015 ; 131 : 23 - 33 . Google Scholar Crossref Search ADS PubMed 6. Wang HE , Muntner P , Chertow GM , Warnock DG . Acute kidney injury and mortality in hospitalized patients . Am J Nephrol. 2012 ; 35 : 349 - 55 . Google Scholar Crossref Search ADS PubMed 7. Kane-Gill SL , Goldstein SL . Drug-induced acute kidney injury: a focus on risk assessment for prevention . Crit Care Clin. 2015 ; 31 : 675 - 84 . Google Scholar Crossref Search ADS PubMed 8. Blix HS , Viktil KK , Moger TA , Reikvam A . Use of renal risk drugs in hospitalized patients with impaired renal function-an underestimated problem ? Nephrol Dial Transplant. 2006 ; 21 : 3164 - 71 . Google Scholar Crossref Search ADS PubMed 9. Joyce EL , Kane-Gill SL , Fuhrman DY , Kellum JA . Drug-associated acute kidney injury: who’s at risk ? Pediatr Nephrol. 2017 ; 32 : 59 - 69 . Google Scholar Crossref Search ADS PubMed 10. Birnie K , Verheyden V , Pagano D et al. ; UK AKI in Cardiac Surgery Collaborators . Predictive models for kidney disease: improving global outcomes (KDIGO) defined acute kidney injury in UK cardiac surgery . Crit Care. 2014 ; 18 : 606 . Google Scholar Crossref Search ADS PubMed 11. Hoste EA , Lameire NH , Vanholder RC et al. Acute renal failure in patients with sepsis in a surgical ICU: predictive factors, incidence, comorbidity, and outcome . J Am Soc Nephrol. 2003 ; 14 : 1022 – 30 . Google Scholar Crossref Search ADS PubMed 12. Englberger L , Suri RM , Li Z et al. Validation of clinical scores predicting severe acute kidney injury after cardiac surgery . Am J Kidney Dis. 2010 ; 56 : 623 - 31 . Google Scholar Crossref Search ADS PubMed 13. Kristovic D , Horvatic I , Husedzinovic I et al. Cardiac surgery-associated acute kidney injury: risk factors analysis and comparison of prediction models . Interact Cardiovasc Thorac Surg. 2015 ; 21 : 366 - 73 . Google Scholar Crossref Search ADS PubMed 14. Kashani K , Steuernagle JH 4th , Akhoundi A et al. Vascular surgery kidney injury predictive score: A historical cohort study . J Cardiothorac Vasc Anesth. 2015 ; 29 : 1588 - 95 . Google Scholar Crossref Search ADS PubMed 15. Jorge-Monjas P , Bustamante-Munguira J , Lorenzo M et al. Predicting cardiac surgery-associated acute kidney injury: the CRATE score . J Crit Care. 2016 ; 31 : 130 - 8 . Google Scholar Crossref Search ADS PubMed 16. Tziakas D , Chalikias G , Stakos D et al. Development of an easily applicable risk score model for contrast-induced nephropathy prediction after percutaneous coronary intervention: a novel approach tailored to current practice . Int J Cardiol. 2013 ; 163 : 46 - 55 . Google Scholar Crossref Search ADS PubMed 17. McMahon GM , Zeng X , Waikar SS . A risk prediction score for kidney failure or mortality in rhabdomyolysis . JAMA Intern Med. 2013 ; 173 : 1821 - 8 . Google Scholar Crossref Search ADS PubMed 18. Roberts G , Phillips D , McCarthy R et al. Acute kidney injury risk assessment at the hospital front door: what is the best measure of risk ? Clin Kidney J. 2015 ; 8 : 673 - 80 . Google Scholar Crossref Search ADS PubMed 19. Khwaja A . KDIGO clinical practice guidelines for acute kidney injury . Nephron Clin Pract. 2012 ; 120 : c179 - 84 . Google Scholar PubMed 20. Falconnier AD , Haefeli WE , Schoenenberger RA et al. Drug dosage in patients with renal failure optimized by immediate concurrent feedback . J Gen Intern Med. 2001 ; 16 : 369 - 75 . Google Scholar Crossref Search ADS PubMed 21. Goldstein SL , Kirkendall E , Nguyen H et al. Electronic health record identification of nephrotoxin exposure and associated acute kidney injury . Pediatrics. 2013 ; 132 : e756 - 67 . Google Scholar Crossref Search ADS PubMed 22. Cronin RM , VanHouten JP , Siew ED et al. National Veterans Health Administration inpatient risk stratification models for hospital-acquired acute kidney injury . J Am Med Informatics Assoc. 2015 ; 22 : 1054 - 71 . Google Scholar Crossref Search ADS 23. Matheny ME , Miller RA , Ikizler TA et al. Development of inpatient risk stratification models of acute kidney injury for use in electronic health records . Med Decis Making. 2010 ; 30 : 639 - 50 . Google Scholar Crossref Search ADS PubMed 24. Lapi F , Azoulay L , Yin H et al. Concurrent use of diuretics, angiotensin converting enzyme inhibitors, and angiotensin receptor blockers with non-steroidal anti-inflammatory drugs and risk of acute kidney injury: nested case-control study . BMJ. 2013 ; 346 : e8525 . Google Scholar Crossref Search ADS PubMed 25. Jeon N , Staley B , Klinker KP et al. Acute kidney injury risk associated with piperacillin/tazobactam compared with cefepime during vancomycin therapy in hospitalised patients: a cohort study stratified by baseline kidney function . Int J Antimicrob Agents. 2017 ; 50 : 63 - 7 . Google Scholar Crossref Search ADS PubMed 26. Boyer A , Gruson D , Bouchet S et al. Aminoglycosides in septic shock: an overview, with specific consideration given to their nephrotoxic risk . Drug Saf. 2013 ; 36 : 217 - 30 . Google Scholar Crossref Search ADS PubMed 27. Rocco M , Montini L , Alessandri E et al. Risk factors for acute kidney injury in critically ill patients receiving high intravenous doses of colistin methanesulfonate and/or other nephrotoxic antibiotics: a retrospective cohort study . Crit Care. 2013 ; 17 : R174 . Google Scholar Crossref Search ADS PubMed 28. Laville M , Juillard L . Contrast-induced acute kidney injury: how should at-risk patients be identified and managed ? J Nephrol. 2010 ; 23 : 387 - 98 . Google Scholar PubMed 29. Naesens M , Kuypers DR , Sarwal M . Calcineurin inhibitor nephrotoxicity . Clin J Am Soc Nephrol. 2009 ; 4 : 481 - 508 . Google Scholar Crossref Search ADS PubMed 30. Lagrange JL , Médecin B , Etienne MC et al. Cisplatin nephrotoxicity: a multivariate analysis of potential predisposing factors . Pharmacotherapy. 1997 ; 17 : 1246 - 53 . Google Scholar PubMed 31. Pannu N , Nadim MK . An overview of drug-induced acute kidney injury . Crit Care Med. 2008 ; 36 ( suppl 4 ). DOI: https://doi.org/10.1097/CCM.0b013e318168e375 . 32. Huerta C , Castellsague J , Varas-Lorenzo C , García Rodríguez LA . Nonsteroidal anti-inflammatory drugs and risk of ARF in the general population . Am J Kidney Dis. 2005 ; 45 : 531 - 9 . Google Scholar Crossref Search ADS PubMed 33. Mendes AE , Lombardi NF , Andrzejevski VS et al. Medication reconciliation at patient admission: a randomized controlled trial . Pharm Pract (Granada). 2016 ; 14 . DOI: https://doi.org/10.18549/PharmPract.2016.01.656 . 34. Donders ART , van der Heijden GJMG , Stijnen T , Moons KGM . Review: a gentle introduction to imputation of missing values . J Clin Epidemiol. 2006 ; 59 : 1087 - 91 . Google Scholar Crossref Search ADS PubMed 35. Steyerberg EW , Vickers AJ , Cook NR et al. Assessing the performance of prediction models: a framework for traditional and novel measures . Epidemiology. 2010 ; 21 : 128 - 38 . Google Scholar Crossref Search ADS PubMed 36. Allen DW , Ma B , Leung KC et al. Risk prediction models for contrast-induced acute kidney injury accompanying cardiac catheterization: systematic review and meta-analysis . Can J Cardiol. 2017 ; 33 : 724 - 36 . Google Scholar Crossref Search ADS PubMed 37. Wilson T , Quan S , Cheema K et al. Risk prediction models for acute kidney injury following major noncardiac surgery: systematic review . Nephrol Dial Transplant. 2015 ; 31 : 231 - 240 . Google Scholar PubMed 38. Huen SC , Parikh CR . Predicting acute kidney injury after cardiac surgery: a systematic review . Ann Thorac Surg. 2012 ; 93 : 337 - 47 . Google Scholar Crossref Search ADS PubMed 39. Pozzoli S , Simonini M , Manunta P . Predicting acute kidney injury: current status and future challenges . J Nephrol. 2018 ; 31 : 209 - 23 . Google Scholar Crossref Search ADS PubMed Appendix A—List of nephrotoxic medications Acyclovir Albumin Allopurinol Amikacina Amphotericin B cholesteryl sulfate complex Amphotericin B deoxylatea Amphotericin B lipid complex Amphotericin B liposomal Aspirin Azacitidine Azilsartan Benazepril Bevacizumab Bumetanide Candesartan Captopril Carboplatin Cefaclor Cefadroxil Cefazolin Cefdinir Cefditoren Cefepime Cefixime Cefotaxime Cefpodoxime Cefprozil Ceftaroline Ceftazidime Ceftibuten Ceftriaxone Cefuroxime Celecoxib Cephalexine Cidofovira Cisplatina Clindamycin Colistina Cyclosporine Dapsone Diclofenac Diflunisal Enalapril Eprosartan Ethacrynic acid Etodolac Fenoprofen Fluoxetine Flurbiprofen Foscarneta Fosinopril Furosemide Ganciclovir Gemcitabine Gentamicina Hetastarch Ibuprofen Ifosfamide Imipenem+cilastatin Indinavir Indomethacin Irbersartan Kanamycina Ketoprofen Ketorolac Lisinopril Lithium Losartan Mannitol Meclofenamate Mefenamic acid Meloxicam Mesalamine Methotrexatea Mitomycin Ca Moexipril Nabumetone Nafcillin Naproxen Neomycina Olmesartan Oxaprozin Paromomycin Perindopril Piperacillin Piroxicam Polymyxin B sulfatea Quinapril Ramipril Rifampin Ritonavir Salsalate Sirolimus Streptomycina Streptozocina Sulfadiazine Sulfasalazine Sulindac Tacrolimusa Telmisartan Tenofovira Tobramycina Tolmetin Torsemide Trandolapril Trimethoprim-sulfamethoxazole Valacyclovir Valganciclovir Valsartan Vancomycina Zonisamide Acyclovir Albumin Allopurinol Amikacina Amphotericin B cholesteryl sulfate complex Amphotericin B deoxylatea Amphotericin B lipid complex Amphotericin B liposomal Aspirin Azacitidine Azilsartan Benazepril Bevacizumab Bumetanide Candesartan Captopril Carboplatin Cefaclor Cefadroxil Cefazolin Cefdinir Cefditoren Cefepime Cefixime Cefotaxime Cefpodoxime Cefprozil Ceftaroline Ceftazidime Ceftibuten Ceftriaxone Cefuroxime Celecoxib Cephalexine Cidofovira Cisplatina Clindamycin Colistina Cyclosporine Dapsone Diclofenac Diflunisal Enalapril Eprosartan Ethacrynic acid Etodolac Fenoprofen Fluoxetine Flurbiprofen Foscarneta Fosinopril Furosemide Ganciclovir Gemcitabine Gentamicina Hetastarch Ibuprofen Ifosfamide Imipenem+cilastatin Indinavir Indomethacin Irbersartan Kanamycina Ketoprofen Ketorolac Lisinopril Lithium Losartan Mannitol Meclofenamate Mefenamic acid Meloxicam Mesalamine Methotrexatea Mitomycin Ca Moexipril Nabumetone Nafcillin Naproxen Neomycina Olmesartan Oxaprozin Paromomycin Perindopril Piperacillin Piroxicam Polymyxin B sulfatea Quinapril Ramipril Rifampin Ritonavir Salsalate Sirolimus Streptomycina Streptozocina Sulfadiazine Sulfasalazine Sulindac Tacrolimusa Telmisartan Tenofovira Tobramycina Tolmetin Torsemide Trandolapril Trimethoprim-sulfamethoxazole Valacyclovir Valganciclovir Valsartan Vancomycina Zonisamide aHigh-risk nephrotoxic medication. View Large Acyclovir Albumin Allopurinol Amikacina Amphotericin B cholesteryl sulfate complex Amphotericin B deoxylatea Amphotericin B lipid complex Amphotericin B liposomal Aspirin Azacitidine Azilsartan Benazepril Bevacizumab Bumetanide Candesartan Captopril Carboplatin Cefaclor Cefadroxil Cefazolin Cefdinir Cefditoren Cefepime Cefixime Cefotaxime Cefpodoxime Cefprozil Ceftaroline Ceftazidime Ceftibuten Ceftriaxone Cefuroxime Celecoxib Cephalexine Cidofovira Cisplatina Clindamycin Colistina Cyclosporine Dapsone Diclofenac Diflunisal Enalapril Eprosartan Ethacrynic acid Etodolac Fenoprofen Fluoxetine Flurbiprofen Foscarneta Fosinopril Furosemide Ganciclovir Gemcitabine Gentamicina Hetastarch Ibuprofen Ifosfamide Imipenem+cilastatin Indinavir Indomethacin Irbersartan Kanamycina Ketoprofen Ketorolac Lisinopril Lithium Losartan Mannitol Meclofenamate Mefenamic acid Meloxicam Mesalamine Methotrexatea Mitomycin Ca Moexipril Nabumetone Nafcillin Naproxen Neomycina Olmesartan Oxaprozin Paromomycin Perindopril Piperacillin Piroxicam Polymyxin B sulfatea Quinapril Ramipril Rifampin Ritonavir Salsalate Sirolimus Streptomycina Streptozocina Sulfadiazine Sulfasalazine Sulindac Tacrolimusa Telmisartan Tenofovira Tobramycina Tolmetin Torsemide Trandolapril Trimethoprim-sulfamethoxazole Valacyclovir Valganciclovir Valsartan Vancomycina Zonisamide Acyclovir Albumin Allopurinol Amikacina Amphotericin B cholesteryl sulfate complex Amphotericin B deoxylatea Amphotericin B lipid complex Amphotericin B liposomal Aspirin Azacitidine Azilsartan Benazepril Bevacizumab Bumetanide Candesartan Captopril Carboplatin Cefaclor Cefadroxil Cefazolin Cefdinir Cefditoren Cefepime Cefixime Cefotaxime Cefpodoxime Cefprozil Ceftaroline Ceftazidime Ceftibuten Ceftriaxone Cefuroxime Celecoxib Cephalexine Cidofovira Cisplatina Clindamycin Colistina Cyclosporine Dapsone Diclofenac Diflunisal Enalapril Eprosartan Ethacrynic acid Etodolac Fenoprofen Fluoxetine Flurbiprofen Foscarneta Fosinopril Furosemide Ganciclovir Gemcitabine Gentamicina Hetastarch Ibuprofen Ifosfamide Imipenem+cilastatin Indinavir Indomethacin Irbersartan Kanamycina Ketoprofen Ketorolac Lisinopril Lithium Losartan Mannitol Meclofenamate Mefenamic acid Meloxicam Mesalamine Methotrexatea Mitomycin Ca Moexipril Nabumetone Nafcillin Naproxen Neomycina Olmesartan Oxaprozin Paromomycin Perindopril Piperacillin Piroxicam Polymyxin B sulfatea Quinapril Ramipril Rifampin Ritonavir Salsalate Sirolimus Streptomycina Streptozocina Sulfadiazine Sulfasalazine Sulindac Tacrolimusa Telmisartan Tenofovira Tobramycina Tolmetin Torsemide Trandolapril Trimethoprim-sulfamethoxazole Valacyclovir Valganciclovir Valsartan Vancomycina Zonisamide aHigh-risk nephrotoxic medication. View Large © American Society of Health-System Pharmacists 2019. All rights reserved. For permissions, please e-mail: [email protected]. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)