Evidence-based practice is the gold standard and should be adhered to at all times—or should it?

Evidence-based practice is the gold standard and should be adhered to at all times—or should it? That health care interventions should be based on sound evidence has become accepted dogma. The evidence is generally graded according to its strength, with the meta-analyses of randomized controlled trials being the gold standard. For many other interventions, such as education, randomized controlled trials are difficult to perform and often little evidence exists. This does not preclude them from being recommended in most guidelines. We would contend that even in this area there are influences that have major effects on what evidence is available, and that these influences are not in line with best patient care. The drug industry performs phase 3 studies to confirm the safety and efficacy of a drug that has survived earlier studies and, importantly, seems to have a commercial future. To make a financial return, a licence that allows the drug to be promoted and used in the most appropriate patients is required. For the drug industry, it will allow patients to to have the most efficient use of money per patient. For the doctors, it will allow treatment for the currently least satisfactorily treated patients. These are usually different populations of patients. However, if there is no thought of the financial return, then there will be no drug, regardless of how important the therapeutic effect. It costs upwards of $2.5 billion [1] to bring a drug to market, and the return needs to pay for all of this development plus the cost of all the drugs that did not make it, and to give a return to investors to make it attractive for them to fund the next drugs. The drug industry cannot afford to do expensive phase 3 studies without having a very good idea of what the results will be. It would be naïve to think that they will not also have an eye to studies that will help with marketing later. Phase 3 studies are therefore designed around predictable outcomes, and study populations are selected by strict inclusion and exclusion criteria to give the best results for the drug. It is for this reason that only 15–30% of patients seen in clinic with a particular diagnosis will fit into any study for that disease [2]. Phase 3 studies are often designed in conjunction with regulators who take a view on what they would like to see in terms of evidence. The regulators are, however, made up of committees and panels of individuals who can hold different and even opposing views. There is also turnover in their ranks, and views may change between those held at the design stage of the study and those held when the data becomes available, maybe 5 years later. For example, Biosimilar infliximab was trialled in RA and AS rather than in Crohn’s disease, which is its major market. Apart from these influences on study design, there is also the effect of publication bias, with the positive studies being more likely to make it to press. It is estimated that only half of the total number of studies done are published [3]. In addition, the licensing authorities can make different decisions, despite looking at substantially the same evidence. If evidence is king, then how could the European Medicines Evaluation Agency (EMEA) turn down Tofacitinib for a licence when the U.S. Food & Drug Administration (FDA) and a variety of other countries approved it? Did evidence-based medicine die when the EMEA approved Baricitinib (the next JAK inhibitor) but the FDA asked for more data? The licence granted to the drug will be based on the studies that have been done. If a drug such as Rituximab has only been trialled with MTX, then the licence will be for use with MTX. If it has only been used after TNF inhibitors (TNFis), then the licence will say that it has to be used only after TNFis. This would occur regardless of how it would have worked had it been tried in mono therapy or as the first biologic. Clinical experience suggests that in this case it would have worked fine. Phase 4 post-marketing studies are done mainly as an aid to marketing, but under the guise of being real studies: tidying up a side-effect signal, taking on a competitor, increasing the prescriber base, etc. Here it is only the ethics committee that needs to be satisfied—and the marketing department. New indications may be explored, but beware, you may find something you were not looking for. A good example of this was the study of the use of coxibs in the prevention of colonic cancer in people with polyps [4]. Extremely large studies against a placebo showed a cardiovascular signal that probably applies to all NSAIDs. Coxibs were tarred with an association that they never really threw off, while drugs that had not been so extensively studied remained under the evidence-based radar. Lack of evidence is not evidence of lack. The next hurdle to widespread use of a drug is the ability of payers to assess the cost effectiveness of the drug in a particular health service. This appraisal is entirely dependent on the published evidence base, which is compromised for all of the above reasons. The payers are unable to go against the licensed indications and are therefore sometimes unable to make sensible extrapolations of data to slightly different use, or for different indications. An example would be the use of Anakinra, licensed for use in RA but more useful in gout and auto-inflammatory conditions [5]. Another problem area for payers is the use of drugs outside their licensed dose range. The trialled doses of drugs are evolved from dose-ranging studies, usually at phase 2. From the results of these studies, the company will make a decision on the doses to be used in phase 3 studies, and the successful ones become the licensed doses. The company will choose doses to maximise effect while keeping to an acceptable toxicity. This will be data averaged over many patients, and within that group there will be individuals who will respond to much lower doses, and also some who will need more. For biologics, the potential financial savings are sufficient for making tapering attractive, and this may also minimise side effects. A variety of studies have shown successful tapering of doses of biologic drugs for proportions of patients that could result in substantial savings [6, 7], but recommending this action is beyond the license and therefore beyond recommendation by the payers. The positioning of the licence can also be affected by the prospect of a cost/effectiveness assessment. New drugs such as Baricitinib have shown excellent efficacy in MTX- naïve RA patients [8], but have no prospect of being cost/effective when compared with MTX, so they are not licenced for use here, even though that is the obvious situation in which to use them and where they will be used when off patent. One interesting area where extrapolation of data is actively encouraged is with the biosimilars. These drugs should be cheaper, not because they are easier to make, but because they do not have to do the extensive trialling of originator drugs. Currently, once they have shown adequate structural similarity, they have to do two studies in disease areas against the originator and are then allowed to inherit all of the indications and presumably all of the originator data. It remains to be seen how prescribing doctors will respond to this freedom. Once licensed and approved for use, then all we need is someone to pay for it. The intervention of purchasers in our prescribing is probably the least welcome of recent developments. This of course varies from purchaser to purchaser. There are stories of purchasers demanding strict adherence to the minutia of the guidance and licence or they will not pay; however, in other areas more sensible pathways are negotiated, allowing some wriggle room for real life. The order in which drugs are used is a legitimate subject for guidelines and purchasers. However, in some areas, we are pushed to use Rituximab as the second biologic because it is perceived to be cheaper. This is not usually a major problem, but when you have a biologic responder who has lost effect due to anti-drug antibodies, it would be nice to try another. Some guidance is probably encouraging use in non-optimal patients. How much more effective would TNFis be in early disease where TNF is probably more important! Once licensed, approved and paid for, we get to use these drugs in real people, for real reasons. That is when we get to find out how representative the evidence base is. This is also where we find that we cannot quite follow the guidance. The patient is intolerant of MTX, but we want to give the Rituximab. Will LEF do? Can it be used alone? Well probably yes. Another can of worms is that we know what we have prescribed, but we do not really have a good handle on what the patient is taking. Adherence to drugs as prescribed is only rated by the WHO at 60%. For MTX, where tolerability is a huge problem [9], how much is actually swallowed? Does that invalidate the guidance? Will the purchaser pay? In conclusion: while it would be nice to have evidence for every intervention we make, it is impractical to have a study that accurately reflects every individual in front of us. Evidence-based practice is always going to be seriously limited, and it is the degree of extrapolation that is allowed that is crucial to us making sensible clinical decisions. We should, however, be reassured that each and every intervention we make potentially becomes an n = 1 study at follow-up—not valued as anecdotal by the evidence grade, but part of our everyday practice and a major driver of our learning. Perhaps any guidelines committees should have a role in validating the sort of deviations we make in clinical practice every day, and not be solely dependent on the so-called evidence base. Possible solutions to this include an increase in emphasis placed on real-world observational data, because this would provide practically both useful evidence for daily practice and guidance for decision-making. Such data could provide information for decision-makers about what the real effect of the drugs or interventions would be. Another solution would be to set up more pragmatic trials with patients who could be comparable with those routinely seen in rheumatology clinics. Such approaches are being supported by organisations like the National Institute for Health Research in the UK, The Netherlands Organisation for Health Research and Development in The Netherlands and Belgian healthcare knowledge centre in Belgium. The problem will still lie in not being able to model all therapeutic decisions in clinical practice, and that is why rheumatologists can never be replaced by computed algorithms. Funding: No specific funding was received from any bodies in the public, commercial or not-for-profit sectors to carry out the work described in this manuscript. Disclosure statement: M.B. has been sponsored to attend regional, national and international meetings by UCB celltech, Roche/Chugai, Pfizer, AbbVie, Merck, Mennarini, Janssen, Bristol-Myers Squib, Novartis and Eli-Lilly. He has received honoraria for speaking and attended advisory boards with Bristol-Myers Squib, UCB celltech, Roche/Chugai, Pfizer, AbbVie, Merck, Mennarini, Sanofi-aventis, Eli-Lilly, Janssen and Novartis. D.W. has performed commercial research for Pfizer, Gilead, Lilly and Janssen and has done speaker meetings and advisory boards for Celgene, Pfizer, AbbVie, Lilly and Janssen. References 1 Mullin, R. Cost to Develop New Pharmaceutical Drug Now Exceeds $2.5B. https://www.scientificamerican.com/article/cost-to-develop-new-pharmaceutical-drug-now-exceeds-2-5b/ (19 September 2017, date last accessed). 2 Kennedy-Martin T, Curtis S, Faries D, Robinson S, Johnston J. A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials  2015; 16: 495. Google Scholar CrossRef Search ADS PubMed  3 Hopewell S, Boutron I, Altman DG, Ravaud P. Deficiencies in the publication and reporting of the results of systematic reviews presented at scientific medical conferences. J Clin Epidemiol  2015; 68: 1488– 95. http://dx.doi.org/10.1016/j.jclinepi.2015.03.006 Google Scholar CrossRef Search ADS PubMed  4 Baron JA, Sandler RS, Bresalier RS et al.   Cardiovascular events associated with rofecoxib: final analysis of the APPROVe trial. Lancet  2008; 372: 1756– 64. http://dx.doi.org/10.1016/S0140-6736(08)61490-7 Google Scholar CrossRef Search ADS PubMed  5 Petite SE. Effectiveness of anakinra in acute gout: a retrospective review of initial and refractory therapy. Am J Ther  2017; 24: e633– 4. Google Scholar CrossRef Search ADS PubMed  6 Lau CS, Gibofsky A, Damjanov N et al.   Down-titration of biologics for the treatment of rheumatoid arthritis: a systematic literature review. Rheumatol Int  2017; 37: 1789– 98. http://dx.doi.org/10.1007/s00296-017-3780-8 Google Scholar CrossRef Search ADS PubMed  7 Vanier A, Mariette X, Tubach F, Fautrel B; STRASS Study Group. Cost-effectiveness of TNF-blocker injection spacing for patients with established rheumatoid arthritis in remission: an economic evaluation from the spacing of TNF-blocker injections in rheumatoid arthritis trial. Value Health  2017; 20: 577– 85. http://dx.doi.org/10.1016/j.jval.2017.01.005 Google Scholar CrossRef Search ADS PubMed  8 Kuriya B, Cohen MD, Keystone E. Baricitinib in rheumatoid arthritis: evidence-to-date and clinical potential. Ther Adv Musculoskelet Dis  2017; 9: 37– 44. http://dx.doi.org/10.1177/1759720X16687481 Google Scholar CrossRef Search ADS PubMed  9 Robinson S, Gibson S, George E et al.   Tolerability and adherence problems in patients on a stable dose of methotrexate: results of a multicentre survey. Musculoskeletal Care  2015; 14: 152– 5. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Rheumatology. All rights reserved. For Permissions, please email: journals.permissions@oup.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Rheumatology Oxford University Press

Evidence-based practice is the gold standard and should be adhered to at all times—or should it?

Rheumatology , Volume Advance Article – Feb 23, 2018

Loading next page...
 
/lp/ou_press/evidence-based-practice-is-the-gold-standard-and-should-be-adhered-to-lQpOhQOypF
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Rheumatology. All rights reserved. For Permissions, please email: journals.permissions@oup.com
ISSN
1462-0324
eISSN
1462-0332
D.O.I.
10.1093/rheumatology/kex509
Publisher site
See Article on Publisher Site

Abstract

That health care interventions should be based on sound evidence has become accepted dogma. The evidence is generally graded according to its strength, with the meta-analyses of randomized controlled trials being the gold standard. For many other interventions, such as education, randomized controlled trials are difficult to perform and often little evidence exists. This does not preclude them from being recommended in most guidelines. We would contend that even in this area there are influences that have major effects on what evidence is available, and that these influences are not in line with best patient care. The drug industry performs phase 3 studies to confirm the safety and efficacy of a drug that has survived earlier studies and, importantly, seems to have a commercial future. To make a financial return, a licence that allows the drug to be promoted and used in the most appropriate patients is required. For the drug industry, it will allow patients to to have the most efficient use of money per patient. For the doctors, it will allow treatment for the currently least satisfactorily treated patients. These are usually different populations of patients. However, if there is no thought of the financial return, then there will be no drug, regardless of how important the therapeutic effect. It costs upwards of $2.5 billion [1] to bring a drug to market, and the return needs to pay for all of this development plus the cost of all the drugs that did not make it, and to give a return to investors to make it attractive for them to fund the next drugs. The drug industry cannot afford to do expensive phase 3 studies without having a very good idea of what the results will be. It would be naïve to think that they will not also have an eye to studies that will help with marketing later. Phase 3 studies are therefore designed around predictable outcomes, and study populations are selected by strict inclusion and exclusion criteria to give the best results for the drug. It is for this reason that only 15–30% of patients seen in clinic with a particular diagnosis will fit into any study for that disease [2]. Phase 3 studies are often designed in conjunction with regulators who take a view on what they would like to see in terms of evidence. The regulators are, however, made up of committees and panels of individuals who can hold different and even opposing views. There is also turnover in their ranks, and views may change between those held at the design stage of the study and those held when the data becomes available, maybe 5 years later. For example, Biosimilar infliximab was trialled in RA and AS rather than in Crohn’s disease, which is its major market. Apart from these influences on study design, there is also the effect of publication bias, with the positive studies being more likely to make it to press. It is estimated that only half of the total number of studies done are published [3]. In addition, the licensing authorities can make different decisions, despite looking at substantially the same evidence. If evidence is king, then how could the European Medicines Evaluation Agency (EMEA) turn down Tofacitinib for a licence when the U.S. Food & Drug Administration (FDA) and a variety of other countries approved it? Did evidence-based medicine die when the EMEA approved Baricitinib (the next JAK inhibitor) but the FDA asked for more data? The licence granted to the drug will be based on the studies that have been done. If a drug such as Rituximab has only been trialled with MTX, then the licence will be for use with MTX. If it has only been used after TNF inhibitors (TNFis), then the licence will say that it has to be used only after TNFis. This would occur regardless of how it would have worked had it been tried in mono therapy or as the first biologic. Clinical experience suggests that in this case it would have worked fine. Phase 4 post-marketing studies are done mainly as an aid to marketing, but under the guise of being real studies: tidying up a side-effect signal, taking on a competitor, increasing the prescriber base, etc. Here it is only the ethics committee that needs to be satisfied—and the marketing department. New indications may be explored, but beware, you may find something you were not looking for. A good example of this was the study of the use of coxibs in the prevention of colonic cancer in people with polyps [4]. Extremely large studies against a placebo showed a cardiovascular signal that probably applies to all NSAIDs. Coxibs were tarred with an association that they never really threw off, while drugs that had not been so extensively studied remained under the evidence-based radar. Lack of evidence is not evidence of lack. The next hurdle to widespread use of a drug is the ability of payers to assess the cost effectiveness of the drug in a particular health service. This appraisal is entirely dependent on the published evidence base, which is compromised for all of the above reasons. The payers are unable to go against the licensed indications and are therefore sometimes unable to make sensible extrapolations of data to slightly different use, or for different indications. An example would be the use of Anakinra, licensed for use in RA but more useful in gout and auto-inflammatory conditions [5]. Another problem area for payers is the use of drugs outside their licensed dose range. The trialled doses of drugs are evolved from dose-ranging studies, usually at phase 2. From the results of these studies, the company will make a decision on the doses to be used in phase 3 studies, and the successful ones become the licensed doses. The company will choose doses to maximise effect while keeping to an acceptable toxicity. This will be data averaged over many patients, and within that group there will be individuals who will respond to much lower doses, and also some who will need more. For biologics, the potential financial savings are sufficient for making tapering attractive, and this may also minimise side effects. A variety of studies have shown successful tapering of doses of biologic drugs for proportions of patients that could result in substantial savings [6, 7], but recommending this action is beyond the license and therefore beyond recommendation by the payers. The positioning of the licence can also be affected by the prospect of a cost/effectiveness assessment. New drugs such as Baricitinib have shown excellent efficacy in MTX- naïve RA patients [8], but have no prospect of being cost/effective when compared with MTX, so they are not licenced for use here, even though that is the obvious situation in which to use them and where they will be used when off patent. One interesting area where extrapolation of data is actively encouraged is with the biosimilars. These drugs should be cheaper, not because they are easier to make, but because they do not have to do the extensive trialling of originator drugs. Currently, once they have shown adequate structural similarity, they have to do two studies in disease areas against the originator and are then allowed to inherit all of the indications and presumably all of the originator data. It remains to be seen how prescribing doctors will respond to this freedom. Once licensed and approved for use, then all we need is someone to pay for it. The intervention of purchasers in our prescribing is probably the least welcome of recent developments. This of course varies from purchaser to purchaser. There are stories of purchasers demanding strict adherence to the minutia of the guidance and licence or they will not pay; however, in other areas more sensible pathways are negotiated, allowing some wriggle room for real life. The order in which drugs are used is a legitimate subject for guidelines and purchasers. However, in some areas, we are pushed to use Rituximab as the second biologic because it is perceived to be cheaper. This is not usually a major problem, but when you have a biologic responder who has lost effect due to anti-drug antibodies, it would be nice to try another. Some guidance is probably encouraging use in non-optimal patients. How much more effective would TNFis be in early disease where TNF is probably more important! Once licensed, approved and paid for, we get to use these drugs in real people, for real reasons. That is when we get to find out how representative the evidence base is. This is also where we find that we cannot quite follow the guidance. The patient is intolerant of MTX, but we want to give the Rituximab. Will LEF do? Can it be used alone? Well probably yes. Another can of worms is that we know what we have prescribed, but we do not really have a good handle on what the patient is taking. Adherence to drugs as prescribed is only rated by the WHO at 60%. For MTX, where tolerability is a huge problem [9], how much is actually swallowed? Does that invalidate the guidance? Will the purchaser pay? In conclusion: while it would be nice to have evidence for every intervention we make, it is impractical to have a study that accurately reflects every individual in front of us. Evidence-based practice is always going to be seriously limited, and it is the degree of extrapolation that is allowed that is crucial to us making sensible clinical decisions. We should, however, be reassured that each and every intervention we make potentially becomes an n = 1 study at follow-up—not valued as anecdotal by the evidence grade, but part of our everyday practice and a major driver of our learning. Perhaps any guidelines committees should have a role in validating the sort of deviations we make in clinical practice every day, and not be solely dependent on the so-called evidence base. Possible solutions to this include an increase in emphasis placed on real-world observational data, because this would provide practically both useful evidence for daily practice and guidance for decision-making. Such data could provide information for decision-makers about what the real effect of the drugs or interventions would be. Another solution would be to set up more pragmatic trials with patients who could be comparable with those routinely seen in rheumatology clinics. Such approaches are being supported by organisations like the National Institute for Health Research in the UK, The Netherlands Organisation for Health Research and Development in The Netherlands and Belgian healthcare knowledge centre in Belgium. The problem will still lie in not being able to model all therapeutic decisions in clinical practice, and that is why rheumatologists can never be replaced by computed algorithms. Funding: No specific funding was received from any bodies in the public, commercial or not-for-profit sectors to carry out the work described in this manuscript. Disclosure statement: M.B. has been sponsored to attend regional, national and international meetings by UCB celltech, Roche/Chugai, Pfizer, AbbVie, Merck, Mennarini, Janssen, Bristol-Myers Squib, Novartis and Eli-Lilly. He has received honoraria for speaking and attended advisory boards with Bristol-Myers Squib, UCB celltech, Roche/Chugai, Pfizer, AbbVie, Merck, Mennarini, Sanofi-aventis, Eli-Lilly, Janssen and Novartis. D.W. has performed commercial research for Pfizer, Gilead, Lilly and Janssen and has done speaker meetings and advisory boards for Celgene, Pfizer, AbbVie, Lilly and Janssen. References 1 Mullin, R. Cost to Develop New Pharmaceutical Drug Now Exceeds $2.5B. https://www.scientificamerican.com/article/cost-to-develop-new-pharmaceutical-drug-now-exceeds-2-5b/ (19 September 2017, date last accessed). 2 Kennedy-Martin T, Curtis S, Faries D, Robinson S, Johnston J. A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials  2015; 16: 495. Google Scholar CrossRef Search ADS PubMed  3 Hopewell S, Boutron I, Altman DG, Ravaud P. Deficiencies in the publication and reporting of the results of systematic reviews presented at scientific medical conferences. J Clin Epidemiol  2015; 68: 1488– 95. http://dx.doi.org/10.1016/j.jclinepi.2015.03.006 Google Scholar CrossRef Search ADS PubMed  4 Baron JA, Sandler RS, Bresalier RS et al.   Cardiovascular events associated with rofecoxib: final analysis of the APPROVe trial. Lancet  2008; 372: 1756– 64. http://dx.doi.org/10.1016/S0140-6736(08)61490-7 Google Scholar CrossRef Search ADS PubMed  5 Petite SE. Effectiveness of anakinra in acute gout: a retrospective review of initial and refractory therapy. Am J Ther  2017; 24: e633– 4. Google Scholar CrossRef Search ADS PubMed  6 Lau CS, Gibofsky A, Damjanov N et al.   Down-titration of biologics for the treatment of rheumatoid arthritis: a systematic literature review. Rheumatol Int  2017; 37: 1789– 98. http://dx.doi.org/10.1007/s00296-017-3780-8 Google Scholar CrossRef Search ADS PubMed  7 Vanier A, Mariette X, Tubach F, Fautrel B; STRASS Study Group. Cost-effectiveness of TNF-blocker injection spacing for patients with established rheumatoid arthritis in remission: an economic evaluation from the spacing of TNF-blocker injections in rheumatoid arthritis trial. Value Health  2017; 20: 577– 85. http://dx.doi.org/10.1016/j.jval.2017.01.005 Google Scholar CrossRef Search ADS PubMed  8 Kuriya B, Cohen MD, Keystone E. Baricitinib in rheumatoid arthritis: evidence-to-date and clinical potential. Ther Adv Musculoskelet Dis  2017; 9: 37– 44. http://dx.doi.org/10.1177/1759720X16687481 Google Scholar CrossRef Search ADS PubMed  9 Robinson S, Gibson S, George E et al.   Tolerability and adherence problems in patients on a stable dose of methotrexate: results of a multicentre survey. Musculoskeletal Care  2015; 14: 152– 5. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Rheumatology. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Journal

RheumatologyOxford University Press

Published: Feb 23, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off