Evidence-Based Evaluation of Practice and Innovation in Physical Therapy Using the IDEAL-Physio Framework

Evidence-Based Evaluation of Practice and Innovation in Physical Therapy Using the IDEAL-Physio... Abstract The IDEAL framework is an established method for initial and ongoing evaluations of innovation and practice for complex health care interventions. First derived for surgical sciences and embedded at a global level for evaluating surgery/surgical devices, the IDEAL framework is based on the principle that innovation and evaluation in clinical practice can, and should, evolve together in an ordered manner: from conception to development and then to validation by appropriate clinical studies and, finally, longer-term follow-up. This framework is highly suited to other complex, nonpharmacological interventions, such as physical therapist interventions. This perspective outlines the application of IDEAL to physical therapy in the new IDEAL-Physio framework. The IDEAL-Physio framework comprises 5 stages. In stage 1, the idea phase, formal data collection should begin. Stage 2a is the phase for iterative improvement and adjustment with thorough data recording. Stage 2b involves the onset of formal evaluation using systematically collected group or cohort data. Stage 3 is the phase for formal comparative assessment of treatment, usually involving randomized studies. Stage 4 involves long-term follow-up. The IDEAL-Physio framework is recommended as a method for guiding and evaluating both innovation and practice in physical therapy, with the overall goal of providing better evidence-based care. Physical therapy has been affected by the introduction and proliferation of treatments with an inadequate scientific basis, little or poor evaluation, and underexposure to rigorous scientific method.1–5 Providing there is a level of consensus on the potential benefit-to-risk ratio, new treatments can be developed and introduced without evidence of efficacy, regulation, or governance. There is no requirement to collect prospective data to support claims or demonstrate efficacy. This approach has resulted in numerous disparate practices that may not stand up to rigorous evaluation or evidence-based commissioning.6 Patients may be unaware of the innovative nature of interventions, and a lack of rigorous evaluation and data may ultimately reduce patients’ autonomy and informed decision-making.7–9 Enabling patients’ realistic assessment of the benefits of interventions with efficacy data and reassuring patients with safety data to mitigate risk is a crucial aspect of ethical, evidence-based practice.7,10 Growing demand for evidence-based practice is transforming physical therapy.3,5,11–16 In the United States (US), the American Physical Therapy Association's (APTA) “Vision and Strategic Plan” statement calls on the profession to commit to establishing and adopting best practice standards across practice, education, and research.15 APTA calls on physical therapists to embrace best practice and to: generate, validate, and disseminate evidence and quality indicators, expand available evidence and translate it into practice, conduct comparative effectiveness research, standardize outcome measurement, and participate in interprofessional research teams.15 The number of controlled clinical trials, mixed-method and observational studies, and systematic reviews evaluating physical therapist practice has increased considerably in the last few decades.5,12,17,18 The Physical Therapy Evidence Database (PEDro) (http://www.pedro.fhs.usyd.edu.au/) contained 2455 randomized controlled trials (RCTs) and 71 systematic reviews in 1989, rising to 4633 clinical trials by 2004, with an additional 18,562 trials and 5674 systematic reviews added by August 2016.5,19 Increased recognition of the important contributions of qualitative and mixed-method research, incorporating both quantitative and qualitative methods, is also transforming physical therapist practice.20–22 Physical therapy addresses often complex movement impairments that are influenced by the interaction of participants’ biological, psychosocial, and cultural factors and provider-patient interactions.20,23 RCTs, although often the gold standard in clinical trials, may not always allow for the exploration of many nuances these interactions introduce, or explore the “how” and why” of how these many variables influence patient outcomes and patient-provider relationships.20,21,23 Although the quantity and quality of evidence available to practitioners continues to improve, the lack of an organized structure and framework for the development and introduction of new interventions has negatively affected the adoption of evidence-based practice within physical therapy.2,24,25 Unstructured introduction of new treatment is not unique to physical therapy, having been experienced and fought by other specialties. The pharmacological industry, for example, has been active in staged rigorous evaluation for years. New drugs, and their application, are controlled by licensing and various governance entities after strict, standardized scientific checks. No drug or pharmacological agent use is permitted without minimal efficacy and safety data. This process requires a dedicated structure, and a whole industry has been aligned to the formal evaluation of pharmacological agents, usually producing safe and usually efficacious drug treatments. The ability to apply such evaluation in pharma has been aided by the nature of drug intervention, where the active ingredients, control, and other components of the therapeutic interaction are readily identifiable.13 This contrasts with complex interventions such as physical therapy and surgery. Complex interventions include multiple facets that may affect the intervention's results. They are “built up from a number of components, which may act both independently and interdependently.”13 Treatment outcomes may rely on operator skill and experience and tailoring of the intervention to the patient, such as in cardiac catheterization or poststroke physical therapy, for example.13,26,27 The type and intensity of intervention in physical therapy, whether standardized or individually tailored, can also interact with patient-specific and environmental factors to influence the results. This dynamic system of factors and components has contributed to the lack of structured evaluation of innovation within physical therapy. Until recently, many surgical procedures also escaped rigorous, scientifically methodical evaluation. Innovative practices could be, and often have been, introduced without proof of efficacy. Recently, expanded investment in generating better evidence from clinical trials and increased interest in demonstrating the worth of various surgical procedures have improved the evaluation of surgery and the requirement to demonstrate efficacy. The recent investment and enthusiasm for surgical evaluation shown by the United Kingdom's (UK’s) Royal College of Surgeons and professional bodies in the US have facilitated this improvement, assisted by the adoption of new frameworks that create ordered and structured introduction and evaluation of new and existing surgical treatments. The IDEAL framework is one such development that has helped considerably. The IDEAL framework, aimed at improving the quality of research in surgery, was first proposed in 2007 at the Balliol Collaboration Conferences, Balliol College, University of Oxford, Oxford, United Kingdom. A group of surgeons and methodologists developed a new framework for describing the stages of development of surgical and interventional innovations. A series of methodological recommendations and research reporting guidelines for each stage were generated. The proposals were directed at research publishers, funders, regulators, and professional organizations.28–30 The framework described a 5-stage process to introduce innovation in surgery: idea, development, exploration, assessment, and long-term study. Research items fit within this ordered structure, helping to provide an evidence-based introduction of innovation, and a transparent method of evaluating existing treatments (Tab.). The IDEAL system has helped researchers and research funders identify purpose and positioning for research projects in the surgical sciences. It facilitates sequential and ordered introduction of new technology, methods, pathways, and procedures. Importantly, although it presently has no regulatory implications, it can prevent the widespread adoption of new surgical treatments lacking adequate efficacy or safety data. Ultimately, IDEAL ensures that new treatments have minimum outcome data, collected in a standardized way. Longer established or more developed interventions are appropriate for more substantive evaluation of efficacy, effectiveness, or cost-effectiveness, such as large multicenter trials. The IDEAL framework for surgery is easily applied to similar, practitioner-based interventional therapies. Physical therapy shares many attributes in complexity with surgery, if not quite sharing the same level of morbid risk. It is similarly challenged by factors that depend on clinician and caregiver, patient biopsychosocial factors, team, and setting, such as skill mix, experience, learning curves, quality variations, and perception of equipoise. The use of the IDEAL framework for physical therapy was explored at the recent International IDEAL Conference (http://www.ideal-collaboration.net/ideal-conference-2016-evaluating-innovation-in-surgery-and-therapeutic-technology-the-ideal-approach/; April 2016), enjoying support from representatives of medical industry, journals, and professional bodies. This paper outlines the application of the IDEAL model for use in physical therapy, proposing a new IDEAL-Physio framework in response to the IDEAL conference. It is adapted from the original IDEAL framework developed for surgery and is proposed here as a way of controlling random and inappropriate introduction of ineffective, and possibly harmful, physical therapist interventions. IDEAL-Physio has the same stages as its surgical counterpart, with appropriate modifications made by the application of principles and core values of physical therapy, informed by APTA’s Guide to Physical Therapist Practice, to suit the discipline better.31 Idea (Stage 1) The idea stage, or stage 1, of evaluation contains “first-in-human” instances of a new practice. Physical therapy innovations are commonplace and may represent advances in technology, technique, or modified thinking. Innovation could represent incremental advances in practice, where new procedures are variants on existing interventions, or be entirely novel, with initial evaluations being the first stage in an incorporation pathway ultimately leading to clinical trials. The critical aspect of initial evaluations (stage 1) is ordered and systematic data collection using appropriate outcome measures. This stage represents less of a formal evaluation of efficacy and much more of a first, public description of new intervention. The patient has understanding that the intervention is new and there is transparent recording and reporting of initial data. Safety is a major focus, and although only a small number of cases may be reported, any early signs of problems or potential adverse events can be reported. The diversity of physical therapist practice requires particular consideration in this stage. Like drug trials and medical devices, some physical therapist interventions (ie, acute care, cardiopulmonary rehabilitation, wound care) may be associated with serious systemic effects and adverse events and require detailed monitoring and recording. Many interventions, such as exercise therapy, are more benign, however, and recording may focus on practical and mechanistic aspects of the intervention and its likelihood of being taken up, before further investment of time and resources are made to develop it further. Physical therapists should also have the opportunity to learn from each other's experiences. This should be embedded in a sensible and ethical model. Unfortunately, this method of knowledge transition has remained many therapists’ main pathway for developing practice for some time.2,5,24 Peer-to-peer knowledge propagation has become the mainstay for many, with therapists often learning treatment techniques mostly through observation. New or popular techniques are also taught at continuing education workshops and conferences and are then reproduced in general practice, often with little high-quality scientific evidence to support their use. IDEAL-Physio stage 1 makes some allowance for this process to continue, as it is often rich with innovation, but in a more controlled manner, providing a framework for improved data collection. Appendix  1 provides an example of a study at the idea stage of IDEAL-Physio. As with all research projects, any new innovation should enjoy local oversight and clinical governance. At present, consent for new or innovative procedures in physical therapy is not required. The wide spectrum of therapeutic agents and practices and the current lack of consensus on best practices often means therapists can include any innovation in a treatment plan without recourse to regulatory framework. Providing no harm is done or the treatment would not seem unreasonable to peers in practice, therapists can attempt almost any intervention. This is unsuitable, and an IDEAL-Physio approach may facilitate improved safety, systematic evaluation of novel practices, and higher levels of informed consent and patient participation in decision-making. Patients should be provided with the best available information about the level of efficacy and possible risk of therapy. Stage 1 can also serve as a useful screening point for unsuccessful forays into new practice, as practitioners systematically gather and record initial data about safety, application of the novel intervention, and patient responses, and document failed attempts of innovative practice. This screening process may help prevent propagation of ineffective practices by others and missed opportunities for patients’ recovery. Reports of failed attempts or innovations are seen as important as successful innovations at this stage. To facilitate this, some form of registration of new practices is required. Registries, though needing further development in physical therapy, can be useful for documenting stage 1 data and could help promote efficiency and evaluation of innovation. APTA is currently upgrading the Physical Therapy Outcomes Registry (http://www.ptoutcomes.com/home.aspx?navID=10737435589), and there may be opportunities to initiate appropriate registries of new practices, similar to clinical trial registers. This will rely on widespread acceptance and contribution to the registry. Standardization of new interventions, reliability and validity of therapists’ employed technique, data set identification, and collection may initially be challenges. Application of electronic data methodologies may prove beneficial, though hosting the service will require planning and commitment at the profession's governance level. Adoption of the IDEAL-Physio framework by such bodies, such as APTA, may also support the implementation of the framework and assist its sustainability. As with other frameworks for gathering and reporting data, such as the CONSORT32 and STROBE33 statements, there is a cost in time and labor to the therapist. However, as IDEAL-Physio seeks to order and guide the process by which therapists evaluate novel practices, these initial costs may be offset by a reduction in health costs to patients and the health care system by the screening out of ineffective practices and their duplication. Development (Stage 2a) The IDEAL development stage in both surgery and physical therapy involves relatively rapid iterative change of news practices in the light of experience alongside preexisting, structured data collection. It is a fluid phase which differentiates the pathway for innovative complex interventions from that for pharmaceutical innovations. In pharma, the product is complete before human testing. In skill-based disciplines, however, indication, techniques, and possibly materials used may need modification. In a specific progressive-resistive exercise, for example, different types of resistive materials or delivery agents (ie, spring resistance, manual resistance) may need to be tried in the course of patient care. The best dosage or intensity can be preliminarily identified, but may be continually titrated with evolving treatment. This IDEAL-Physio stage can require extra care in reporting, especially if the interventions’ iterations are rapid in sequence. The lack of current reporting in the development stages of new physical therapist practices can create challenges for IDEAL-Physio that can be ameliorated as part of an overall adoption strategy. A particularly important consideration in IDEAL-Physio is adequate description of treatment details, and how they change as the treatment is developed. Achieving a level of standardization for the intervention is of the upmost importance by the end of stage 2a. The detailed reporting of the intervention is also important and challenging in this stage of IDEAL-Physio, as it was for surgery.28 Lack of reporting in this stage, or its omission once subsequent stages have been reached, can deprive others of the opportunity to learn from the development process. Detailed recording of the specifics of interventions and patients’ response to them, familiar to both surgeons and therapists from routine documentation of interventions and treatment sessions, are the foundations of developing interventions in an iterative fashion. Communication of the individual's findings beyond immediate colleagues may be less commonplace in routine practice. As with stage 1, online registries could facilitate the step-change needed in data recording. Stage 2a data are collected prospectively. No strong judgment should be made about the interventions’ success or failure at this point. Not only is the emphasis distanced from evaluation of outcome, but the quantity and granularity of data is likely to be insufficient to form evidenced opinion. For example, a new form of cryotherapy for postoperative care may initially seem effective and useful, but this will only be shown in the later IDEAL-Physio stages. Results can be reported, and it is important they are in this stage, but without conclusions about efficacy. The underpinning principle here is transparency. All changes or iterations should be documented and explained. As the intervention is skill-based, consideration should be given to learning curves and how dependent the intervention might be on therapists’ skill. For example, a new manual therapy intervention might require high levels of palpation and visualization skills, and this may not initially be a strong attribute for all therapists. Clear reporting of the intervention mechanisms and techniques at stage 1, as in 2a, can help minimize variation in the content and scope of the intervention at this stage. The IDEAL nomenclature and notation is often useful when submitting grant proposals to funding bodies. It provides scope and limits to the research plan and is particularly helpful for developmental work (stage 2a). The use of IDEAL to position specific research topics and areas within the broad research landscape is now commonplace in surgery and has helped review bodies/panels quickly identify the context of any proposed study. Physical therapy–based submissions, if adopted, may also benefit from a similar strategy. The use of an IDEAL notation will also facilitate standardized nomenclature in physical therapy research, which has been called for repeatedly in recent years.34,35 Physical therapists are encouraged to adhere to the initiatives for standardized outcomes measures, such as the COMET (Core Outcome Measures in Effectiveness Trials) Group (http://www.comet-initiative.org). The COMET Initiative focuses on the development and application of agreed, standardized sets of outcomes, fostering their collection and reporting in order to make it easier for the results of trials to be compared, contrasted, and combined.36,37 These outcome sets represent the minimum that should be measured and reported in all clinical trials of a specific condition, and are suitable for use in clinical audit or research. Although designed primarily for trials, the COMET guidelines are a useful adjunct to aid reporting for stage 2a studies. Core outcomes should always be collected and reported to avoid selective reporting of studies and outcomes.36,37 Other initiatives, such as ICHOM (International Consortium for Health Outcomes Measurement; http://www.ichom.org) and PROMIS (Patient-Reported Outcomes Measurement Information System; http://www.healthmeasures.net/explore-measurement-systems/promis), are also available and may be used by physical therapists. In physical therapy, there will be a great deal of expertise-based opinion in this stage and the ability to harness this with consensus meetings and qualitative research will be critical to the development and evidencing of interventions.2,38 Qualitative and mixed-method research will also be of great importance in exploring the dynamic and complex nature of patients’ interactions with physical therapy, the intervention, and the physical therapist and in exploring the impact these interactions can have on patient outcomes. Qualitative research approaches examine phenomena from the perspectives of the participants, who are dynamic forces within their social, cultural, physical, and economic environments, and are inseparable from these.20,23 All of these complement the goals of IDEAL-Physio. Development of working groups and publication of consensus statements, with recognition of standard items and definitions, will be helpful in guiding further developments. Certain aspects will require particular attention for physical therapy (Appendix  2). Exploration (Stage 2b) In this stage, “new” practices, now relatively stable in development, are exposed to early formal evaluation and testing (Appendix  3). Promising evidence of the intervention's safety and beneficial, short-term outcomes will have been generated in stages 1 and 2a, or further development would have been halted. The exploration stage bridges the gap between early (or no formal evaluation for efficacy) and future RCTs (or equivalents) that would firmly establish treatments’ effectiveness. The stage involves the intervention being more widely used than previously throughout clinical practice. At the end of this stage, the intervention itself is deemed to be stable in content and description. Further iterations in the development of any technique or practice are much reduced or nonexistent compared to stage 2a. The population they are applied to should be well defined, though the treatment can be administered more universally, rather than to a special selection of patients. The treatment is likely to be adopted by multiple therapists, possibly in different centers, making issues of learning-curve evaluation and mentoring particularly important. Although interventions are usually deemed sufficiently safe by this stage, data should be collected systematically for every patient treated with the innovative intervention, ensuring that benefits and adverse outcomes are documented. At this stage, observational studies can collect prospective data, preferably from several sites, contributing to the body of evidence supporting or refuting a practice. Stage 2b evidence should not include retrospective studies, and exploratory studies will usually be a cohort design without a contemporaneous comparison group(s). Often, as in surgery, the principle here is to evaluate whether the intervention is worthy of more rigorous, formal evaluation, such as RCT. With that said, the RCT may not always be the most appropriate design for addressing the nuances and complexities of interventions, given the multifactorial nature of rehabilitation.38,39 Qualitative and mixed-method research exploring nature of participants’ experience of rehabilitation may allow therapists to see further than just treatment techniques.40–42 Exploration of patients’ perspectives and values may improve the therapists’ understanding of the experience of individuals in rehabilitation and the effectiveness of therapeutic interventions, further guiding research and innovation.41,42 Although this 2b IDEAL stage has often represented a “tipping point” in surgical development, wherein new practices go on to obtain RCT evidence to become established practice, for physical therapy the situation may be different. Higher-quality research is needed, but not all questions may be best answered by RCTs.5,11,16 If IDEAL-Physio is fully adopted, a substantial proportion of evidence for the profession may be derived from well-conducted stage 2b studies. Therefore, any increase in stage 2b studies would be welcomed, with only specific and appropriate research questions being answered by the controlled trial design. Stage 2b is a useful place in which to explore clinicians’ experience of the intervention and the requirements for its application. As physical therapist intervention is practitioner-based, operator learning curves, therapists’ skill mix, and levels of experience (including experience with the intervention) may strongly influence efficacy. Documenting this for further exploration will be helpful. Where possible, the effect of intertherapist skill differences and learning should be assessed. The exploration stage comprises any robust evidence, outside of formal RCTs, supporting or refuting the intended practice. Evidence should be of the highest quality possible, and studies should be designed with the prospect of an ensuing RCT. Mixed-method and nonrandomized designs, such as observational studies, are important, and registry data may also add value. Registries such as APTA’s Physical Therapy Outcomes Registry (http://www.ptoutcomes.com/home.aspx?navID=10737435589) may be helpful sources of outcomes data for novel interventions. There is risk of bias with stage 2b studies, but efforts can be made to anticipate and minimize these. Data should be collected on consecutive patients, avoiding “cherry picking” of patients most likely to respond positively to the intervention, biasing the results.43 Assessment (Stage 3) By stage 3, interventions have shown promise and will usually have been widely accepted by the clinical community (Appendix  4). There should be good documentation of efficacy and data sets available with which to justify continued use and further assessment. Stage 3 requires larger and more authoritative bodies of evidence, free of bias, on which to continue practice or desist. Usually, this is achieved through randomized evidence, often in the form of clinical trials. Alternately, quasi-randomized, nonrandomized, interrupted time series, or observational designs are possible when randomized trials are not feasible.39 Although there is great need of quality, trial-based evidence for many physical therapist interventions, the situation continues to improve. In the UK and US, several large-scale National Institute for Health Research (NIHR) and National Institutes of Health (NIH) physical therapy projects have been funded. The lack of RCT evidence for physical therapist practice has several implications for the use of the IDEAL-Physio framework. The vast majority of current physical therapist treatments may be in need of stage 3 evaluation. Substantial evaluation is required for practices in frequent use with poor evidence. As these are numerous, there is a rich, open landscape for developing studies and submissions, though a completely evidence-based platform for therapeutic intervention seems some way off.3,38 This paper is not the correct place to describe the need for clinical trials in medicine and physical therapy, nor a detailed description of their nuances. These are well documented elsewhere. It is sufficient to note that high-quality multicenter RCTs for IDEAL-Physio stage 3 are required, and these should include adequate descriptions of population, intervention, comparison, and outcomes (PICO), a trial model that is fully compatible with the IDEAL structure. If trials are conducted, a pragmatic design, which is more generalizable and can accommodate greater variation in practice, is recommended over a more experimental/mechanistic design. Although a robust RCT with a strong control comparator is the design of choice, it should not be seen as the only evidence-generating platform for physical therapy. Given the expense and difficulty of conducting RCTs in rehabilitation, other types of IDEAL classified studies may provide the best evidence available for the interventions that physical therapists use. A sensible and practical approach is required. Clinical effectiveness research, including well-conducted stage 2b studies, in which patient benefit is shown longitudinally using a valid outcome measure (such as change in pain or function) and set against historical control data, could also be authoritative. Long-Term Monitoring (Stage 4) Long-term monitoring may initially seem less salient in physical therapy than in surgery, for example. When medical devices are implanted, anatomy changed, or long drug usage required, it is imperative to follow up patients and collect reliable data sets. Further intervention may be required in the event of the initial intervention failing, posing significant risk to the patient. The long-term influence or failure of physical therapist intervention is not likely to be life-threatening. However, there are physical therapist practices that will require long-term evaluation for both efficacy and risk (Appendix  5). For example, low-level laser (LLT) treatment or other electro-physical agents are used by physical therapists and new types of treatment in these areas should have a follow-up strategy for safety.44 Adaptive equipment, such as mobility and assistive devices, designed or prescribed by physical therapists have long-term effects on patients’ structure and function.45–47 Similarly, inadequate adaptive seating for the mobility-impaired can lead to serious long-term consequences, from structural deformity to ulcerations, if improperly fitted or designed. Additionally, there can be negative long-term structural and functional consequences from inappropriate use, or use of inappropriately fitted, splints and orthoses, such as foot, hand, and upper limb orthoses. All of these may be designed or fabricated by physical therapists, and long-term follow-up in IDEAL-Physio stage 4 is essential. In neurological, developmental, and orthopedic rehabilitation, treatment effects may last over a long time period and practices may lend themselves to longer registration. Patients may enter physical therapist care directly, without prior referral, for musculoskeletal or functional limitations. For example, patients with low back pain may first seek care directly through a physical therapist who will evaluate, diagnose the impairment, and design a treatment plan. Patients and caregivers may be given therapeutic activities to carry out on a long-term basis, returning for follow-up with the therapist long after the period of care has ended. Given the chronic nature of many impairments, the long-term follow-up of IDEAL-Physio stage 4 is an important aspect of the evaluation of treatments in many populations. Physical therapists play an increasing role in wellness, health promotion, and preventative care, providing education and direct intervention in clinical and community settings and collaborative consultation.48 Lifestyle changes, such as increased physical activity, have been shown to assist in the primary and secondary prevention and management of many chronic diseases, such as obesity, diabetes, and cardiovascular disease, and lead to an increased quality of life.49 These lifestyle modifications may have a long-lasting effect on patients’ health and function, and require long-term follow-up for safety and efficacy. Many factors, both patient-specific biopsychosocial, and related to the interaction between the therapist and patient, may greatly influence long-term outcomes.23,50 Patients with greater resources, motivation, or familial support may be better able to continue carrying out therapists’ prescribed interventions over the long term than those lacking these resources.50 Similarly, patients with poorer health or access to health services may have less ability to do so. Therapists more skilled in patient education and with greater ability to engage patients’ social and familial supports, or greater ability to follow up with patients with chronic conditions, may be better able to influence long-term outcomes than other physical therapists.50–52 This is an important consideration in evaluation of the long-term impact and success of any intervention, and is not unique to interventions guided by IDEAL-Physio.23,51 Careful documentation and consideration of these factors when assessing the long-term efficacy of interventions is of the upmost importance, and may ultimately require modification of aspects of the intervention, or new research, to address the potential influence of these factors.38,42 A major challenge with long-term monitoring of physical therapist interventions lies in defining the end-point of the intervention and determining treatment efficacy relative to this. A specific, therapeutic-exercise prescription for anterior knee pain in a middle-aged man may have proven measurably beneficial, but how long should this benefit last? Is future diagnosis and surgical intervention for patella-femoral osteoarthritis a failure of the therapist intervention, or a new pathology that renders the patient as lost to further follow-up? It is likely that multiple “longer-term” outcome databases will be created by various groups for specific areas of interest. Some degree of standardization of methodology would be beneficial, but requires a level of regulatory oversight, the infrastructure of which does not currently exist. Here, professional regulating bodies, such as APTA, could lend invaluable support and provide both consensus and guidance to the clinician. Discussion The IDEAL framework is based on the principle that innovation and evaluation in clinical practice can and should evolve together in an ordered manner, from conception to validation by appropriate clinical trials.28 The IDEAL framework is now successfully embedded in the surgical sciences and has also been adapted for evaluating and regulating the use of medical devices.6 IDEAL lends itself to other evaluations and introductions of new treatment. Physical therapy, as a complex intervention, has some very similar characteristics to surgery in terms of skill requirements, expertise, and variation in treatment. The IDEAL-Physio framework is recommended as a method of guiding innovation and the evaluation of practice and constructing a much more evidence-based profession. If, as recommended, IDEAL-Physio is adopted (at a global level) similar to surgery, this move should involve editors, professional and funding bodies, and governments. A partnership between physical therapists, academics, and professional bodies could facilitate its adoption without the need for major new resources. It will be useful to allocate every study to an appropriate IDEAL- Physio stage. This will create order, registration, and stability for the assessment of existing interventions and the introduction of new modalities and treatment. Although there should be the ambition to do so, not all work will progress to, or be best addressed by, RCTs, and the mainstay may lie in stages 1, 2a, and 2b or in quasi-randomized or mixed-method designs. IDEAL-Physio may be particularly useful in facilitating the development of successful RCTs. IDEAL has been shown to be a feasible method for documenting the development and implementation of new procedures in patient care.53,54 Guided by IDEAL-Physio, major gains in physical therapy innovation could be achieved through the development of reporting systems (stage 1), protocol registries (stage 2a), comprehensive, disease-specific research databases (stages 2b and 3), and population registries (stage 4).28 We believe that physical therapy can be improved, and safe and efficient innovation in practice fostered, by the adoption of the IDEAL-Physio framework. Author Contributions Concept/idea/research design: D. Beard, D. Hamilton, J. Cook, A. Hirst, P. McCulloch, A. Paez Writing: D. Beard, A. Paez, D. Hamilton, J. Cook, A. Hirst, P. McCulloch Data collection: A. Paez Project management: D. Beard, A. Paez Clerical/secretarial support: A. Paez Consultation (including review of manuscript before submitting): D. Beard, A. Paez, D. Hamilton, L. Davies, A. Hirst, P. McCulloch Disclosure The authors completed the ICJME Form for Disclosure of Potential Conflicts of Interest and reported no conflicts of interest.
A. Hirst is employed by the University of Oxford as Project Manager/Researcher for the IDEAL Collaboration to develop research methods in complex interventions in surgery and in new devices. References 1 Miller PA, McKibbon KA, Haynes RB. A quantitative analysis of research. Phys Ther . 2003; 83: 123– 131. Google Scholar PubMed  2 Turner PAT, Trobe L, Whitfield TW. Physiotherapists’ use of evidence based practice: a cross-national study. Physiother Res Int . 1997; 2: 17– 29. Google Scholar CrossRef Search ADS PubMed  3 Scurlock-Evans L, Upton P, Upton D. Evidence-based practice in physiotherapy: a systematic review of barriers, enablers and interventions. Physiotherapy . 2014; 100: 208– 219. Google Scholar CrossRef Search ADS PubMed  4 Metcalfe C, Lewin R, Wisher S, Perry S, Bannigan K, Moffett JK. Barriers to implementing the evidence base in four NHS therapies. Physiotherapy . 2001; 87: 433– 441. Google Scholar CrossRef Search ADS   5 Maher CG, Sherrington C, Elkins M, Herbert RD, Moseley AM. Challenges for evidence-based physical therapy: accessing and interpreting high-quality evidence on therapy. Phys Ther . 2004; 84: 644– 654. Google Scholar PubMed  6 Sedrakyan A, Campbell B, Merino JG, Kuntz R, Hirst A, McCulloch P. IDEAL-D: a rational framework for evaluating and regulating the use of medical devices. BMJ . 2016; 353: i2372. Google Scholar CrossRef Search ADS PubMed  7 Bracken-Roche D, Bell E, Karpowicz L, Racine E. Disclosure, consent, and the exercise of patient autonomy in surgical innovation: a systematic content analysis of the conceptual literature. Account Res . 2014; 21: 331– 352. Google Scholar CrossRef Search ADS PubMed  8 Johnson J, Rogers W, Lotz M, Townley C, Meyerson D, Tomossy G. Ethical challenges of innovative surgery: a response to the IDEAL recommendations. Lancet . 2010; 376: 1113– 1115. Google Scholar CrossRef Search ADS PubMed  9 Brower V. The ethics of innovation: should innovative surgery be exempt from clinical trials and regulations? EMBO Rep . 2003; 4: 338– 340. Google Scholar CrossRef Search ADS PubMed  10 Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ . 1996; 312: 71– 72. Google Scholar CrossRef Search ADS PubMed  11 Dannapfel P, Peolsson A, Nilsen P. What supports physiotherapists’ use of research in clinical practice? A qualitative study in Sweden. Implement Sci . 2013; 8: 1– 13. Google Scholar CrossRef Search ADS PubMed  12 Kunz R, Autti-Rämö I, Anttila H, Malmivaara A, Mäkelä M. A systematic review finds that methodological quality is better than its reputation but can be improved in physiotherapy trials in childhood cerebral palsy. J Clin Epidemiol . 2006; 59: 1239. e1–1239.e12. Google Scholar CrossRef Search ADS PubMed  13 Moulding NT, Silagy CA, Weller DP. A framework for effective management of change in clinical practice: dissemination and implementation of clinical practice guidelines. Qual Health Care . 1999; 8: 177– 183. Google Scholar CrossRef Search ADS PubMed  14 Paci M, Briganti G, Lombardi B. Levels of evidence of articles published in physical and rehabilitation medicine journals. J Rehabil Med . 2011; 43: 264– 267. Google Scholar CrossRef Search ADS PubMed  15 American Physical Therapy Association. Vision statement for the physical therapy profession and guiding principles to achieve the vision. 16 American Physical Therapy Association website. http://www.apta.org/Vision/ 17 Delitto A. What will it take? The need for large-scale trials. Phys Ther . 2005; 85: 466. Google Scholar PubMed  18 Paci M, Landi N, Marchettini M, Baccini M. Randomized controlled trial quality in pediatric physical therapy. Phys Occup Ther Pediatr . 2014; 34: 260– 270. Google Scholar CrossRef Search ADS PubMed  19 Kumar SP. Professional discussion. Physical therapy: past, present and future—a paradigm shift. J Phys Ther . 2010; 1: 58– 67. 20 Musculoskeletal Health Sydney, School of Public Health, University of Sydney. Physiotherapy Evidence Database. http://www.pedro.fhs.usyd.edu.au. Last updated September 4, 2017. Accessed September 12, 2017. 21 Tashakkori A, Creswell JW. Editorial: the new era of mixed methods. Journal of Mixed Methods Research . 2007; 1: 3– 7. Google Scholar CrossRef Search ADS   22 Rauscher L, Greenfield BH. Advancements in contemporary physical therapy research: use of mixed methods designs. Phys Ther . 2009; 89: 91– 100. Google Scholar CrossRef Search ADS PubMed  23 Deyle GD, Allison SC, Matekel RL et al.   Physical therapy treatment effectiveness for osteoarthritis of the knee: a randomized comparison of supervised clinical exercise and manual therapy procedures versus a home exercise program. Phys Ther . 2005; 85: 1301– 1317. Google Scholar PubMed  24 Bartlett DJ, Macnab J, Macarthur C et al.   Advancing rehabilitation research: an interactionist perspective to guide question and design. Disabil Rehabil . 2006; 28: 1169– 1176. Google Scholar CrossRef Search ADS PubMed  25 Bohannon RW, LeVeau BF. Clinicians’ use of research findings: a review of the literature with implications for physical therapists. Phys Ther . 1986; 66: 45– 50. Google Scholar CrossRef Search ADS PubMed  26 Jette DU, Bacon K, Batty C et al.   Evidence-based practice: beliefs, attitudes, knowledge, and behaviors of physical therapists. Phys Ther . 2003; 83: 786– 805. Google Scholar PubMed  27 McCulloch P, Cook JA, Altman DG, Heneghan C, Diener MK IDEAL Group. IDEAL framework for surgical innovation, 1: the idea and development stages. BMJ . 2013; 346: f3012. Google Scholar CrossRef Search ADS PubMed  28 Sheill A, Hawe P, Gold L. Complex interventions or complex systems? Implications for health economic evaluation. BMJ . 2008; 336: 1281– 1283. Google Scholar CrossRef Search ADS PubMed  29 Mcculloch P, Altman DG, Campbell WB et al.   No surgical innovation without evaluation: the IDEAL recommendations. Lancet . 2009; 374: 1105– 1112. Google Scholar CrossRef Search ADS PubMed  30 Barkun JS, Aronson JK, Feldman LS et al.   Evaluation and stages of surgical innovations. Lancet . 2009; 374: 1089– 1096. Google Scholar CrossRef Search ADS PubMed  31 Ergina PL, Cook JA, Blazeby JM et al.   Challenges in evaluating surgical innovation. Lancet . 2009; 374: 1097– 1104. Google Scholar CrossRef Search ADS PubMed  32 American Physical Therapy Association. Guide to physical therapist practice 3.0. http://guidetoptpractice.apta.org/. Published 2014. Last updated November 27, 2016. Accessed September 12, 2017. 33 Moher D, Hopewell S, Schulz KF et al.   CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Int J Surg . 2012; 10: 28– 55. Google Scholar CrossRef Search ADS PubMed  34 von Elm E, Altman DG, Egger M et al.   The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. J Clin Epidemiol . 2008; 61: 344– 349. Google Scholar CrossRef Search ADS PubMed  35 Mintken PE, DeRosa C, Little T, Smith B American Academy of Orthopaedic Manual Physical Therapists. A model for standardizing manipulation terminology in physical therapy practice. J Orthop Sports Phys Ther . 2008; 38: A1– A6. Google Scholar CrossRef Search ADS PubMed  36 Jette AM. Toward a common language for function, disability, and health. Phys Ther . 2006; 86: 726– 734. Google Scholar CrossRef Search ADS PubMed  37 COMET Initiative. http://www.comet-initiative.org. Accessed September 12, 2017. 38 Gargon E, Williamson PR, Altman DG, Blazeby JM, Clarke M. The COMET Initiative database: progress and activities from 2011 to 2013. Trials . 2014; 15: 279. Google Scholar CrossRef Search ADS PubMed  39 Dijkers MP, Murphy SL, Krellman J. Evidence-based practice for rehabilitation professionals: concepts and controversies. Arch Phys Med Rehabil . 2012; 93( 8 suppl): S164– S176. Google Scholar CrossRef Search ADS PubMed  40 Whyte J. Traumatic brain injury rehabilitation: are there alternatives to randomized clinical trials? Arch Phys Med Rehabil . 2002; 83: 1320– 1322. Google Scholar CrossRef Search ADS PubMed  41 Johnson R, Waterfield J. Making words count: the value of qualitative research. Physiother Res Int.  2004; 9: 121– 131. Google Scholar CrossRef Search ADS PubMed  42 Hammell KW, Carpenter C, Dyck I. Using Qualitative Research: A Practical Introduction for Occupational and Physical Therapists . Edinburgh: Churchill Livingstone; 2000: 59– 71. 43 Peters DJ. Qualitative inquiry: expanding rehabilitation medicine's research repetoire—a commentary. Am J Phys Med Rehabil.  1996; 75: 144– 148. Google Scholar CrossRef Search ADS PubMed  44 Bjorn M, Brendstrup C, Karlsen S, Carlsen JE. Consecutive screening and enrollment in clinical trials: the way to representative patient samples? J Card Fail . 1998; 4: 225– 230. Google Scholar CrossRef Search ADS PubMed  45 Hutchison AM, Pallister I, Evans RM et al.   Intense pulsed light treatment of chronic mid-body Achilles tendinopathy: a double blind randomised placebo-controlled trial. Bone Joint J . 2013; 95-B: 504– 509. Google Scholar CrossRef Search ADS PubMed  46 Reddy M, Gill SS, Rochon PA. Clinician's corner: preventing pressure ulcers. JAMA . 2006; 296: 974– 984. Google Scholar CrossRef Search ADS PubMed  47 Chung J, Evans J, Lee C et al.   Effectiveness of adaptive seating on sitting posture and postural control in children with cerebral palsy. Pediatr Phys Ther . 2008; 20: 303– 317. Google Scholar CrossRef Search ADS PubMed  48 Cassel J, Cassel R, Down F et al.   Best practice guidelines: wheelchair provision for children and adults with muscular dystrophy and other neuromuscular conditions. Muscular Dystrophy Campaign. http://www.musculardystrophyuk.org/app/uploads/2015/02/wheelchair-guidelines.pdf. Published March 2011. Accessed September 12, 2017. 49 American Physical Therapy Association. Physical therapists’ role in prevention, wellness, fitness, health promotion and management of disease and disability. HOD P06-16-06-05. http://www.apta.org/uploadedFiles/APTAorg/About_Us/Policies/Practice/PTRoleAdvocacy.pdf. Last updated September 2, 2016. Accessed September 12, 2017. 50 Warburton DER, Nicol CW, Bredin SSD. Health benefits of physical activity: the evidence. Can Med Assoc J.  2006; 174: 801– 809. Google Scholar CrossRef Search ADS   51 DiMatteo MR. Social support and patient adherence to medical treatment: a meta-analysis. Health Psychol . 2004; 23: 207– 218. Google Scholar CrossRef Search ADS PubMed  52 Stewart MA. Effective physician-patient communication and health outcomes: a review. Can Med Assoc J . 1995; 152: 1423– 1433. 53 Siminoff LA. Incorporating patient and family preferences into evidence-based medicine. BMC Medical Informatics and Decision Making . 2013; 13( suppl 3): S6. Google Scholar CrossRef Search ADS PubMed  54 Blazeby JM, Blencowe NS, Titcomb DR, Metcalfe C, Hollowood AD, Barham CP. Demonstration of the IDEAL recommendations for evaluating and reporting surgical innovation in minimally invasive oesophagectomy. Br J Surg . 2011; 98: 544– 551. Google Scholar CrossRef Search ADS PubMed  55 Manca A, Cabboi MP, Ortu E et al.   Effect of contralateral strength training on muscle weakness in people with multiple sclerosis: proof-of-concept case series. Phys Ther . 2016; 96: 828– 838. Google Scholar CrossRef Search ADS PubMed  56 Perme C, Nalty T, Winkelman C, Kenji Nawa R, Masud F. Safety and efficacy of mobility interventions in patients with femoral catheters in the ICU: a prospective observational study. Cardiopulm Phys Ther J . 2013; 24: 12– 17. Google Scholar PubMed  57 Donaldson C, Tallis R, Miller S, Sunderland A, Lemon R, Pomeroy V. Effects of conventional physical therapy and functional strength training on upper limb motor recovery after stroke: a randomized phase II study. Neurorehabil Neural Repair . 2009; 23: 389– 397. Google Scholar CrossRef Search ADS PubMed  58 Wolf SL, Winstein CJ, Miller JP et al.   Effect of constraint-induced movement therapy on upper extremity function 3 to 9 months after stroke: the EXCITE randomized clinical trial. JAMA . 2006; 296: 2095– 2104. Google Scholar CrossRef Search ADS PubMed  59 Wolf SL, Winstein CJ, Miller JP et al.   Retention of upper limb function in stroke survivors who have received constraint-induced movement therapy: the EXCITE randomised trial. Lancet Neurol . 2008; 7: 33– 40. Google Scholar CrossRef Search ADS PubMed  Appendix 1. IDEAL-Physio Stage 1: Idea This stage includes first-in-human and proof-of-concept studies. Here, a first, public description of a new intervention or application of a concept is presented. Accuracy, transparency, and completeness of reporting and data collection are important. Reporting standards are developed and agreed upon. Goals and outcomes Explanation and description of the new intervention or application to a new population. Transparent, quality recording of initial data with standard outcomes. Definitions for key outcomes and factors are agreed upon and standardized. Identification of early signs, problems, and adverse effects. Safety and adherence or uptake are emphasized. Example: Manca et al55 Multiple sclerosis (MS) is a neurological condition characterized by reduced muscle and decreased ability to fully activate motor units in the lower limb muscles. Muscle weakness reduces the ability of people with MS to perform physical exercise, with a consequent decrease in the level of activities of daily living (ADL). Resistance training (RT) reduces muscle weakness, improves balance, and decreases self-reported fatigue during ADL in people with MS, improving quality of life. When strength impairment is lateralized to 1 limb, RT is conventionally addressed on the weaker side. However, this approach may not be applicable to a severely weakened limb that is too compromised to sustain it. Contralateral strength training (CST) may represent a viable alternative to the conventional direct approach. CST has been well studied in people who are healthy and in people with orthopedic conditions, but has been poorly addressed in people with neurological disorders. Design: Proof of concept Staged program of multidisciplinary research beginning with detailed preclinical studies and literature review. Explicit clinical and data collection protocol written in advance and submitted for ethical review, with detailed informed consent process. A series of replicated single-system case studies completed using a pretest-posttest design with a short-term follow-up and repeated recording of outcome measures. Findings After intervention, the contralateral limb shows an increase in maximal strength and improvements in mobility and walking performance. No adverse events identified in response to the intervention. Authors believe that CST is a promising approach for addressing unilateral muscle weakness in people with neurological impairment and that the new treatment protocol is feasible in the health care system. Proof of concept is shown. Further study is warranted, and important factors needing further exploration for the next phase are identified. Appendix 2. Standardization and Contextual Factors in IDEAL-Physio Stage 2a: Developmenta In this stage, the early evaluation in the development stage (2a) will use small observational studies without contemporaneous comparison groups in highly selected cohorts of patients. Pilot data are being collected. Terminology is being standardized for the intervention. Judgment about the success or failure of the intervention is based on short-term outcomes that might not reflect the important effects of the intervention. Prospective rather than retrospective studies are preferred for greater validity and reliability. Data may be insufficient for meaningful statistical analysis, although trends may be identified. Online registries could facilitate data recording. Operator learning curves and aspects of the intervention requiring further description and standardization are being identified. Key elements of stage 2a studies Prior, clearly defined protocol. Clearly defined, objective outcomes. Transparent, sequential reporting of cases. Clear, detailed description of when changes to the intervention are made, whether indication or technique. Continued emphasis on transparency and safety. Continued standardization of terminology. Example: Perme et al56 Limited research exists on the safety of mobility activities for patients with femoral catheters. Most intensive care units (ICUs) recommend that patients with femoral catheters remain immobilized, in bed. This approach might expose those patients to unnecessary adverse effects of bed rest. There is a clinical need for investigating the feasibility and safety of mobilizing. Patients in ICUs with 1 or more femoral catheters. A prospective, observational study in a cardiovascular surgical ICU at a large metropolitan teaching hospital from 2009 to 2010. No adverse events occurred during a physical therapy session; evidence for safety was reported. Physical therapy sessions including standing and walking were feasible and safe in patients in a cardiovascular ICU who had femoral catheters and who met the criteria for mobility interventions. The data from this study provided new information to clinicians and researchers about feasibility and safety regarding sitting, standing, and walking activities in adults who are critically ill. Recommendations are made for larger, future studies, including a larger cohort, a longer observation period following mobility activities, unit-based guidelines for initiation and progression of activities, and greater standardization. aAdapted from McCulloch et al (2013).26 Appendix 3. Recommendations for Observational Studies at Stage 2b: Exploration By this stage, promising evidence of the intervention's safety and beneficial, short-term outcomes will have been generated (in stages 1 and 2a), or further development would have been halted. Exploratory studies collect prospective data from several sites or multiple therapists, contributing to the body of evidence supporting or refuting a practice. These include protocol-driven observational studies and exploratory studies, such as phase 2 trials, often with a cohort design, with or without a contemporaneous comparison group(s). The exploration stage (2b) offers the opportunity to obtain higher-quality evidence in a more representative patient population and to deal with factors that could hinder the conduct of a proper methodological evaluation in the next stages. Outcomes Standardized outcomes are important. Universally accepted, validated outcome measures for pathology-specific problems, general health evaluation, and therapy-specific interventions. Reported consistently, perhaps with a minimal data set. Goals Identify and carefully document intervention benefits and adverse outcomes. Explore clinicians’ experience of the intervention and the requirements for its application. Determine whether the intervention may be worthy of more rigorous, formal evaluation. Identify issues in need of further exploration, within intervention and among operators. Example: Donaldson et al57 This study was an observer-masked, randomized, controlled, phase 2 trial. A random-allocation order was generated using a computer program. An exploratory trial was designed as a precursor to a definitive phase 3 trial. Thirty participants in this phase 2 trial (10 per treatment arm). Results would be interpreted cautiously in this phase 2 study, as it was underpowered, but authors sought to determine whether the intervention was worthy of further study. Intervention was compared to best practice or conventional physical therapy (CPT). Three study arms: CPT, CPT plus enhanced-intensity CPT, and CPT plus functional strength training (FST). Observer and participant masking to reduce bias. Standardized treatment schedules and standardized outcome measures were used: Action Research Arm Test (ARAT), Nine-Hole Peg Test, and upper limb strength measured by myometer. Measures were taken before intervention and 6 and 12 weeks after intervention. Results and conclusion: the intervention was safe, may be beneficial, and was worthy of further study. Outcome: CPT + FST group showed the greatest increases in ARAT and other scores, above the clinically important level for tests. Benefit in function was identified. No adverse events or outcomes were identified. Authors concluded that further work toward a phase 3 clinical trial appeared justifiable. Authors calculated the estimated sample size for an adequately powered subsequent phase 3 trial and identified important factors needing further exploration for phase 3. Appendix 4. Considerations for Randomized Studies at Phase 3: Assessmenta In stage 3, full evaluation of efficacy is taking place. Formal, comparative studies—always prefaced by systematic review of the literature and, if possible, meta-analysis—explore the state of the evidence for the intervention or the condition: randomized controlled trial, if possible, with direct and/or indirect comparisons between intervention and no treatment, best practice, or placebo; and quasi-randomized or observational designs when randomized trials are not feasible. Examples of these include nonrandomized and interrupted time series studies. Interrupted time series (a quasi-experimental design) use a temporal (or sequential) rather than a concurrent control group. In this stage: Quality documentation of efficacy and data sets with which to justify continued use and further assessment. Stage 3 requires larger, more authoritative bodies of evidence for continuing practice or desisting. Studies should be designed to provide definitive evaluation of the intervention at this stage. Reporting guidelines, such as CONSORT or STROBE, can guide recording and reporting of data. Outcomes registries can be helpful for reporting outcomes. Example: Wolf et al58 Previous, single-site studies suggested that a 2-week program of constraint-induced movement therapy (CIMT) for patients more than 1 year after stroke can improve upper extremity function that persists for at least 1 year. The EXCITE trial, a prospective, single-blind, randomized, multisite clinical trial, was conducted at 7 US academic institutions between January 2001 and January 2003. Two hundred twenty-two individuals with predominantly ischemic stroke participated. The control group received “usual and customary care (standard care).” The intervention group received CIMT, including shaping (adaptive task practice) and standard task training of the paretic limb for up to 6 hours daily over 2 weeks. Participants in the control group were offered the same CIMT regimen after the 12-month evaluation session. Outcomes and interventions were standardized, validated, and reported consistently. The Wolf Motor Function Test (WMFT), a measure of laboratory time and strength-based ability and quality of movement (functional ability), and the Motor Activity Log, a measure of how well and how often 30 common daily activities are performed, were used. Follow-up took place at 1 year after intervention. The results indicated that CIMT produced statistically significant and clinically relevant improvements in arm motor function that persisted for at least 1 year. Further research on CIMT, including alternate models of CIMT, is warranted. aAdapted from McCulloch et al (2013).26 CONSORT = Consolidated Standards of Reporting Trials, STROBE = Strengthening the Reporting of Observational Studies in Epidemiology. Appendix 5. IDEAL-Physio Stage 4: Long-Term Follow-Upa In this stage, interventions have become established by previous stages and are now being assessed for long-term outcomes, rare events, variations in outcomes, and any influence of biopsychosocial patient and practitioner factors on long-term outcomes. Large numbers of cases (patients treated with the intervention) may now be available, adding to the ability to assess the impact of comorbidities and trends over time. Type of study: Observational studies, phase 4 trials, randomized controlled trials with a long follow-up arm, and mixed-method and qualitative studies for monitoring of patients’ responses and ability to carry through with prescribed interventions and long-term influence on activity and participation. Long-term evaluations of both efficacy and risk are taking place and may help reveal effects of interventions not immediately evident and beyond the immediate effect on the target impairment or condition. Registries, American Physical Therapy Association, National Institute for Health Research, and National Institutes of Health can be helpful for recording data and modeling new registries of outcomes. Example: Wolf et al59 A previous study by Wolf et al58 in this series had follow-up at 1 year after intervention. The results indicated that constraint-induced movement therapy (CIMT) produced statistically significant and clinically relevant improvements in arm motor function that persisted for at least 1 year. The authors concluded that further research on CIMT, including a longer follow-up, was warranted. In the current study, the authors assessed the retention of improvements from the EXCITE CIMT intervention through 24 months. The authors reported: “There was no observed regression from the treatment effects observed at 12 months after treatment during the next 12 months for the primary outcome measures of WMFT and MAL. In fact, the additional changes were in the direction of increased therapeutic effect.”59 The authors concluded that the intervention (CIMT) had persistent benefits. Additional improvements in strength, performance of ADL/IADL, and Social Participation domains, not noted at the initial 12-month follow-up in the previous study, were now apparent. Long-term follow-up in clinical trials in rehabilitation may help reveal the full extent of effects of therapeutic interventions, beyond the immediate impact of the interventions. Biopsychosocial and other factors influencing outcomes may be elucidated in long-term follow-up studies. aADL = activities of daily living, IADL = instrumental ADL, MAL = Motor Activity Log, WMFT = Wolf Motor Function Test. Table. Five Stages of IDEAL-Physio Frameworka Stage 1: Idea  Stage 2a: Development  Stage 2b: Exploration  Stage 3: Assessment  Stage 4: Long-Term Follow-Up  First-in-human studies and proof-of-concept studies  Prospective development studies including mixed-method and qualitative data  Exploratory prospective observational studies, often with a cohort design  Full evaluation of efficacy; formal, comparative studies including RCTs and observational designs  Longitudinal observational studies  Focus on explanation and description of the new intervention  Continued emphasis on structured data collection and quality recording  No necessity for comparison group(s)  Prefaced by systematic review/meta-analysis; RCT if possible  Can be longer-term follow-up of RCTs or mixed-method and qualitative studies  Focus on safety, with reporting of early adverse events  Iterative development  Data from several sites or therapists  Direct and/or indirect comparisons between intervention and no Rx, best practice, or placebo  Influence of biopsychosocial factors and pragmatic aspects  Transparent, quality recording of initial data with standard outcomes  Description of changes in treatment  Data collected systematically from consecutive patients  Reporting guidelines, CONSORT, or STROBE  Registries, ATPA, National Institute for Health Research, and National Institutes of Health helpful  Determination of whether the intervention is efficacious and worthy of further study  Detailed recording of patients’ responses to intervention  Recording of clinicians’ experience          Use of refined outcome measures      Stage 1: Idea  Stage 2a: Development  Stage 2b: Exploration  Stage 3: Assessment  Stage 4: Long-Term Follow-Up  First-in-human studies and proof-of-concept studies  Prospective development studies including mixed-method and qualitative data  Exploratory prospective observational studies, often with a cohort design  Full evaluation of efficacy; formal, comparative studies including RCTs and observational designs  Longitudinal observational studies  Focus on explanation and description of the new intervention  Continued emphasis on structured data collection and quality recording  No necessity for comparison group(s)  Prefaced by systematic review/meta-analysis; RCT if possible  Can be longer-term follow-up of RCTs or mixed-method and qualitative studies  Focus on safety, with reporting of early adverse events  Iterative development  Data from several sites or therapists  Direct and/or indirect comparisons between intervention and no Rx, best practice, or placebo  Influence of biopsychosocial factors and pragmatic aspects  Transparent, quality recording of initial data with standard outcomes  Description of changes in treatment  Data collected systematically from consecutive patients  Reporting guidelines, CONSORT, or STROBE  Registries, ATPA, National Institute for Health Research, and National Institutes of Health helpful  Determination of whether the intervention is efficacious and worthy of further study  Detailed recording of patients’ responses to intervention  Recording of clinicians’ experience          Use of refined outcome measures      aModified from McCulloch et al (2013).26 APTA = American Physical Therapy Association, CONSORT = Consolidated Standards of Reporting Trials, 
RCT = randomized controlled trial, Rx = prescription, STROBE = Strengthening the Reporting of Observational Studies in Epidemiology. View Large © 2017 American Physical Therapy Association http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Physical Therapy Oxford University Press

Evidence-Based Evaluation of Practice and Innovation in Physical Therapy Using the IDEAL-Physio Framework

Loading next page...
 
/lp/ou_press/evidence-based-evaluation-of-practice-and-innovation-in-physical-xRJ060zWQi
Publisher
American Physical Therapy Association
Copyright
© 2017 American Physical Therapy Association
ISSN
0031-9023
eISSN
1538-6724
D.O.I.
10.1093/ptj/pzx103
Publisher site
See Article on Publisher Site

Abstract

Abstract The IDEAL framework is an established method for initial and ongoing evaluations of innovation and practice for complex health care interventions. First derived for surgical sciences and embedded at a global level for evaluating surgery/surgical devices, the IDEAL framework is based on the principle that innovation and evaluation in clinical practice can, and should, evolve together in an ordered manner: from conception to development and then to validation by appropriate clinical studies and, finally, longer-term follow-up. This framework is highly suited to other complex, nonpharmacological interventions, such as physical therapist interventions. This perspective outlines the application of IDEAL to physical therapy in the new IDEAL-Physio framework. The IDEAL-Physio framework comprises 5 stages. In stage 1, the idea phase, formal data collection should begin. Stage 2a is the phase for iterative improvement and adjustment with thorough data recording. Stage 2b involves the onset of formal evaluation using systematically collected group or cohort data. Stage 3 is the phase for formal comparative assessment of treatment, usually involving randomized studies. Stage 4 involves long-term follow-up. The IDEAL-Physio framework is recommended as a method for guiding and evaluating both innovation and practice in physical therapy, with the overall goal of providing better evidence-based care. Physical therapy has been affected by the introduction and proliferation of treatments with an inadequate scientific basis, little or poor evaluation, and underexposure to rigorous scientific method.1–5 Providing there is a level of consensus on the potential benefit-to-risk ratio, new treatments can be developed and introduced without evidence of efficacy, regulation, or governance. There is no requirement to collect prospective data to support claims or demonstrate efficacy. This approach has resulted in numerous disparate practices that may not stand up to rigorous evaluation or evidence-based commissioning.6 Patients may be unaware of the innovative nature of interventions, and a lack of rigorous evaluation and data may ultimately reduce patients’ autonomy and informed decision-making.7–9 Enabling patients’ realistic assessment of the benefits of interventions with efficacy data and reassuring patients with safety data to mitigate risk is a crucial aspect of ethical, evidence-based practice.7,10 Growing demand for evidence-based practice is transforming physical therapy.3,5,11–16 In the United States (US), the American Physical Therapy Association's (APTA) “Vision and Strategic Plan” statement calls on the profession to commit to establishing and adopting best practice standards across practice, education, and research.15 APTA calls on physical therapists to embrace best practice and to: generate, validate, and disseminate evidence and quality indicators, expand available evidence and translate it into practice, conduct comparative effectiveness research, standardize outcome measurement, and participate in interprofessional research teams.15 The number of controlled clinical trials, mixed-method and observational studies, and systematic reviews evaluating physical therapist practice has increased considerably in the last few decades.5,12,17,18 The Physical Therapy Evidence Database (PEDro) (http://www.pedro.fhs.usyd.edu.au/) contained 2455 randomized controlled trials (RCTs) and 71 systematic reviews in 1989, rising to 4633 clinical trials by 2004, with an additional 18,562 trials and 5674 systematic reviews added by August 2016.5,19 Increased recognition of the important contributions of qualitative and mixed-method research, incorporating both quantitative and qualitative methods, is also transforming physical therapist practice.20–22 Physical therapy addresses often complex movement impairments that are influenced by the interaction of participants’ biological, psychosocial, and cultural factors and provider-patient interactions.20,23 RCTs, although often the gold standard in clinical trials, may not always allow for the exploration of many nuances these interactions introduce, or explore the “how” and why” of how these many variables influence patient outcomes and patient-provider relationships.20,21,23 Although the quantity and quality of evidence available to practitioners continues to improve, the lack of an organized structure and framework for the development and introduction of new interventions has negatively affected the adoption of evidence-based practice within physical therapy.2,24,25 Unstructured introduction of new treatment is not unique to physical therapy, having been experienced and fought by other specialties. The pharmacological industry, for example, has been active in staged rigorous evaluation for years. New drugs, and their application, are controlled by licensing and various governance entities after strict, standardized scientific checks. No drug or pharmacological agent use is permitted without minimal efficacy and safety data. This process requires a dedicated structure, and a whole industry has been aligned to the formal evaluation of pharmacological agents, usually producing safe and usually efficacious drug treatments. The ability to apply such evaluation in pharma has been aided by the nature of drug intervention, where the active ingredients, control, and other components of the therapeutic interaction are readily identifiable.13 This contrasts with complex interventions such as physical therapy and surgery. Complex interventions include multiple facets that may affect the intervention's results. They are “built up from a number of components, which may act both independently and interdependently.”13 Treatment outcomes may rely on operator skill and experience and tailoring of the intervention to the patient, such as in cardiac catheterization or poststroke physical therapy, for example.13,26,27 The type and intensity of intervention in physical therapy, whether standardized or individually tailored, can also interact with patient-specific and environmental factors to influence the results. This dynamic system of factors and components has contributed to the lack of structured evaluation of innovation within physical therapy. Until recently, many surgical procedures also escaped rigorous, scientifically methodical evaluation. Innovative practices could be, and often have been, introduced without proof of efficacy. Recently, expanded investment in generating better evidence from clinical trials and increased interest in demonstrating the worth of various surgical procedures have improved the evaluation of surgery and the requirement to demonstrate efficacy. The recent investment and enthusiasm for surgical evaluation shown by the United Kingdom's (UK’s) Royal College of Surgeons and professional bodies in the US have facilitated this improvement, assisted by the adoption of new frameworks that create ordered and structured introduction and evaluation of new and existing surgical treatments. The IDEAL framework is one such development that has helped considerably. The IDEAL framework, aimed at improving the quality of research in surgery, was first proposed in 2007 at the Balliol Collaboration Conferences, Balliol College, University of Oxford, Oxford, United Kingdom. A group of surgeons and methodologists developed a new framework for describing the stages of development of surgical and interventional innovations. A series of methodological recommendations and research reporting guidelines for each stage were generated. The proposals were directed at research publishers, funders, regulators, and professional organizations.28–30 The framework described a 5-stage process to introduce innovation in surgery: idea, development, exploration, assessment, and long-term study. Research items fit within this ordered structure, helping to provide an evidence-based introduction of innovation, and a transparent method of evaluating existing treatments (Tab.). The IDEAL system has helped researchers and research funders identify purpose and positioning for research projects in the surgical sciences. It facilitates sequential and ordered introduction of new technology, methods, pathways, and procedures. Importantly, although it presently has no regulatory implications, it can prevent the widespread adoption of new surgical treatments lacking adequate efficacy or safety data. Ultimately, IDEAL ensures that new treatments have minimum outcome data, collected in a standardized way. Longer established or more developed interventions are appropriate for more substantive evaluation of efficacy, effectiveness, or cost-effectiveness, such as large multicenter trials. The IDEAL framework for surgery is easily applied to similar, practitioner-based interventional therapies. Physical therapy shares many attributes in complexity with surgery, if not quite sharing the same level of morbid risk. It is similarly challenged by factors that depend on clinician and caregiver, patient biopsychosocial factors, team, and setting, such as skill mix, experience, learning curves, quality variations, and perception of equipoise. The use of the IDEAL framework for physical therapy was explored at the recent International IDEAL Conference (http://www.ideal-collaboration.net/ideal-conference-2016-evaluating-innovation-in-surgery-and-therapeutic-technology-the-ideal-approach/; April 2016), enjoying support from representatives of medical industry, journals, and professional bodies. This paper outlines the application of the IDEAL model for use in physical therapy, proposing a new IDEAL-Physio framework in response to the IDEAL conference. It is adapted from the original IDEAL framework developed for surgery and is proposed here as a way of controlling random and inappropriate introduction of ineffective, and possibly harmful, physical therapist interventions. IDEAL-Physio has the same stages as its surgical counterpart, with appropriate modifications made by the application of principles and core values of physical therapy, informed by APTA’s Guide to Physical Therapist Practice, to suit the discipline better.31 Idea (Stage 1) The idea stage, or stage 1, of evaluation contains “first-in-human” instances of a new practice. Physical therapy innovations are commonplace and may represent advances in technology, technique, or modified thinking. Innovation could represent incremental advances in practice, where new procedures are variants on existing interventions, or be entirely novel, with initial evaluations being the first stage in an incorporation pathway ultimately leading to clinical trials. The critical aspect of initial evaluations (stage 1) is ordered and systematic data collection using appropriate outcome measures. This stage represents less of a formal evaluation of efficacy and much more of a first, public description of new intervention. The patient has understanding that the intervention is new and there is transparent recording and reporting of initial data. Safety is a major focus, and although only a small number of cases may be reported, any early signs of problems or potential adverse events can be reported. The diversity of physical therapist practice requires particular consideration in this stage. Like drug trials and medical devices, some physical therapist interventions (ie, acute care, cardiopulmonary rehabilitation, wound care) may be associated with serious systemic effects and adverse events and require detailed monitoring and recording. Many interventions, such as exercise therapy, are more benign, however, and recording may focus on practical and mechanistic aspects of the intervention and its likelihood of being taken up, before further investment of time and resources are made to develop it further. Physical therapists should also have the opportunity to learn from each other's experiences. This should be embedded in a sensible and ethical model. Unfortunately, this method of knowledge transition has remained many therapists’ main pathway for developing practice for some time.2,5,24 Peer-to-peer knowledge propagation has become the mainstay for many, with therapists often learning treatment techniques mostly through observation. New or popular techniques are also taught at continuing education workshops and conferences and are then reproduced in general practice, often with little high-quality scientific evidence to support their use. IDEAL-Physio stage 1 makes some allowance for this process to continue, as it is often rich with innovation, but in a more controlled manner, providing a framework for improved data collection. Appendix  1 provides an example of a study at the idea stage of IDEAL-Physio. As with all research projects, any new innovation should enjoy local oversight and clinical governance. At present, consent for new or innovative procedures in physical therapy is not required. The wide spectrum of therapeutic agents and practices and the current lack of consensus on best practices often means therapists can include any innovation in a treatment plan without recourse to regulatory framework. Providing no harm is done or the treatment would not seem unreasonable to peers in practice, therapists can attempt almost any intervention. This is unsuitable, and an IDEAL-Physio approach may facilitate improved safety, systematic evaluation of novel practices, and higher levels of informed consent and patient participation in decision-making. Patients should be provided with the best available information about the level of efficacy and possible risk of therapy. Stage 1 can also serve as a useful screening point for unsuccessful forays into new practice, as practitioners systematically gather and record initial data about safety, application of the novel intervention, and patient responses, and document failed attempts of innovative practice. This screening process may help prevent propagation of ineffective practices by others and missed opportunities for patients’ recovery. Reports of failed attempts or innovations are seen as important as successful innovations at this stage. To facilitate this, some form of registration of new practices is required. Registries, though needing further development in physical therapy, can be useful for documenting stage 1 data and could help promote efficiency and evaluation of innovation. APTA is currently upgrading the Physical Therapy Outcomes Registry (http://www.ptoutcomes.com/home.aspx?navID=10737435589), and there may be opportunities to initiate appropriate registries of new practices, similar to clinical trial registers. This will rely on widespread acceptance and contribution to the registry. Standardization of new interventions, reliability and validity of therapists’ employed technique, data set identification, and collection may initially be challenges. Application of electronic data methodologies may prove beneficial, though hosting the service will require planning and commitment at the profession's governance level. Adoption of the IDEAL-Physio framework by such bodies, such as APTA, may also support the implementation of the framework and assist its sustainability. As with other frameworks for gathering and reporting data, such as the CONSORT32 and STROBE33 statements, there is a cost in time and labor to the therapist. However, as IDEAL-Physio seeks to order and guide the process by which therapists evaluate novel practices, these initial costs may be offset by a reduction in health costs to patients and the health care system by the screening out of ineffective practices and their duplication. Development (Stage 2a) The IDEAL development stage in both surgery and physical therapy involves relatively rapid iterative change of news practices in the light of experience alongside preexisting, structured data collection. It is a fluid phase which differentiates the pathway for innovative complex interventions from that for pharmaceutical innovations. In pharma, the product is complete before human testing. In skill-based disciplines, however, indication, techniques, and possibly materials used may need modification. In a specific progressive-resistive exercise, for example, different types of resistive materials or delivery agents (ie, spring resistance, manual resistance) may need to be tried in the course of patient care. The best dosage or intensity can be preliminarily identified, but may be continually titrated with evolving treatment. This IDEAL-Physio stage can require extra care in reporting, especially if the interventions’ iterations are rapid in sequence. The lack of current reporting in the development stages of new physical therapist practices can create challenges for IDEAL-Physio that can be ameliorated as part of an overall adoption strategy. A particularly important consideration in IDEAL-Physio is adequate description of treatment details, and how they change as the treatment is developed. Achieving a level of standardization for the intervention is of the upmost importance by the end of stage 2a. The detailed reporting of the intervention is also important and challenging in this stage of IDEAL-Physio, as it was for surgery.28 Lack of reporting in this stage, or its omission once subsequent stages have been reached, can deprive others of the opportunity to learn from the development process. Detailed recording of the specifics of interventions and patients’ response to them, familiar to both surgeons and therapists from routine documentation of interventions and treatment sessions, are the foundations of developing interventions in an iterative fashion. Communication of the individual's findings beyond immediate colleagues may be less commonplace in routine practice. As with stage 1, online registries could facilitate the step-change needed in data recording. Stage 2a data are collected prospectively. No strong judgment should be made about the interventions’ success or failure at this point. Not only is the emphasis distanced from evaluation of outcome, but the quantity and granularity of data is likely to be insufficient to form evidenced opinion. For example, a new form of cryotherapy for postoperative care may initially seem effective and useful, but this will only be shown in the later IDEAL-Physio stages. Results can be reported, and it is important they are in this stage, but without conclusions about efficacy. The underpinning principle here is transparency. All changes or iterations should be documented and explained. As the intervention is skill-based, consideration should be given to learning curves and how dependent the intervention might be on therapists’ skill. For example, a new manual therapy intervention might require high levels of palpation and visualization skills, and this may not initially be a strong attribute for all therapists. Clear reporting of the intervention mechanisms and techniques at stage 1, as in 2a, can help minimize variation in the content and scope of the intervention at this stage. The IDEAL nomenclature and notation is often useful when submitting grant proposals to funding bodies. It provides scope and limits to the research plan and is particularly helpful for developmental work (stage 2a). The use of IDEAL to position specific research topics and areas within the broad research landscape is now commonplace in surgery and has helped review bodies/panels quickly identify the context of any proposed study. Physical therapy–based submissions, if adopted, may also benefit from a similar strategy. The use of an IDEAL notation will also facilitate standardized nomenclature in physical therapy research, which has been called for repeatedly in recent years.34,35 Physical therapists are encouraged to adhere to the initiatives for standardized outcomes measures, such as the COMET (Core Outcome Measures in Effectiveness Trials) Group (http://www.comet-initiative.org). The COMET Initiative focuses on the development and application of agreed, standardized sets of outcomes, fostering their collection and reporting in order to make it easier for the results of trials to be compared, contrasted, and combined.36,37 These outcome sets represent the minimum that should be measured and reported in all clinical trials of a specific condition, and are suitable for use in clinical audit or research. Although designed primarily for trials, the COMET guidelines are a useful adjunct to aid reporting for stage 2a studies. Core outcomes should always be collected and reported to avoid selective reporting of studies and outcomes.36,37 Other initiatives, such as ICHOM (International Consortium for Health Outcomes Measurement; http://www.ichom.org) and PROMIS (Patient-Reported Outcomes Measurement Information System; http://www.healthmeasures.net/explore-measurement-systems/promis), are also available and may be used by physical therapists. In physical therapy, there will be a great deal of expertise-based opinion in this stage and the ability to harness this with consensus meetings and qualitative research will be critical to the development and evidencing of interventions.2,38 Qualitative and mixed-method research will also be of great importance in exploring the dynamic and complex nature of patients’ interactions with physical therapy, the intervention, and the physical therapist and in exploring the impact these interactions can have on patient outcomes. Qualitative research approaches examine phenomena from the perspectives of the participants, who are dynamic forces within their social, cultural, physical, and economic environments, and are inseparable from these.20,23 All of these complement the goals of IDEAL-Physio. Development of working groups and publication of consensus statements, with recognition of standard items and definitions, will be helpful in guiding further developments. Certain aspects will require particular attention for physical therapy (Appendix  2). Exploration (Stage 2b) In this stage, “new” practices, now relatively stable in development, are exposed to early formal evaluation and testing (Appendix  3). Promising evidence of the intervention's safety and beneficial, short-term outcomes will have been generated in stages 1 and 2a, or further development would have been halted. The exploration stage bridges the gap between early (or no formal evaluation for efficacy) and future RCTs (or equivalents) that would firmly establish treatments’ effectiveness. The stage involves the intervention being more widely used than previously throughout clinical practice. At the end of this stage, the intervention itself is deemed to be stable in content and description. Further iterations in the development of any technique or practice are much reduced or nonexistent compared to stage 2a. The population they are applied to should be well defined, though the treatment can be administered more universally, rather than to a special selection of patients. The treatment is likely to be adopted by multiple therapists, possibly in different centers, making issues of learning-curve evaluation and mentoring particularly important. Although interventions are usually deemed sufficiently safe by this stage, data should be collected systematically for every patient treated with the innovative intervention, ensuring that benefits and adverse outcomes are documented. At this stage, observational studies can collect prospective data, preferably from several sites, contributing to the body of evidence supporting or refuting a practice. Stage 2b evidence should not include retrospective studies, and exploratory studies will usually be a cohort design without a contemporaneous comparison group(s). Often, as in surgery, the principle here is to evaluate whether the intervention is worthy of more rigorous, formal evaluation, such as RCT. With that said, the RCT may not always be the most appropriate design for addressing the nuances and complexities of interventions, given the multifactorial nature of rehabilitation.38,39 Qualitative and mixed-method research exploring nature of participants’ experience of rehabilitation may allow therapists to see further than just treatment techniques.40–42 Exploration of patients’ perspectives and values may improve the therapists’ understanding of the experience of individuals in rehabilitation and the effectiveness of therapeutic interventions, further guiding research and innovation.41,42 Although this 2b IDEAL stage has often represented a “tipping point” in surgical development, wherein new practices go on to obtain RCT evidence to become established practice, for physical therapy the situation may be different. Higher-quality research is needed, but not all questions may be best answered by RCTs.5,11,16 If IDEAL-Physio is fully adopted, a substantial proportion of evidence for the profession may be derived from well-conducted stage 2b studies. Therefore, any increase in stage 2b studies would be welcomed, with only specific and appropriate research questions being answered by the controlled trial design. Stage 2b is a useful place in which to explore clinicians’ experience of the intervention and the requirements for its application. As physical therapist intervention is practitioner-based, operator learning curves, therapists’ skill mix, and levels of experience (including experience with the intervention) may strongly influence efficacy. Documenting this for further exploration will be helpful. Where possible, the effect of intertherapist skill differences and learning should be assessed. The exploration stage comprises any robust evidence, outside of formal RCTs, supporting or refuting the intended practice. Evidence should be of the highest quality possible, and studies should be designed with the prospect of an ensuing RCT. Mixed-method and nonrandomized designs, such as observational studies, are important, and registry data may also add value. Registries such as APTA’s Physical Therapy Outcomes Registry (http://www.ptoutcomes.com/home.aspx?navID=10737435589) may be helpful sources of outcomes data for novel interventions. There is risk of bias with stage 2b studies, but efforts can be made to anticipate and minimize these. Data should be collected on consecutive patients, avoiding “cherry picking” of patients most likely to respond positively to the intervention, biasing the results.43 Assessment (Stage 3) By stage 3, interventions have shown promise and will usually have been widely accepted by the clinical community (Appendix  4). There should be good documentation of efficacy and data sets available with which to justify continued use and further assessment. Stage 3 requires larger and more authoritative bodies of evidence, free of bias, on which to continue practice or desist. Usually, this is achieved through randomized evidence, often in the form of clinical trials. Alternately, quasi-randomized, nonrandomized, interrupted time series, or observational designs are possible when randomized trials are not feasible.39 Although there is great need of quality, trial-based evidence for many physical therapist interventions, the situation continues to improve. In the UK and US, several large-scale National Institute for Health Research (NIHR) and National Institutes of Health (NIH) physical therapy projects have been funded. The lack of RCT evidence for physical therapist practice has several implications for the use of the IDEAL-Physio framework. The vast majority of current physical therapist treatments may be in need of stage 3 evaluation. Substantial evaluation is required for practices in frequent use with poor evidence. As these are numerous, there is a rich, open landscape for developing studies and submissions, though a completely evidence-based platform for therapeutic intervention seems some way off.3,38 This paper is not the correct place to describe the need for clinical trials in medicine and physical therapy, nor a detailed description of their nuances. These are well documented elsewhere. It is sufficient to note that high-quality multicenter RCTs for IDEAL-Physio stage 3 are required, and these should include adequate descriptions of population, intervention, comparison, and outcomes (PICO), a trial model that is fully compatible with the IDEAL structure. If trials are conducted, a pragmatic design, which is more generalizable and can accommodate greater variation in practice, is recommended over a more experimental/mechanistic design. Although a robust RCT with a strong control comparator is the design of choice, it should not be seen as the only evidence-generating platform for physical therapy. Given the expense and difficulty of conducting RCTs in rehabilitation, other types of IDEAL classified studies may provide the best evidence available for the interventions that physical therapists use. A sensible and practical approach is required. Clinical effectiveness research, including well-conducted stage 2b studies, in which patient benefit is shown longitudinally using a valid outcome measure (such as change in pain or function) and set against historical control data, could also be authoritative. Long-Term Monitoring (Stage 4) Long-term monitoring may initially seem less salient in physical therapy than in surgery, for example. When medical devices are implanted, anatomy changed, or long drug usage required, it is imperative to follow up patients and collect reliable data sets. Further intervention may be required in the event of the initial intervention failing, posing significant risk to the patient. The long-term influence or failure of physical therapist intervention is not likely to be life-threatening. However, there are physical therapist practices that will require long-term evaluation for both efficacy and risk (Appendix  5). For example, low-level laser (LLT) treatment or other electro-physical agents are used by physical therapists and new types of treatment in these areas should have a follow-up strategy for safety.44 Adaptive equipment, such as mobility and assistive devices, designed or prescribed by physical therapists have long-term effects on patients’ structure and function.45–47 Similarly, inadequate adaptive seating for the mobility-impaired can lead to serious long-term consequences, from structural deformity to ulcerations, if improperly fitted or designed. Additionally, there can be negative long-term structural and functional consequences from inappropriate use, or use of inappropriately fitted, splints and orthoses, such as foot, hand, and upper limb orthoses. All of these may be designed or fabricated by physical therapists, and long-term follow-up in IDEAL-Physio stage 4 is essential. In neurological, developmental, and orthopedic rehabilitation, treatment effects may last over a long time period and practices may lend themselves to longer registration. Patients may enter physical therapist care directly, without prior referral, for musculoskeletal or functional limitations. For example, patients with low back pain may first seek care directly through a physical therapist who will evaluate, diagnose the impairment, and design a treatment plan. Patients and caregivers may be given therapeutic activities to carry out on a long-term basis, returning for follow-up with the therapist long after the period of care has ended. Given the chronic nature of many impairments, the long-term follow-up of IDEAL-Physio stage 4 is an important aspect of the evaluation of treatments in many populations. Physical therapists play an increasing role in wellness, health promotion, and preventative care, providing education and direct intervention in clinical and community settings and collaborative consultation.48 Lifestyle changes, such as increased physical activity, have been shown to assist in the primary and secondary prevention and management of many chronic diseases, such as obesity, diabetes, and cardiovascular disease, and lead to an increased quality of life.49 These lifestyle modifications may have a long-lasting effect on patients’ health and function, and require long-term follow-up for safety and efficacy. Many factors, both patient-specific biopsychosocial, and related to the interaction between the therapist and patient, may greatly influence long-term outcomes.23,50 Patients with greater resources, motivation, or familial support may be better able to continue carrying out therapists’ prescribed interventions over the long term than those lacking these resources.50 Similarly, patients with poorer health or access to health services may have less ability to do so. Therapists more skilled in patient education and with greater ability to engage patients’ social and familial supports, or greater ability to follow up with patients with chronic conditions, may be better able to influence long-term outcomes than other physical therapists.50–52 This is an important consideration in evaluation of the long-term impact and success of any intervention, and is not unique to interventions guided by IDEAL-Physio.23,51 Careful documentation and consideration of these factors when assessing the long-term efficacy of interventions is of the upmost importance, and may ultimately require modification of aspects of the intervention, or new research, to address the potential influence of these factors.38,42 A major challenge with long-term monitoring of physical therapist interventions lies in defining the end-point of the intervention and determining treatment efficacy relative to this. A specific, therapeutic-exercise prescription for anterior knee pain in a middle-aged man may have proven measurably beneficial, but how long should this benefit last? Is future diagnosis and surgical intervention for patella-femoral osteoarthritis a failure of the therapist intervention, or a new pathology that renders the patient as lost to further follow-up? It is likely that multiple “longer-term” outcome databases will be created by various groups for specific areas of interest. Some degree of standardization of methodology would be beneficial, but requires a level of regulatory oversight, the infrastructure of which does not currently exist. Here, professional regulating bodies, such as APTA, could lend invaluable support and provide both consensus and guidance to the clinician. Discussion The IDEAL framework is based on the principle that innovation and evaluation in clinical practice can and should evolve together in an ordered manner, from conception to validation by appropriate clinical trials.28 The IDEAL framework is now successfully embedded in the surgical sciences and has also been adapted for evaluating and regulating the use of medical devices.6 IDEAL lends itself to other evaluations and introductions of new treatment. Physical therapy, as a complex intervention, has some very similar characteristics to surgery in terms of skill requirements, expertise, and variation in treatment. The IDEAL-Physio framework is recommended as a method of guiding innovation and the evaluation of practice and constructing a much more evidence-based profession. If, as recommended, IDEAL-Physio is adopted (at a global level) similar to surgery, this move should involve editors, professional and funding bodies, and governments. A partnership between physical therapists, academics, and professional bodies could facilitate its adoption without the need for major new resources. It will be useful to allocate every study to an appropriate IDEAL- Physio stage. This will create order, registration, and stability for the assessment of existing interventions and the introduction of new modalities and treatment. Although there should be the ambition to do so, not all work will progress to, or be best addressed by, RCTs, and the mainstay may lie in stages 1, 2a, and 2b or in quasi-randomized or mixed-method designs. IDEAL-Physio may be particularly useful in facilitating the development of successful RCTs. IDEAL has been shown to be a feasible method for documenting the development and implementation of new procedures in patient care.53,54 Guided by IDEAL-Physio, major gains in physical therapy innovation could be achieved through the development of reporting systems (stage 1), protocol registries (stage 2a), comprehensive, disease-specific research databases (stages 2b and 3), and population registries (stage 4).28 We believe that physical therapy can be improved, and safe and efficient innovation in practice fostered, by the adoption of the IDEAL-Physio framework. Author Contributions Concept/idea/research design: D. Beard, D. Hamilton, J. Cook, A. Hirst, P. McCulloch, A. Paez Writing: D. Beard, A. Paez, D. Hamilton, J. Cook, A. Hirst, P. McCulloch Data collection: A. Paez Project management: D. Beard, A. Paez Clerical/secretarial support: A. Paez Consultation (including review of manuscript before submitting): D. Beard, A. Paez, D. Hamilton, L. Davies, A. Hirst, P. McCulloch Disclosure The authors completed the ICJME Form for Disclosure of Potential Conflicts of Interest and reported no conflicts of interest.
A. Hirst is employed by the University of Oxford as Project Manager/Researcher for the IDEAL Collaboration to develop research methods in complex interventions in surgery and in new devices. References 1 Miller PA, McKibbon KA, Haynes RB. A quantitative analysis of research. Phys Ther . 2003; 83: 123– 131. Google Scholar PubMed  2 Turner PAT, Trobe L, Whitfield TW. Physiotherapists’ use of evidence based practice: a cross-national study. Physiother Res Int . 1997; 2: 17– 29. Google Scholar CrossRef Search ADS PubMed  3 Scurlock-Evans L, Upton P, Upton D. Evidence-based practice in physiotherapy: a systematic review of barriers, enablers and interventions. Physiotherapy . 2014; 100: 208– 219. Google Scholar CrossRef Search ADS PubMed  4 Metcalfe C, Lewin R, Wisher S, Perry S, Bannigan K, Moffett JK. Barriers to implementing the evidence base in four NHS therapies. Physiotherapy . 2001; 87: 433– 441. Google Scholar CrossRef Search ADS   5 Maher CG, Sherrington C, Elkins M, Herbert RD, Moseley AM. Challenges for evidence-based physical therapy: accessing and interpreting high-quality evidence on therapy. Phys Ther . 2004; 84: 644– 654. Google Scholar PubMed  6 Sedrakyan A, Campbell B, Merino JG, Kuntz R, Hirst A, McCulloch P. IDEAL-D: a rational framework for evaluating and regulating the use of medical devices. BMJ . 2016; 353: i2372. Google Scholar CrossRef Search ADS PubMed  7 Bracken-Roche D, Bell E, Karpowicz L, Racine E. Disclosure, consent, and the exercise of patient autonomy in surgical innovation: a systematic content analysis of the conceptual literature. Account Res . 2014; 21: 331– 352. Google Scholar CrossRef Search ADS PubMed  8 Johnson J, Rogers W, Lotz M, Townley C, Meyerson D, Tomossy G. Ethical challenges of innovative surgery: a response to the IDEAL recommendations. Lancet . 2010; 376: 1113– 1115. Google Scholar CrossRef Search ADS PubMed  9 Brower V. The ethics of innovation: should innovative surgery be exempt from clinical trials and regulations? EMBO Rep . 2003; 4: 338– 340. Google Scholar CrossRef Search ADS PubMed  10 Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ . 1996; 312: 71– 72. Google Scholar CrossRef Search ADS PubMed  11 Dannapfel P, Peolsson A, Nilsen P. What supports physiotherapists’ use of research in clinical practice? A qualitative study in Sweden. Implement Sci . 2013; 8: 1– 13. Google Scholar CrossRef Search ADS PubMed  12 Kunz R, Autti-Rämö I, Anttila H, Malmivaara A, Mäkelä M. A systematic review finds that methodological quality is better than its reputation but can be improved in physiotherapy trials in childhood cerebral palsy. J Clin Epidemiol . 2006; 59: 1239. e1–1239.e12. Google Scholar CrossRef Search ADS PubMed  13 Moulding NT, Silagy CA, Weller DP. A framework for effective management of change in clinical practice: dissemination and implementation of clinical practice guidelines. Qual Health Care . 1999; 8: 177– 183. Google Scholar CrossRef Search ADS PubMed  14 Paci M, Briganti G, Lombardi B. Levels of evidence of articles published in physical and rehabilitation medicine journals. J Rehabil Med . 2011; 43: 264– 267. Google Scholar CrossRef Search ADS PubMed  15 American Physical Therapy Association. Vision statement for the physical therapy profession and guiding principles to achieve the vision. 16 American Physical Therapy Association website. http://www.apta.org/Vision/ 17 Delitto A. What will it take? The need for large-scale trials. Phys Ther . 2005; 85: 466. Google Scholar PubMed  18 Paci M, Landi N, Marchettini M, Baccini M. Randomized controlled trial quality in pediatric physical therapy. Phys Occup Ther Pediatr . 2014; 34: 260– 270. Google Scholar CrossRef Search ADS PubMed  19 Kumar SP. Professional discussion. Physical therapy: past, present and future—a paradigm shift. J Phys Ther . 2010; 1: 58– 67. 20 Musculoskeletal Health Sydney, School of Public Health, University of Sydney. Physiotherapy Evidence Database. http://www.pedro.fhs.usyd.edu.au. Last updated September 4, 2017. Accessed September 12, 2017. 21 Tashakkori A, Creswell JW. Editorial: the new era of mixed methods. Journal of Mixed Methods Research . 2007; 1: 3– 7. Google Scholar CrossRef Search ADS   22 Rauscher L, Greenfield BH. Advancements in contemporary physical therapy research: use of mixed methods designs. Phys Ther . 2009; 89: 91– 100. Google Scholar CrossRef Search ADS PubMed  23 Deyle GD, Allison SC, Matekel RL et al.   Physical therapy treatment effectiveness for osteoarthritis of the knee: a randomized comparison of supervised clinical exercise and manual therapy procedures versus a home exercise program. Phys Ther . 2005; 85: 1301– 1317. Google Scholar PubMed  24 Bartlett DJ, Macnab J, Macarthur C et al.   Advancing rehabilitation research: an interactionist perspective to guide question and design. Disabil Rehabil . 2006; 28: 1169– 1176. Google Scholar CrossRef Search ADS PubMed  25 Bohannon RW, LeVeau BF. Clinicians’ use of research findings: a review of the literature with implications for physical therapists. Phys Ther . 1986; 66: 45– 50. Google Scholar CrossRef Search ADS PubMed  26 Jette DU, Bacon K, Batty C et al.   Evidence-based practice: beliefs, attitudes, knowledge, and behaviors of physical therapists. Phys Ther . 2003; 83: 786– 805. Google Scholar PubMed  27 McCulloch P, Cook JA, Altman DG, Heneghan C, Diener MK IDEAL Group. IDEAL framework for surgical innovation, 1: the idea and development stages. BMJ . 2013; 346: f3012. Google Scholar CrossRef Search ADS PubMed  28 Sheill A, Hawe P, Gold L. Complex interventions or complex systems? Implications for health economic evaluation. BMJ . 2008; 336: 1281– 1283. Google Scholar CrossRef Search ADS PubMed  29 Mcculloch P, Altman DG, Campbell WB et al.   No surgical innovation without evaluation: the IDEAL recommendations. Lancet . 2009; 374: 1105– 1112. Google Scholar CrossRef Search ADS PubMed  30 Barkun JS, Aronson JK, Feldman LS et al.   Evaluation and stages of surgical innovations. Lancet . 2009; 374: 1089– 1096. Google Scholar CrossRef Search ADS PubMed  31 Ergina PL, Cook JA, Blazeby JM et al.   Challenges in evaluating surgical innovation. Lancet . 2009; 374: 1097– 1104. Google Scholar CrossRef Search ADS PubMed  32 American Physical Therapy Association. Guide to physical therapist practice 3.0. http://guidetoptpractice.apta.org/. Published 2014. Last updated November 27, 2016. Accessed September 12, 2017. 33 Moher D, Hopewell S, Schulz KF et al.   CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Int J Surg . 2012; 10: 28– 55. Google Scholar CrossRef Search ADS PubMed  34 von Elm E, Altman DG, Egger M et al.   The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. J Clin Epidemiol . 2008; 61: 344– 349. Google Scholar CrossRef Search ADS PubMed  35 Mintken PE, DeRosa C, Little T, Smith B American Academy of Orthopaedic Manual Physical Therapists. A model for standardizing manipulation terminology in physical therapy practice. J Orthop Sports Phys Ther . 2008; 38: A1– A6. Google Scholar CrossRef Search ADS PubMed  36 Jette AM. Toward a common language for function, disability, and health. Phys Ther . 2006; 86: 726– 734. Google Scholar CrossRef Search ADS PubMed  37 COMET Initiative. http://www.comet-initiative.org. Accessed September 12, 2017. 38 Gargon E, Williamson PR, Altman DG, Blazeby JM, Clarke M. The COMET Initiative database: progress and activities from 2011 to 2013. Trials . 2014; 15: 279. Google Scholar CrossRef Search ADS PubMed  39 Dijkers MP, Murphy SL, Krellman J. Evidence-based practice for rehabilitation professionals: concepts and controversies. Arch Phys Med Rehabil . 2012; 93( 8 suppl): S164– S176. Google Scholar CrossRef Search ADS PubMed  40 Whyte J. Traumatic brain injury rehabilitation: are there alternatives to randomized clinical trials? Arch Phys Med Rehabil . 2002; 83: 1320– 1322. Google Scholar CrossRef Search ADS PubMed  41 Johnson R, Waterfield J. Making words count: the value of qualitative research. Physiother Res Int.  2004; 9: 121– 131. Google Scholar CrossRef Search ADS PubMed  42 Hammell KW, Carpenter C, Dyck I. Using Qualitative Research: A Practical Introduction for Occupational and Physical Therapists . Edinburgh: Churchill Livingstone; 2000: 59– 71. 43 Peters DJ. Qualitative inquiry: expanding rehabilitation medicine's research repetoire—a commentary. Am J Phys Med Rehabil.  1996; 75: 144– 148. Google Scholar CrossRef Search ADS PubMed  44 Bjorn M, Brendstrup C, Karlsen S, Carlsen JE. Consecutive screening and enrollment in clinical trials: the way to representative patient samples? J Card Fail . 1998; 4: 225– 230. Google Scholar CrossRef Search ADS PubMed  45 Hutchison AM, Pallister I, Evans RM et al.   Intense pulsed light treatment of chronic mid-body Achilles tendinopathy: a double blind randomised placebo-controlled trial. Bone Joint J . 2013; 95-B: 504– 509. Google Scholar CrossRef Search ADS PubMed  46 Reddy M, Gill SS, Rochon PA. Clinician's corner: preventing pressure ulcers. JAMA . 2006; 296: 974– 984. Google Scholar CrossRef Search ADS PubMed  47 Chung J, Evans J, Lee C et al.   Effectiveness of adaptive seating on sitting posture and postural control in children with cerebral palsy. Pediatr Phys Ther . 2008; 20: 303– 317. Google Scholar CrossRef Search ADS PubMed  48 Cassel J, Cassel R, Down F et al.   Best practice guidelines: wheelchair provision for children and adults with muscular dystrophy and other neuromuscular conditions. Muscular Dystrophy Campaign. http://www.musculardystrophyuk.org/app/uploads/2015/02/wheelchair-guidelines.pdf. Published March 2011. Accessed September 12, 2017. 49 American Physical Therapy Association. Physical therapists’ role in prevention, wellness, fitness, health promotion and management of disease and disability. HOD P06-16-06-05. http://www.apta.org/uploadedFiles/APTAorg/About_Us/Policies/Practice/PTRoleAdvocacy.pdf. Last updated September 2, 2016. Accessed September 12, 2017. 50 Warburton DER, Nicol CW, Bredin SSD. Health benefits of physical activity: the evidence. Can Med Assoc J.  2006; 174: 801– 809. Google Scholar CrossRef Search ADS   51 DiMatteo MR. Social support and patient adherence to medical treatment: a meta-analysis. Health Psychol . 2004; 23: 207– 218. Google Scholar CrossRef Search ADS PubMed  52 Stewart MA. Effective physician-patient communication and health outcomes: a review. Can Med Assoc J . 1995; 152: 1423– 1433. 53 Siminoff LA. Incorporating patient and family preferences into evidence-based medicine. BMC Medical Informatics and Decision Making . 2013; 13( suppl 3): S6. Google Scholar CrossRef Search ADS PubMed  54 Blazeby JM, Blencowe NS, Titcomb DR, Metcalfe C, Hollowood AD, Barham CP. Demonstration of the IDEAL recommendations for evaluating and reporting surgical innovation in minimally invasive oesophagectomy. Br J Surg . 2011; 98: 544– 551. Google Scholar CrossRef Search ADS PubMed  55 Manca A, Cabboi MP, Ortu E et al.   Effect of contralateral strength training on muscle weakness in people with multiple sclerosis: proof-of-concept case series. Phys Ther . 2016; 96: 828– 838. Google Scholar CrossRef Search ADS PubMed  56 Perme C, Nalty T, Winkelman C, Kenji Nawa R, Masud F. Safety and efficacy of mobility interventions in patients with femoral catheters in the ICU: a prospective observational study. Cardiopulm Phys Ther J . 2013; 24: 12– 17. Google Scholar PubMed  57 Donaldson C, Tallis R, Miller S, Sunderland A, Lemon R, Pomeroy V. Effects of conventional physical therapy and functional strength training on upper limb motor recovery after stroke: a randomized phase II study. Neurorehabil Neural Repair . 2009; 23: 389– 397. Google Scholar CrossRef Search ADS PubMed  58 Wolf SL, Winstein CJ, Miller JP et al.   Effect of constraint-induced movement therapy on upper extremity function 3 to 9 months after stroke: the EXCITE randomized clinical trial. JAMA . 2006; 296: 2095– 2104. Google Scholar CrossRef Search ADS PubMed  59 Wolf SL, Winstein CJ, Miller JP et al.   Retention of upper limb function in stroke survivors who have received constraint-induced movement therapy: the EXCITE randomised trial. Lancet Neurol . 2008; 7: 33– 40. Google Scholar CrossRef Search ADS PubMed  Appendix 1. IDEAL-Physio Stage 1: Idea This stage includes first-in-human and proof-of-concept studies. Here, a first, public description of a new intervention or application of a concept is presented. Accuracy, transparency, and completeness of reporting and data collection are important. Reporting standards are developed and agreed upon. Goals and outcomes Explanation and description of the new intervention or application to a new population. Transparent, quality recording of initial data with standard outcomes. Definitions for key outcomes and factors are agreed upon and standardized. Identification of early signs, problems, and adverse effects. Safety and adherence or uptake are emphasized. Example: Manca et al55 Multiple sclerosis (MS) is a neurological condition characterized by reduced muscle and decreased ability to fully activate motor units in the lower limb muscles. Muscle weakness reduces the ability of people with MS to perform physical exercise, with a consequent decrease in the level of activities of daily living (ADL). Resistance training (RT) reduces muscle weakness, improves balance, and decreases self-reported fatigue during ADL in people with MS, improving quality of life. When strength impairment is lateralized to 1 limb, RT is conventionally addressed on the weaker side. However, this approach may not be applicable to a severely weakened limb that is too compromised to sustain it. Contralateral strength training (CST) may represent a viable alternative to the conventional direct approach. CST has been well studied in people who are healthy and in people with orthopedic conditions, but has been poorly addressed in people with neurological disorders. Design: Proof of concept Staged program of multidisciplinary research beginning with detailed preclinical studies and literature review. Explicit clinical and data collection protocol written in advance and submitted for ethical review, with detailed informed consent process. A series of replicated single-system case studies completed using a pretest-posttest design with a short-term follow-up and repeated recording of outcome measures. Findings After intervention, the contralateral limb shows an increase in maximal strength and improvements in mobility and walking performance. No adverse events identified in response to the intervention. Authors believe that CST is a promising approach for addressing unilateral muscle weakness in people with neurological impairment and that the new treatment protocol is feasible in the health care system. Proof of concept is shown. Further study is warranted, and important factors needing further exploration for the next phase are identified. Appendix 2. Standardization and Contextual Factors in IDEAL-Physio Stage 2a: Developmenta In this stage, the early evaluation in the development stage (2a) will use small observational studies without contemporaneous comparison groups in highly selected cohorts of patients. Pilot data are being collected. Terminology is being standardized for the intervention. Judgment about the success or failure of the intervention is based on short-term outcomes that might not reflect the important effects of the intervention. Prospective rather than retrospective studies are preferred for greater validity and reliability. Data may be insufficient for meaningful statistical analysis, although trends may be identified. Online registries could facilitate data recording. Operator learning curves and aspects of the intervention requiring further description and standardization are being identified. Key elements of stage 2a studies Prior, clearly defined protocol. Clearly defined, objective outcomes. Transparent, sequential reporting of cases. Clear, detailed description of when changes to the intervention are made, whether indication or technique. Continued emphasis on transparency and safety. Continued standardization of terminology. Example: Perme et al56 Limited research exists on the safety of mobility activities for patients with femoral catheters. Most intensive care units (ICUs) recommend that patients with femoral catheters remain immobilized, in bed. This approach might expose those patients to unnecessary adverse effects of bed rest. There is a clinical need for investigating the feasibility and safety of mobilizing. Patients in ICUs with 1 or more femoral catheters. A prospective, observational study in a cardiovascular surgical ICU at a large metropolitan teaching hospital from 2009 to 2010. No adverse events occurred during a physical therapy session; evidence for safety was reported. Physical therapy sessions including standing and walking were feasible and safe in patients in a cardiovascular ICU who had femoral catheters and who met the criteria for mobility interventions. The data from this study provided new information to clinicians and researchers about feasibility and safety regarding sitting, standing, and walking activities in adults who are critically ill. Recommendations are made for larger, future studies, including a larger cohort, a longer observation period following mobility activities, unit-based guidelines for initiation and progression of activities, and greater standardization. aAdapted from McCulloch et al (2013).26 Appendix 3. Recommendations for Observational Studies at Stage 2b: Exploration By this stage, promising evidence of the intervention's safety and beneficial, short-term outcomes will have been generated (in stages 1 and 2a), or further development would have been halted. Exploratory studies collect prospective data from several sites or multiple therapists, contributing to the body of evidence supporting or refuting a practice. These include protocol-driven observational studies and exploratory studies, such as phase 2 trials, often with a cohort design, with or without a contemporaneous comparison group(s). The exploration stage (2b) offers the opportunity to obtain higher-quality evidence in a more representative patient population and to deal with factors that could hinder the conduct of a proper methodological evaluation in the next stages. Outcomes Standardized outcomes are important. Universally accepted, validated outcome measures for pathology-specific problems, general health evaluation, and therapy-specific interventions. Reported consistently, perhaps with a minimal data set. Goals Identify and carefully document intervention benefits and adverse outcomes. Explore clinicians’ experience of the intervention and the requirements for its application. Determine whether the intervention may be worthy of more rigorous, formal evaluation. Identify issues in need of further exploration, within intervention and among operators. Example: Donaldson et al57 This study was an observer-masked, randomized, controlled, phase 2 trial. A random-allocation order was generated using a computer program. An exploratory trial was designed as a precursor to a definitive phase 3 trial. Thirty participants in this phase 2 trial (10 per treatment arm). Results would be interpreted cautiously in this phase 2 study, as it was underpowered, but authors sought to determine whether the intervention was worthy of further study. Intervention was compared to best practice or conventional physical therapy (CPT). Three study arms: CPT, CPT plus enhanced-intensity CPT, and CPT plus functional strength training (FST). Observer and participant masking to reduce bias. Standardized treatment schedules and standardized outcome measures were used: Action Research Arm Test (ARAT), Nine-Hole Peg Test, and upper limb strength measured by myometer. Measures were taken before intervention and 6 and 12 weeks after intervention. Results and conclusion: the intervention was safe, may be beneficial, and was worthy of further study. Outcome: CPT + FST group showed the greatest increases in ARAT and other scores, above the clinically important level for tests. Benefit in function was identified. No adverse events or outcomes were identified. Authors concluded that further work toward a phase 3 clinical trial appeared justifiable. Authors calculated the estimated sample size for an adequately powered subsequent phase 3 trial and identified important factors needing further exploration for phase 3. Appendix 4. Considerations for Randomized Studies at Phase 3: Assessmenta In stage 3, full evaluation of efficacy is taking place. Formal, comparative studies—always prefaced by systematic review of the literature and, if possible, meta-analysis—explore the state of the evidence for the intervention or the condition: randomized controlled trial, if possible, with direct and/or indirect comparisons between intervention and no treatment, best practice, or placebo; and quasi-randomized or observational designs when randomized trials are not feasible. Examples of these include nonrandomized and interrupted time series studies. Interrupted time series (a quasi-experimental design) use a temporal (or sequential) rather than a concurrent control group. In this stage: Quality documentation of efficacy and data sets with which to justify continued use and further assessment. Stage 3 requires larger, more authoritative bodies of evidence for continuing practice or desisting. Studies should be designed to provide definitive evaluation of the intervention at this stage. Reporting guidelines, such as CONSORT or STROBE, can guide recording and reporting of data. Outcomes registries can be helpful for reporting outcomes. Example: Wolf et al58 Previous, single-site studies suggested that a 2-week program of constraint-induced movement therapy (CIMT) for patients more than 1 year after stroke can improve upper extremity function that persists for at least 1 year. The EXCITE trial, a prospective, single-blind, randomized, multisite clinical trial, was conducted at 7 US academic institutions between January 2001 and January 2003. Two hundred twenty-two individuals with predominantly ischemic stroke participated. The control group received “usual and customary care (standard care).” The intervention group received CIMT, including shaping (adaptive task practice) and standard task training of the paretic limb for up to 6 hours daily over 2 weeks. Participants in the control group were offered the same CIMT regimen after the 12-month evaluation session. Outcomes and interventions were standardized, validated, and reported consistently. The Wolf Motor Function Test (WMFT), a measure of laboratory time and strength-based ability and quality of movement (functional ability), and the Motor Activity Log, a measure of how well and how often 30 common daily activities are performed, were used. Follow-up took place at 1 year after intervention. The results indicated that CIMT produced statistically significant and clinically relevant improvements in arm motor function that persisted for at least 1 year. Further research on CIMT, including alternate models of CIMT, is warranted. aAdapted from McCulloch et al (2013).26 CONSORT = Consolidated Standards of Reporting Trials, STROBE = Strengthening the Reporting of Observational Studies in Epidemiology. Appendix 5. IDEAL-Physio Stage 4: Long-Term Follow-Upa In this stage, interventions have become established by previous stages and are now being assessed for long-term outcomes, rare events, variations in outcomes, and any influence of biopsychosocial patient and practitioner factors on long-term outcomes. Large numbers of cases (patients treated with the intervention) may now be available, adding to the ability to assess the impact of comorbidities and trends over time. Type of study: Observational studies, phase 4 trials, randomized controlled trials with a long follow-up arm, and mixed-method and qualitative studies for monitoring of patients’ responses and ability to carry through with prescribed interventions and long-term influence on activity and participation. Long-term evaluations of both efficacy and risk are taking place and may help reveal effects of interventions not immediately evident and beyond the immediate effect on the target impairment or condition. Registries, American Physical Therapy Association, National Institute for Health Research, and National Institutes of Health can be helpful for recording data and modeling new registries of outcomes. Example: Wolf et al59 A previous study by Wolf et al58 in this series had follow-up at 1 year after intervention. The results indicated that constraint-induced movement therapy (CIMT) produced statistically significant and clinically relevant improvements in arm motor function that persisted for at least 1 year. The authors concluded that further research on CIMT, including a longer follow-up, was warranted. In the current study, the authors assessed the retention of improvements from the EXCITE CIMT intervention through 24 months. The authors reported: “There was no observed regression from the treatment effects observed at 12 months after treatment during the next 12 months for the primary outcome measures of WMFT and MAL. In fact, the additional changes were in the direction of increased therapeutic effect.”59 The authors concluded that the intervention (CIMT) had persistent benefits. Additional improvements in strength, performance of ADL/IADL, and Social Participation domains, not noted at the initial 12-month follow-up in the previous study, were now apparent. Long-term follow-up in clinical trials in rehabilitation may help reveal the full extent of effects of therapeutic interventions, beyond the immediate impact of the interventions. Biopsychosocial and other factors influencing outcomes may be elucidated in long-term follow-up studies. aADL = activities of daily living, IADL = instrumental ADL, MAL = Motor Activity Log, WMFT = Wolf Motor Function Test. Table. Five Stages of IDEAL-Physio Frameworka Stage 1: Idea  Stage 2a: Development  Stage 2b: Exploration  Stage 3: Assessment  Stage 4: Long-Term Follow-Up  First-in-human studies and proof-of-concept studies  Prospective development studies including mixed-method and qualitative data  Exploratory prospective observational studies, often with a cohort design  Full evaluation of efficacy; formal, comparative studies including RCTs and observational designs  Longitudinal observational studies  Focus on explanation and description of the new intervention  Continued emphasis on structured data collection and quality recording  No necessity for comparison group(s)  Prefaced by systematic review/meta-analysis; RCT if possible  Can be longer-term follow-up of RCTs or mixed-method and qualitative studies  Focus on safety, with reporting of early adverse events  Iterative development  Data from several sites or therapists  Direct and/or indirect comparisons between intervention and no Rx, best practice, or placebo  Influence of biopsychosocial factors and pragmatic aspects  Transparent, quality recording of initial data with standard outcomes  Description of changes in treatment  Data collected systematically from consecutive patients  Reporting guidelines, CONSORT, or STROBE  Registries, ATPA, National Institute for Health Research, and National Institutes of Health helpful  Determination of whether the intervention is efficacious and worthy of further study  Detailed recording of patients’ responses to intervention  Recording of clinicians’ experience          Use of refined outcome measures      Stage 1: Idea  Stage 2a: Development  Stage 2b: Exploration  Stage 3: Assessment  Stage 4: Long-Term Follow-Up  First-in-human studies and proof-of-concept studies  Prospective development studies including mixed-method and qualitative data  Exploratory prospective observational studies, often with a cohort design  Full evaluation of efficacy; formal, comparative studies including RCTs and observational designs  Longitudinal observational studies  Focus on explanation and description of the new intervention  Continued emphasis on structured data collection and quality recording  No necessity for comparison group(s)  Prefaced by systematic review/meta-analysis; RCT if possible  Can be longer-term follow-up of RCTs or mixed-method and qualitative studies  Focus on safety, with reporting of early adverse events  Iterative development  Data from several sites or therapists  Direct and/or indirect comparisons between intervention and no Rx, best practice, or placebo  Influence of biopsychosocial factors and pragmatic aspects  Transparent, quality recording of initial data with standard outcomes  Description of changes in treatment  Data collected systematically from consecutive patients  Reporting guidelines, CONSORT, or STROBE  Registries, ATPA, National Institute for Health Research, and National Institutes of Health helpful  Determination of whether the intervention is efficacious and worthy of further study  Detailed recording of patients’ responses to intervention  Recording of clinicians’ experience          Use of refined outcome measures      aModified from McCulloch et al (2013).26 APTA = American Physical Therapy Association, CONSORT = Consolidated Standards of Reporting Trials, 
RCT = randomized controlled trial, Rx = prescription, STROBE = Strengthening the Reporting of Observational Studies in Epidemiology. View Large © 2017 American Physical Therapy Association

Journal

Physical TherapyOxford University Press

Published: Feb 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 12 million articles from more than
10,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Unlimited reading

Read as many articles as you need. Full articles with original layout, charts and figures. Read online, from anywhere.

Stay up to date

Keep up with your field with Personalized Recommendations and Follow Journals to get automatic updates.

Organize your research

It’s easy to organize your research with our built-in tools.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

Monthly Plan

  • Read unlimited articles
  • Personalized recommendations
  • No expiration
  • Print 20 pages per month
  • 20% off on PDF purchases
  • Organize your research
  • Get updates on your journals and topic searches

$49/month

Start Free Trial

14-day Free Trial

Best Deal — 39% off

Annual Plan

  • All the features of the Professional Plan, but for 39% off!
  • Billed annually
  • No expiration
  • For the normal price of 10 articles elsewhere, you get one full year of unlimited access to articles.

$588

$360/year

billed annually
Start Free Trial

14-day Free Trial