Background: Randomised controlled clinical trials typically have a relatively brief in-trial follow-up period which can underestimate safety signals and fail to detect long-term hazards, which may take years to appear. Extended follow-up after the scheduled closure of the trial allows detection of both persistent or enhanced beneficial effects following cessation of study treatment (i.e. a legacy effect) and the emergence of possible adverse effects (e.g. development of cancer). Methods: A systematic review was conducted following PRISMA guidelines to qualitatively compare post-trial follow-up methods used in large randomised controlled trials. Five bibliographic databases, including Medline and the Cochrane Library, and one trial registry were searched. All large randomised controlled trials (more than 1000 adult participants) published from March 2006 to April 2017 were evaluated. Two reviewers screened and extracted data attaining > 95% concordance of papers checked. Assessment of bias in the trials was evaluated using the Cochrane Risk of Bias tool. Results: Fifty-seven thousand three hundred and fifty-two papers were identified and 65 trials which had post-trial follow-up (PTFU) were included in the analysis. The majority of trials used more than one type of follow-up. There was no evidence of an association between the retention rates of participants in the PTFU period and the type of follow-up used. Costs of PTFU varied widely with data linkage being the most economical. It was not possible to assess associations between risk of bias during the in-trial period and proportions lost to follow-up during the PTFU period. Discussion: Data captured during the post-trial follow-up period can add scientific value to a trial. However, there are logistical and financial barriers to overcome. Where available, data linkage via electronic registries and records is a cost- effective method which can provide data on a range of endpoints. Systematic review registration: Not applicable for PROSPERO registration. Keywords: Methodology, Post-trial, Retention, Randomised controlled trial, Cost, Long-term, Follow-up, Effective Background is important as persistent effects may be detected years Randomised controlled trials (RCTs) are considered to later after treatment cessation or even enhanced benefits be the ‘gold standard‘ for assessing the effects of a treat- observed decades later – a so-called ‘legacy effect‘ [1, 2]. ment. However, these trials usually report results follow- Furthermore, delayed hazards may only emerge several ing a relatively brief exposure to the intervention under years after exposure to certain treatments. Therefore, investigation. Longer-term follow-up of trial participants PTFU may add significant scientific value to the evalu- ation of many healthcare interventions. We define post-trial follow-up (PTFU) as extended * Correspondence: email@example.com follow-up which starts after the end of the scheduled MRC Population Health Research Unit, Clinical Trial Service Unit (CTSU), Nuffield Department of Population Health, University of Oxford, Richard Doll period of the trial. Such follow-up, regardless of the pri- Building, Roosevelt Drive, Oxford OX3 7LF, UK mary in-trial outcome, provides important information Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Llewellyn-Bennett et al. Trials (2018) 19:298 Page 2 of 12 including safety of the intervention, identification of de- provided in Additional file 2. In addition, a database layed hazards and long-term beneficial effects. search for completed and ongoing studies was con- Retention of participants in PTFU is important since ducted at ClinicalTrials.gov (https://clinicaltrials.gov/). high rates of attrition may introduce bias if reasons for Studies which were not yet published ‘grey literature’ withdrawal are related to the intervention . There are were not included in the search strategy. a variety of methods for PTFU, but little research has been done to evaluate which methods for PTFU leads to Data collection the best retention rates . Choice of follow-up method Papers identified from the ClinicalTrials.gov registry is often determined by the funding for the trial and the were imported into a MS Excel spreadsheet. Duplicates local availability of relevant data. Telephone calls, postal and studies which had less than 1000 participants were questionnaires and face-to face interviews are the more removed using a filter option. The selection of eligible traditional approach to follow-up. Web-based ap- papers followed a concordance strategy between two re- proaches and use of routine health records and elec- viewers (RLB and DE) which ensured that concordance tronic registries are becoming more popular due to was > 95% (Fig. 1). advancing technology and options for accessing the in- Medical interventions were defined as an intervention formation inexpensively [5, 6]. that was consumed orally, inhaled, or administered by This systematic review compares methods used in ap- intravenous or intramuscular injections including vac- proaches to PTFU and aims to inform the design of cines. A surgical intervention was defined as any inter- PTFU for a wide range of randomised trials. The main vention which was invasive (apart from those mentioned objective was to evaluate the retention rates (or levels of above and including blood transfusions). Potential stud- attrition) of the participants followed up during PTFU ies were checked for eligibility by two reviewers who ini- and to compare this to the type of methodology used. A tially reviewed abstracts and then proceeded to full secondary objective was to compare the costs of post- paper review in a step-wise process (Fig. 1). trial methodology as funding is often limited. In addition to those described in the protocol, some additional exclusions which were not originally listed Methods were identified during the process of performing the The methods used in this systematic review have been systematic review in keeping with our definition of described in detail previously  and follow PRISMA PTFU. This was required due to the heterogeneity of guidelines Additional file 1. PTFUs. These include: (1) trials that were stopped be- fore the scheduled closure of the trial; (2) cancer tri- Eligibility criteria als which had an open endpoint (e.g. survival as an Briefly, all large (> 1000 adult participants) RCTs which endpoint with no clear scheduled plan of duration); investigated a healthcare intervention (i.e. medicine, sur- (3) trials which continued with active intervention in gery or psychiatric in nature) and involved PTFU were the PTFU period with the primary outcome of safety included. Only studies published between 2006 and 2017 and (4) trials eligible for inclusion but which did not were included. Alternative medicines (e.g. acupuncture) contribute novel data as they only published add- or holistic interventions including physical therapy were itional subgroup or post-hoc analyses. A table of ex- excluded from the review. Large RCTs were only in- cluded trials is provided in Additional file 3. cluded due to the reduced risk of random error in the Full papers deemed eligible for inclusion in the sys- outcomes. tematic review were extracted using a standardised Excel PTFU was defined as passive follow-up which had oc- spreadsheet. Data was extracted by DE and RLB and curred either after the scheduled closure of the trial or concordance was checked. Primary outcome, healthcare after the primary results had been published. intervention and attrition rates were tabulated for each study. Lead trialists were contacted via email to inform Search strategy them of the systematic review and to clarify information The search was conducted in five bibliographic data- where necessary. The papers included in the review were bases on 13 April 2017, including Embase (OvidSP) (1 diverse with a range of interventions and different out- March 1974 to 12 April 2017), Medline (OvidSP) (1946 come measures. Due to the high level of clinical hetero- to present), PubMed, Cochrane Central Register of Con- geneity a meta-analysis was not possible. trolled Trials (Cochrane Library, Wiley) (issue 3 of 12, Retention rates were calculated as the proportion of March 2017) and Cochrane Methods Register (CEN- participants who were lost to follow-up compared to the TRAL) (Cochrane Library, Wiley) (issue 3 of 4, July overall total of those who started the PTFU period. In- 2017). Searches were then restricted to articles published formation about the cost of the PTFU was sought from in English since 2006. Full details of strategies are study publications or via personal communication. Two Llewellyn-Bennett et al. Trials (2018) 19:298 Page 3 of 12 Fig. 1 PRISMA flow diagram detailing the process of study selection and data extraction. HCI healthcare intervention, PTFU post-trial follow-up, RCT randomised controlled trial attempts were made to contact the trialist via email and Review Manager 5 (RevMan 5) for graphical represen- if there was no response or inadequate data, the trial tation . was excluded from the cost analysis. Results From 57,352 papers identified, 65 studies with PTFU Assessment of risk of bias were included in the systematic review (Fig. 1). Fifty Risk of bias was assessed for each included RCT on trials involved medical interventions and 15 involved their primary results using the Cochrane Risk of Bias surgical interventions. There were no eligible psychi- tool. Covdence.org was used to assess the levels of bias atric trials which had (all > 1000 participants). The (low risk, high risk or unclear risk) in each methodo- duration of PTFU ranged from 1 to 20 years, with a logical domain (sequence generation, allocation of se- median of 4.5 years of follow-up. The number of par- quence concealment, blinding, incomplete outcome ticipants followed during the post-trial period ranged data, selective reporting bias and other bias) and deci- from 575 to 29 862. sions checked by one of the senior authors . The Five methods of follow-up were identified: postal cor- data recorded from Covidence.org was imported into respondence/questionnaire (19%); clinic appointments Llewellyn-Bennett et al. Trials (2018) 19:298 Page 4 of 12 Table 1 Post-trial follow-up (PTFU) in eligible medical trials. Note retention of participants expressed as % lost to follow-up 1st author, Primary outcome RCT name No. Intervention No. No. at the % Type of PTFU for year for PTFU (PTFU name) years randomised start of participants primary outcome PTFU in-trial PTFU lost in PTFU Post/ Clinic Telephone Data Paper Other Q linkage records Alan, 2015 Mortality ProHOSP 6 CAP antibiotics 1359 925 6 Y Y Arbel, 2016 Mortality BIP 20 Bezafibrate 3090 3090 – Y Arber, 2011 Cancer, safety PreSAP 2 Celecoxib 1561 1043 12 Y Avenell, Mortality RECORD 3 Vitamin D, Calcium 5292 4394 – Y Breitner, Alzheimer’s ADAPT 2 Naproxen, 2528 2233 1 Y 2011 disease Celecoxib Bulbulia, Mortality and HPS 6 Simvastatin 20,536 17519 0 Y Y 2011 morbidity Cauley, 2013 Hip fractures, WHI 5 Calcium plus 36,282 29862 1 Y cancers, CVE vitamin D and mortality Cherry, 2014 Mortality, cancer ESPIRIT 12 Oestrogen 1017 1017 – Y Chew, 2013 Progression AREDS 5 Antioxidants 4757 3549 – YY Y of age-related macular degeneration Chowdhury, Diabetes mellitus, ANBP2 7 ACE inhibitor, 6083 5678 (6083 linked – YY 2014 mortality, MACE Thiazide to death registry) Cushman, MACE, mortality ALLHAT 13 Amlodipine, 32,804 17,722 (CVD), – Y 2012 lisinopril 27,755 (mortality) Dienstag, Progression HALT-C 4 Peginterferon 1050 743 – Y 2011 of Hep C Eastell, 2015 Bone mineral HORIZON-PFT 3 Zoledronic acid 7765 1223 – YY density Ebbing, Mortality NORVIT, WENBIT 4 B vitamins 6845 6261 0 Y Einstein, Safety, – 2 HPV vaccine 1106 671 0 Y – 2011 immunogenicity Erdmann, Mortality, MI, stroke, PROactive 3 Pioglitazone 5238 3599 9 Y Y Y Y 2014 MACE, (composite) Ezzedine, Skin cancer SU.VI.MAX 5 Antioxidant 12,741 11054 2 Y Y 2010 vitamins Flossman, Colorectal cancer UK-TIA 20 Aspirin 2449 2249 – YY Colorectal cancer BDAT 20 Aspirin 5139 5139 – YY Ford, 2016 Mortality and morbidity WOSCOPS 20 Pravastatin 6595 5778 – Y Llewellyn-Bennett et al. Trials (2018) 19:298 Page 5 of 12 Table 1 Post-trial follow-up (PTFU) in eligible medical trials. Note retention of participants expressed as % lost to follow-up (Continued) 1st author, Primary outcome RCT name No. Intervention No. No. at the % Type of PTFU for year for PTFU (PTFU name) years randomised start of participants primary outcome PTFU in-trial PTFU lost in PTFU Post/ Clinic Telephone Data Paper Other Q linkage records Gerstein, MACE, mortality ACCORD 3 Intensive glucose 10,251 8601 – YY 2016 (composite) (ACCORDIAN) control Gluud, 2008 Mortality CLARICOR 3 Clarithromycin 4373 4029 1 Y Gordon, Efficacy and REVEAL 2 Adalimumab 1212 575 7 Y 2012 safety Grau, 2009 Adenomas AFPPS 4 Aspirin 1121 1007 14 Y Y Grubb, 2013 Cancer REDUCE 2 Dutasteride 8231 2751 – YY Hackshaw, Event-free survival OVER 50S TRIAL 10 Tamoxifen 3449 3449 – Y Hague, 2016 Mortality, cancer LIPID 10 Pravastatin 9014 7721 0 Y Y Y Y Y Hayashino, Diabetes mellitus PHI1 17 Aspirin 22,071 22,071 – YY Hayward, MACE VADT 5 Intensive glucose 1791 1791 22 Y Y 2015 lowering vs standard therapy Holman, Macrovascular UKPDS 10 Intensive glycaemic 3867 3277 20 Y Y Y 2008 outcomes control Hornslien, Stroke, MI, mortality SCAST 3 Candesartan 2029 1286 2 Y Investigators, Diabetes mellitus DREAM 2 Rosiglitazone, 5269 1653 18 Y 2011 (DREAM ON) ramipril Johnson, Vaccine efficacy SPS (LTPS) 4 Vaccine 38,543 6867 6 Y Y Jones, 2015 Cancer, bone RECORD 4 Rosiglitazone 4447 2546 1 Y Y Y fractures Kostis, 2011 Mortality SHEP 13 Chlorthalidone 4736 –– Y Krane, 2016 MACE, mortality 4D 8 Atorvastatin 1255 637 3 Y (composite) Lai, 2014 Mortality, liver cancer ATBC 16 α-tocopherol, 29,133 29105 – Y β-carotene Laterre, 2007 Mortality ADDRESS 1 Drotrecogin-α 2640 2621 9 Y Y Y Leslie, 2011 Mortality ENIGMA 4 Nitrous oxide 2050 2002 17 Y Y Leslie, 2015 MACE, mortality ENIGMA-II 1 Nitrous oxide 7112 6651 12 Y Y Lewis, 2011 MACE CAIFOS 5 Calcium 1510 1510 – Y Llewellyn-Bennett et al. Trials (2018) 19:298 Page 6 of 12 Table 1 Post-trial follow-up (PTFU) in eligible medical trials. Note retention of participants expressed as % lost to follow-up (Continued) 1st author, Primary outcome RCT name No. Intervention No. No. at the % Type of PTFU for year for PTFU (PTFU name) years randomised start of participants primary outcome PTFU in-trial PTFU lost in PTFU Post/ Clinic Telephone Data Paper Other Q linkage records Lloyd, 2013 MACE, cancers, PROSPER 3 Pravastatin 5804 5188 – Y mortality Menne, 2014 Long-term micro, ROADMAP 3 Olmesartan 4449 2198 0 Y macrovascular (ROADMAP OFU) medoxomil benefit Ogihara, MACE, cancer, mortality CASE-J (CASE-J Ex) 3 Candesartan, 4728 2232 2 Y 2011 amlodipine Radford, Bone mineral Auckland Calcium 5 Calcium 1471 1408 17 Y Y 2014 density Study Rothwell, Colorectal cancer Thrombosis Prev Trial, 12, 13, 17, Aspirin 16,488 14033 – YY Y 2010 Swedish Aspirin Low 18, 20 Dose Trial, Dutch TIA Aspirin Trial, UK-Tia Aspirin Trial, British Doctors Aspirin Trial Tenkanen, MACE, cancer, Helsinki Heart Study 10 Gemfibrozil 4081 4081 0 Y Y 2006 mortality Wang, 2015 Fracture incidence NIT 16 Vitamins (14), 3318 3318 1 Y Y minerals (12) Weston, Persistence of 106316 3 Vaccine dip, 2284 1505 – Y 2011 antibodies pert, tetanus Whiteley, Disability IST-3 1 Alteplase 3035 2348 2 Y Zoungas, Mortality ADVANCE 6 Perindopril, 11,140 8494 – YY 2014 (ADVANCE-ON) indapamide where is number of years (median/mean/max) published in the cited paper, years followed up to the nearest whole number, % participants lost to the nearest whole number,‘–’ no data available or not applicable where mortality records were sought, CVD cardiovascular disease, MACE major adverse cardiovascular events ± revascularisation, MI myocardial infarction. Where 0 participants have been lost to follow-up this has been confirmed either in the cited paper or directly with the corresponding trialist Llewellyn-Bennett et al. Trials (2018) 19:298 Page 7 of 12 Table 2 Post-trial follow-up (PTFU) in eligible surgical trials. Note, retention of participants is expressed as % lost to follow-up 1st author, year Primary outcome for PTFU RCT name No. Intervention No. No. % Method of PTFU for primary (PTFU name) years participants participants participants outcome PTFU randomised at the start lost in PTFU Post Clinic Telephone Data Paper in trial of PTFU /Q linkage records Carson, 2015 Mortality FOCUS 3 Blood transfusion 2016 2002 – Y Cho 2017 Mortality, MI, stroke, revascularisation RISPO 4 RIPC, RIPostC 1328 1280 15 Y Y Y Gada, 2013 Safety, efficacy, mortality SPIRIT III 5 EES, PES 1002 –– Y Gallagher, 2014 Mortality RENAL 4 Renal replacement therapy 1508 1464 – Y (POST-RENAL) Halliday, 2010 Mortality, stroke ACST-1 4 CEA or deferement 3120 3120 – Y Henderson, 2015 Mortality RITA-3 5 PCI 1810 1810 0 Y Hirsch, 2007 Mortality, MACE ICTUS 4 PCI 1200 1124 3 Y Y Y Hochman, 2011 Mortality, MACE OAT 3 PCI 2201 1504 – YY Y Investigators, 2007 Mortality BARI 5 PTCA 1829 1829 4 Y Y Y Milojevic, 2016 Mortality SYNTAX 5 PCI 1800 847 – YY Y Naunheim, 2006 Mortality NETT 2 Lung-volume surgery 1218 70% – YY Patel, 2016 Mortality EVAR-1 13 EVAR 1252 1252 2 Y Y Y Powell, 2007 Mortality UKSAT 12 Early AAA repair 1090 1090 0 Y Sedlis, 2015 Mortality COURAGE 6 PCI 2287 1211 – Y Wallentein, 2016 Mortality, MI (composite) FRISC-II 15 PCI 2457 2421 1 Y Y where is number of years (median/mean/max) published in the cited paper, years followed up to the nearest whole number, PCI percutaneous coronary intervention ± revascularisation, PTCA percutaneous transluminal coronary balloon angioplasty, EES everolimus-eluting stents, PES paclitaxel-eluting stents, EVAR endovascular aneurysm repair, CEA carotid endarterectomy, AAA abdominal aortic aneurysm, RIPC remote ischaemic preconditioning, RIPostC RIPC with postconditioning, MI myocardial infarction, MACE major adverse cardiovascular events ± revascularisation, Postal/Q postal communication or questionnaire, years followed up to the nearest whole number, % participants lost to the nearest whole number, 70% provided by trialist. Where 0 participants have been lost to follow-up this has been confirmed either in the cited paper or directly with the corresponding trialist Llewellyn-Bennett et al. Trials (2018) 19:298 Page 8 of 12 (35%); telephone interviews (26%); electronic data link- apparent in Canada and for some North American par- age (52%); and review of paper medical records (26%). In ticipants (Medicare and Veteran Affairs) where a specific addition, in individual cases, specific follow-up was per- health ID number is required to access national data formed, e.g. endoscopy follow-up only . Electronic (Table 3). Trials experienced difficulty in accessing rou- data linkage and medical records review were used ex- tinely collected health records in 3% of included papers clusively together in 11% of papers; either were used in and PTFU was restricted to those countries with robust combination with other methods in 74%. Overall, 48% of and accessible centrally held records and registries (e.g. trials used more than one method to follow-up partici- Sweden and Scotland) [12, 13]. pants in the post-trial period (Tables 1 and 2). On aver- age, two methods were used for each PTFU follow-up. Retention rates Where data linkage was used, it was not always feasible Unfortunately, retention rates were often poorly re- to follow up all participants . Some trials experi- ported in the PTFU, limiting the ability to assess the im- enced difficulty accessing national electronic data in cer- pact of methods used in relation to the proportion lost tain countries; for example, stricter regulations are to follow-up. Table 3 Registries used for data linkage during post-trial follow-up (PTFU) Country Registry Dataset Website USA United States Renal Data System Renal www.usrds.org (USRDS) Centres for Medicare and Medicaid Non-fatal events www.cms.gov Services (CMS ([formerly HCFA)) National Death Index Plus Database Cause- specific mortality https://www.cdc.gov/nchs/ndi/ National Death Index and Social All-cause mortality https://www.cdc.gov/nchs/nvss/deaths.htm Security Administration The Central Veterans Affairs Medical All-cause morbidity https://www.va.gov/directory/guide/ Information files facility.asp?ID=5380 The Veterans Affairs Death Files All-cause mortality https://www.archives.gov/research/ alic/reference/vital-records.html Canada Statistics Canada Mortality Database All-cause mortality http://www23.statcan.gc.ca/imdb/ p2SV.pl?Function=getSurvey& SDDS=3233 England NHS Digital (formerly HSCIC and Office Non-fatal events, https://digital.nhs.uk/ of National Statistics) all-cause mortality Scotland Information and Statistical Division of the All-cause morbidity, http://www.isdscotland.org/ National Health Service for Scotland mortality (Scottish Morbidity Record, General Register Office Death Record) Israel Ministry of Health from the Israeli All-cause mortality https://www.health.gov.il/English/ Population Registry Pages/HomePage.aspx Israel National Cancer Registry Cancer https://www.health.gov.il/English/ MinistryUnits/HealthDivision/Icdc/ Icr/Pages/default.aspx Holland Dutch Central Bureau of Statistics All-cause mortality http://www.iamexpat.nl/expat-page/ official-issues/organisations/statistics- netherlands-cbs Norway Cardiovascular Disease in Norway Cause-specific https://cvdnor.b.uib.no/ (CVDNOR) project (for data < 2008) morbidity Finland Cause-of-Death Register All-cause mortality http://tilastokeskus.fi/til/ksyyt/ (Statistics Finland) index_en.html Population Register Centre Demographics http://vrk.fi/en/frontpage Finnish Cancer Registry Cancer http://www.cancer.fi/ syoparekisteri/en/ Australia Western Australia Data Linkage Non-fatal events, http://www.datalinkage-wa.org.au/ System (WADLS) all-cause mortality a b Data only available for those with a valid Medicare or Social Security number (65% of all participants in the ALLHAT long-term follow-up), Registry linkage in Norway only available from 2008, A personal identification number issued to each Finnish resident accesses demographic and medical records Llewellyn-Bennett et al. Trials (2018) 19:298 Page 9 of 12 All surgical trials investigated mortality as the primary due to the use of mortality records where only notifica- outcome and, where data was available, the proportion of tions of deaths were fed back to the trialists. participants lost to follow-up in surgical trials ranged from 0.4 to 15.5%. However, data was not available for 53% of Cost surgical trials. In medical trials, the primary outcome in- Financial information was available for one third of vestigated varied more widely, although mortality as an the included trials. Consequently, it was not possible endpoint was common and the proportion of participants to provide a direct comparison between cost of PTFU lost to follow-up ranged from 0 to 22%. Data on loss to and the different methodologies used due to the small follow-up was not available in 44% of trials. Where sample size. The cost of PTFU ranged from £6000 to mortality was the primary outcome, the number of partici- £14,600,000 (Table 4). Cost of PTFU per participant pants lost to follow-up was not available in 32% of trials per year showed that IST-3 was the most economical Table 4 Comparing post-trial follow-up (PTFU) costs (where disclosed), by different follow-up methodologies Type of follow-up, Number of Duration Incentive for participant Cost of PTFU/grant received Cost per name of participants of PTFU* follow-up participant RCT or PTFU in PTFU per year Clinical appointment only ROADMAP 2198 3.3 Travel reimbursement €20 per visit €3,000,000 €413.60 Clinical appointment + telephone LTPS 6867 4 – US$14,600,000 US$531.53 Data linkage/medical records only RECORD 4394 3 No £6,000 £0.46 FOCUS 2002 3 No US$75,000 US$12.49 NORWIT, WENBIT 6261 4 Letters sent to offer NOK 16,000 NOK 0.64 withdrawal from PTFU (registry follow-up) RENAL 1464 4 No Undisclosed – original recruiting sites paid for finding and contacting participants CLARICOR 4029 3 – £1,100,000 £91.01 ‘Over 50s’ 3449 10 no £14,000 £0.41 RITA-3 1810 5 – £359,577 £39.73 SCAST 1286 3 no £7,000 €1.81 CAIFOS 1510 4.5 no AUD 848,206 AUD 124.83 IST-3 2348 1 no £500 £0.21 Telephone + data linkage/medical records ProHOSP 925 6 no Negligible. Students conducted telephone – follow-up as part of their training OAT 1504 3 no US$100 administrative start-up, US$50 per call – for each follow-up, US$30 per subject for re-consent payment, US$300 per event completing reporting ENIGMA 2002 3.5 no AUD 53,807 AUD 7.68 ENIGMA-II 6651 1 no AUD 60,000 AUD 9.02 Postal correspondence + data linkage/medical records HPS 17,519 6 – £250,000 £2.38 ANBP2 6983 6.9 no AUD 18,000 AUD 0.37 ACST-1 3120 4 – £120,000 £9.62 VADT 1791 5 US$10 per survey gift card US$10,00,000 US$111.67 Postal correspondence +telephone + medical records ADDRESS 2621 1 no US$13,10,500 US$500 where ; median/max/range published in the cited paper, RCT randomised controlled trial, PTFU post-trial follow-up, NOK Norwegian Krone, AUD Australian dollar; ‘-’ no data available/ declined by corresponding trialist, ‘~‘ ,estimate; + RCT number as PTFU data not available. Results to 2 decimal places for cost per participant Llewellyn-Bennett et al. Trials (2018) 19:298 Page 10 of 12 Fig. 2 Cochrane Risk of Bias graph. Review authors’ judgements about each risk of bias item presented as percentages across all included studies costing £0.21 per participant per year using data all domains was found in 43 (74%) of those assessed. linkage/medical records, closely followed by ‘Over Only seven trials (12%) had at least one domain which 50s’ (£0.41) and RECORD trials (£0.46) which also was high risk of bias, of which one had two domains at used data linkage. LTPS was the most expensive high risk (Fig. 2). Details of the individual risk of bias PTFU per participant per year (US$531.53) using clin- domains for each included study are provided in ical appointments and telephone follow-up. ROAD- Additional file 4. MAP which also followed up participants by clinic Given the small number of trials found to have a high appointment only had a cost of €413.60 per partici- risk of bias in at least one domain and the highly vari- pant per year. able retention rates observed during PTFU (Table 5), it is not possible to draw any clear conclusions with Cochrane Risk of Bias respect to possible associations between risk of bias and We hypothesised that those RCTs which had poor its potential impact on the proportion of participants methodology or ‘high risk of bias’ might subse- that were lost to follow-up in the post-trial period. quently have a PTFU which was poorly organised and, therefore, have low retention rates (or a high proportion lost to PTFU). Of the 65 papers included Discussion in the systematic review, seven were excluded from This systematic review identified that PTFU methods the risk of bias assessment: these were PTFUs which varied and many trials used overlapping approaches followed-up an amalgamation of data from more which were more costly than needed. Data was limited than one trial or were part of a systematic review on retention rates and so it was difficult to draw any and, therefore, not suitable to be included in the firm conclusions on which method was best for PTFU. analysis (the risks of bias from individual component Our main findings suggest that most PTFU published in trials could not be combined). the last 11 years does not appear to be designed in a cost- Of the 58 trials considered, the risk of bias could not effective manner. Cost of PTFU was shown to vary widely be fully assessed in 11 trials due to lack of information and not many trials used incentives to retain participants. in at least one domain. Low (or unclear) risk of bias in Despite only a third of trialists providing complete financial Table 5 Comparison of randomised controlled trials (RCTs) which had high risk of bias compared to the proportion lost to follow-up during post-trial follow-up (PTFU). A summary of those RCTs with no risk of bias are also detailed High-risk domain Number of studies with high-risk domain Proportion of participants lost to follow-up during PTFU (%) Blinding participants and personnel 3 3.96–6.16 Incomplete outcome data 2 – Other sources of bias 3 1.21–11.79 Selective outcome reporting 1 1.2 Low risk of bias in all domains 43 (no high/unclear risk of bias) 0–19.90 Llewellyn-Bennett et al. Trials (2018) 19:298 Page 11 of 12 information for PTFU, follow-up by clinical appointment Conclusions appeared to be the most expensive method, as might be ex- Post-trial follow-up of large RCTs can contribute signifi- pected given the resource implications. Postal or telephone cantly to the scientific value of a trial by determining the correspondence in addition to data linkage did not appear longer-term magnitude of the effects of an intervention. to increase the cost per participant per year considerably. PTFU is valuable to ensure that there are no long-term However, the effect of inflation over the 11 years included hazards or beneficial effects which have been missed due in this systematic review, makes quantitative comparison of to the common short in-trial periods for following up cost differences difficult. Given the limited data available we participants. However, it is not widely used as shown by have not attempted to adjust for inflation. the small number of eligible trials which had PTFU from Data linkage or access to medical records is likely to be the original search strategy. the most cost-effective method of following participants Data linkage and the use of registries appear to be the due to minimal staff required. However, a number of trial- most plausible and economical approach to PTFU. These ists highlighted the limitations of this approach, noting it methods also have the advantage of providing data for a to be time-consuming and frustrating with increasing wide range of endpoints. Improvement of electronic regulatory costs and country-specific restrictions. In reporting and informatics could lead to better reporting the UK, the process of accessing data electronically and allow this type of method to be widely used. has become more stringent and costly, and markedly different to the processes which were encountered Additional files 10 years ago. There is also an issue of the data lag- ging behind by up to 2 years in some countries, Additional file 1: PRISMA Checklist. (DOCX 30 kb) which can impact on the completeness of results for Additional file 2: Search strategies. Key to operators used in Medline/ a trial. Despite this, data linkages to national regis- Ovid: where .pt. is publication type, (?) represents any single character, (*) is a group of characters, .mp.is multi-purpose search, /is Medical tries and electronic health records have been shown Subject Headings (MeSH), exp. is explode subject heading, .sh. is subject to be a valid and reliable method of PTFU [12–15]. heading, (““) is phrase search. Comments: all results were downloaded When designing this systematic review, we anticipated with all fields displayed and in a tab delimited format. This file was then opened in Excel. Duplicates were removed. The spreadsheet sort order that papers published in the early half of the last 11 years was changed to Enrollment A–Z and studies with fewer than 1000 would choose more traditional methods of PTFU, e.g. enrollees will be removed. (PDF 396 kb) clinic- and telephone-based approaches, and more re- Additional file 3: Trials with long-term follow-up excluded from final cent trials may increasingly use data linkage where avail- analysis. *open-label study investigating safety doses of intervention. Extension study of two previous RCTs (Philipp T et al. Clin Ther 2007; able. However, this has not been the case. The majority 29:563–80). (PDF 147 kb) of trials have used a variety of different methods to cap- Additional file 4: Risk of bias shown in each domain for an individual ture data for the same primary outcome. We were, randomised controlled trial (RCT). Red indicates high risk, yellow indicates therefore, unable to compare retention rates by each unsure and green indicates low risk. (PDF 173 kb) type of method used. In addition, sparsity of complete data in the review (typically poor reporting of the final Abbreviations number of participants at the end of the follow-up CENTRAL: Cochrane Central Register of Controlled Trials; EMBASE: Excerpta Medica database; PTFU: Post-trial follow-up; RCT: Randomised controlled trial period) limited the ability to assess retention rates achieved with different PTFU methods. Acknowledgements We found limited evidence of high risk of bias in the Thank you to all corresponding trialists. methodology of the in-trial periods. A likely explanation for this is that the majority of the trials included in this Funding review were well-designed, large RCTs in which results RLB has received funding from the Royal College of Surgeons of England Research Fellowship. were published in high-impact journals. Furthermore, trials which employ poor methodology or have had Availability of data and materials negative results are more likely not to engage in PTFU The datasets used and/or analysed during the current study are available due to lack of funding or interest. from the corresponding author on reasonable request. Due to new guidelines (Consolidated Standards of Reporting Trials (CONSORT)) recommending increas- Authors’ contributions RLB designed, carried out the systematic review including screening, data ing transparency in the reporting of RCTs, a more capture, data analysis, interpretation of results and wrote the paper. DE complete capture of data would be likely for any future screened the papers from the search strategy and identified relevant papers study . Research into appropriate methods in PTFU including checking 10% of data from the extraction stage. NR assisted in the design of the search strategy and completed the search strategy. RB, LB can only occur if there is transparency of the logistical assisted in the search strategy and in decisions relevant to the review. LB and financial implications including number of partici- and RB assisted in drafting the review. AH assisted in the discussions of the pants lost to follow-up. review and drafting. All authors read and approved the final manuscript. Llewellyn-Bennett et al. Trials (2018) 19:298 Page 12 of 12 Ethics approval and consent to participate 14. Kivimaki M, et al. Validity of cardiovascular disease event ascertainment Not applicable. using linkage to UK hospital records. Epidemiology. 2017;28(5):735–9. 15. Herbert A, et al. Data Resource Profile: Hospital Episode Statistics Admitted Patient Care (HES APC). LID – https://doi.org/10.1093/ije/dyx015 [doi] Consent for publication FAU – Herbert, Annie. (1464–3685 (Electronic)). RLB, DE, NR, AH, LB and RB consent for publication. 16. CONSORT Transparent Reporting of Trials. Available from: http://www. consort-statement.org/. Accessed 30 Dec 2017. Competing interests The authors declare that they have no competing interests. Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Author details MRC Population Health Research Unit, Clinical Trial Service Unit (CTSU), Nuffield Department of Population Health, University of Oxford, Richard Doll Building, Roosevelt Drive, Oxford OX3 7LF, UK. Bodleian Health Care Libraries, University of Oxford, Roosevelt Drive, Oxford OX3 7LF, UK. Molecular and Clinical Sciences Research Institute, St George’s University of London, Cranmer Terrace, London SW17 0RE, UK. Medical Research Council Population Health Research Unit, Nuffield Department of Population Health, University of Oxford, Oxford, UK. Clinical Trial Service Unit and Epidemiological Studies Unit, Nuffield Department of Population Health, University of Oxford, Oxford, UK. Received: 30 December 2017 Accepted: 13 April 2018 References 1. Ford I., et al., Long-term safety and efficacy of lowering low-density lipoprotein cholesterol with statin therapy: 20-year follow-up of West of Scotland Coronary Prevention Study. (1524–4539 (Electronic)). 2. Heart Protection Study Collaborative Group, et al. Effects on 11-year mortality and morbidity of lowering LDL cholesterol with simvastatin for about 5 years in 20,536 high-risk individuals: a randomised controlled trial. Lancet. 2011;378(9808):2013–20. 3. Fergusson D, et al. Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis. BMJ. 2002;325(7365):652–4. 4. Brueton VC, et al., Strategies to improve retention in randomised trials: a Cochrane systematic review and meta-analysis. (2044–6055 (Linking)). 5. Barton J, Young A, Lay M., Introduction of electronic data capture method using participant-completed online web-based follow up questionnaire in mail-based study achieves expected benefits and positive participant feedback. Trials, 2015. 16(2): p44. 6. Scuffham P, Chaplin S, Legood R. Incidence and costs of unintentional falls in older people in the United Kingdom. J Epidemiol Community Health, 2003. 57(9): p. 740. 7. Llewellyn-Bennett R, Bowman L, Bulbulia R Post-trial follow-up methodology in large randomized controlled trials: a systematic review protocol. Systematic Reviews, 2016. 5: p. 214. 8. Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia. Available at www.covidence.org. 9. (RevMan), R.M., [Computer program] Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration. 2014. http://community. cochrane.org/tools/review-production-tools/revman-5/about-revman-5. Accessed 30 Dec 2017. 10. Arber N, et al. Five-year analysis of the prevention of colorectal sporadic adenomatous polyps trial. Am J Gastroenterol. 2011;106(6):1135–46. 11. Margolis KL, et al. Long-term follow-up of moderately hypercholesterolemic hypertensive patients following randomization to pravastatin vs usual care: the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT-LLT). J Clin Hypertens (Greenwich). 2013;15(8):542–54. (1751–7176 (Electronic)). 12. Hornslien AG, et al. Effects of candesartan in acute stroke on vascular events during long-term follow-up: results from the Scandinavian Candesartan Acute Stroke Trial (SCAST). Int J Stroke. 2015;10:830–5. https://doi.org/10. 1111/ijs.12477. 13. Lloyd SM, et al. Long-term effects of statin treatment in elderly people: extended follow-up of the PROspective Study of Pravastatin in the Elderly at Risk (PROSPER). PLoS ONE [Electronic Resource]. 2013;8(9):e72642.
Trials – Springer Journals
Published: May 30, 2018
It’s your single place to instantly
discover and read the research
that matters to you.
Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.
All for just $49/month
Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly
Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.
Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.
Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.
All the latest content is available, no embargo periods.
“Hi guys, I cannot tell you how much I love this resource. Incredible. I really believe you've hit the nail on the head with this site in regards to solving the research-purchase issue.”Daniel C.
“Whoa! It’s like Spotify but for academic articles.”@Phil_Robichaud
“I must say, @deepdyve is a fabulous solution to the independent researcher's problem of #access to #information.”@deepthiw
“My last article couldn't be possible without the platform @deepdyve that makes journal papers cheaper.”@JoseServera