The reporting of blinding in orthodontic randomized controlled trials: where do we stand?

The reporting of blinding in orthodontic randomized controlled trials: where do we stand? Summary Objective: To analyse in 10 orthodontic journals how many randomized controlled trials (RCTs) performed ‘single-’, ‘double-’, ‘triple-’, or ‘outcome assessors blinding’ and to evaluate, from the number of RCTs that did not conduct blinding, how many could actually have achieved it. Material and methods: Randomized controlled trials published in 10 orthodontic journals between 1 September 2012 and 28 February 2018 were included. A search was performed in PubMed and conducted for publication type ‘randomized controlled trial’ for each journal. Two reviewers independently analysed each RCT and registered that blinding was performed and included which specific type. It was also evaluated whether misclassifications of blinding items occurred and whether it was possible to achieve blinding among the RCTs that did not perform blinding. Results: After applying the inclusion criteria, 203 RCTs were assessed, and 61.6 per cent of them had used blinding, with the main type being ‘outcome assessors blinding’ (40.4%) followed by ‘single-blinding’ (15.3%), ‘double-blinding’ (2.5%), and ‘triple-blinding’ (3.4%). In 38.4 per cent of the trials, no blinding was performed; however, 79.4 per cent of them could have achieved blinding. Fifteen RCTs (7.3%) misclassified the blinding in relation to single-, double-, or triple-blinding. Journals followed the CONSORT (AJODO, EJO, JO, OCR) published together significantly more RCTs that performed blinding than journals not following the CONSORT. Conclusions: Blinding of outcome assessors was the most frequent type, as orthodontic trials are often of intervention design and thereby difficult to mask for patients and trial staff. The misclassifications of blinding items may indicate suboptimal knowledge among researchers and peer-reviewers regarding the definitions for diverse blinding types. Introduction In clinical research, the randomized controlled trial (RCT) is the acknowledged golden standard and is considered to generate the highest level of evidence for assessing competing interventions clinically, followed by controlled trials (1, 2). The advantages of using RCT methodology include elimination of bias in treatment assignment, specifically, selection bias (without therapists or patient’s preferences), and most importantly, the confounding variables—those which are hidden or out of one’s control—will be of equal value and evenly distributed between the groups the subjects are randomized to. However, all methods have their weaknesses, and an RCT can be affected by poor or inadequate methodology. Some of the biases that can occur in RCTs are wrong randomization procedures, broken concealments, missing data or dropouts, and the lack of blinding (3–6). The risk of bias in RCTs has been described and analysed in several methodological investigations (4–8), but few have focused on the bias that may occur because of inadequate blinding. Nevertheless, in a systematic review of 250 RCTs, investigators noticed a significant disparity in the magnitude of the accounted treatment outcomes between studies that described double-blinding in comparison to those that did not (9). In addition, Dechartres et al.’s (6) study shows that the proportion of RCTs with a high risk of bias was 33.1 per cent for blinding of participants and personnel while 22.6 per cent for blinding of outcome assessors. In clinical investigations, the condition of ‘blinding’ may also be proclaimed masking, and this relates to actions to conceal certain key circumstances in an investigation until it is completed and the results are being processed. Consequently, in a blinded clinical trial, it is unknown which participants are receiving each of the treatments/interventions tested. An important aspect to consider is which party the data is being kept secret from (i.e. participants, trial managers, trial staff, and/or statisticians). In a single blinded trial, either the participants/patients or the trial staff are unaware of what treatment/intervention has been performed. Usually, the participants/patients are those who are unaware, while the trial staff is informed because it is often difficult to blind the staff (doctors, nurses, and others). In a double-blind test, neither the participants/patients nor the trial staff know which of the treatments/interventions a particular participant/patient has received. Finally, in a triple-blind trial, the blinding includes participants/patients, trial staff, and the outcome assessors and/or statisticians (10). Frequently clinical investigators mix up ‘blinding’ with ‘allocation concealment’ through the course of setting up comparison groups. It is important to note that the concealment of the allocation focuses on inhibiting selection and confounding biases, and hence, protects the allocation sequence up to the assignment. In contrast, the primary purpose of blinding is to avoid bias related to the participants/patients’ and researchers’ expectations throughout the test (11). The importance of blinding is particularly significant when the response criteria is subjective, for instance, with the relief of pain. Although single- or double-blinding cannot always be achieved in a trial, this does not necessarily reflect low quality. These trials have to be judged on other merits. When various surgical interventions are to be compared or when various orthodontic treatments are evaluated, it is often impossible to perform single- or double-blinding. Nevertheless, in such ‘open’ trials, it may be possible to conduct blinding during the outcome assessment. The CONSORT statement endorses the declaration of the findings of an appraisal of blinding if it was accomplished (12). However, the presenting of blinding conditions or blinding actions is often deficient or lacking (13) despite scientific journals frequently requesting that authors adhere the CONSORT guidelines to publish an RCT. To reduce bias in clinical trial outcomes in orthodontics, the blinding effect can be a significant component, but to our understanding, nothing has been published which highlights to what extent blinding is regarded in orthodontic RCTs. Therefore, the primary aim of this investigation was to analyse and highlight, by selecting 10 different orthodontic journals, how many RCTs reported and performed single-, double-, triple- or outcome assessors blinding only. The secondary aim was to evaluate, of the RCTs that did not perform blinding, how many could have actually achieved it. Material and methods All the RCTs published in 10 orthodontic journals between 1 September 2012 and 28 February 2018 were included (a 5.5-year period). The journals were American Journal of Orthodontics and Dentofacial Orthopedics (AJODO), The Angle Orthodontist (AO), Australian Orthodontic Journal (AOJ), Dental Press Journal of Orthodontics (DPJO), European Journal of Orthodontics (EJO), Journal of Orthodontics (JO), Journal of Orofacial Orthopaedics (JOO), Korean Journal of Orthodontics (KJO), Orthodontics and Craniofacial Research (OCR), and Progress in Orthodontics (PO). The journals AJODO, EJO, JO, and OCR have instructions that authors adhere the CONSORT guidelines to publish an RCT. The exclusion criteria were RCTs that were not performed on humans and non-randomized trials (i.e. prospective and retrospective control trials, observational studies, prospective, and retrospective case series and case reports). A database search was performed in Medline, via PubMed, which was conducted for publication type ‘randomized controlled trial’ for each journal separately. A manual search was also performed for additional trials. Two independent reviewers selected and categorized each article and then extracted the data. If randomization was evident from the title and abstract of the articles, the full text of the study was acquired and read to verify and apply the inclusion and exclusion criteria. The journals were also hand-searched for the actual search period to cross-check against the database search. After inclusion of the RCTs, the two authors independently registered what type of blinding was performed for each trial or if no blinding occurred. Then, determination was performed regarding blinding for participants/patients (single-blinding), both participants/patients and trial staff (double-blinding), participants/patients and trial staff as well as outcome assessors (triple-blinding) or if only blinding was carried out for outcome assessors. Of the included RCTs, the two authors also evaluated how many RCTs did not perform any blinding at all, and of those, determined if it may have been possible to perform a type of blinding. It was also evaluated if misclassification of the blinding items occurred (e.g. if a trial claimed to be double-blinded, but it becomes apparent that blinding was only performed on participants/patients and outcome assessors, thus indicating a single-blinded trial). At the title, abstract and full-text screening stages, as well as with any dichotomous and categorical variables during the data extraction, the number of inter-examiner agreements and disagreements were noted and calculated for the assessment of agreement. Then, all the disagreements were discussed and resolved by discussion to reach consensus. Statistical analysis Chi square test and Fisher’s exact test was used to determine differences in categorical data. Differences with probabilities of less than 5 per cent (P-value < 0.05) were considered to be statistically significant. Kappa score was used to measure the level of agreement, and the degree of agreement was 0.96 (i.e. an excellent agreement) (14). Results The initial screening yielded 240 potentially relevant articles all identified via the electronic database search and without any further trials added by manual searching. After applying the inclusion and exclusion criteria, 37 articles were excluded (11 in vitro studies, 18 prospective controlled trials, and 8 retrospective controlled studies). Thus, 203 RCTs were assessed and the numbers of RCTs per journal are presented in Table 1. Table 1. Number of randomized trials (RCTs) per journal and number of different blinding items performed as well as number of RCTs that did not performed any blinding at all. Also the number RCTs that could have achieved blinding is given. Single-blinding includes either participants/patients or the trial staff; double-blinding includes both participants/patients and the trial staff; triple-blinding includes participants/patients, trial staff and outcome assessors. Journal Number of RCTs per journal Number of RCTs that performed single-blinding Number of RCTs that performed double-blinding Number of RCTs that performed tripple-blinding Number of RCTs that performed outcome assessors blinding Number of RCTs that did not performed any blinding at all Number of RCTs that could have achieved blinding among those that did not AJODO 70 (34.4%) 17 0 1 28 24 23 of 24 AO 45 (22.1%) 4 0 1 21 19 15 of 19 AOJ 6 (3%) 0 2 0 0 4 3 of 4 DPJO 9 (4.4%) 2 0 0 0 7 5 of 7 EJO 28 (13.7%) 2 1 4 14 7 5 of 7 JO 10 (4.9%) 2 0 0 7 1 1 of 1 JOO 10 (4.9%) 0 1 0 0 9 4 of 9 KJO 3 (1.4%) 0 0 0 2 1 0 of 1 OCR 6 (3%) 0 0 0 4 2 2 of 2 PO 16 (7.9%) 4 1 1 6 4 4 of 4 Total 203 (100%) 31 (15.3%) 5 (2.5%) 7 (3.4%) 82 (40.4%) 78 (38.4%) 62 of 78 Journal Number of RCTs per journal Number of RCTs that performed single-blinding Number of RCTs that performed double-blinding Number of RCTs that performed tripple-blinding Number of RCTs that performed outcome assessors blinding Number of RCTs that did not performed any blinding at all Number of RCTs that could have achieved blinding among those that did not AJODO 70 (34.4%) 17 0 1 28 24 23 of 24 AO 45 (22.1%) 4 0 1 21 19 15 of 19 AOJ 6 (3%) 0 2 0 0 4 3 of 4 DPJO 9 (4.4%) 2 0 0 0 7 5 of 7 EJO 28 (13.7%) 2 1 4 14 7 5 of 7 JO 10 (4.9%) 2 0 0 7 1 1 of 1 JOO 10 (4.9%) 0 1 0 0 9 4 of 9 KJO 3 (1.4%) 0 0 0 2 1 0 of 1 OCR 6 (3%) 0 0 0 4 2 2 of 2 PO 16 (7.9%) 4 1 1 6 4 4 of 4 Total 203 (100%) 31 (15.3%) 5 (2.5%) 7 (3.4%) 82 (40.4%) 78 (38.4%) 62 of 78 AJODO = American Journal of Orthodontics and Dentofacial Orthopedics; AO = The Angle Orthodontist; AOJ = Australian Orthodontic Journal; DPJO = Dental Press Journal of Orthodontics; EJO = European Journal of Orthodontics; JO = Journal of Orthodontics; JOO = Journal of Orofacial Orthopedics; KJO = Korean Journal of Orthodontics; OCR = Orthodontics and Craniofacial Research; PO = Progress in Orthodontics. View Large Table 1. Number of randomized trials (RCTs) per journal and number of different blinding items performed as well as number of RCTs that did not performed any blinding at all. Also the number RCTs that could have achieved blinding is given. Single-blinding includes either participants/patients or the trial staff; double-blinding includes both participants/patients and the trial staff; triple-blinding includes participants/patients, trial staff and outcome assessors. Journal Number of RCTs per journal Number of RCTs that performed single-blinding Number of RCTs that performed double-blinding Number of RCTs that performed tripple-blinding Number of RCTs that performed outcome assessors blinding Number of RCTs that did not performed any blinding at all Number of RCTs that could have achieved blinding among those that did not AJODO 70 (34.4%) 17 0 1 28 24 23 of 24 AO 45 (22.1%) 4 0 1 21 19 15 of 19 AOJ 6 (3%) 0 2 0 0 4 3 of 4 DPJO 9 (4.4%) 2 0 0 0 7 5 of 7 EJO 28 (13.7%) 2 1 4 14 7 5 of 7 JO 10 (4.9%) 2 0 0 7 1 1 of 1 JOO 10 (4.9%) 0 1 0 0 9 4 of 9 KJO 3 (1.4%) 0 0 0 2 1 0 of 1 OCR 6 (3%) 0 0 0 4 2 2 of 2 PO 16 (7.9%) 4 1 1 6 4 4 of 4 Total 203 (100%) 31 (15.3%) 5 (2.5%) 7 (3.4%) 82 (40.4%) 78 (38.4%) 62 of 78 Journal Number of RCTs per journal Number of RCTs that performed single-blinding Number of RCTs that performed double-blinding Number of RCTs that performed tripple-blinding Number of RCTs that performed outcome assessors blinding Number of RCTs that did not performed any blinding at all Number of RCTs that could have achieved blinding among those that did not AJODO 70 (34.4%) 17 0 1 28 24 23 of 24 AO 45 (22.1%) 4 0 1 21 19 15 of 19 AOJ 6 (3%) 0 2 0 0 4 3 of 4 DPJO 9 (4.4%) 2 0 0 0 7 5 of 7 EJO 28 (13.7%) 2 1 4 14 7 5 of 7 JO 10 (4.9%) 2 0 0 7 1 1 of 1 JOO 10 (4.9%) 0 1 0 0 9 4 of 9 KJO 3 (1.4%) 0 0 0 2 1 0 of 1 OCR 6 (3%) 0 0 0 4 2 2 of 2 PO 16 (7.9%) 4 1 1 6 4 4 of 4 Total 203 (100%) 31 (15.3%) 5 (2.5%) 7 (3.4%) 82 (40.4%) 78 (38.4%) 62 of 78 AJODO = American Journal of Orthodontics and Dentofacial Orthopedics; AO = The Angle Orthodontist; AOJ = Australian Orthodontic Journal; DPJO = Dental Press Journal of Orthodontics; EJO = European Journal of Orthodontics; JO = Journal of Orthodontics; JOO = Journal of Orofacial Orthopedics; KJO = Korean Journal of Orthodontics; OCR = Orthodontics and Craniofacial Research; PO = Progress in Orthodontics. View Large Blinding, of any type, occurred in 125 (61.6%) of the RCTs, while in 78 (38.4%) of the trials, no blinding was performed at all (Table 1). Single-blinding was performed in 31 (15.3%) of the trials, while 5 (2.5%) of the trials performed double-blinding and 7 (3.4%) of the trials performed triple-blinding. In 82 (40.4%) of the trials, outcome assessors blinding was performed without any other type of blinding (Table 1). In total, the blinding of outcome assessors was performed in 87 of 125 trials (69.6%). No main differences were found between the specific journals regarding blinding and type of blinding. However, the journals (AJODO, EJO, JO, and OCR) that require authors adhere CONSORT guidelines, published together significantly more RCTs that performed blinding than journals that not follow the CONSORT, i.e. 70.1 versus 45.5 per cent (P = 0.004). Of the 78 RCTs that did not perform any blinding, 62 trials (79.4%) could have achieved blinding to at least one of the three blinding categories (participants, trial staff and/or outcomes). The other 16 (20.6%) RCTs were of such a design that blinding was not possible whatsoever. The distribution of RCTs that could have achieved blinding among the different journals was fairly equal, with no significant differences between them (P = 0.246). In 15 RCTs (7.4%), the blinding was misclassified. Five RCTs claimed in their articles that single-blinding was performed, when actually, the authors had only blinded the outcome assessors (two trials in JO, and one trial each in AJODO, AO, EJO). A further six RCTs overestimated the blinding and described them as double-blinded when they were single-blinded because only the blinding of the participants/patients or trial staff and the outcome assessors was performed (two trials in AO and one trial each in EJO, JO, JOO, PO). Finally, four RCTs underestimated their blinding and reported double-blinding while they had actually carried out triple-blinding because the participants/patients, the trial staff and the outcome assessors were all blinded (three trials in EJO and one in PO). Discussion It was important and encouraging to note in this methodological investigation that the majority of the RCTs (61.6%) had used blinding for at least one of the three categories to reduce bias within the trial. The most common blinding item produced in these trials was the blinding of outcome assessors (69.6%). Another study investigated the risk of bias in dentistry RCTs published from 1955 to 2013 and blinding of participants was judged to be adequate in 71.5 per cent of the trials, and blinding of the outcome assessment was judged to be adequate in 59.4 per cent of the trials (15). In our study, it was found that blinding was adequate in 69.5 per cent of the RCTs, while misclassifications occurred in 7.4 per cent. It is not surprising that the blinding of outcome assessors was the most frequent type of blinding due to the nature of orthodontic trials, which are often of intervention design. Consequently, blinding is often impossible because the various orthodontic appliances (or no appliance intervention) are obvious and difficult to mask for patients and trial staff. This also helps explain why only 43 (21.2%) RCTs out of 203 performed single-, double-, or triple-blinding. Another important finding was that 38.4 per cent of the RCTs did not perform any blinding at all (Table 1), although in 16 (20.5%) of those, blinding was impossible due to the character or design of the trial. Consequently, in 62 trials, blinding may have been achieved; at least, the blinding of the outcome assessors. This fact also means that the RCTs without blinding are at risk of bias and may produce overestimated results (9). It is also important to mention that the RCTs without blinding will be graded as having a substantial risk for bias in relation to blinding when they are pooled into a systematic review, and for example, evaluated by the Cochrane Risk of Bias Assessment Tool (16). This means the trial will be downgraded in quality and the contribution of the trial to the evidence will also be diminished. When it is not possible to achieve blinding, researchers can still make arrangements to compensate as much as possible for the lack of blinding. For example, in one RCT (17) investigating the effect of the maxillary primary canines extraction effect on the eruption of palatally impacted maxillary permanent canines, blinding of any type was not possible. The patients were randomized into two groups: one received extraction and one served as a control group, but given that half of the patients will receive extractions and the other half of the patients will not, those who have had extractions and those who have not will be obvious to the patients and trial staff. This will also be obvious to the outcome assessors, who will easily notice the empty spaces after the extractions clinically and on the radiographs. Nevertheless, the authors of this study still managed to achieve blinding of the baseline radiographs (15). In contrast, orthodontic RCTs that investigate different drugs for relieving pain, fluoride varnishes for preventing white spot lesions, or altered resin materials for bonding often permit the triple-blinding design. In one study with the aim to evaluate and compare the effect of pre-procedural administration of acetaminophen, ibuprofen, and meloxicam in reducing pain after orthodontic separator placement, a blinding design was performed by covering the tablets with identical gelatine covers, which then permitted the blinding of the patients and trial staff as well as the outcome assessors (17). However, another study did not take the opportunity to achieve blinding when comparing ibuprofen with acetaminophen for the effect on prostaglandin E2 levels of the gingival crevicular fluid (18). Here, the authors could also have covered the tablets with gelatine covers, as was the case in the aforementioned study (17). Another important finding was that 15 RCTs reported the blinding wrongly in relation to single-, double-, or triple-blinding. This misclassification indicates not only that the researchers have insufficient knowledge regarding the definitions for diverse types of blinding but also that the peer-reviewers had obviously not reported the misclassifications. However, it should be pointed out that there can be confusion regarding the definition of blinding, and in particular, the terms ‘single-’, ‘double-’, and ‘triple-blinding’. Therefore, it is recommended that one make precise and explicit definitions and descriptions of what type of blinding has been achieved. Thus, blinding can be based on a protocol covering up to five different categories of blinding, namely, the blinding of: 1. participants/patients; 2. trial staff/ health care providers; 3. data collectors; 4. outcome assessors; and 5. data analysts including statisticians. If blinding cannot be performed for any reason, then acknowledge this limitation and possible bias in the discussion of the trial and explain that the assignment groups, aside from the interventions, were treated as identically as possible. When data collectors or outcome assessors cannot be blinded, use as much as possible objective and/or reliable outcomes as well as arrange to use duplicate assessments of outcomes and declare the degree of inter-assessor concordance obtained (19). All the journals included in this survey were peer-reviewed journals, and four of them (AJODO, EJO, JO, and OCR) request that authors adhere to the CONSORT guidelines to publish an RCT. One of the CONSORT guidelines considers the act of reporting blinding in RCTs (12). Interestingly, this survey shows that those four journals as a group had significant more RCTs that had performed blinding than those that did not follow the CONSORT guidelines. Thus, it may be advisable for other orthodontic journals to request authors to follow the CONSORT guidelines, and hence, a possible increase in the number of RCTs performing blinding. Although the impact factor differed between the journals, no clear pattern was found that the journals with a higher impact factor would be any better at achieving blinding or reporting correctly classified blinding. As mentioned, the blinding of patients, trial staff or outcome assessors is effective in reducing bias within the trial and increases the quality strength of the trial. Starting with the blinding of patients, it prevents reporting bias in patient-reported outcome measures (20). It has been proven that non-blinded patients exaggerate the effect size by 0.56 standard deviations (21). This exaggerated effect is probably due to the patient’s expectations of the treatment effect. In addition, the blinding of patients may also reduce adherence bias; e.g. when the control subjects are aware of their status as members of the control group and are able to compare their performance with those of the treatment group, it may result in members of the control group not following the protocol. Instead, the control subjects actively work harder to overcome the ‘disadvantage’ of being in the control group (20). Finally, the blinding of patients may also decrease the risk of a participant’s attrition from the trial, as it was reported that the risk of attrition in the blinded trials was 4 per cent, while in non-blinded trials it was 7 per cent (21). The blinding of trial staff is also highly important, as the patient and trial staff relationship can have an important effect on the treatment response. Previous research has shown that the behaviour and attitudes of the trial staff often influence patient response (22, 23); thus in general, it is important to keep the relationship between the trial staff and the patients as standardised as possible. Moreover, if the trial staff give the same attention to all of the trial groups or patients (achieved by blinding the trial staff), then this will prevent them from both intentional and unintentional changes in behaviour towards a specific trial group or patient (20). The third category of blinding is the ‘outcome assessors blinding’. Blinding of assessors prevents observer-related bias, as it was demonstrated that non-blinded assessors over-rate patients in the treatment group while under-rating patients in the control group (24). This survey was carried out in 10 well-recognized orthodontic journals and the high number of journals included as well as the relatively long (5.5 years) search period give strength to this study. However, a limitation was that when assessing the trials, we did not remove the name of the authors to avoid bias to be influenced by well-respected research authorship, mentorship or even friendship. Furthermore, one of the assessors had produced many RCTs included in the survey. Nevertheless, the selection and assessment of the RCTs were performed independently by the two authors and the Kappa value was measured and it was 0.96 (excellent agreement). Moreover, the assessment concerned more or less dichotomous decisions, i.e. blinding or not, type of blinding, and thus, limited subjective assessments had to be performed and all evaluations were based on what was written in the trials. Finally, blinding had been ineffective when assessments were to be made by the assessors’ own studies, since then simply the text would be recognizable. Conclusions The majority of the RCTs (61.6%) had used blinding for at least one of the three categories to reduce bias within the trial, and the prevailing blinding item produced was ‘blinding of outcome assessors’ (69.6%). It is not surprising that ‘blinding of outcome assessors’ was the most frequent type since the trials are often of intervention design and difficult to mask for patients and trial staff. A total of 78 RCTs (38.4%) did not perform any blinding; however, of those, 62 (79.4%) could have performed blinding, in particular, ‘outcome assessors blinding’. Fifteen RCTs (7.3%) reported the blinding wrongly in relation to single-, double-, or triple-blinding. The misclassifications of blinding items may indicate suboptimal knowledge among researchers and peer reviewers regarding the definitions of diverse types of blinding. Since confusions existed regarding the classification of single, double and triple blinding it can be recommended to make precise and explicit descriptions of what type of blinding has been achieved. Thus, describe if blinding has been performed of participants/patients; trial staff/health care providers; data collectors; outcome assessors; and data analysts including statisticians. Funding This survey was funded by the authors’ academic institution. Conflict of Interest None to declare. References 1. O’Brien , K. and Craven , R . ( 1995 ) Pitfalls in orthodontic health service research . British Journal of Orthodontics , 22 , 353 – 356 . Google Scholar CrossRef Search ADS PubMed 2. Bondemark , L. and Ruf , S . ( 2015 ) Randomized controlled trial: the gold standard or an unobtainable fallacy ? European Journal of Orthodontics , 37 , 457 – 461 . Google Scholar CrossRef Search ADS PubMed 3. Moher , D. , Jones , A. and Lepage , L .; CONSORT Group (Consolidated Standards for Reporting of Trials) . ( 2001 ) Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation . JAMA , 285 , 1992 – 1995 . Google Scholar CrossRef Search ADS PubMed 4. Lempesi , E. , Koletsi , D. , Fleming , P.S. and Pandis , N . ( 2014 ) The reporting quality of randomized controlled trials in orthodontics . The Journal of Evidence-Based Dental Practice , 14 , 46 – 52 . Google Scholar CrossRef Search ADS PubMed 5. Bondemark , L. and Abdulraheem , S . ( 2017 ) Intention to treat (ITT) analysis as reported in orthodontic randomized controlled trials-evaluations of methodology and recommendations for the accurate use of ITT analysis and handling dropouts . European Journal of Orthodontics . doi: 10.1093/ejo/cjx084 . 6. Dechartres , A. , Trinquart , L. , Atal , I. , Moher , D. , Dickersin , K. , Boutron , I. , Perrodeau , E. , Altman , D.G. and Ravaud , P . ( 2017 ) Evolution of poor reporting and inadequate methods over time in 20920 randomised controlled trials included in Cochrane reviews: research on research study . BMJ , 357 , j2490 . Erratum in: BMJ. 2017, 358, j3806. Google Scholar CrossRef Search ADS PubMed 7. Dechartres , A. , Charles , P. , Hopewell , S. , Ravaud , P. and Altman , D.G . ( 2011 ) Reviews assessing the quality or the reporting of randomized controlled trials are increasing over time but raised questions about how quality is assessed . Journal of Clinical Epidemiology , 64 , 136 – 144 . Google Scholar CrossRef Search ADS PubMed 8. Kloukos , D. , Papageorgiou , S.N. , Doulis , I. , Petridis , H. and Pandis , N . ( 2015 ) Reporting quality of randomised controlled trials published in prosthodontic and implantology journals . Journal of Oral Rehabilitation , 42 , 914 – 925 . Google Scholar CrossRef Search ADS PubMed 9. Schulz , K.F. , Chalmers , I. , Hayes , R.J. and Altman , D.G . ( 1995 ) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials . JAMA , 273 , 408 – 412 . Google Scholar CrossRef Search ADS PubMed 10. Devereaux , P.J. , Manns , B.J. , Ghali , W.A. , Quan , H. , Lacchetti , C. , Montori , V.M. , Bhandari , M. and Guyatt , G.H . ( 2001 ) Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials . JAMA , 285 , 2000 – 2003 . Google Scholar CrossRef Search ADS PubMed 11. Jüni , P. , Altman , D.G. and Egger , M . ( 2001 ) Assessing the quality of controlled clinical trials . BMJ , 323 , 42 – 46 . Google Scholar CrossRef Search ADS PubMed 12. CONSORT . ( 2010 )http://www.equator-network.org/reporting-guidelines/consort/ and http://www.consort-statement.org/ (31 July 2017, date last accessed). 13. Montori , V.M. , Bhandari , M. , Devereaux , P.J. , Manns , B.J. , Ghali , W.A. and Guyatt , G.H . ( 2002 ) In the dark: the reporting of blinding status in randomized controlled trials . Journal of Clinical Epidemiology , 55 , 787 – 790 . Google Scholar CrossRef Search ADS PubMed 14. Fleiss , J.L . ( 1981 ) Statistical Methods for Rates and Proportions . John Wiley , New York , 2nd edn , pp. 38 – 46 . 15. Saltaji , H. , Armijo-Olivo , S. , Cummings , G.G. , Amin , M. and Flores-Mir , C . ( 2017 ) Randomized clinical trials in dentistry: Risks of bias, risks of random errors, reporting quality, and methodologic quality over the years 1955-2013 . PloS One , 12 , e0190089 . Google Scholar CrossRef Search ADS PubMed 16. Naoumova , J. , Kürol , J. and Kjellberg , H . ( 2015 ) Extraction of the deciduous canine as an interceptive treatment in children with palatally displaced canines - part II: possible predictors of success and cut-off points for a spontaneous eruption . European Journal of Orthodontics , 37 , 219 – 229 . Google Scholar CrossRef Search ADS PubMed 17. Zarif Najafi , H. , Oshagh , M. , Salehi , P. , Babanouri , N. and Torkan , S . ( 2015 ) Comparison of the effects of preemptive acetaminophen, ibuprofen, and meloxicam on pain after separator placement: a randomized clinical trial . Progress in Orthodontics , 16 , 34 . Google Scholar CrossRef Search ADS PubMed 18. Shetty , N. , Patil , A.K. , Ganeshkar , S.V. and Hegde , S . ( 2013 ) Comparison of the effects of ibuprofen and acetaminophen on PGE2 levels in the GCF during orthodontic tooth movement: a human study . Progress in Orthodontics , 14 , 6 . Google Scholar CrossRef Search ADS PubMed 19. Karanicolas , P.J. , Farrokhyar , F. and Bhandari , M . ( 2010 ) Practical tips for surgical research: blinding: who, what, when, why, how ? Canadian Journal of Surgery , 53 , 345 – 348 . 20. Wartolowska , K. , Beard , D. and Carr , A . ( 2017 ) Blinding in trials of interventional procedures is possible and worthwhile . F1000Research , 6 , 1663 . Google Scholar CrossRef Search ADS PubMed 21. Hróbjartsson , A. , Emanuelsson , F. , Skou Thomsen , A.S. , Hilden , J. and Brorson , S . ( 2014 ) Bias due to lack of patient blinding in clinical trials. a systematic review of trials randomizing patients to blind and nonblind sub-studies . International Journal of Epidemiology , 43 , 1272 – 1283 . Google Scholar CrossRef Search ADS PubMed 22. Benedetti , F . ( 2002 ) How the doctor’s words affect the patient’s brain . Evaluation & The Health Professions , 25 , 369 – 386 . Google Scholar CrossRef Search ADS PubMed 23. Thomas , K.B . ( 1987 ) General practice consultations: is there any point in being positive ? British Medical Journal (Clinical research ed.) , 294 , 1200 – 1202 . Google Scholar CrossRef Search ADS PubMed 24. Hróbjartsson , A. , Thomsen , A.S. , Emanuelsson , F. , Tendal , B. , Hilden , J. , Boutron , I. , Ravaud , P. and Brorson , S . ( 2012 ) Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors . British Medical Journal (Clinical research ed.) , 344 , e1119 . Google Scholar CrossRef Search ADS © The Author(s) 2018. Published by Oxford University Press on behalf of the European Orthodontic Society. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png The European Journal of Orthodontics Oxford University Press

The reporting of blinding in orthodontic randomized controlled trials: where do we stand?

Loading next page...
 
/lp/ou_press/the-reporting-of-blinding-in-orthodontic-randomized-controlled-trials-5ULmJhRNja
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press on behalf of the European Orthodontic Society. All rights reserved. For permissions, please email: journals.permissions@oup.com
ISSN
0141-5387
eISSN
1460-2210
D.O.I.
10.1093/ejo/cjy021
Publisher site
See Article on Publisher Site

Abstract

Summary Objective: To analyse in 10 orthodontic journals how many randomized controlled trials (RCTs) performed ‘single-’, ‘double-’, ‘triple-’, or ‘outcome assessors blinding’ and to evaluate, from the number of RCTs that did not conduct blinding, how many could actually have achieved it. Material and methods: Randomized controlled trials published in 10 orthodontic journals between 1 September 2012 and 28 February 2018 were included. A search was performed in PubMed and conducted for publication type ‘randomized controlled trial’ for each journal. Two reviewers independently analysed each RCT and registered that blinding was performed and included which specific type. It was also evaluated whether misclassifications of blinding items occurred and whether it was possible to achieve blinding among the RCTs that did not perform blinding. Results: After applying the inclusion criteria, 203 RCTs were assessed, and 61.6 per cent of them had used blinding, with the main type being ‘outcome assessors blinding’ (40.4%) followed by ‘single-blinding’ (15.3%), ‘double-blinding’ (2.5%), and ‘triple-blinding’ (3.4%). In 38.4 per cent of the trials, no blinding was performed; however, 79.4 per cent of them could have achieved blinding. Fifteen RCTs (7.3%) misclassified the blinding in relation to single-, double-, or triple-blinding. Journals followed the CONSORT (AJODO, EJO, JO, OCR) published together significantly more RCTs that performed blinding than journals not following the CONSORT. Conclusions: Blinding of outcome assessors was the most frequent type, as orthodontic trials are often of intervention design and thereby difficult to mask for patients and trial staff. The misclassifications of blinding items may indicate suboptimal knowledge among researchers and peer-reviewers regarding the definitions for diverse blinding types. Introduction In clinical research, the randomized controlled trial (RCT) is the acknowledged golden standard and is considered to generate the highest level of evidence for assessing competing interventions clinically, followed by controlled trials (1, 2). The advantages of using RCT methodology include elimination of bias in treatment assignment, specifically, selection bias (without therapists or patient’s preferences), and most importantly, the confounding variables—those which are hidden or out of one’s control—will be of equal value and evenly distributed between the groups the subjects are randomized to. However, all methods have their weaknesses, and an RCT can be affected by poor or inadequate methodology. Some of the biases that can occur in RCTs are wrong randomization procedures, broken concealments, missing data or dropouts, and the lack of blinding (3–6). The risk of bias in RCTs has been described and analysed in several methodological investigations (4–8), but few have focused on the bias that may occur because of inadequate blinding. Nevertheless, in a systematic review of 250 RCTs, investigators noticed a significant disparity in the magnitude of the accounted treatment outcomes between studies that described double-blinding in comparison to those that did not (9). In addition, Dechartres et al.’s (6) study shows that the proportion of RCTs with a high risk of bias was 33.1 per cent for blinding of participants and personnel while 22.6 per cent for blinding of outcome assessors. In clinical investigations, the condition of ‘blinding’ may also be proclaimed masking, and this relates to actions to conceal certain key circumstances in an investigation until it is completed and the results are being processed. Consequently, in a blinded clinical trial, it is unknown which participants are receiving each of the treatments/interventions tested. An important aspect to consider is which party the data is being kept secret from (i.e. participants, trial managers, trial staff, and/or statisticians). In a single blinded trial, either the participants/patients or the trial staff are unaware of what treatment/intervention has been performed. Usually, the participants/patients are those who are unaware, while the trial staff is informed because it is often difficult to blind the staff (doctors, nurses, and others). In a double-blind test, neither the participants/patients nor the trial staff know which of the treatments/interventions a particular participant/patient has received. Finally, in a triple-blind trial, the blinding includes participants/patients, trial staff, and the outcome assessors and/or statisticians (10). Frequently clinical investigators mix up ‘blinding’ with ‘allocation concealment’ through the course of setting up comparison groups. It is important to note that the concealment of the allocation focuses on inhibiting selection and confounding biases, and hence, protects the allocation sequence up to the assignment. In contrast, the primary purpose of blinding is to avoid bias related to the participants/patients’ and researchers’ expectations throughout the test (11). The importance of blinding is particularly significant when the response criteria is subjective, for instance, with the relief of pain. Although single- or double-blinding cannot always be achieved in a trial, this does not necessarily reflect low quality. These trials have to be judged on other merits. When various surgical interventions are to be compared or when various orthodontic treatments are evaluated, it is often impossible to perform single- or double-blinding. Nevertheless, in such ‘open’ trials, it may be possible to conduct blinding during the outcome assessment. The CONSORT statement endorses the declaration of the findings of an appraisal of blinding if it was accomplished (12). However, the presenting of blinding conditions or blinding actions is often deficient or lacking (13) despite scientific journals frequently requesting that authors adhere the CONSORT guidelines to publish an RCT. To reduce bias in clinical trial outcomes in orthodontics, the blinding effect can be a significant component, but to our understanding, nothing has been published which highlights to what extent blinding is regarded in orthodontic RCTs. Therefore, the primary aim of this investigation was to analyse and highlight, by selecting 10 different orthodontic journals, how many RCTs reported and performed single-, double-, triple- or outcome assessors blinding only. The secondary aim was to evaluate, of the RCTs that did not perform blinding, how many could have actually achieved it. Material and methods All the RCTs published in 10 orthodontic journals between 1 September 2012 and 28 February 2018 were included (a 5.5-year period). The journals were American Journal of Orthodontics and Dentofacial Orthopedics (AJODO), The Angle Orthodontist (AO), Australian Orthodontic Journal (AOJ), Dental Press Journal of Orthodontics (DPJO), European Journal of Orthodontics (EJO), Journal of Orthodontics (JO), Journal of Orofacial Orthopaedics (JOO), Korean Journal of Orthodontics (KJO), Orthodontics and Craniofacial Research (OCR), and Progress in Orthodontics (PO). The journals AJODO, EJO, JO, and OCR have instructions that authors adhere the CONSORT guidelines to publish an RCT. The exclusion criteria were RCTs that were not performed on humans and non-randomized trials (i.e. prospective and retrospective control trials, observational studies, prospective, and retrospective case series and case reports). A database search was performed in Medline, via PubMed, which was conducted for publication type ‘randomized controlled trial’ for each journal separately. A manual search was also performed for additional trials. Two independent reviewers selected and categorized each article and then extracted the data. If randomization was evident from the title and abstract of the articles, the full text of the study was acquired and read to verify and apply the inclusion and exclusion criteria. The journals were also hand-searched for the actual search period to cross-check against the database search. After inclusion of the RCTs, the two authors independently registered what type of blinding was performed for each trial or if no blinding occurred. Then, determination was performed regarding blinding for participants/patients (single-blinding), both participants/patients and trial staff (double-blinding), participants/patients and trial staff as well as outcome assessors (triple-blinding) or if only blinding was carried out for outcome assessors. Of the included RCTs, the two authors also evaluated how many RCTs did not perform any blinding at all, and of those, determined if it may have been possible to perform a type of blinding. It was also evaluated if misclassification of the blinding items occurred (e.g. if a trial claimed to be double-blinded, but it becomes apparent that blinding was only performed on participants/patients and outcome assessors, thus indicating a single-blinded trial). At the title, abstract and full-text screening stages, as well as with any dichotomous and categorical variables during the data extraction, the number of inter-examiner agreements and disagreements were noted and calculated for the assessment of agreement. Then, all the disagreements were discussed and resolved by discussion to reach consensus. Statistical analysis Chi square test and Fisher’s exact test was used to determine differences in categorical data. Differences with probabilities of less than 5 per cent (P-value < 0.05) were considered to be statistically significant. Kappa score was used to measure the level of agreement, and the degree of agreement was 0.96 (i.e. an excellent agreement) (14). Results The initial screening yielded 240 potentially relevant articles all identified via the electronic database search and without any further trials added by manual searching. After applying the inclusion and exclusion criteria, 37 articles were excluded (11 in vitro studies, 18 prospective controlled trials, and 8 retrospective controlled studies). Thus, 203 RCTs were assessed and the numbers of RCTs per journal are presented in Table 1. Table 1. Number of randomized trials (RCTs) per journal and number of different blinding items performed as well as number of RCTs that did not performed any blinding at all. Also the number RCTs that could have achieved blinding is given. Single-blinding includes either participants/patients or the trial staff; double-blinding includes both participants/patients and the trial staff; triple-blinding includes participants/patients, trial staff and outcome assessors. Journal Number of RCTs per journal Number of RCTs that performed single-blinding Number of RCTs that performed double-blinding Number of RCTs that performed tripple-blinding Number of RCTs that performed outcome assessors blinding Number of RCTs that did not performed any blinding at all Number of RCTs that could have achieved blinding among those that did not AJODO 70 (34.4%) 17 0 1 28 24 23 of 24 AO 45 (22.1%) 4 0 1 21 19 15 of 19 AOJ 6 (3%) 0 2 0 0 4 3 of 4 DPJO 9 (4.4%) 2 0 0 0 7 5 of 7 EJO 28 (13.7%) 2 1 4 14 7 5 of 7 JO 10 (4.9%) 2 0 0 7 1 1 of 1 JOO 10 (4.9%) 0 1 0 0 9 4 of 9 KJO 3 (1.4%) 0 0 0 2 1 0 of 1 OCR 6 (3%) 0 0 0 4 2 2 of 2 PO 16 (7.9%) 4 1 1 6 4 4 of 4 Total 203 (100%) 31 (15.3%) 5 (2.5%) 7 (3.4%) 82 (40.4%) 78 (38.4%) 62 of 78 Journal Number of RCTs per journal Number of RCTs that performed single-blinding Number of RCTs that performed double-blinding Number of RCTs that performed tripple-blinding Number of RCTs that performed outcome assessors blinding Number of RCTs that did not performed any blinding at all Number of RCTs that could have achieved blinding among those that did not AJODO 70 (34.4%) 17 0 1 28 24 23 of 24 AO 45 (22.1%) 4 0 1 21 19 15 of 19 AOJ 6 (3%) 0 2 0 0 4 3 of 4 DPJO 9 (4.4%) 2 0 0 0 7 5 of 7 EJO 28 (13.7%) 2 1 4 14 7 5 of 7 JO 10 (4.9%) 2 0 0 7 1 1 of 1 JOO 10 (4.9%) 0 1 0 0 9 4 of 9 KJO 3 (1.4%) 0 0 0 2 1 0 of 1 OCR 6 (3%) 0 0 0 4 2 2 of 2 PO 16 (7.9%) 4 1 1 6 4 4 of 4 Total 203 (100%) 31 (15.3%) 5 (2.5%) 7 (3.4%) 82 (40.4%) 78 (38.4%) 62 of 78 AJODO = American Journal of Orthodontics and Dentofacial Orthopedics; AO = The Angle Orthodontist; AOJ = Australian Orthodontic Journal; DPJO = Dental Press Journal of Orthodontics; EJO = European Journal of Orthodontics; JO = Journal of Orthodontics; JOO = Journal of Orofacial Orthopedics; KJO = Korean Journal of Orthodontics; OCR = Orthodontics and Craniofacial Research; PO = Progress in Orthodontics. View Large Table 1. Number of randomized trials (RCTs) per journal and number of different blinding items performed as well as number of RCTs that did not performed any blinding at all. Also the number RCTs that could have achieved blinding is given. Single-blinding includes either participants/patients or the trial staff; double-blinding includes both participants/patients and the trial staff; triple-blinding includes participants/patients, trial staff and outcome assessors. Journal Number of RCTs per journal Number of RCTs that performed single-blinding Number of RCTs that performed double-blinding Number of RCTs that performed tripple-blinding Number of RCTs that performed outcome assessors blinding Number of RCTs that did not performed any blinding at all Number of RCTs that could have achieved blinding among those that did not AJODO 70 (34.4%) 17 0 1 28 24 23 of 24 AO 45 (22.1%) 4 0 1 21 19 15 of 19 AOJ 6 (3%) 0 2 0 0 4 3 of 4 DPJO 9 (4.4%) 2 0 0 0 7 5 of 7 EJO 28 (13.7%) 2 1 4 14 7 5 of 7 JO 10 (4.9%) 2 0 0 7 1 1 of 1 JOO 10 (4.9%) 0 1 0 0 9 4 of 9 KJO 3 (1.4%) 0 0 0 2 1 0 of 1 OCR 6 (3%) 0 0 0 4 2 2 of 2 PO 16 (7.9%) 4 1 1 6 4 4 of 4 Total 203 (100%) 31 (15.3%) 5 (2.5%) 7 (3.4%) 82 (40.4%) 78 (38.4%) 62 of 78 Journal Number of RCTs per journal Number of RCTs that performed single-blinding Number of RCTs that performed double-blinding Number of RCTs that performed tripple-blinding Number of RCTs that performed outcome assessors blinding Number of RCTs that did not performed any blinding at all Number of RCTs that could have achieved blinding among those that did not AJODO 70 (34.4%) 17 0 1 28 24 23 of 24 AO 45 (22.1%) 4 0 1 21 19 15 of 19 AOJ 6 (3%) 0 2 0 0 4 3 of 4 DPJO 9 (4.4%) 2 0 0 0 7 5 of 7 EJO 28 (13.7%) 2 1 4 14 7 5 of 7 JO 10 (4.9%) 2 0 0 7 1 1 of 1 JOO 10 (4.9%) 0 1 0 0 9 4 of 9 KJO 3 (1.4%) 0 0 0 2 1 0 of 1 OCR 6 (3%) 0 0 0 4 2 2 of 2 PO 16 (7.9%) 4 1 1 6 4 4 of 4 Total 203 (100%) 31 (15.3%) 5 (2.5%) 7 (3.4%) 82 (40.4%) 78 (38.4%) 62 of 78 AJODO = American Journal of Orthodontics and Dentofacial Orthopedics; AO = The Angle Orthodontist; AOJ = Australian Orthodontic Journal; DPJO = Dental Press Journal of Orthodontics; EJO = European Journal of Orthodontics; JO = Journal of Orthodontics; JOO = Journal of Orofacial Orthopedics; KJO = Korean Journal of Orthodontics; OCR = Orthodontics and Craniofacial Research; PO = Progress in Orthodontics. View Large Blinding, of any type, occurred in 125 (61.6%) of the RCTs, while in 78 (38.4%) of the trials, no blinding was performed at all (Table 1). Single-blinding was performed in 31 (15.3%) of the trials, while 5 (2.5%) of the trials performed double-blinding and 7 (3.4%) of the trials performed triple-blinding. In 82 (40.4%) of the trials, outcome assessors blinding was performed without any other type of blinding (Table 1). In total, the blinding of outcome assessors was performed in 87 of 125 trials (69.6%). No main differences were found between the specific journals regarding blinding and type of blinding. However, the journals (AJODO, EJO, JO, and OCR) that require authors adhere CONSORT guidelines, published together significantly more RCTs that performed blinding than journals that not follow the CONSORT, i.e. 70.1 versus 45.5 per cent (P = 0.004). Of the 78 RCTs that did not perform any blinding, 62 trials (79.4%) could have achieved blinding to at least one of the three blinding categories (participants, trial staff and/or outcomes). The other 16 (20.6%) RCTs were of such a design that blinding was not possible whatsoever. The distribution of RCTs that could have achieved blinding among the different journals was fairly equal, with no significant differences between them (P = 0.246). In 15 RCTs (7.4%), the blinding was misclassified. Five RCTs claimed in their articles that single-blinding was performed, when actually, the authors had only blinded the outcome assessors (two trials in JO, and one trial each in AJODO, AO, EJO). A further six RCTs overestimated the blinding and described them as double-blinded when they were single-blinded because only the blinding of the participants/patients or trial staff and the outcome assessors was performed (two trials in AO and one trial each in EJO, JO, JOO, PO). Finally, four RCTs underestimated their blinding and reported double-blinding while they had actually carried out triple-blinding because the participants/patients, the trial staff and the outcome assessors were all blinded (three trials in EJO and one in PO). Discussion It was important and encouraging to note in this methodological investigation that the majority of the RCTs (61.6%) had used blinding for at least one of the three categories to reduce bias within the trial. The most common blinding item produced in these trials was the blinding of outcome assessors (69.6%). Another study investigated the risk of bias in dentistry RCTs published from 1955 to 2013 and blinding of participants was judged to be adequate in 71.5 per cent of the trials, and blinding of the outcome assessment was judged to be adequate in 59.4 per cent of the trials (15). In our study, it was found that blinding was adequate in 69.5 per cent of the RCTs, while misclassifications occurred in 7.4 per cent. It is not surprising that the blinding of outcome assessors was the most frequent type of blinding due to the nature of orthodontic trials, which are often of intervention design. Consequently, blinding is often impossible because the various orthodontic appliances (or no appliance intervention) are obvious and difficult to mask for patients and trial staff. This also helps explain why only 43 (21.2%) RCTs out of 203 performed single-, double-, or triple-blinding. Another important finding was that 38.4 per cent of the RCTs did not perform any blinding at all (Table 1), although in 16 (20.5%) of those, blinding was impossible due to the character or design of the trial. Consequently, in 62 trials, blinding may have been achieved; at least, the blinding of the outcome assessors. This fact also means that the RCTs without blinding are at risk of bias and may produce overestimated results (9). It is also important to mention that the RCTs without blinding will be graded as having a substantial risk for bias in relation to blinding when they are pooled into a systematic review, and for example, evaluated by the Cochrane Risk of Bias Assessment Tool (16). This means the trial will be downgraded in quality and the contribution of the trial to the evidence will also be diminished. When it is not possible to achieve blinding, researchers can still make arrangements to compensate as much as possible for the lack of blinding. For example, in one RCT (17) investigating the effect of the maxillary primary canines extraction effect on the eruption of palatally impacted maxillary permanent canines, blinding of any type was not possible. The patients were randomized into two groups: one received extraction and one served as a control group, but given that half of the patients will receive extractions and the other half of the patients will not, those who have had extractions and those who have not will be obvious to the patients and trial staff. This will also be obvious to the outcome assessors, who will easily notice the empty spaces after the extractions clinically and on the radiographs. Nevertheless, the authors of this study still managed to achieve blinding of the baseline radiographs (15). In contrast, orthodontic RCTs that investigate different drugs for relieving pain, fluoride varnishes for preventing white spot lesions, or altered resin materials for bonding often permit the triple-blinding design. In one study with the aim to evaluate and compare the effect of pre-procedural administration of acetaminophen, ibuprofen, and meloxicam in reducing pain after orthodontic separator placement, a blinding design was performed by covering the tablets with identical gelatine covers, which then permitted the blinding of the patients and trial staff as well as the outcome assessors (17). However, another study did not take the opportunity to achieve blinding when comparing ibuprofen with acetaminophen for the effect on prostaglandin E2 levels of the gingival crevicular fluid (18). Here, the authors could also have covered the tablets with gelatine covers, as was the case in the aforementioned study (17). Another important finding was that 15 RCTs reported the blinding wrongly in relation to single-, double-, or triple-blinding. This misclassification indicates not only that the researchers have insufficient knowledge regarding the definitions for diverse types of blinding but also that the peer-reviewers had obviously not reported the misclassifications. However, it should be pointed out that there can be confusion regarding the definition of blinding, and in particular, the terms ‘single-’, ‘double-’, and ‘triple-blinding’. Therefore, it is recommended that one make precise and explicit definitions and descriptions of what type of blinding has been achieved. Thus, blinding can be based on a protocol covering up to five different categories of blinding, namely, the blinding of: 1. participants/patients; 2. trial staff/ health care providers; 3. data collectors; 4. outcome assessors; and 5. data analysts including statisticians. If blinding cannot be performed for any reason, then acknowledge this limitation and possible bias in the discussion of the trial and explain that the assignment groups, aside from the interventions, were treated as identically as possible. When data collectors or outcome assessors cannot be blinded, use as much as possible objective and/or reliable outcomes as well as arrange to use duplicate assessments of outcomes and declare the degree of inter-assessor concordance obtained (19). All the journals included in this survey were peer-reviewed journals, and four of them (AJODO, EJO, JO, and OCR) request that authors adhere to the CONSORT guidelines to publish an RCT. One of the CONSORT guidelines considers the act of reporting blinding in RCTs (12). Interestingly, this survey shows that those four journals as a group had significant more RCTs that had performed blinding than those that did not follow the CONSORT guidelines. Thus, it may be advisable for other orthodontic journals to request authors to follow the CONSORT guidelines, and hence, a possible increase in the number of RCTs performing blinding. Although the impact factor differed between the journals, no clear pattern was found that the journals with a higher impact factor would be any better at achieving blinding or reporting correctly classified blinding. As mentioned, the blinding of patients, trial staff or outcome assessors is effective in reducing bias within the trial and increases the quality strength of the trial. Starting with the blinding of patients, it prevents reporting bias in patient-reported outcome measures (20). It has been proven that non-blinded patients exaggerate the effect size by 0.56 standard deviations (21). This exaggerated effect is probably due to the patient’s expectations of the treatment effect. In addition, the blinding of patients may also reduce adherence bias; e.g. when the control subjects are aware of their status as members of the control group and are able to compare their performance with those of the treatment group, it may result in members of the control group not following the protocol. Instead, the control subjects actively work harder to overcome the ‘disadvantage’ of being in the control group (20). Finally, the blinding of patients may also decrease the risk of a participant’s attrition from the trial, as it was reported that the risk of attrition in the blinded trials was 4 per cent, while in non-blinded trials it was 7 per cent (21). The blinding of trial staff is also highly important, as the patient and trial staff relationship can have an important effect on the treatment response. Previous research has shown that the behaviour and attitudes of the trial staff often influence patient response (22, 23); thus in general, it is important to keep the relationship between the trial staff and the patients as standardised as possible. Moreover, if the trial staff give the same attention to all of the trial groups or patients (achieved by blinding the trial staff), then this will prevent them from both intentional and unintentional changes in behaviour towards a specific trial group or patient (20). The third category of blinding is the ‘outcome assessors blinding’. Blinding of assessors prevents observer-related bias, as it was demonstrated that non-blinded assessors over-rate patients in the treatment group while under-rating patients in the control group (24). This survey was carried out in 10 well-recognized orthodontic journals and the high number of journals included as well as the relatively long (5.5 years) search period give strength to this study. However, a limitation was that when assessing the trials, we did not remove the name of the authors to avoid bias to be influenced by well-respected research authorship, mentorship or even friendship. Furthermore, one of the assessors had produced many RCTs included in the survey. Nevertheless, the selection and assessment of the RCTs were performed independently by the two authors and the Kappa value was measured and it was 0.96 (excellent agreement). Moreover, the assessment concerned more or less dichotomous decisions, i.e. blinding or not, type of blinding, and thus, limited subjective assessments had to be performed and all evaluations were based on what was written in the trials. Finally, blinding had been ineffective when assessments were to be made by the assessors’ own studies, since then simply the text would be recognizable. Conclusions The majority of the RCTs (61.6%) had used blinding for at least one of the three categories to reduce bias within the trial, and the prevailing blinding item produced was ‘blinding of outcome assessors’ (69.6%). It is not surprising that ‘blinding of outcome assessors’ was the most frequent type since the trials are often of intervention design and difficult to mask for patients and trial staff. A total of 78 RCTs (38.4%) did not perform any blinding; however, of those, 62 (79.4%) could have performed blinding, in particular, ‘outcome assessors blinding’. Fifteen RCTs (7.3%) reported the blinding wrongly in relation to single-, double-, or triple-blinding. The misclassifications of blinding items may indicate suboptimal knowledge among researchers and peer reviewers regarding the definitions of diverse types of blinding. Since confusions existed regarding the classification of single, double and triple blinding it can be recommended to make precise and explicit descriptions of what type of blinding has been achieved. Thus, describe if blinding has been performed of participants/patients; trial staff/health care providers; data collectors; outcome assessors; and data analysts including statisticians. Funding This survey was funded by the authors’ academic institution. Conflict of Interest None to declare. References 1. O’Brien , K. and Craven , R . ( 1995 ) Pitfalls in orthodontic health service research . British Journal of Orthodontics , 22 , 353 – 356 . Google Scholar CrossRef Search ADS PubMed 2. Bondemark , L. and Ruf , S . ( 2015 ) Randomized controlled trial: the gold standard or an unobtainable fallacy ? European Journal of Orthodontics , 37 , 457 – 461 . Google Scholar CrossRef Search ADS PubMed 3. Moher , D. , Jones , A. and Lepage , L .; CONSORT Group (Consolidated Standards for Reporting of Trials) . ( 2001 ) Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation . JAMA , 285 , 1992 – 1995 . Google Scholar CrossRef Search ADS PubMed 4. Lempesi , E. , Koletsi , D. , Fleming , P.S. and Pandis , N . ( 2014 ) The reporting quality of randomized controlled trials in orthodontics . The Journal of Evidence-Based Dental Practice , 14 , 46 – 52 . Google Scholar CrossRef Search ADS PubMed 5. Bondemark , L. and Abdulraheem , S . ( 2017 ) Intention to treat (ITT) analysis as reported in orthodontic randomized controlled trials-evaluations of methodology and recommendations for the accurate use of ITT analysis and handling dropouts . European Journal of Orthodontics . doi: 10.1093/ejo/cjx084 . 6. Dechartres , A. , Trinquart , L. , Atal , I. , Moher , D. , Dickersin , K. , Boutron , I. , Perrodeau , E. , Altman , D.G. and Ravaud , P . ( 2017 ) Evolution of poor reporting and inadequate methods over time in 20920 randomised controlled trials included in Cochrane reviews: research on research study . BMJ , 357 , j2490 . Erratum in: BMJ. 2017, 358, j3806. Google Scholar CrossRef Search ADS PubMed 7. Dechartres , A. , Charles , P. , Hopewell , S. , Ravaud , P. and Altman , D.G . ( 2011 ) Reviews assessing the quality or the reporting of randomized controlled trials are increasing over time but raised questions about how quality is assessed . Journal of Clinical Epidemiology , 64 , 136 – 144 . Google Scholar CrossRef Search ADS PubMed 8. Kloukos , D. , Papageorgiou , S.N. , Doulis , I. , Petridis , H. and Pandis , N . ( 2015 ) Reporting quality of randomised controlled trials published in prosthodontic and implantology journals . Journal of Oral Rehabilitation , 42 , 914 – 925 . Google Scholar CrossRef Search ADS PubMed 9. Schulz , K.F. , Chalmers , I. , Hayes , R.J. and Altman , D.G . ( 1995 ) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials . JAMA , 273 , 408 – 412 . Google Scholar CrossRef Search ADS PubMed 10. Devereaux , P.J. , Manns , B.J. , Ghali , W.A. , Quan , H. , Lacchetti , C. , Montori , V.M. , Bhandari , M. and Guyatt , G.H . ( 2001 ) Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials . JAMA , 285 , 2000 – 2003 . Google Scholar CrossRef Search ADS PubMed 11. Jüni , P. , Altman , D.G. and Egger , M . ( 2001 ) Assessing the quality of controlled clinical trials . BMJ , 323 , 42 – 46 . Google Scholar CrossRef Search ADS PubMed 12. CONSORT . ( 2010 )http://www.equator-network.org/reporting-guidelines/consort/ and http://www.consort-statement.org/ (31 July 2017, date last accessed). 13. Montori , V.M. , Bhandari , M. , Devereaux , P.J. , Manns , B.J. , Ghali , W.A. and Guyatt , G.H . ( 2002 ) In the dark: the reporting of blinding status in randomized controlled trials . Journal of Clinical Epidemiology , 55 , 787 – 790 . Google Scholar CrossRef Search ADS PubMed 14. Fleiss , J.L . ( 1981 ) Statistical Methods for Rates and Proportions . John Wiley , New York , 2nd edn , pp. 38 – 46 . 15. Saltaji , H. , Armijo-Olivo , S. , Cummings , G.G. , Amin , M. and Flores-Mir , C . ( 2017 ) Randomized clinical trials in dentistry: Risks of bias, risks of random errors, reporting quality, and methodologic quality over the years 1955-2013 . PloS One , 12 , e0190089 . Google Scholar CrossRef Search ADS PubMed 16. Naoumova , J. , Kürol , J. and Kjellberg , H . ( 2015 ) Extraction of the deciduous canine as an interceptive treatment in children with palatally displaced canines - part II: possible predictors of success and cut-off points for a spontaneous eruption . European Journal of Orthodontics , 37 , 219 – 229 . Google Scholar CrossRef Search ADS PubMed 17. Zarif Najafi , H. , Oshagh , M. , Salehi , P. , Babanouri , N. and Torkan , S . ( 2015 ) Comparison of the effects of preemptive acetaminophen, ibuprofen, and meloxicam on pain after separator placement: a randomized clinical trial . Progress in Orthodontics , 16 , 34 . Google Scholar CrossRef Search ADS PubMed 18. Shetty , N. , Patil , A.K. , Ganeshkar , S.V. and Hegde , S . ( 2013 ) Comparison of the effects of ibuprofen and acetaminophen on PGE2 levels in the GCF during orthodontic tooth movement: a human study . Progress in Orthodontics , 14 , 6 . Google Scholar CrossRef Search ADS PubMed 19. Karanicolas , P.J. , Farrokhyar , F. and Bhandari , M . ( 2010 ) Practical tips for surgical research: blinding: who, what, when, why, how ? Canadian Journal of Surgery , 53 , 345 – 348 . 20. Wartolowska , K. , Beard , D. and Carr , A . ( 2017 ) Blinding in trials of interventional procedures is possible and worthwhile . F1000Research , 6 , 1663 . Google Scholar CrossRef Search ADS PubMed 21. Hróbjartsson , A. , Emanuelsson , F. , Skou Thomsen , A.S. , Hilden , J. and Brorson , S . ( 2014 ) Bias due to lack of patient blinding in clinical trials. a systematic review of trials randomizing patients to blind and nonblind sub-studies . International Journal of Epidemiology , 43 , 1272 – 1283 . Google Scholar CrossRef Search ADS PubMed 22. Benedetti , F . ( 2002 ) How the doctor’s words affect the patient’s brain . Evaluation & The Health Professions , 25 , 369 – 386 . Google Scholar CrossRef Search ADS PubMed 23. Thomas , K.B . ( 1987 ) General practice consultations: is there any point in being positive ? British Medical Journal (Clinical research ed.) , 294 , 1200 – 1202 . Google Scholar CrossRef Search ADS PubMed 24. Hróbjartsson , A. , Thomsen , A.S. , Emanuelsson , F. , Tendal , B. , Hilden , J. , Boutron , I. , Ravaud , P. and Brorson , S . ( 2012 ) Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors . British Medical Journal (Clinical research ed.) , 344 , e1119 . Google Scholar CrossRef Search ADS © The Author(s) 2018. Published by Oxford University Press on behalf of the European Orthodontic Society. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

The European Journal of OrthodonticsOxford University Press

Published: Apr 24, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off