Hawthorne effect reporting in orthodontic randomized controlled trials: truth or myth? Blessing or curse?

Hawthorne effect reporting in orthodontic randomized controlled trials: truth or myth? Blessing... Summary Objective To investigate in 10 orthodontic journals how many randomized controlled trials (RCTs) considered the Hawthorne effect, and if considered, to determine whether it was related to the patients or the therapists involved in the trial and, finally, to discuss the Hawthorne effect in an educational way. Materials and methods A search was performed on the Medline database, via PubMed, for publication type ‘randomized controlled trial’ published for each journal between 1 August 2007 and 31 July 2017. The American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, Australian Orthodontic Journal, Dental Press Journal of Orthodontics, European Journal of Orthodontics, Journal of Orthodontics, Journal of Orofacial Orthopedics, Korean Journal of Orthodontics, Orthodontics and Craniofacial Research and Progress in Orthodontics were assessed. Two independent reviewers extracted the data and identified whether the Hawthorne effect was considered or discussed in the articles and whether the Hawthorne effect was related to the behaviour of the patients, the therapists, or both. Results The initial search generated 502 possible trials. After applying the inclusion and exclusion criteria, 290 RCTs were included and assessed. The Hawthorne effect was considered or discussed in 10 of 290 RCTs (3.4%), and all were related to the patients’ and none to the therapists’ behaviour. Conclusions The Hawthorne effect reported in orthodontic RCTs was suboptimal. The researchers’ lack of knowledge about this phenomenon is evident, despite evidence that the Hawthorne effect may cause over-optimistic results or false-positive bias. Introduction It is well known that rigorously conducted randomized controlled trials (RCTs) to assess the effectiveness of different treatments may be valuable for the medical and dental care of our patients, as well as for the whole community. The results from RCTs mean that detrimental or ineffective treatments are avoided and replaced by more efficient ones. However, when performing and analysing a trial, researchers must take into account the possibility that the participants’ behavioural changes may affect the results of the trial. Four examples of these behavioural changes are the Hawthorne effect, Novelty effect, John Henry effect, and the Pygmalion effect (1–4). The Hawthorne effect is defined as changes in patients’ or therapists’ behaviour when involved in a trial because of increased knowledge or interest or else due to feeling observed in the trial (1). The Novelty effect is explained by a tendency for the subjects to want to do better with a new or improved technology because they are intrigued by it rather than the effect being truly based on the variables under study (2). The John Henry effect is defined as when the control subjects are aware of their status as members of the control group and are able to compare their performance with that of the treatment group, which means that members of the control group may actively work harder to overcome the ‘disadvantage’ of being in the control group (3). This may happen when an innovative technology is evaluated and the control group participants behave in an unnatural way to try to outwit the new technology (3). Finally, the Pygmalion effect is defined as when we expect certain behaviours of others, we are likely to act in a way that makes the expected behaviour more likely to occur (4). Of the phenomena described above, the Hawthorne effect is the most discussed phenomenon in the literature as having an effect and bias in the results of RCTs in the medical and dental fields (5). The type of bias caused by this phenomenon is a false-positive bias that may lead to over-optimistic results. The name, Hawthorne, originates from a factory called Hawthorne Works, which was part of the Western Electric Company outside of Chicago. A series of extensive research was conducted focusing on the workers in the factory to investigate the effects that changing the lighting conditions would have on the workers’ productivity. It was found that by changing the work environment by increasing or decreasing the lighting conditions, the workers productivity was altered (6). Since then, the Hawthorne effect has appeared in discussions about clinical trials and has been accused of causing positive bias on the results of medical and dental RCTs (7, 8). In 2008, Sandler et al. (8) claimed to have witnessed the Hawthorne effect when performing an RCT comparing palatal implants with the headgear in supplementing anchorage during orthodontic treatment. They wrote in the discussion that ‘The headgear patients in the study surprised the clinicians with the speed and efficiency of this method of anchorage supplementation. In some cases, the headgear was discontinued after only five months; on average, headgear wear was continued for nine months. We believed that we had witnessed the Hawthorne effect’ (8). In this example, the investigators noticed the Hawthorne effect, reported it and related the phenomenon to patients’ behaviour. However, they did not stress another important consideration or relation of the Hawthorne effect, namely, that the therapist’s behaviour may also have changed during the trial. According to the definition mentioned above, the phenomenon affects both patients and therapists. Nevertheless, an ongoing debate is present in the literature to determine whether this phenomenon is a myth or true and whether it smears our trial results with false-positive bias and a wrong presentation of real clinical situations. A review (5) was assessed to determine whether the Hawthorne effect exists, and if so, to explore under what conditions and estimate the size of such an effect. The conclusion confirmed the presence of this phenomenon but without any conclusion about the condition that led to it or its magnitude (5). Consequently, the Hawthorne effect may be an important factor affecting clinical trial results in orthodontics, but to our knowledge, no attempt has ever been made to evaluate to what extent the Hawthorne effect is considered in RCTs in orthodontics. Therefore, the aim of this study was to investigate, in 10 orthodontic journals, how many RCTs considered the Hawthorne effect; if it was considered, to determine whether the Hawthorne effect was related to just the patients or also to the therapist involved in the trial; and, finally, to discuss the role of the Hawthorne effect from an educational point of view and from an orthodontic research perspective. Materials and methods The inclusion criteria were human RCTs that used the individual as a unit, and thus, RCTs applying split-mouth design were excluded. A search was performed on the database Medline, via PubMed, for publication type ‘randomized controlled trial’ published for each journal between 1 August 2007 and 31 July 2017. The 10 orthodontic journals assessed were the American Journal of Orthodontics and Dentofacial Orthopedics (AJODO), The Angle Orthodontist (AO), Australian Orthodontic Journal (AOJ), Dental Press Journal of Orthodontics (DPJO), European Journal of Orthodontics (EJO), Journal of Orthodontics (JO), Journal of Orofacial Orthopedics (JOO), Korean Journal of Orthodontics (KJO), Orthodontics and Craniofacial Research (OCR), and Progress in Orthodontics (PO). Each article was selected and the data were extracted by two independent reviewers. If randomization was obvious from the title and abstract of the trial, the full text of the trial was read to verify and clarify the study design according to human RCTs. Then, the full text was reviewed to identify whether the Hawthorne effect was considered or discussed in the article and whether the Hawthorne effect was related to the behaviour of the patients, therapists, or both. And finally, to cross-check against the database search, the 10 journals were hand searched for the actual search period. At the title, abstract and full-text screening stages as well as for dichotomous and categorical variables during data extraction, the number of inter-examiner agreements and disagreements were noted and calculated for the assessment of agreement. Then, all the disagreements were discussed and resolved by discussion to reach consensus. Kappa score was used to measure the level of agreement, and the degree of agreement was 0.91 (i.e. an excellent agreement) (9). Results The initial search generated 502 possible concordant trials. Two-hundred and twelve trials were excluded based on the inclusion and exclusion criteria, i.e. 93 split mouth design, 75 in vitro studies, and 44 prospective controlled or retrospective studies. Consequently, 290 RCTs were included and assessed, and the number of RCTs per journal is presented in Table 1. Table 1. Number of randomized controlled trials (RCTs) per journal and number of RCTs that considered the Hawthorne effect and related it to patients and therapists. Journals Number of RCTs per journal (%) Number of the total RCTs (%) that reported Hawthorne effect related to therapists Number of the total RCTs (%) that reported the Hawthorne effect Number of the total RCTs (%) that reported the Hawthorne effect related to patients AJODO 105 (36.2) 5 (1.7) 5 (1.7) 0 (0.0) AO 55 (19.0) 3 (1.0) 3 (1.0) 0 (0.0) AOJ 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) DPJO 5 (1.7) 0 (0.0) 0 (0.0) 0 (0.0) EJO 51 (17.6) 2 (0.7) 2 (0.7) 0 (0.0) JO 21 (7.2) 0 (0.0) 0 (0.0) 0 (0.0) JOO 18 (6.2) 0 (0.0) 0 (0.0) 0 (0.0) KJO 2 (0.7) 0 (0.0) 0 (0.0) 0 (0.0) OCR 9 (3.1) 0 (0.0) 0 (0.0) 0 (0.0) PO 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) Total 290 (100) 10 (3.4) 10 (3.4) 0 (0.0) Journals Number of RCTs per journal (%) Number of the total RCTs (%) that reported Hawthorne effect related to therapists Number of the total RCTs (%) that reported the Hawthorne effect Number of the total RCTs (%) that reported the Hawthorne effect related to patients AJODO 105 (36.2) 5 (1.7) 5 (1.7) 0 (0.0) AO 55 (19.0) 3 (1.0) 3 (1.0) 0 (0.0) AOJ 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) DPJO 5 (1.7) 0 (0.0) 0 (0.0) 0 (0.0) EJO 51 (17.6) 2 (0.7) 2 (0.7) 0 (0.0) JO 21 (7.2) 0 (0.0) 0 (0.0) 0 (0.0) JOO 18 (6.2) 0 (0.0) 0 (0.0) 0 (0.0) KJO 2 (0.7) 0 (0.0) 0 (0.0) 0 (0.0) OCR 9 (3.1) 0 (0.0) 0 (0.0) 0 (0.0) PO 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) Total 290 (100) 10 (3.4) 10 (3.4) 0 (0.0) AJODO, American Journal of Orthodontics and Dentofacial Orthopedics; AO, The Angle Orthodontist; AOJ, Australian Orthodontic Journal; DPJO, Dental Press Journal of Orthodontics; EJO, European Journal of Orthodontics; JO, Journal of Orthodontics; JOO, Journal of Orofacial Orthopedics; KJO, Korean Journal of Orthodontics; OCR, Orthodontics and Craniofacial Research; PO, Progress in Orthodontics. View Large Table 1. Number of randomized controlled trials (RCTs) per journal and number of RCTs that considered the Hawthorne effect and related it to patients and therapists. Journals Number of RCTs per journal (%) Number of the total RCTs (%) that reported Hawthorne effect related to therapists Number of the total RCTs (%) that reported the Hawthorne effect Number of the total RCTs (%) that reported the Hawthorne effect related to patients AJODO 105 (36.2) 5 (1.7) 5 (1.7) 0 (0.0) AO 55 (19.0) 3 (1.0) 3 (1.0) 0 (0.0) AOJ 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) DPJO 5 (1.7) 0 (0.0) 0 (0.0) 0 (0.0) EJO 51 (17.6) 2 (0.7) 2 (0.7) 0 (0.0) JO 21 (7.2) 0 (0.0) 0 (0.0) 0 (0.0) JOO 18 (6.2) 0 (0.0) 0 (0.0) 0 (0.0) KJO 2 (0.7) 0 (0.0) 0 (0.0) 0 (0.0) OCR 9 (3.1) 0 (0.0) 0 (0.0) 0 (0.0) PO 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) Total 290 (100) 10 (3.4) 10 (3.4) 0 (0.0) Journals Number of RCTs per journal (%) Number of the total RCTs (%) that reported Hawthorne effect related to therapists Number of the total RCTs (%) that reported the Hawthorne effect Number of the total RCTs (%) that reported the Hawthorne effect related to patients AJODO 105 (36.2) 5 (1.7) 5 (1.7) 0 (0.0) AO 55 (19.0) 3 (1.0) 3 (1.0) 0 (0.0) AOJ 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) DPJO 5 (1.7) 0 (0.0) 0 (0.0) 0 (0.0) EJO 51 (17.6) 2 (0.7) 2 (0.7) 0 (0.0) JO 21 (7.2) 0 (0.0) 0 (0.0) 0 (0.0) JOO 18 (6.2) 0 (0.0) 0 (0.0) 0 (0.0) KJO 2 (0.7) 0 (0.0) 0 (0.0) 0 (0.0) OCR 9 (3.1) 0 (0.0) 0 (0.0) 0 (0.0) PO 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) Total 290 (100) 10 (3.4) 10 (3.4) 0 (0.0) AJODO, American Journal of Orthodontics and Dentofacial Orthopedics; AO, The Angle Orthodontist; AOJ, Australian Orthodontic Journal; DPJO, Dental Press Journal of Orthodontics; EJO, European Journal of Orthodontics; JO, Journal of Orthodontics; JOO, Journal of Orofacial Orthopedics; KJO, Korean Journal of Orthodontics; OCR, Orthodontics and Craniofacial Research; PO, Progress in Orthodontics. View Large The Hawthorne effect was considered or discussed in 10 of 290 RCTs (3.4%). Five RCTs were found in AJODO, three in AO, and two in EJO. All 10 RCTs related the Hawthorne effect to the patients’ behavioural changes without including the therapists (2, 8, 10–17) (Table 2). Four of the RCTs evaluated different oral hygiene protocols or instruments (2, 10, 15, 16), one analysed Streptococcus mutans counts in two groups of different retainers (11), another trial investigated the influence of different bracket types on halitosis and periodontal conditions (12), while Wiedel and Bondemark (13) compared fixed and removable appliances for correction of anterior cross bite (Table 2). In the trial by O′Rourke et al. (14), the Hawthorne effect was discussed in the context of not using special monitoring sensors (TheraMon), which are usually used to check patients’ compliance. The trial avoided using the sensors because it was thought that the patients would feel followed or monitored if used. In the trial by Sandler et al. (8), the Hawthorne effect was related to the unexpected excellent compliance with the headgear use, and finally, in the study by Yaqoob et al. (17), the Hawthorne effect was discussed in relation to the overall good compliance with the daily use of functional appliances (Table 2). Table 2. How the Hawthorne effect considered or discussed in the 10 randomized controlled trials (RCTs) of the survey. The studies are tabulated by increasing number as the studies have in the reference list. RCT Hawthorne effect Bowen et al. 2015 (2) It is possible that oral hygiene improved because of performed bias that could be introduced in this study from the Hawthorne and novelty effects. The Hawthorne effect is a form of reactivity in which subjects improve or modify an aspect of their behaviour being experimentally measured simply in response to the fact that they are being studied, not in response to any particular experimental manipulation. Sandler et al. 2008 (8) The headgear patients in the study surprised the clinicians with the speed and efficiency of this method of anchorage supplementation. In some cases, the headgear was discontinued after only five months; on average, headgear wear was continued for nine months. We believed that we had witnessed the Hawthorne effect. Zinger et al. 2014 (10) In addition to the impact of overall oral hygiene level of the current study population, MacGregor and Rugg- Gunn (1986) pointed out that awareness of being observed may change brushing behaviour (Hawthorne effect). Farhadian et al. 2016 (11) Patients who participate in such a clinical trial pay more attention to their oral hygiene, independent of the treatment modality (Hawthorne effect). However, it seems that the use of silver nanoparticles has a significant impact on reducing the colonization of cariogenic bacteria. Kaygisiz et al. 2015 (12) The results of this study showed no significant differences in plaque index, gingival index, pocket depth, and bleeding on probing values changes during the treatment or observation time among the groups. This may be attributed to the effect of an oral hygiene supplement given before the treatment/observation and motivation of the patients. It can also be attributed to the Hawthorne Effect (patient awareness of being examined and evaluated). Wiedel and Bondemark 2015 (13) However, it must be acknowledged that clinical trials run the risk of the Hawthorne effect (positive bias), whereby subjects tend to perform better when they are participants in an experiment. Consequently, the Hawthorne effect has influence on both groups, i.e. the patients were more compliant than the average orthodontic patients in everyday orthodontic practice. O′Rourke et al. 2016 (14) The Hawthorne effect was discussed from the perspective not to use special monitoring sensors (TheraMon), which usually are applicated to check patients’ compliance. The sensors were avoided to be used in the trial because if used, it was regarded that the patients should feel followed or monitored. Sharma et al. 2008 (15) The plaque reduction in the dental water jet group was significantly greater than in the flossing and manual toothbrush groups, but the reduction of interproximal bleeding did not reach significance at 28 days. This might be attributed to the subjects’ response to the reduction of plaque or the quality of their plaque, the required twice daily brushing for 2 min, and the possible improved technique for brushing and flossing because of being subjects in a clinical study (Hawthorne effect). Costa et al. 2010 (16) The sequence distribution of the patients was done to eliminate the influence of the Hawthorne effect on the results, after each type of brush was used by a group of patients in the three periods of the study. Yaqoob et al. 2012 (17) Compliance overall in this study compares favourably with that in previous studies, and although it is difficult to ascertain why this is so, it is probably, at least in part, due to careful clinical management, the experience of the supervising clinician (Dr DiBiase) with the appliance, and possibly the fact that the patients knew they were in a study. RCT Hawthorne effect Bowen et al. 2015 (2) It is possible that oral hygiene improved because of performed bias that could be introduced in this study from the Hawthorne and novelty effects. The Hawthorne effect is a form of reactivity in which subjects improve or modify an aspect of their behaviour being experimentally measured simply in response to the fact that they are being studied, not in response to any particular experimental manipulation. Sandler et al. 2008 (8) The headgear patients in the study surprised the clinicians with the speed and efficiency of this method of anchorage supplementation. In some cases, the headgear was discontinued after only five months; on average, headgear wear was continued for nine months. We believed that we had witnessed the Hawthorne effect. Zinger et al. 2014 (10) In addition to the impact of overall oral hygiene level of the current study population, MacGregor and Rugg- Gunn (1986) pointed out that awareness of being observed may change brushing behaviour (Hawthorne effect). Farhadian et al. 2016 (11) Patients who participate in such a clinical trial pay more attention to their oral hygiene, independent of the treatment modality (Hawthorne effect). However, it seems that the use of silver nanoparticles has a significant impact on reducing the colonization of cariogenic bacteria. Kaygisiz et al. 2015 (12) The results of this study showed no significant differences in plaque index, gingival index, pocket depth, and bleeding on probing values changes during the treatment or observation time among the groups. This may be attributed to the effect of an oral hygiene supplement given before the treatment/observation and motivation of the patients. It can also be attributed to the Hawthorne Effect (patient awareness of being examined and evaluated). Wiedel and Bondemark 2015 (13) However, it must be acknowledged that clinical trials run the risk of the Hawthorne effect (positive bias), whereby subjects tend to perform better when they are participants in an experiment. Consequently, the Hawthorne effect has influence on both groups, i.e. the patients were more compliant than the average orthodontic patients in everyday orthodontic practice. O′Rourke et al. 2016 (14) The Hawthorne effect was discussed from the perspective not to use special monitoring sensors (TheraMon), which usually are applicated to check patients’ compliance. The sensors were avoided to be used in the trial because if used, it was regarded that the patients should feel followed or monitored. Sharma et al. 2008 (15) The plaque reduction in the dental water jet group was significantly greater than in the flossing and manual toothbrush groups, but the reduction of interproximal bleeding did not reach significance at 28 days. This might be attributed to the subjects’ response to the reduction of plaque or the quality of their plaque, the required twice daily brushing for 2 min, and the possible improved technique for brushing and flossing because of being subjects in a clinical study (Hawthorne effect). Costa et al. 2010 (16) The sequence distribution of the patients was done to eliminate the influence of the Hawthorne effect on the results, after each type of brush was used by a group of patients in the three periods of the study. Yaqoob et al. 2012 (17) Compliance overall in this study compares favourably with that in previous studies, and although it is difficult to ascertain why this is so, it is probably, at least in part, due to careful clinical management, the experience of the supervising clinician (Dr DiBiase) with the appliance, and possibly the fact that the patients knew they were in a study. View Large Table 2. How the Hawthorne effect considered or discussed in the 10 randomized controlled trials (RCTs) of the survey. The studies are tabulated by increasing number as the studies have in the reference list. RCT Hawthorne effect Bowen et al. 2015 (2) It is possible that oral hygiene improved because of performed bias that could be introduced in this study from the Hawthorne and novelty effects. The Hawthorne effect is a form of reactivity in which subjects improve or modify an aspect of their behaviour being experimentally measured simply in response to the fact that they are being studied, not in response to any particular experimental manipulation. Sandler et al. 2008 (8) The headgear patients in the study surprised the clinicians with the speed and efficiency of this method of anchorage supplementation. In some cases, the headgear was discontinued after only five months; on average, headgear wear was continued for nine months. We believed that we had witnessed the Hawthorne effect. Zinger et al. 2014 (10) In addition to the impact of overall oral hygiene level of the current study population, MacGregor and Rugg- Gunn (1986) pointed out that awareness of being observed may change brushing behaviour (Hawthorne effect). Farhadian et al. 2016 (11) Patients who participate in such a clinical trial pay more attention to their oral hygiene, independent of the treatment modality (Hawthorne effect). However, it seems that the use of silver nanoparticles has a significant impact on reducing the colonization of cariogenic bacteria. Kaygisiz et al. 2015 (12) The results of this study showed no significant differences in plaque index, gingival index, pocket depth, and bleeding on probing values changes during the treatment or observation time among the groups. This may be attributed to the effect of an oral hygiene supplement given before the treatment/observation and motivation of the patients. It can also be attributed to the Hawthorne Effect (patient awareness of being examined and evaluated). Wiedel and Bondemark 2015 (13) However, it must be acknowledged that clinical trials run the risk of the Hawthorne effect (positive bias), whereby subjects tend to perform better when they are participants in an experiment. Consequently, the Hawthorne effect has influence on both groups, i.e. the patients were more compliant than the average orthodontic patients in everyday orthodontic practice. O′Rourke et al. 2016 (14) The Hawthorne effect was discussed from the perspective not to use special monitoring sensors (TheraMon), which usually are applicated to check patients’ compliance. The sensors were avoided to be used in the trial because if used, it was regarded that the patients should feel followed or monitored. Sharma et al. 2008 (15) The plaque reduction in the dental water jet group was significantly greater than in the flossing and manual toothbrush groups, but the reduction of interproximal bleeding did not reach significance at 28 days. This might be attributed to the subjects’ response to the reduction of plaque or the quality of their plaque, the required twice daily brushing for 2 min, and the possible improved technique for brushing and flossing because of being subjects in a clinical study (Hawthorne effect). Costa et al. 2010 (16) The sequence distribution of the patients was done to eliminate the influence of the Hawthorne effect on the results, after each type of brush was used by a group of patients in the three periods of the study. Yaqoob et al. 2012 (17) Compliance overall in this study compares favourably with that in previous studies, and although it is difficult to ascertain why this is so, it is probably, at least in part, due to careful clinical management, the experience of the supervising clinician (Dr DiBiase) with the appliance, and possibly the fact that the patients knew they were in a study. RCT Hawthorne effect Bowen et al. 2015 (2) It is possible that oral hygiene improved because of performed bias that could be introduced in this study from the Hawthorne and novelty effects. The Hawthorne effect is a form of reactivity in which subjects improve or modify an aspect of their behaviour being experimentally measured simply in response to the fact that they are being studied, not in response to any particular experimental manipulation. Sandler et al. 2008 (8) The headgear patients in the study surprised the clinicians with the speed and efficiency of this method of anchorage supplementation. In some cases, the headgear was discontinued after only five months; on average, headgear wear was continued for nine months. We believed that we had witnessed the Hawthorne effect. Zinger et al. 2014 (10) In addition to the impact of overall oral hygiene level of the current study population, MacGregor and Rugg- Gunn (1986) pointed out that awareness of being observed may change brushing behaviour (Hawthorne effect). Farhadian et al. 2016 (11) Patients who participate in such a clinical trial pay more attention to their oral hygiene, independent of the treatment modality (Hawthorne effect). However, it seems that the use of silver nanoparticles has a significant impact on reducing the colonization of cariogenic bacteria. Kaygisiz et al. 2015 (12) The results of this study showed no significant differences in plaque index, gingival index, pocket depth, and bleeding on probing values changes during the treatment or observation time among the groups. This may be attributed to the effect of an oral hygiene supplement given before the treatment/observation and motivation of the patients. It can also be attributed to the Hawthorne Effect (patient awareness of being examined and evaluated). Wiedel and Bondemark 2015 (13) However, it must be acknowledged that clinical trials run the risk of the Hawthorne effect (positive bias), whereby subjects tend to perform better when they are participants in an experiment. Consequently, the Hawthorne effect has influence on both groups, i.e. the patients were more compliant than the average orthodontic patients in everyday orthodontic practice. O′Rourke et al. 2016 (14) The Hawthorne effect was discussed from the perspective not to use special monitoring sensors (TheraMon), which usually are applicated to check patients’ compliance. The sensors were avoided to be used in the trial because if used, it was regarded that the patients should feel followed or monitored. Sharma et al. 2008 (15) The plaque reduction in the dental water jet group was significantly greater than in the flossing and manual toothbrush groups, but the reduction of interproximal bleeding did not reach significance at 28 days. This might be attributed to the subjects’ response to the reduction of plaque or the quality of their plaque, the required twice daily brushing for 2 min, and the possible improved technique for brushing and flossing because of being subjects in a clinical study (Hawthorne effect). Costa et al. 2010 (16) The sequence distribution of the patients was done to eliminate the influence of the Hawthorne effect on the results, after each type of brush was used by a group of patients in the three periods of the study. Yaqoob et al. 2012 (17) Compliance overall in this study compares favourably with that in previous studies, and although it is difficult to ascertain why this is so, it is probably, at least in part, due to careful clinical management, the experience of the supervising clinician (Dr DiBiase) with the appliance, and possibly the fact that the patients knew they were in a study. View Large Discussion The main findings of this survey were that very few orthodontic RCTs considered the Hawthorne effect (3.4%). Furthermore, the few RCTs that did consider the Hawthorne effect related it to only the patients’ behaviour and never to the therapist involved in the trial, even though it is clearly mentioned in the definition of the Hawthorne effect that the therapist’s behaviour can also be included. It was noticed that trials investigating oral hygiene methods reported the Hawthorne effect more frequently, however, without any real explanation for this fact. Despite RCTs are considered the best way to minimize bias, wrong randomization procedures, missing data, and noncompliance are some of the biases that can occur within the RCT’s design affecting their external and internal validity (18–20). All these types of bias are involved in the CONSORT guidelines, which most of the orthodontic journals require adherence to before publishing a paper. However, the Hawthorne effect is not included in the CONSORT guideline keys (21), and conceivably, this fact may explain why so few RCTs reported or considered the Hawthorne effect. The reason for excluding RCTs with split-mouth design was that the Hawthorne effect will always affect the treatment/treatment or treatment/control simultaneously within the mouth, while in an RCT using the individual as a unit in one group and not included in another group, the Hawthorne effect can affect the groups in a varying degree, and thereby, the consequences can be different per group. When we analysed in detail the RCTs that did not consider the Hawthorne effect, a majority of those still reported or witnessed short-term unusual treatment results without considering the Hawthorne effect as a possible cause. Thus, it can be recommended to be aware of and take into account the Hawthorne effect in RCTs, especially when unusual compliance rates or results are reached. Sandler et al. (8) noticed patients’ unusual compliance with headgear, i.e. the headgear patients in the study surprised the clinicians with the speed and efficiency of headgear anchorage supplementation, and reported that they witnessed the Hawthorne effect. A similar finding was reported by Feldmann (22), who reported that the headgear even moved the molars distally during the first 6 months of using the headgear anchorage, but during the following 12 month treatment period, anchorage loss of just over 1 mm was found in the headgear group. Feldmann (22) claimed that the high level of patient cooperation during the first 6 months could partly be attributed to the Hawthorne effect. An interesting finding of the different intensities of the Hawthorne effect was evident in a trial that evaluated the effect of a sensor (TheraMon) that recorded how long (in hours) the patients used a removable orthodontic appliance (23). There were two groups. In one group, the patients were aware of the sensor that measured how long they used the appliance, while in the other group, the patients had no knowledge of the sensor. It was found that the patients who were aware of the sensor used the appliance significantly more than the patients in the other group (20). This example indicates that the patients in both groups were influenced by the Hawthorne effect. However, the patients in the group aware that their use of the removable appliance would be evaluated with a sensor (in addition to being aware of participating in the trial), adapted even more by using the removable appliance more often. Thus, researchers who incorporate sensors to check patients’ compliance in a trial should be aware of a particular behaviour phenomenon, i.e. the Hawthorne effect and a sensor effect. One strength of this study was that two independent reviewers searched for and critically brought data to this survey. An excellent agreement (Kappa score of 0.91) was found for the reviewers. Other strengths were the long time span of 10 years search duration and that 10 orthodontic journals have been included and reviewed. Is the Hawthorne effect true or a myth? As earlier mentioned, Sandler et al. (8) claimed to have witnessed the Hawthorne effect when performing an RCT comparing palatal implants with the headgear in supplementing anchorage during orthodontic treatment, and in the ten RCTs found in this survey, the Hawthorne effect was discussed and related to patients’ behaviour. Also, many studies in the medical fields have investigated the Hawthorne effect. One study (7) attempted to quantify the Hawthorne effect, and in that study, the aim was to compare minimal follow-up to intensive follow-up in participants in a placebo-controlled trial of Ginkgo biloba for treatment of mild-to-moderate dementia. It was found that the group with intensive follow-up had significantly better results, as measured by their cognitive functioning, and this was attributed to the Hawthorne effect (7). However, in another study (24), the Hawthorne effect on the therapists was analysed by determination of obstetricians’ clinical estimation of foetal weight (EFW). The obstetricians in the experimental group were informed that their EFW accuracy should be studied but without them being aware that the investigators were evaluating the Hawthorne effect. The control group consisted of obstetricians who made their EFW before the study was conducted and without knowing that they were observed. The conclusion was that EFW accuracies were similar between the groups without any bias caused by the Hawthorne effect (24). Finally, a systematic review concluded that the Hawthorne effect was considered a true effect and not a myth. Thus, the Hawthorne effect may bias our study results but without knowing its magnitude and under which conditions it may be raised (5). Is the Hawthorne effect a blessing or a curse? When performing RCTs, the Hawthorne effect is often regarded as a curse because the phenomenon can cause changes in participants’ behaviour during the trial, thus making them perform better when compared with real-life clinical situations. As a result, overestimated results, i.e. false-positive bias, may be a consequence of this phenomenon (5, 7). In contrast, the Hawthorne effect may be viewed as a blessing when the manipulation of the results is wanted. A recent systematic review concluded that the main behaviour attributed to Hawthorne effect disappeared after a 6 month trial period (5). This also means that if the researchers shorten the observation period as much as possible (<6 months), the participants will easily adapt and fulfil their ‘good trial behaviour’ during this short observation period. Thus, a shortening of the trial period will result in a relatively greater Hawthorne effect, which may affect the results implying a higher positive bias than if a long observation period had been chosen (5). Therefore, if we want to decrease the Hawthorne effect on the behaviour of study participants and therapists in a trial, setting an observation period that is longer than 6 months is recommended. Given that orthodontic studies may include observation periods from a few days to several decades of follow-up, it may therefore be important to be aware and understand that the Hawthorne effect is likely to be the greatest for short-term studies and practically non-existent for long observation periods of several years. In any case, the Hawthorne effect should be regarded as neither a blessing nor a curse, but instead, depending on the study design, considered a factor that can contribute to positive bias of the results obtained. Recommendations Always analyse and report unexpected overestimated results and discuss whether the Hawthorne effect is a possible explanation for these unexpected results. Be aware that a short observation period often implies the increased risk of the Hawthorne effect, and if the study design allows for it, plan a longer observation period than 6 months to minimize the Hawthorne effect. Conclusion In this survey, only 3.4% of the RCTs reported the Hawthorne effect despite indications that the other RCTs that did not consider the Hawthorne effect should have regarded this effect as a possible source of false-positive bias. All the RCTs related the Hawthorne effect to the patients’ behavioural changes without including the therapists. Evidence exists that the Hawthorne effect on patients’ performance and behaviour is to be considered a true effect and not a myth; however, there is no evidence that the therapists’ behaviour may also be affected. The Hawthorne effect should be regarded as neither a blessing nor a curse, but it is important to be aware that the Hawthorne effect is likely to be the greatest for short-term trials and practically non-existent during long observation periods of several years. Funding This survey was funded by the authors’ academic institution. Conflict of Interest None to declare. References 1. Braunholtz , D.A. , Edwards , S.J. and Lilford , R.J . ( 2001 ) Are randomized clinical trials good for us (in the short term)? Evidence for a “trial effect” . Journal of Clinical Epidemiology , 54 , 217 – 224 . Google Scholar CrossRef Search ADS PubMed 2. Bowen , T.B. , Rinchuse , D.J. , Zullo , T. and DeMaria , M.E . ( 2015 ) The influence of text messaging on oral hygiene effectiveness . The Angle Orthodontist , 85 , 543 – 548 . Google Scholar CrossRef Search ADS PubMed 3. Salkind , N.J . ( 2010 ) John Henry effect . In Salkind, N.J. (ed.), Encyclopaedia of Research Design . SAGE Publications, Inc ., London, UK . Google Scholar CrossRef Search ADS 4. McNatt , D.B . ( 2000 ) Ancient Pygmalion joins contemporary management: a meta-analysis of the result . The Journal of Applied Psychology , 85 , 314 – 322 . Google Scholar CrossRef Search ADS PubMed 5. McCambridge , J. , Witton , J. and Elbourne , D.R . ( 2014 ) Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects . Journal of Clinical Epidemiology , 67 , 267 – 277 . Google Scholar CrossRef Search ADS PubMed 6. Landsberger , H.A . ( 1958 ) Hawthorne Revisited: Management and the Worker: Its Critics, and Developments in Human Relations in Industry . Cornell University , Ithaca, NY . 7. McCarney , R. , Warner , J. , Iliffe , S. , van Haselen , R. , Griffin , M. and Fisher , P . ( 2007 ) The Hawthorne effect: a randomised, controlled trial . BMC Medical Research Methodology , 7 , 30 . Google Scholar CrossRef Search ADS PubMed 8. Sandler , J. , Benson , P.E. , Doyle , P. , Majumder , A. , O’Dwyer , J. , Speight , P. , Thiruvenkatachari , B. and Tinsley , D . ( 2008 ) Palatal implants are a good alternative to headgear: a randomized trial . American Journal of Orthodontics and Dentofacial Orthopedics , 133 , 51 – 57 . Google Scholar CrossRef Search ADS PubMed 9. Landis , J.R. and Koch , G.G . ( 1977 ) The measurement of observer agreement for categorical data . Biometrics , 33 , 159 – 174 . Google Scholar CrossRef Search ADS PubMed 10. Zingler , S. , Pritsch , M. , Wrede , D.J. , Ludwig , B. , Bister , D. , Kneist , S. and Lux , C.J . ( 2014 ) A randomized clinical trial comparing the impact of different oral hygiene protocols and sealant applications on plaque, gingival, and caries index scores . European Journal of Orthodontics , 36 , 150 – 163 . Google Scholar CrossRef Search ADS PubMed 11. Farhadian , N. , Usefi Mashoof , R. , Khanizadeh , S. , Ghaderi , E. , Farhadian , M. and Miresmaeili , A . ( 2016 ) Streptococcus mutans counts in patients wearing removable retainers with silver nanoparticles vs those wearing conventional retainers: a randomized clinical trial . American Journal of Orthodontics and Dentofacial Orthopedics , 149 , 155 – 160 . Google Scholar CrossRef Search ADS PubMed 12. Kaygisiz , E. , Uzuner , F.D. , Yuksel , S. , Taner , L. , Çulhaoğlu , R. , Sezgin , Y. and Ateş , C . ( 2015 ) Effects of self-ligating and conventional brackets on halitosis and periodontal conditions . The Angle Orthodontist , 85 , 468 – 473 . Google Scholar CrossRef Search ADS PubMed 13. Wiedel , A.P. and Bondemark , L . ( 2015 ) Fixed versus removable orthodontic appliances to correct anterior crossbite in the mixed dentition—a randomized controlled trial . European Journal of Orthodontics , 37 , 123 – 127 . Google Scholar CrossRef Search ADS PubMed 14. O’Rourke , N. , Albeedh , H. , Sharma , P. and Johal , A . ( 2016 ) Effectiveness of bonded and vacuum-formed retainers: a prospective randomized controlled clinical trial . American Journal of Orthodontics and Dentofacial Orthopedics , 150 , 406 – 415 . Google Scholar CrossRef Search ADS PubMed 15. Sharma , N.C. , Lyle , D.M. , Qaqish , J.G. , Galustians , J. and Schuller , R . ( 2008 ) Effect of a dental water jet with orthodontic tip on plaque and bleeding in adolescent patients with fixed orthodontic appliances . American Journal of Orthodontics and Dentofacial Orthopedics , 133 , 565 –5 71 . Google Scholar CrossRef Search ADS PubMed 16. Costa , M.R. , da Silva , V.C. , Miqui , M.N. , Colombo , A.P. and Cirelli , J.A . ( 2010 ) Effects of ultrasonic, electric, and manual toothbrushes on subgingival plaque composition in orthodontically banded molars . American Journal of Orthodontics and Dentofacial Orthopedics , 137 , 229 – 235 . Google Scholar CrossRef Search ADS PubMed 17. Yaqoob , O. , Dibiase , A.T. , Fleming , P.S. and Cobourne , M.T . ( 2012 ) Use of the Clark Twin Block functional appliance with and without an upper labial bow: a randomized controlled trial . The Angle Orthodontist , 82 , 363 – 369 . Google Scholar CrossRef Search ADS PubMed 18. Lempesi , E. , Koletsi , D. , Fleming , P.S. and Pandis , N . ( 2014 ) The reporting quality of randomized controlled trials in orthodontics . The Journal of Evidence-based Dental Practice , 14 , 46 – 52 . Google Scholar CrossRef Search ADS PubMed 19. O’Brien , K. and Craven , R . ( 1995 ) Pitfalls in orthodontic health service research . British Journal of Orthodontics , 22 , 353 – 356 . Google Scholar CrossRef Search ADS PubMed 20. Bondemark , L. and Ruf , S . ( 2015 ) Randomized controlled trial: the gold standard or an unobtainable fallacy? European Journal of Orthodontics , 37 , 457 – 461 . Google Scholar CrossRef Search ADS PubMed 21. CONSORT . ( 2010 ) http://www.equator-network.org/reporting-guidelines/consort/ and http://www.consort-statement.org/ (31 July 2017, date last accessed) 22. Feldmann , I . ( 2007 ) Orthodontic anchorage—evidence-based evaluation of anchorage capacity and patients’ perceptions . Swedish Dental Journal (Supplement) (191), 10 – 86 . 23. Ackerman , M.B. and Thornton , B . ( 2011 ) Posttreatment compliance with removable maxillary retention in a teenage population: a short-term randomized clinical trial . Orthodontics , 12 , 22 – 27 . Google Scholar PubMed 24. Fox , N.S. , Brennan , J.S. and Chasen , S.T . ( 2008 ) Clinical estimation of fetal weight and the Hawthorne effect . European Journal of Obstetrics, Gynecology, and Reproductive Biology , 141 , 111 – 114 . Google Scholar CrossRef Search ADS PubMed © The Author(s) 2017. Published by Oxford University Press on behalf of the European Orthodontic Society. All rights reserved. For permissions, please email: journals.permissions@oup.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png The European Journal of Orthodontics Oxford University Press

Hawthorne effect reporting in orthodontic randomized controlled trials: truth or myth? Blessing or curse?

Loading next page...
 
/lp/ou_press/hawthorne-effect-reporting-in-orthodontic-randomized-controlled-trials-Zmb1LvRHjF
Publisher
Oxford University Press
Copyright
© The Author(s) 2017. Published by Oxford University Press on behalf of the European Orthodontic Society. All rights reserved. For permissions, please email: journals.permissions@oup.com
ISSN
0141-5387
eISSN
1460-2210
D.O.I.
10.1093/ejo/cjx089
Publisher site
See Article on Publisher Site

Abstract

Summary Objective To investigate in 10 orthodontic journals how many randomized controlled trials (RCTs) considered the Hawthorne effect, and if considered, to determine whether it was related to the patients or the therapists involved in the trial and, finally, to discuss the Hawthorne effect in an educational way. Materials and methods A search was performed on the Medline database, via PubMed, for publication type ‘randomized controlled trial’ published for each journal between 1 August 2007 and 31 July 2017. The American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, Australian Orthodontic Journal, Dental Press Journal of Orthodontics, European Journal of Orthodontics, Journal of Orthodontics, Journal of Orofacial Orthopedics, Korean Journal of Orthodontics, Orthodontics and Craniofacial Research and Progress in Orthodontics were assessed. Two independent reviewers extracted the data and identified whether the Hawthorne effect was considered or discussed in the articles and whether the Hawthorne effect was related to the behaviour of the patients, the therapists, or both. Results The initial search generated 502 possible trials. After applying the inclusion and exclusion criteria, 290 RCTs were included and assessed. The Hawthorne effect was considered or discussed in 10 of 290 RCTs (3.4%), and all were related to the patients’ and none to the therapists’ behaviour. Conclusions The Hawthorne effect reported in orthodontic RCTs was suboptimal. The researchers’ lack of knowledge about this phenomenon is evident, despite evidence that the Hawthorne effect may cause over-optimistic results or false-positive bias. Introduction It is well known that rigorously conducted randomized controlled trials (RCTs) to assess the effectiveness of different treatments may be valuable for the medical and dental care of our patients, as well as for the whole community. The results from RCTs mean that detrimental or ineffective treatments are avoided and replaced by more efficient ones. However, when performing and analysing a trial, researchers must take into account the possibility that the participants’ behavioural changes may affect the results of the trial. Four examples of these behavioural changes are the Hawthorne effect, Novelty effect, John Henry effect, and the Pygmalion effect (1–4). The Hawthorne effect is defined as changes in patients’ or therapists’ behaviour when involved in a trial because of increased knowledge or interest or else due to feeling observed in the trial (1). The Novelty effect is explained by a tendency for the subjects to want to do better with a new or improved technology because they are intrigued by it rather than the effect being truly based on the variables under study (2). The John Henry effect is defined as when the control subjects are aware of their status as members of the control group and are able to compare their performance with that of the treatment group, which means that members of the control group may actively work harder to overcome the ‘disadvantage’ of being in the control group (3). This may happen when an innovative technology is evaluated and the control group participants behave in an unnatural way to try to outwit the new technology (3). Finally, the Pygmalion effect is defined as when we expect certain behaviours of others, we are likely to act in a way that makes the expected behaviour more likely to occur (4). Of the phenomena described above, the Hawthorne effect is the most discussed phenomenon in the literature as having an effect and bias in the results of RCTs in the medical and dental fields (5). The type of bias caused by this phenomenon is a false-positive bias that may lead to over-optimistic results. The name, Hawthorne, originates from a factory called Hawthorne Works, which was part of the Western Electric Company outside of Chicago. A series of extensive research was conducted focusing on the workers in the factory to investigate the effects that changing the lighting conditions would have on the workers’ productivity. It was found that by changing the work environment by increasing or decreasing the lighting conditions, the workers productivity was altered (6). Since then, the Hawthorne effect has appeared in discussions about clinical trials and has been accused of causing positive bias on the results of medical and dental RCTs (7, 8). In 2008, Sandler et al. (8) claimed to have witnessed the Hawthorne effect when performing an RCT comparing palatal implants with the headgear in supplementing anchorage during orthodontic treatment. They wrote in the discussion that ‘The headgear patients in the study surprised the clinicians with the speed and efficiency of this method of anchorage supplementation. In some cases, the headgear was discontinued after only five months; on average, headgear wear was continued for nine months. We believed that we had witnessed the Hawthorne effect’ (8). In this example, the investigators noticed the Hawthorne effect, reported it and related the phenomenon to patients’ behaviour. However, they did not stress another important consideration or relation of the Hawthorne effect, namely, that the therapist’s behaviour may also have changed during the trial. According to the definition mentioned above, the phenomenon affects both patients and therapists. Nevertheless, an ongoing debate is present in the literature to determine whether this phenomenon is a myth or true and whether it smears our trial results with false-positive bias and a wrong presentation of real clinical situations. A review (5) was assessed to determine whether the Hawthorne effect exists, and if so, to explore under what conditions and estimate the size of such an effect. The conclusion confirmed the presence of this phenomenon but without any conclusion about the condition that led to it or its magnitude (5). Consequently, the Hawthorne effect may be an important factor affecting clinical trial results in orthodontics, but to our knowledge, no attempt has ever been made to evaluate to what extent the Hawthorne effect is considered in RCTs in orthodontics. Therefore, the aim of this study was to investigate, in 10 orthodontic journals, how many RCTs considered the Hawthorne effect; if it was considered, to determine whether the Hawthorne effect was related to just the patients or also to the therapist involved in the trial; and, finally, to discuss the role of the Hawthorne effect from an educational point of view and from an orthodontic research perspective. Materials and methods The inclusion criteria were human RCTs that used the individual as a unit, and thus, RCTs applying split-mouth design were excluded. A search was performed on the database Medline, via PubMed, for publication type ‘randomized controlled trial’ published for each journal between 1 August 2007 and 31 July 2017. The 10 orthodontic journals assessed were the American Journal of Orthodontics and Dentofacial Orthopedics (AJODO), The Angle Orthodontist (AO), Australian Orthodontic Journal (AOJ), Dental Press Journal of Orthodontics (DPJO), European Journal of Orthodontics (EJO), Journal of Orthodontics (JO), Journal of Orofacial Orthopedics (JOO), Korean Journal of Orthodontics (KJO), Orthodontics and Craniofacial Research (OCR), and Progress in Orthodontics (PO). Each article was selected and the data were extracted by two independent reviewers. If randomization was obvious from the title and abstract of the trial, the full text of the trial was read to verify and clarify the study design according to human RCTs. Then, the full text was reviewed to identify whether the Hawthorne effect was considered or discussed in the article and whether the Hawthorne effect was related to the behaviour of the patients, therapists, or both. And finally, to cross-check against the database search, the 10 journals were hand searched for the actual search period. At the title, abstract and full-text screening stages as well as for dichotomous and categorical variables during data extraction, the number of inter-examiner agreements and disagreements were noted and calculated for the assessment of agreement. Then, all the disagreements were discussed and resolved by discussion to reach consensus. Kappa score was used to measure the level of agreement, and the degree of agreement was 0.91 (i.e. an excellent agreement) (9). Results The initial search generated 502 possible concordant trials. Two-hundred and twelve trials were excluded based on the inclusion and exclusion criteria, i.e. 93 split mouth design, 75 in vitro studies, and 44 prospective controlled or retrospective studies. Consequently, 290 RCTs were included and assessed, and the number of RCTs per journal is presented in Table 1. Table 1. Number of randomized controlled trials (RCTs) per journal and number of RCTs that considered the Hawthorne effect and related it to patients and therapists. Journals Number of RCTs per journal (%) Number of the total RCTs (%) that reported Hawthorne effect related to therapists Number of the total RCTs (%) that reported the Hawthorne effect Number of the total RCTs (%) that reported the Hawthorne effect related to patients AJODO 105 (36.2) 5 (1.7) 5 (1.7) 0 (0.0) AO 55 (19.0) 3 (1.0) 3 (1.0) 0 (0.0) AOJ 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) DPJO 5 (1.7) 0 (0.0) 0 (0.0) 0 (0.0) EJO 51 (17.6) 2 (0.7) 2 (0.7) 0 (0.0) JO 21 (7.2) 0 (0.0) 0 (0.0) 0 (0.0) JOO 18 (6.2) 0 (0.0) 0 (0.0) 0 (0.0) KJO 2 (0.7) 0 (0.0) 0 (0.0) 0 (0.0) OCR 9 (3.1) 0 (0.0) 0 (0.0) 0 (0.0) PO 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) Total 290 (100) 10 (3.4) 10 (3.4) 0 (0.0) Journals Number of RCTs per journal (%) Number of the total RCTs (%) that reported Hawthorne effect related to therapists Number of the total RCTs (%) that reported the Hawthorne effect Number of the total RCTs (%) that reported the Hawthorne effect related to patients AJODO 105 (36.2) 5 (1.7) 5 (1.7) 0 (0.0) AO 55 (19.0) 3 (1.0) 3 (1.0) 0 (0.0) AOJ 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) DPJO 5 (1.7) 0 (0.0) 0 (0.0) 0 (0.0) EJO 51 (17.6) 2 (0.7) 2 (0.7) 0 (0.0) JO 21 (7.2) 0 (0.0) 0 (0.0) 0 (0.0) JOO 18 (6.2) 0 (0.0) 0 (0.0) 0 (0.0) KJO 2 (0.7) 0 (0.0) 0 (0.0) 0 (0.0) OCR 9 (3.1) 0 (0.0) 0 (0.0) 0 (0.0) PO 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) Total 290 (100) 10 (3.4) 10 (3.4) 0 (0.0) AJODO, American Journal of Orthodontics and Dentofacial Orthopedics; AO, The Angle Orthodontist; AOJ, Australian Orthodontic Journal; DPJO, Dental Press Journal of Orthodontics; EJO, European Journal of Orthodontics; JO, Journal of Orthodontics; JOO, Journal of Orofacial Orthopedics; KJO, Korean Journal of Orthodontics; OCR, Orthodontics and Craniofacial Research; PO, Progress in Orthodontics. View Large Table 1. Number of randomized controlled trials (RCTs) per journal and number of RCTs that considered the Hawthorne effect and related it to patients and therapists. Journals Number of RCTs per journal (%) Number of the total RCTs (%) that reported Hawthorne effect related to therapists Number of the total RCTs (%) that reported the Hawthorne effect Number of the total RCTs (%) that reported the Hawthorne effect related to patients AJODO 105 (36.2) 5 (1.7) 5 (1.7) 0 (0.0) AO 55 (19.0) 3 (1.0) 3 (1.0) 0 (0.0) AOJ 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) DPJO 5 (1.7) 0 (0.0) 0 (0.0) 0 (0.0) EJO 51 (17.6) 2 (0.7) 2 (0.7) 0 (0.0) JO 21 (7.2) 0 (0.0) 0 (0.0) 0 (0.0) JOO 18 (6.2) 0 (0.0) 0 (0.0) 0 (0.0) KJO 2 (0.7) 0 (0.0) 0 (0.0) 0 (0.0) OCR 9 (3.1) 0 (0.0) 0 (0.0) 0 (0.0) PO 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) Total 290 (100) 10 (3.4) 10 (3.4) 0 (0.0) Journals Number of RCTs per journal (%) Number of the total RCTs (%) that reported Hawthorne effect related to therapists Number of the total RCTs (%) that reported the Hawthorne effect Number of the total RCTs (%) that reported the Hawthorne effect related to patients AJODO 105 (36.2) 5 (1.7) 5 (1.7) 0 (0.0) AO 55 (19.0) 3 (1.0) 3 (1.0) 0 (0.0) AOJ 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) DPJO 5 (1.7) 0 (0.0) 0 (0.0) 0 (0.0) EJO 51 (17.6) 2 (0.7) 2 (0.7) 0 (0.0) JO 21 (7.2) 0 (0.0) 0 (0.0) 0 (0.0) JOO 18 (6.2) 0 (0.0) 0 (0.0) 0 (0.0) KJO 2 (0.7) 0 (0.0) 0 (0.0) 0 (0.0) OCR 9 (3.1) 0 (0.0) 0 (0.0) 0 (0.0) PO 12 (4.1) 0 (0.0) 0 (0.0) 0 (0.0) Total 290 (100) 10 (3.4) 10 (3.4) 0 (0.0) AJODO, American Journal of Orthodontics and Dentofacial Orthopedics; AO, The Angle Orthodontist; AOJ, Australian Orthodontic Journal; DPJO, Dental Press Journal of Orthodontics; EJO, European Journal of Orthodontics; JO, Journal of Orthodontics; JOO, Journal of Orofacial Orthopedics; KJO, Korean Journal of Orthodontics; OCR, Orthodontics and Craniofacial Research; PO, Progress in Orthodontics. View Large The Hawthorne effect was considered or discussed in 10 of 290 RCTs (3.4%). Five RCTs were found in AJODO, three in AO, and two in EJO. All 10 RCTs related the Hawthorne effect to the patients’ behavioural changes without including the therapists (2, 8, 10–17) (Table 2). Four of the RCTs evaluated different oral hygiene protocols or instruments (2, 10, 15, 16), one analysed Streptococcus mutans counts in two groups of different retainers (11), another trial investigated the influence of different bracket types on halitosis and periodontal conditions (12), while Wiedel and Bondemark (13) compared fixed and removable appliances for correction of anterior cross bite (Table 2). In the trial by O′Rourke et al. (14), the Hawthorne effect was discussed in the context of not using special monitoring sensors (TheraMon), which are usually used to check patients’ compliance. The trial avoided using the sensors because it was thought that the patients would feel followed or monitored if used. In the trial by Sandler et al. (8), the Hawthorne effect was related to the unexpected excellent compliance with the headgear use, and finally, in the study by Yaqoob et al. (17), the Hawthorne effect was discussed in relation to the overall good compliance with the daily use of functional appliances (Table 2). Table 2. How the Hawthorne effect considered or discussed in the 10 randomized controlled trials (RCTs) of the survey. The studies are tabulated by increasing number as the studies have in the reference list. RCT Hawthorne effect Bowen et al. 2015 (2) It is possible that oral hygiene improved because of performed bias that could be introduced in this study from the Hawthorne and novelty effects. The Hawthorne effect is a form of reactivity in which subjects improve or modify an aspect of their behaviour being experimentally measured simply in response to the fact that they are being studied, not in response to any particular experimental manipulation. Sandler et al. 2008 (8) The headgear patients in the study surprised the clinicians with the speed and efficiency of this method of anchorage supplementation. In some cases, the headgear was discontinued after only five months; on average, headgear wear was continued for nine months. We believed that we had witnessed the Hawthorne effect. Zinger et al. 2014 (10) In addition to the impact of overall oral hygiene level of the current study population, MacGregor and Rugg- Gunn (1986) pointed out that awareness of being observed may change brushing behaviour (Hawthorne effect). Farhadian et al. 2016 (11) Patients who participate in such a clinical trial pay more attention to their oral hygiene, independent of the treatment modality (Hawthorne effect). However, it seems that the use of silver nanoparticles has a significant impact on reducing the colonization of cariogenic bacteria. Kaygisiz et al. 2015 (12) The results of this study showed no significant differences in plaque index, gingival index, pocket depth, and bleeding on probing values changes during the treatment or observation time among the groups. This may be attributed to the effect of an oral hygiene supplement given before the treatment/observation and motivation of the patients. It can also be attributed to the Hawthorne Effect (patient awareness of being examined and evaluated). Wiedel and Bondemark 2015 (13) However, it must be acknowledged that clinical trials run the risk of the Hawthorne effect (positive bias), whereby subjects tend to perform better when they are participants in an experiment. Consequently, the Hawthorne effect has influence on both groups, i.e. the patients were more compliant than the average orthodontic patients in everyday orthodontic practice. O′Rourke et al. 2016 (14) The Hawthorne effect was discussed from the perspective not to use special monitoring sensors (TheraMon), which usually are applicated to check patients’ compliance. The sensors were avoided to be used in the trial because if used, it was regarded that the patients should feel followed or monitored. Sharma et al. 2008 (15) The plaque reduction in the dental water jet group was significantly greater than in the flossing and manual toothbrush groups, but the reduction of interproximal bleeding did not reach significance at 28 days. This might be attributed to the subjects’ response to the reduction of plaque or the quality of their plaque, the required twice daily brushing for 2 min, and the possible improved technique for brushing and flossing because of being subjects in a clinical study (Hawthorne effect). Costa et al. 2010 (16) The sequence distribution of the patients was done to eliminate the influence of the Hawthorne effect on the results, after each type of brush was used by a group of patients in the three periods of the study. Yaqoob et al. 2012 (17) Compliance overall in this study compares favourably with that in previous studies, and although it is difficult to ascertain why this is so, it is probably, at least in part, due to careful clinical management, the experience of the supervising clinician (Dr DiBiase) with the appliance, and possibly the fact that the patients knew they were in a study. RCT Hawthorne effect Bowen et al. 2015 (2) It is possible that oral hygiene improved because of performed bias that could be introduced in this study from the Hawthorne and novelty effects. The Hawthorne effect is a form of reactivity in which subjects improve or modify an aspect of their behaviour being experimentally measured simply in response to the fact that they are being studied, not in response to any particular experimental manipulation. Sandler et al. 2008 (8) The headgear patients in the study surprised the clinicians with the speed and efficiency of this method of anchorage supplementation. In some cases, the headgear was discontinued after only five months; on average, headgear wear was continued for nine months. We believed that we had witnessed the Hawthorne effect. Zinger et al. 2014 (10) In addition to the impact of overall oral hygiene level of the current study population, MacGregor and Rugg- Gunn (1986) pointed out that awareness of being observed may change brushing behaviour (Hawthorne effect). Farhadian et al. 2016 (11) Patients who participate in such a clinical trial pay more attention to their oral hygiene, independent of the treatment modality (Hawthorne effect). However, it seems that the use of silver nanoparticles has a significant impact on reducing the colonization of cariogenic bacteria. Kaygisiz et al. 2015 (12) The results of this study showed no significant differences in plaque index, gingival index, pocket depth, and bleeding on probing values changes during the treatment or observation time among the groups. This may be attributed to the effect of an oral hygiene supplement given before the treatment/observation and motivation of the patients. It can also be attributed to the Hawthorne Effect (patient awareness of being examined and evaluated). Wiedel and Bondemark 2015 (13) However, it must be acknowledged that clinical trials run the risk of the Hawthorne effect (positive bias), whereby subjects tend to perform better when they are participants in an experiment. Consequently, the Hawthorne effect has influence on both groups, i.e. the patients were more compliant than the average orthodontic patients in everyday orthodontic practice. O′Rourke et al. 2016 (14) The Hawthorne effect was discussed from the perspective not to use special monitoring sensors (TheraMon), which usually are applicated to check patients’ compliance. The sensors were avoided to be used in the trial because if used, it was regarded that the patients should feel followed or monitored. Sharma et al. 2008 (15) The plaque reduction in the dental water jet group was significantly greater than in the flossing and manual toothbrush groups, but the reduction of interproximal bleeding did not reach significance at 28 days. This might be attributed to the subjects’ response to the reduction of plaque or the quality of their plaque, the required twice daily brushing for 2 min, and the possible improved technique for brushing and flossing because of being subjects in a clinical study (Hawthorne effect). Costa et al. 2010 (16) The sequence distribution of the patients was done to eliminate the influence of the Hawthorne effect on the results, after each type of brush was used by a group of patients in the three periods of the study. Yaqoob et al. 2012 (17) Compliance overall in this study compares favourably with that in previous studies, and although it is difficult to ascertain why this is so, it is probably, at least in part, due to careful clinical management, the experience of the supervising clinician (Dr DiBiase) with the appliance, and possibly the fact that the patients knew they were in a study. View Large Table 2. How the Hawthorne effect considered or discussed in the 10 randomized controlled trials (RCTs) of the survey. The studies are tabulated by increasing number as the studies have in the reference list. RCT Hawthorne effect Bowen et al. 2015 (2) It is possible that oral hygiene improved because of performed bias that could be introduced in this study from the Hawthorne and novelty effects. The Hawthorne effect is a form of reactivity in which subjects improve or modify an aspect of their behaviour being experimentally measured simply in response to the fact that they are being studied, not in response to any particular experimental manipulation. Sandler et al. 2008 (8) The headgear patients in the study surprised the clinicians with the speed and efficiency of this method of anchorage supplementation. In some cases, the headgear was discontinued after only five months; on average, headgear wear was continued for nine months. We believed that we had witnessed the Hawthorne effect. Zinger et al. 2014 (10) In addition to the impact of overall oral hygiene level of the current study population, MacGregor and Rugg- Gunn (1986) pointed out that awareness of being observed may change brushing behaviour (Hawthorne effect). Farhadian et al. 2016 (11) Patients who participate in such a clinical trial pay more attention to their oral hygiene, independent of the treatment modality (Hawthorne effect). However, it seems that the use of silver nanoparticles has a significant impact on reducing the colonization of cariogenic bacteria. Kaygisiz et al. 2015 (12) The results of this study showed no significant differences in plaque index, gingival index, pocket depth, and bleeding on probing values changes during the treatment or observation time among the groups. This may be attributed to the effect of an oral hygiene supplement given before the treatment/observation and motivation of the patients. It can also be attributed to the Hawthorne Effect (patient awareness of being examined and evaluated). Wiedel and Bondemark 2015 (13) However, it must be acknowledged that clinical trials run the risk of the Hawthorne effect (positive bias), whereby subjects tend to perform better when they are participants in an experiment. Consequently, the Hawthorne effect has influence on both groups, i.e. the patients were more compliant than the average orthodontic patients in everyday orthodontic practice. O′Rourke et al. 2016 (14) The Hawthorne effect was discussed from the perspective not to use special monitoring sensors (TheraMon), which usually are applicated to check patients’ compliance. The sensors were avoided to be used in the trial because if used, it was regarded that the patients should feel followed or monitored. Sharma et al. 2008 (15) The plaque reduction in the dental water jet group was significantly greater than in the flossing and manual toothbrush groups, but the reduction of interproximal bleeding did not reach significance at 28 days. This might be attributed to the subjects’ response to the reduction of plaque or the quality of their plaque, the required twice daily brushing for 2 min, and the possible improved technique for brushing and flossing because of being subjects in a clinical study (Hawthorne effect). Costa et al. 2010 (16) The sequence distribution of the patients was done to eliminate the influence of the Hawthorne effect on the results, after each type of brush was used by a group of patients in the three periods of the study. Yaqoob et al. 2012 (17) Compliance overall in this study compares favourably with that in previous studies, and although it is difficult to ascertain why this is so, it is probably, at least in part, due to careful clinical management, the experience of the supervising clinician (Dr DiBiase) with the appliance, and possibly the fact that the patients knew they were in a study. RCT Hawthorne effect Bowen et al. 2015 (2) It is possible that oral hygiene improved because of performed bias that could be introduced in this study from the Hawthorne and novelty effects. The Hawthorne effect is a form of reactivity in which subjects improve or modify an aspect of their behaviour being experimentally measured simply in response to the fact that they are being studied, not in response to any particular experimental manipulation. Sandler et al. 2008 (8) The headgear patients in the study surprised the clinicians with the speed and efficiency of this method of anchorage supplementation. In some cases, the headgear was discontinued after only five months; on average, headgear wear was continued for nine months. We believed that we had witnessed the Hawthorne effect. Zinger et al. 2014 (10) In addition to the impact of overall oral hygiene level of the current study population, MacGregor and Rugg- Gunn (1986) pointed out that awareness of being observed may change brushing behaviour (Hawthorne effect). Farhadian et al. 2016 (11) Patients who participate in such a clinical trial pay more attention to their oral hygiene, independent of the treatment modality (Hawthorne effect). However, it seems that the use of silver nanoparticles has a significant impact on reducing the colonization of cariogenic bacteria. Kaygisiz et al. 2015 (12) The results of this study showed no significant differences in plaque index, gingival index, pocket depth, and bleeding on probing values changes during the treatment or observation time among the groups. This may be attributed to the effect of an oral hygiene supplement given before the treatment/observation and motivation of the patients. It can also be attributed to the Hawthorne Effect (patient awareness of being examined and evaluated). Wiedel and Bondemark 2015 (13) However, it must be acknowledged that clinical trials run the risk of the Hawthorne effect (positive bias), whereby subjects tend to perform better when they are participants in an experiment. Consequently, the Hawthorne effect has influence on both groups, i.e. the patients were more compliant than the average orthodontic patients in everyday orthodontic practice. O′Rourke et al. 2016 (14) The Hawthorne effect was discussed from the perspective not to use special monitoring sensors (TheraMon), which usually are applicated to check patients’ compliance. The sensors were avoided to be used in the trial because if used, it was regarded that the patients should feel followed or monitored. Sharma et al. 2008 (15) The plaque reduction in the dental water jet group was significantly greater than in the flossing and manual toothbrush groups, but the reduction of interproximal bleeding did not reach significance at 28 days. This might be attributed to the subjects’ response to the reduction of plaque or the quality of their plaque, the required twice daily brushing for 2 min, and the possible improved technique for brushing and flossing because of being subjects in a clinical study (Hawthorne effect). Costa et al. 2010 (16) The sequence distribution of the patients was done to eliminate the influence of the Hawthorne effect on the results, after each type of brush was used by a group of patients in the three periods of the study. Yaqoob et al. 2012 (17) Compliance overall in this study compares favourably with that in previous studies, and although it is difficult to ascertain why this is so, it is probably, at least in part, due to careful clinical management, the experience of the supervising clinician (Dr DiBiase) with the appliance, and possibly the fact that the patients knew they were in a study. View Large Discussion The main findings of this survey were that very few orthodontic RCTs considered the Hawthorne effect (3.4%). Furthermore, the few RCTs that did consider the Hawthorne effect related it to only the patients’ behaviour and never to the therapist involved in the trial, even though it is clearly mentioned in the definition of the Hawthorne effect that the therapist’s behaviour can also be included. It was noticed that trials investigating oral hygiene methods reported the Hawthorne effect more frequently, however, without any real explanation for this fact. Despite RCTs are considered the best way to minimize bias, wrong randomization procedures, missing data, and noncompliance are some of the biases that can occur within the RCT’s design affecting their external and internal validity (18–20). All these types of bias are involved in the CONSORT guidelines, which most of the orthodontic journals require adherence to before publishing a paper. However, the Hawthorne effect is not included in the CONSORT guideline keys (21), and conceivably, this fact may explain why so few RCTs reported or considered the Hawthorne effect. The reason for excluding RCTs with split-mouth design was that the Hawthorne effect will always affect the treatment/treatment or treatment/control simultaneously within the mouth, while in an RCT using the individual as a unit in one group and not included in another group, the Hawthorne effect can affect the groups in a varying degree, and thereby, the consequences can be different per group. When we analysed in detail the RCTs that did not consider the Hawthorne effect, a majority of those still reported or witnessed short-term unusual treatment results without considering the Hawthorne effect as a possible cause. Thus, it can be recommended to be aware of and take into account the Hawthorne effect in RCTs, especially when unusual compliance rates or results are reached. Sandler et al. (8) noticed patients’ unusual compliance with headgear, i.e. the headgear patients in the study surprised the clinicians with the speed and efficiency of headgear anchorage supplementation, and reported that they witnessed the Hawthorne effect. A similar finding was reported by Feldmann (22), who reported that the headgear even moved the molars distally during the first 6 months of using the headgear anchorage, but during the following 12 month treatment period, anchorage loss of just over 1 mm was found in the headgear group. Feldmann (22) claimed that the high level of patient cooperation during the first 6 months could partly be attributed to the Hawthorne effect. An interesting finding of the different intensities of the Hawthorne effect was evident in a trial that evaluated the effect of a sensor (TheraMon) that recorded how long (in hours) the patients used a removable orthodontic appliance (23). There were two groups. In one group, the patients were aware of the sensor that measured how long they used the appliance, while in the other group, the patients had no knowledge of the sensor. It was found that the patients who were aware of the sensor used the appliance significantly more than the patients in the other group (20). This example indicates that the patients in both groups were influenced by the Hawthorne effect. However, the patients in the group aware that their use of the removable appliance would be evaluated with a sensor (in addition to being aware of participating in the trial), adapted even more by using the removable appliance more often. Thus, researchers who incorporate sensors to check patients’ compliance in a trial should be aware of a particular behaviour phenomenon, i.e. the Hawthorne effect and a sensor effect. One strength of this study was that two independent reviewers searched for and critically brought data to this survey. An excellent agreement (Kappa score of 0.91) was found for the reviewers. Other strengths were the long time span of 10 years search duration and that 10 orthodontic journals have been included and reviewed. Is the Hawthorne effect true or a myth? As earlier mentioned, Sandler et al. (8) claimed to have witnessed the Hawthorne effect when performing an RCT comparing palatal implants with the headgear in supplementing anchorage during orthodontic treatment, and in the ten RCTs found in this survey, the Hawthorne effect was discussed and related to patients’ behaviour. Also, many studies in the medical fields have investigated the Hawthorne effect. One study (7) attempted to quantify the Hawthorne effect, and in that study, the aim was to compare minimal follow-up to intensive follow-up in participants in a placebo-controlled trial of Ginkgo biloba for treatment of mild-to-moderate dementia. It was found that the group with intensive follow-up had significantly better results, as measured by their cognitive functioning, and this was attributed to the Hawthorne effect (7). However, in another study (24), the Hawthorne effect on the therapists was analysed by determination of obstetricians’ clinical estimation of foetal weight (EFW). The obstetricians in the experimental group were informed that their EFW accuracy should be studied but without them being aware that the investigators were evaluating the Hawthorne effect. The control group consisted of obstetricians who made their EFW before the study was conducted and without knowing that they were observed. The conclusion was that EFW accuracies were similar between the groups without any bias caused by the Hawthorne effect (24). Finally, a systematic review concluded that the Hawthorne effect was considered a true effect and not a myth. Thus, the Hawthorne effect may bias our study results but without knowing its magnitude and under which conditions it may be raised (5). Is the Hawthorne effect a blessing or a curse? When performing RCTs, the Hawthorne effect is often regarded as a curse because the phenomenon can cause changes in participants’ behaviour during the trial, thus making them perform better when compared with real-life clinical situations. As a result, overestimated results, i.e. false-positive bias, may be a consequence of this phenomenon (5, 7). In contrast, the Hawthorne effect may be viewed as a blessing when the manipulation of the results is wanted. A recent systematic review concluded that the main behaviour attributed to Hawthorne effect disappeared after a 6 month trial period (5). This also means that if the researchers shorten the observation period as much as possible (<6 months), the participants will easily adapt and fulfil their ‘good trial behaviour’ during this short observation period. Thus, a shortening of the trial period will result in a relatively greater Hawthorne effect, which may affect the results implying a higher positive bias than if a long observation period had been chosen (5). Therefore, if we want to decrease the Hawthorne effect on the behaviour of study participants and therapists in a trial, setting an observation period that is longer than 6 months is recommended. Given that orthodontic studies may include observation periods from a few days to several decades of follow-up, it may therefore be important to be aware and understand that the Hawthorne effect is likely to be the greatest for short-term studies and practically non-existent for long observation periods of several years. In any case, the Hawthorne effect should be regarded as neither a blessing nor a curse, but instead, depending on the study design, considered a factor that can contribute to positive bias of the results obtained. Recommendations Always analyse and report unexpected overestimated results and discuss whether the Hawthorne effect is a possible explanation for these unexpected results. Be aware that a short observation period often implies the increased risk of the Hawthorne effect, and if the study design allows for it, plan a longer observation period than 6 months to minimize the Hawthorne effect. Conclusion In this survey, only 3.4% of the RCTs reported the Hawthorne effect despite indications that the other RCTs that did not consider the Hawthorne effect should have regarded this effect as a possible source of false-positive bias. All the RCTs related the Hawthorne effect to the patients’ behavioural changes without including the therapists. Evidence exists that the Hawthorne effect on patients’ performance and behaviour is to be considered a true effect and not a myth; however, there is no evidence that the therapists’ behaviour may also be affected. The Hawthorne effect should be regarded as neither a blessing nor a curse, but it is important to be aware that the Hawthorne effect is likely to be the greatest for short-term trials and practically non-existent during long observation periods of several years. Funding This survey was funded by the authors’ academic institution. Conflict of Interest None to declare. References 1. Braunholtz , D.A. , Edwards , S.J. and Lilford , R.J . ( 2001 ) Are randomized clinical trials good for us (in the short term)? Evidence for a “trial effect” . Journal of Clinical Epidemiology , 54 , 217 – 224 . Google Scholar CrossRef Search ADS PubMed 2. Bowen , T.B. , Rinchuse , D.J. , Zullo , T. and DeMaria , M.E . ( 2015 ) The influence of text messaging on oral hygiene effectiveness . The Angle Orthodontist , 85 , 543 – 548 . Google Scholar CrossRef Search ADS PubMed 3. Salkind , N.J . ( 2010 ) John Henry effect . In Salkind, N.J. (ed.), Encyclopaedia of Research Design . SAGE Publications, Inc ., London, UK . Google Scholar CrossRef Search ADS 4. McNatt , D.B . ( 2000 ) Ancient Pygmalion joins contemporary management: a meta-analysis of the result . The Journal of Applied Psychology , 85 , 314 – 322 . Google Scholar CrossRef Search ADS PubMed 5. McCambridge , J. , Witton , J. and Elbourne , D.R . ( 2014 ) Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects . Journal of Clinical Epidemiology , 67 , 267 – 277 . Google Scholar CrossRef Search ADS PubMed 6. Landsberger , H.A . ( 1958 ) Hawthorne Revisited: Management and the Worker: Its Critics, and Developments in Human Relations in Industry . Cornell University , Ithaca, NY . 7. McCarney , R. , Warner , J. , Iliffe , S. , van Haselen , R. , Griffin , M. and Fisher , P . ( 2007 ) The Hawthorne effect: a randomised, controlled trial . BMC Medical Research Methodology , 7 , 30 . Google Scholar CrossRef Search ADS PubMed 8. Sandler , J. , Benson , P.E. , Doyle , P. , Majumder , A. , O’Dwyer , J. , Speight , P. , Thiruvenkatachari , B. and Tinsley , D . ( 2008 ) Palatal implants are a good alternative to headgear: a randomized trial . American Journal of Orthodontics and Dentofacial Orthopedics , 133 , 51 – 57 . Google Scholar CrossRef Search ADS PubMed 9. Landis , J.R. and Koch , G.G . ( 1977 ) The measurement of observer agreement for categorical data . Biometrics , 33 , 159 – 174 . Google Scholar CrossRef Search ADS PubMed 10. Zingler , S. , Pritsch , M. , Wrede , D.J. , Ludwig , B. , Bister , D. , Kneist , S. and Lux , C.J . ( 2014 ) A randomized clinical trial comparing the impact of different oral hygiene protocols and sealant applications on plaque, gingival, and caries index scores . European Journal of Orthodontics , 36 , 150 – 163 . Google Scholar CrossRef Search ADS PubMed 11. Farhadian , N. , Usefi Mashoof , R. , Khanizadeh , S. , Ghaderi , E. , Farhadian , M. and Miresmaeili , A . ( 2016 ) Streptococcus mutans counts in patients wearing removable retainers with silver nanoparticles vs those wearing conventional retainers: a randomized clinical trial . American Journal of Orthodontics and Dentofacial Orthopedics , 149 , 155 – 160 . Google Scholar CrossRef Search ADS PubMed 12. Kaygisiz , E. , Uzuner , F.D. , Yuksel , S. , Taner , L. , Çulhaoğlu , R. , Sezgin , Y. and Ateş , C . ( 2015 ) Effects of self-ligating and conventional brackets on halitosis and periodontal conditions . The Angle Orthodontist , 85 , 468 – 473 . Google Scholar CrossRef Search ADS PubMed 13. Wiedel , A.P. and Bondemark , L . ( 2015 ) Fixed versus removable orthodontic appliances to correct anterior crossbite in the mixed dentition—a randomized controlled trial . European Journal of Orthodontics , 37 , 123 – 127 . Google Scholar CrossRef Search ADS PubMed 14. O’Rourke , N. , Albeedh , H. , Sharma , P. and Johal , A . ( 2016 ) Effectiveness of bonded and vacuum-formed retainers: a prospective randomized controlled clinical trial . American Journal of Orthodontics and Dentofacial Orthopedics , 150 , 406 – 415 . Google Scholar CrossRef Search ADS PubMed 15. Sharma , N.C. , Lyle , D.M. , Qaqish , J.G. , Galustians , J. and Schuller , R . ( 2008 ) Effect of a dental water jet with orthodontic tip on plaque and bleeding in adolescent patients with fixed orthodontic appliances . American Journal of Orthodontics and Dentofacial Orthopedics , 133 , 565 –5 71 . Google Scholar CrossRef Search ADS PubMed 16. Costa , M.R. , da Silva , V.C. , Miqui , M.N. , Colombo , A.P. and Cirelli , J.A . ( 2010 ) Effects of ultrasonic, electric, and manual toothbrushes on subgingival plaque composition in orthodontically banded molars . American Journal of Orthodontics and Dentofacial Orthopedics , 137 , 229 – 235 . Google Scholar CrossRef Search ADS PubMed 17. Yaqoob , O. , Dibiase , A.T. , Fleming , P.S. and Cobourne , M.T . ( 2012 ) Use of the Clark Twin Block functional appliance with and without an upper labial bow: a randomized controlled trial . The Angle Orthodontist , 82 , 363 – 369 . Google Scholar CrossRef Search ADS PubMed 18. Lempesi , E. , Koletsi , D. , Fleming , P.S. and Pandis , N . ( 2014 ) The reporting quality of randomized controlled trials in orthodontics . The Journal of Evidence-based Dental Practice , 14 , 46 – 52 . Google Scholar CrossRef Search ADS PubMed 19. O’Brien , K. and Craven , R . ( 1995 ) Pitfalls in orthodontic health service research . British Journal of Orthodontics , 22 , 353 – 356 . Google Scholar CrossRef Search ADS PubMed 20. Bondemark , L. and Ruf , S . ( 2015 ) Randomized controlled trial: the gold standard or an unobtainable fallacy? European Journal of Orthodontics , 37 , 457 – 461 . Google Scholar CrossRef Search ADS PubMed 21. CONSORT . ( 2010 ) http://www.equator-network.org/reporting-guidelines/consort/ and http://www.consort-statement.org/ (31 July 2017, date last accessed) 22. Feldmann , I . ( 2007 ) Orthodontic anchorage—evidence-based evaluation of anchorage capacity and patients’ perceptions . Swedish Dental Journal (Supplement) (191), 10 – 86 . 23. Ackerman , M.B. and Thornton , B . ( 2011 ) Posttreatment compliance with removable maxillary retention in a teenage population: a short-term randomized clinical trial . Orthodontics , 12 , 22 – 27 . Google Scholar PubMed 24. Fox , N.S. , Brennan , J.S. and Chasen , S.T . ( 2008 ) Clinical estimation of fetal weight and the Hawthorne effect . European Journal of Obstetrics, Gynecology, and Reproductive Biology , 141 , 111 – 114 . Google Scholar CrossRef Search ADS PubMed © The Author(s) 2017. Published by Oxford University Press on behalf of the European Orthodontic Society. All rights reserved. For permissions, please email: journals.permissions@oup.com

Journal

The European Journal of OrthodonticsOxford University Press

Published: Nov 28, 2017

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off