maastrichtuniversity.nl Department of Educational Background: Public health leaders are confronted with complex problems, and Development and Research and developing effective leadership competencies is essential. The teaching of leadership Graduate School of Health Professions Education, Faculty of is still not common in public health training programs around the world. A Health, Medicine and Life Sciences, reconceptualization of professional training is needed and can benefit from Maastricht University, PO Box 616, innovative educational approaches. Our aim was to explore learners’ perceptions of 6200, MD, Maastricht, the Netherlands the effectiveness and appeal of a public health leadership course using problem- Full list of author information is based, blended learning methods that used virtual learning environment available at the end of the article technologies. Case presentation: In this cross-sectional evaluative study, the Self-Assessment Instrument of Competencies for Public Health Leaders was administered before and after an online, blended-learning, problem-based (PBL) leadership course. An evaluation questionnaire was also used to measure perceptions of blended learning, problem-based learning, and tutor functioning among 19 public health professionals from The Netherlands (n = 8), Lithuania (n = 5), and Austria (n = 6). Participants showed overall satisfaction and knowledge gains related to public health leadership competencies in six of eight measured areas, especially Political Leadership and Systems Thinking. Some perceptions of blended learning and PBL varied between the institutions. This might have been caused by lack of experience of the educational approaches, differing professional backgrounds, inexperience of communicating in the online setting, and different expectations towards the course. Conclusions: Blended, problem-based learning might be an effective way to develop leadership competencies among public health professionals in international and interdisciplinary context. Keywords: Leadership, Public health, Problem-based learning, Blended learning, Competency self-assessment, Evaluation © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/ publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Könings et al. Public Health Reviews (2018) 39:13 Page 2 of 12 Background Public health leaders in many countries are faced with the challenges like the aging populations with chronic disease and major issues regarding emerging or re-emerging infectious diseases, including uncontrolled epidemics, distrust towards vaccination, or antibiotic resistance. There is little doubt that effective public health leadership is essential given the significant financial pressures on health services and the need to deliver more with less resources . However, few educational courses deliver the ne- cessary competences and specific training in leadership [2–4]. A recent debate on public health leadership featured in The Lancet pointed out that leadership is still not common in most public health training programs at undergraduate, postgraduate, and continuing professional development level and asserted that every public health organization should be engaged in developing more leaders at every level [4, 5]. There is a need for substantial investment in leadership training for public health profes- sionals . This raises a question of how higher education institutions can provide “content and context to initiate a major reconsideration of working and learning patterns which incorporate novel forms, based on the principles of interprofessional collaboration and transcend the confines of the classroom” . Blended learning, which is a combination of face-to-face and online learning , improves access to education for learners who need to organize their education around professional roles or domestic responsibilities or live far from universities [8–11]. Further, it is possible to provide the same educational program to learners in different countries [11, 12] and enable learning in groups despite geographical boundaries , which is important in the context of globalization in higher education and continuing education [14, 15]. This should facilitate multidisciplinary learning and nurture a spirit of teamwork, particularly with regard to leadership skills . Blended learning has proven to be at least as effective and equally satisfying for learners as traditional learn- ing methods [17–19]. Problem-based learning (PBL)  is widely used in medical education and stimu- lates the development of leadership competencies as learners are self-directed and collaborate in small groups to work on authentic, complex tasks to explore prob- lems and consider possible solutions [21, 22]. In response to the need to develop effective public health leaders, we explored the perceptions of the effectiveness and appeal of a newly developed public health leadership course using problem-based, blended-learning methods using virtual learning environment technologies. It was developed by international experts in public health and implemented in an inter- national context. As diversity between learners from different countries may influence the adaptation and use of educational innovations , we also investigated possible differ- ences between three different European academic settings: the Netherlands, Austria, and Lithuania. Case presentation This cross-sectional, evaluative study was carried out in two phases. At the beginning of the course, participants from all three countries filled out the Self-Assessment Instrument of Competencies for Public Health Leaders (SAIC-PHL) . At the end, they again filled out an evaluation questionnaire and the SAIC-PHL. Könings et al. Public Health Reviews (2018) 39:13 Page 3 of 12 Course description The Leadership in Public Health course was designed by international experts to intro- duce diverse European perspectives of leadership in the modern public health environ- ment in Europe. Public health is broadly made up of a number of specific disciplines such as methods in public health; population health and its social and economic determinants; population health and its material-physical, radiological, chemical, and biological environ- mental determinants; health policy; economics; organizational theory and management; health promotion; health education; health protection; and disease prevention and ethics . A starting point was to conceptualize public health in a way that it was relevant to all European states. For this reason, the project adopted the definition of public health in line with the European Public Health Operations (EPHOs) which constitute, “aset of fundamental actions that address determinants of health, and maintain and protect popu- lation health through organized efforts of society .” It was built around a thematic framework of public health leadership competencies, based on a systematic literature review , consisting of 52 competencies distributed among eight domains: Systems Thinking, Political Leadership, Building and Leading Interdisciplinary Teams, Leadership and Communication, Leading Change, Emotional Intelligence and Leadership in Team- based Organisations, Leadership Organisational Learning and Development and Ethics and Professionalism. After being first piloted by Sheffield University in the UK, the course was implemented at Maastricht University (Netherlands), Kaunas University (Lithuania), and the University of Graz (Austria) which were partners in the EU Erasmus Curriculum Development project “Leaders for Public health in Europe.” It was a part-time course, delivered over a period of 8 weeks. The official language was English. PBL was used as the instructional model and implemented as blended learning. The course began with one and a half days of face-to-face learning which included introductions to blended learning and PBL, tutorial group meetings for the first and second PBL task, and a lecture. All other tutorial group meetings and lectures were delivered online during six half-day sessions within a period of 8 weeks. The online sessions were interactive. Participants could interact by using the microphone or the chat function. Collaboration is one of the key learning principles of PBL and played a central role during the tutorial group meetings. During these meetings, participants were continu- ously in interaction with each other. Knowledge and experiences were shared, like in proper face-to-face PBL meetings. All participants attended the online lectures as one group. In some lectures, small group events were organized: Participants were divided into different online breakout rooms for discussions and afterwards plenary reported the results of that discussion in the main online lecture hall. The taught content was based on the competency domains and included systems thinking, political leadership, collaborative leadership, building and leading interdiscip- linary teams, leadership and communication, leading change, emotional intelligence and leadership in teams, and leadership, organizational learning, and development. These topics correspond with domains and competencies associated with leadership in the public health domain and cover the topics that were identified in the literature as most important and relevant for public health leaders . Each session was delivered by a teacher(s) responsible for one of these components. Teachers were from the UK, Austria, Lithuania, and the Netherlands and represented various academic fields: public health, psychology, nursing, political science, education, Könings et al. Public Health Reviews (2018) 39:13 Page 4 of 12 and social science. All teachers underwent PBL training and blended learning training prior to the delivery of the course. They also served as tutors in the online tutorial groups. A virtual learning environment was constructed which contained announcements, course information including course handbook, information on e-learning, and infor- mation for the sessions including all teaching materials, such as assignments, briefs, hand-outs, additional references, and literature cited or used during each session. There was also a discussion board, which was used for informal communication between participants. E-mail was used for questions to the course coordinator. Participants Nineteen participants (4 males, 15 females) completed the course: eight from Maastricht University (The Netherlands), five from Kaunas University (Lithuania), and six from the Medical University of Graz (Austria). The participating universities offered the existent courses in which this leadership course in a blended learning format could be included. The participants from Maastricht followed the European Public Health program and were used to PBL tutorial groups and familiar to some online practices. Participants from Kaunas followed a public health PhD program and had no prior experience of PBL or on- line learning, but they did have experience of team-based learning. Participants from Graz were working professionals, following a Master’s Program in Health and Nursing Science and were usually taught in lectures and small group seminars, but also had some experi- ence of PBL as well as some experience with web-based-training and online lectures. Instruments The Evaluation Questionnaire consisted of 54 items from several existing scales. Gen- eral satisfaction with course and instructor quality was measured with three items . Thirteen items were used for measuring instructiveness, productivity of tutorial groups, applicability of new knowledge, and difficulty of the course. Tutorial group functioning was measured by six subscales: elaboration, interaction, motivation, and sponging, co- hesion, and withdrawing . To evaluate tutor functioning, four subscales were used : stimulating constructive/active learning, stimulating self-directed learning, stimu- lating contextual learning, and stimulating collaborative learning. Motivation of the tutor to fulfill this role and stimulating professional behavior by the tutor are two single-item scales. Quality of different aspects of the e-learning was evaluated with three subscales : Evaluation of e-teaching, evaluation of e-resources, and interac- tions between learners consisted. Items use a 5-point Likert scale ranging from strongly disagree to strongly agree or a 10-point rating scale. One additional item asked to rate their computer skills on a 5-point rating scale ranging from “very poor” to “excellent.” The Self-Assessment Instrument of Competencies for Public Health Leaders (SAIC- PHL)  was used to measure learners’ own perceptions of their competencies before and after the course. This consisted of 52 items describing competencies essential for public health leaders. These competency descriptions are developed based on a litera- ture review and refined and validated in a consensus development panel and two rounds of a Delphi survey . For each competency, learners had to assess how well they thought they were doing on a 5-point scale ranging from “acting as a novice” to “acting as an expert.” This scale has been adapted from Dreyfus and Dreyfus . The Könings et al. Public Health Reviews (2018) 39:13 Page 5 of 12 items were organized into eight competency domains of the public health leadership framework , reflected in the eight subscales of the SAIC-PHL. Data collection and analyses All data were collected online. SPSS version 19 was used to analyze the data. Cronbach’s alpha was calculated for the different scales from the Evaluation Questionnaire and the SAIC-PHL to check whether it was acceptable to use scale scores (alpha from .70 consid- ered as acceptable, from .80 as good). We also explored the results on scales with lower alpha scores, while interpreting the results with more caution. Descriptive statistics for items and scales of the Evaluation Questionnaire, in the form of percentages and means, were used to examine the participants’ evaluations of the course. For negatively formu- lated items and scales consisting of only negatively formulated items, smaller means were interpreted as more positive. For scales containing both negatively and positively formu- lated items, the negatively formulated items were recoded before scale scores were calcu- lated. For the SAIC-PHL, pretest and posttest scores were compared with paired t tests. Differences in evaluation scores and in mean gains on the SAIC-PHL between partic- ipants from different locations were analyzed with non-parametric Kruskal-Wallis tests and post hoc Mann-Whitney tests. Non-parametric tests were used, because the sample sizes were small due to splitting up the sample into three groups for these analyses. To correct for multiple testing, Bonferroni-corrected alpha levels were used with the paired t tests, Kruskal-Wallis tests, and the Mann-Whitney tests. For the Kruskal- Wallis tests, Bonferroni correction was applied for each instrument separately. For the Mann-Whitney tests, the correction was applied for each pair of post hoc tests after a significant Kruskal-Wallis test. Bonferroni correction was also applied for the eight paired t tests computed for the SAIC-PHL. Results The response rate for pretest and posttest was 100%. With regard to the Evaluation Questionnaire, the subscale for global rating of course and instructor quality showed good internal consistency (α = .89). Subscales on tutorial group functioning showed acceptable to good Cronbach’s alphas consistency for elaboration (α =.85), interaction (α = .76), motivation (α = .93), and sponging (α = .73). Because for cohe- sion and withdrawing found alphas were below .50, scores on separate items were used in the analyses. Concerning tutor functioning, the results were the following: stimulating constructive/active learning (α = .85), stimulating self-directed learning (α = .58), stimulating contextual learning (α = .81), and stimulating collaborative learning (α = .89). With respect to e-learning, it showed evaluation of e-teaching (α =.90), evaluation of e-resources (α = .85), and interactions between learners consisted (α =.66). Table 1 presents Cronbach’s alphas of the SAIC-PHL. Evaluation of the course General satisfaction The course content was just right according to 15 of the participants (79%), while two of them (11%) thought contents were easy and two others (11%) found them difficult. The average grade given to different aspects of the course ranged from 6.79 (SD = 2.66) to 7.79 Könings et al. Public Health Reviews (2018) 39:13 Page 6 of 12 Table 1 Cronbach’s alphas for the scales from the self-assessment of competencies for public health leaders on pretest and posttest N of items α pretest α posttest Systems Thinking 7 .89 .91 Political Leadership 8 .94 .95 Inspiring and Motivating Others 7 .93 .96 Building and Leading Interdisciplinary Teams 5 .82 .95 Leadership and Effective Communication 7 .86 .89 Leading Change 6 .95 .94 Emotional Intelligence and Leadership in Teams 6 .91 .94 Ethics and Professionalism 6 .95 .93 (SD = 2.04) on a scale from 1 to 10. Because these means are on the positive half of the scale, these results indicate that on average participants were satisfied. Scores for the items using a 5-point scale differed between 3.47 (SD = 1.07) and 3.95 (SD = 0.97); they were on the positive side of the neutral value of 3. Means for the two negatively formulated aspects were 1.74 (SD = 0.93) and 2.05 (SD = 1.35), indicating positive evaluations. On the scale “Global rating of course and instructor quality,” the score was 3.53 (SD = 1.04). Table 2 shows results separately for differences between the locations. Due to the Bonferroni correction (accounting for conducting 14 tests), an alpha of .0036 is used for the Kruskal-Wallis tests. For the Mann-Whitney tests, the corrected alpha was .025. Participants from Graz (M = 4.56, SD = 0.66) gave higher global ratings of course Graz and instructor quality than their peers (M = 3.33, SD = .82, U = 5.50, p = .01) Maastricht (M = 2.60, SD = 0.64, U = 1.00, p = .0087). Kaunas Group functioning The means on functioning of the tutorial group were 3.56 (SD = 0.79) for interaction, 3.58 (SD = 1.07) for motivation, and 3.87 (SD = 0.78) for elaboration. Also, on cohesion, the Table 2 Student evaluation of course on leadership: results of Kruskal-Wallis comparing results at different locations Evaluation items Hp The organization of the block 3.64 .16 The instructiveness of the block 2.43 .31 The link of the block with their prior knowledge 0.79 .69 The productivity of the tutorial group 9.09 .0052 The link between the block and the assessment 3.74 .16 The objectives of the course were clear 4.74 .08 The literature fitted to the objectives of the course 6.55 .03 I can apply what I have learned in my daily work 2.88 .23 The e-lectures were instructive 7.35 .02 The tasks in the course handbook were instructive 6.27 .04 There were problems in collaboration because of differences in cultural background 5.28 .07 There were difficulties because different universities were involved 5.18 .07 Difficulty of the block contents 1.46 .62 Scale “global rating of course and instructor quality” 9.71 .0031 Grades given on a 10-point scale 5-point Likert scale Könings et al. Public Health Reviews (2018) 39:13 Page 7 of 12 means (3.11, SD = 1.37 and 4.26, SD = 0.65) were on the positive side of the scale or close to the neutral value of 3. For the scale on negative aspects of group functioning (i.e., spon- ging) the mean score was 2.58 (SD = 1.02); a lower score is a positive outcome on this scale. For the two items on withdrawing, the means were 3.58 (SD = 1.22) and 2.32 (SD = .82). Table 3 presents the differences between locations. Due to the Bonferroni correction (accounting for conducting 8 tests), an alpha of .0063 is used for the Kruskal-Wallis tests of group functioning. Significant differences were found for the scales interaction and sponging and for one of the items on withdrawing. Participants from Graz were more posi- tive than the others about interaction and about the absence of sponging and withdrawing. Tutor functioning The means for the tutor evaluation scales ranged from 3.11 (SD = 1.15) for stimulating collaborative learning to 3.75 (SD = 0.95) for stimulating constructive/active learning, indicating neutral to positive perceptions. The Bonferroni-corrected alpha for the Kruskal-Wallis tests of tutor functioning was .0083 (six tests conducted). Differences between locations were found on stimulating contextual learning and stimulating pro- fessional behavior (see Table 3). Participants from Graz were more positive about the tutors than participants from Maastricht on stimulating contextual learning and stimu- lating professional behavior. They were also more positive about stimulating contextual learning compared to participants from Kaunas. Quality of blended learning The mean regarding perceived quality of e-teaching was 3.61 (SD = 0.84), regarding quality of e-resources 3.49 (SD = .94), and regarding online student interaction 3.54 (SD = 0.71). The Bonferroni-corrected alpha for the Kruskal-Wallis tests of quality of blended learning was .017 (three tests conducted). Participants from Graz were more satisfied with the e-learning resources (H  = 9.23, p .0045; M = 4.33, SD = 0.63) than the participants from Kaunas (M = 2.60, SD = 0.86, U = 1.00, p = .01). On the other two scales, there were no differences. Required computer skills The average rating for own computer skills was 4.16 (SD = 0.90), ranging from poor to ex- cellent. Because only one Kruskal-Wallis test was performed for required skills, alpha was not adjusted. Differences between the universities for computer skills were found, H = 10.18, p = .003, indicating that participants from Graz (M = 3.17, SD = 0.75) rated their computer skills as poorer than those of the other participants (M =4.63, SD = Maastricht 0.52, U =3.00, p =.005; M = 4.60, SD = 0.55, U =2.00, p = .02). No differences were Kaunas found between participants from Maastricht and Kaunas, U =19.50, p =1.00. Self-assessment of competencies for public health leaders On the pretest, the SAIC-PHL scores on self-assessed competencies varied between 1.91 (SD = 0.83) and 3.02 (SD = 0.84). On the posttest, scores were between 2.89 (SD = 0.82) and 3.28 (SD = 0.74). The gains between pretest and posttest varied between 0.26 (SD = 0.96) and 1.04 (SD = 0.56). The Bonferroni-corrected alpha level for the parametric t tests analyz- ing the significance of the perceived learning gains and for the non-parametric Kruskal- Wallis tests analyzing the differences between the different locations was .0063. Significant learning gains were found for six of the eight competency domains: systems thinking, polit- ical leadership, inspiring and motivating others, building and leading interdisciplinary teams, Könings et al. Public Health Reviews (2018) 39:13 Page 8 of 12 Table 3 Evaluation of functioning of the tutorial group and tutor functioning: descriptive statistics and results of Kruskal-Wallis tests and Mann-Whitney tests for different locations M (SD) M (SD) M (SD) KW MW MW MW Maastricht Kaunas Graz Maastricht-Kaunas Graz-Maastricht Graz-Kaunas Scales H (p) U (p) U (p) U (p) Functioning of tutorial group Elaboration 3.44 (0.68) 3.70 (0.45) 4.58 (0.66) 6.61 (.03) Interaction 3.00 (0.59) 3.47 (0.38) 4.39 (0.53) 11.34 (.0005) 10.50 (.17) 1.00 (.0013) 2.00 (.02) Motivation 3.31 (0.88) 2.90 (1.19) 4.50 (0.55) 7.67 (.01) Sponging 2.75 (0.76) 3.50 (0.71) 1.58 (0.66) 10.90 (.001) 9.50 (.13) 5.50 (.0097) .00 (.0043) Cohesion_item 1 4.38 (0.52) 3.80 (0.45) 4.50 (0.84) 3.94 (.14) Cohesion_item 2 3.25 (1.28) 3.40 (1.34) 2.67 (1.63) 0.86 (.68) Withdrawing_item 1 4.00 (1.07) 4.20 (0.84) 2.50 (1.05) 6.70 (.03) Withdrawing_item 2 2.63 (0.74) 2.80 (0.45) 1.50 (0.55) 9.31 (.0047) 18.50 (.84) 6.00 (.02) 1.50 (.0087) Tutor functioning Stimulating constructive/active learning 3.38 (1.16) 3.47 (0.56) 4.50 (0.41) 7.50 (.02) Stimulating self-directed learning 3.13 (1.16) 3.40 (0.65) 3.92 (0.74) 1.86 (.41) Stimulating contextual learning 3.00 (0.80) 3.10 (0.82) 4.33 (0.52) 8.92 (.0064) 18.50 (.84) 4.00 (.0073) 2.00 (.01) Stimulating collaborative learning 2.31 (1.16) 3.40 (0.65) 3.92 (0.80) 6.93 (.02) Motivation of the tutor 3.38 (0.74) 3.40 (0.55) 4.50 (0.84) 6.28 (.04) Stimulating professional behavior 2.63 (0.92) 3.20 (0.84) 4.33 (0.82) 8.28 (.00832) 13.00 (.36) 4.00 (.0087) 5.00 (.07) Könings et al. Public Health Reviews (2018) 39:13 Page 9 of 12 leadership and effective communication, and leading change (see Table 4). Kruskal-Wallis tests showed that scores on different locations did not differ from each other. Discussion and conclusions The study showed that participants were overall positive about the effectiveness and delivery of the public health leadership course using blended learning and PBL method- ologies. It seems that they valued both group functioning and tutor functioning in PBL as well as different aspects of blended learning (online interaction, e-teaching, e- resources). The self-reported level of leadership competencies increased over the period of the course for participants at all three locations. They gained most in the area of Polit- ical Leadership, Systems Thinking, and Inspiring and motivating others. The perceptions of blended learning and PBL partly varied between the participating locations. Differences in educational background between learners from different countries might have influence the use of educational innovations  as well as the perceptions of blended learning curricula [32, 33]. Keller and colleagues reported that in their study of blended education delivered in Lithuania, Sweden, and Norway, the students from Lithuania evaluated the virtual learning most positively, which is not confirmed in our study with participants from Kaunas University. We consider several possible explanations for the differences we found between evaluations of participants at different locations. Lack of experience with PBL and online learning might have led to less positive eval- uations of the course. Furthermore, participants’ expectations of the course might have differed from their experiences during the course due to perceived pressure and inse- curity resulting from the need to perform in the group online. Discrepancies between expectations and actual experiences might have negatively affected their evaluation of the course . The demanding nature of blended learning may negatively affect student reactions to these courses [17, 35], and lower activity of some participants in online discussions might be due to the fact that they were not used to actively partici- pate in their learning at their home institution. Furthermore, the participants had different professional status. Participants who were clinical workers might have seen the relevance of the content more than participants who were mainly master students and PhD students. While the participants might have experienced the tension between the leadership theory and public health content, which constituted a kind of comfort zone for them, the resulting reflection contributed to a positive learning experience and Table 4 Self-assessment of competencies on leadership: descriptive statistics for pretest, posttest, and learning gains and statistics for paired t tests Leadership domain (topic of teaching) M (SD) M (SD) posttest M (SD) gains t p pretest (df = 18) Systems Thinking 2.28 (0.65) 3.26 (0.59) 0.98 (0.69) 6.21 .0000 Political Leadership 1.91 (0.83) 2.95 (0.75) 1.04 (0.56) 8.10 .0000 Inspiring and Motivating Others 2.29 (0.74) 3.15 (0.81) 0.86 (0.76) 4.92 .0001 Building and Leading Interdisciplinary Teams 2.49 (0.57) 3.21 (0.84) 0.72 (0.53) 5.84 .0000 Leadership and Effective Communication 2.59 (0.63) 3.11 (0.73) 0.51 (0.59) 3.79 .0013 Leading Change 2.17 (0.92) 2.89 (0.82) 0.73 (0.83) 3.82 .0013 Emotional Intelligence and Leadership in Teams 3.02 (0.84) 3.28 (0.74) 0.26 (0.96) 1.20 .25 Ethics and Professionalism 2.81 (0.89) 3.10 (0.82) 0.29 (0.94) 1.35 .20 Könings et al. Public Health Reviews (2018) 39:13 Page 10 of 12 understanding. In order to successfully implement effective public health interventions and change, collaboration with and participation of diverse groups of stakeholders is vital. Although the results of the study can be direction setting for the blended PBL course designers, there are limitations to this study. The sample size of our study was small and selected conveniently using the available cohorts of students. The results could have been different if we offered the course on the open enrollment basis. It is a strength of our study that a pretest-posttest design was used for measuring competency gains, although based on self-assessment. Using objective learning data and standard assessment of partic- ipants’ competencies could be an interesting addition for future studies as well as follow- ing the impact of the course of a longer period of time and within an experimental design. Taken together, this study suggests that organizing continuing education for profes- sionals in an international context has potential . Course designers might benefit from our results when designing blended learning courses. Bringing together inter- national experts in the field as teachers in a blended course, with ample opportunities for group work and discussions among participants, contributes to the development of professional competencies, in our case on leadership in public health . Differences between the evaluations of participants from universities from different countries point to the relevance of participants’ expectations, previous learning experiences, and educa- tional context. The findings may also suggest that although the blended learning format of course delivery was well accepted in this small-scale study, there is a need to support students’ shift to the more active independent mode of learning including coaching, mentoring, and personal development planning. Such approaches are successfully included in other hybrid inline leadership courses, for example, University of North Carolina and Gillings School of Global Public Health Doctoral Program in Health Leadership . The course applies modern technology and flexible teaching ap- proaches with the emphasis on development of leadership competencies for experi- enced health professionals working full-time anywhere in the world. Similar to our course, leadership is learned through interaction, debate and collaboration, and mentorship which is a key to lifelong learning. Future research on best ways to support the learning process may also be relevant to the other academic environments. We recommend invest- ing in training problem-based learning and blended learning skills for both students and staff to prevent that learning of these skills interferes with studying the course content. Conclusion It seems that problem-based learning and blended learning can be an effective way to de- velop public health leadership competencies among professionals in international and interdisciplinary context if the specificity and educational background of the learners is properly addressed. Endnote The study “Leaders for European Public Health” (LEPHIE) Project was carried out with the support of the European Commission Lifelong Learning Program. Acknowledgements Not applicable Könings et al. Public Health Reviews (2018) 39:13 Page 11 of 12 Funding ERASMUS Multilateral Curriculum Development project: Leaders for European Public Health (LEPHIE). Project no. 510176-LLP-1-2010-1-NL-ERASMUS-ECDCE—supported by the Lifelong Learning Program of the European Commission. Funding was received for developing the course. The funding body did not have a role in design of the study and col- lection, analysis, and interpretation of data, nor in writing the manuscript. Availability of data and materials The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. Authors’ contributions All authors but IS contributed to the conception and the design of the study (KK, NdJ, CL, LS, TS, SO’C, JvM, KC). The development of the measurement instruments was done by KK, NdJ, TS, SO’C, JvM, and KC. The data collection at location was realized by NdJ, CL, LS, and TS. Data analyses was conducted by IS, KK, and JvM. In interpreting the data and writing the paper, KK, IS, and KC took the lead. All authors read and approved the final manuscript. Ethics approval and consent to participate We invited the participants of the course to fill out the evaluation questionnaire within the framework of university satisfaction evaluations, for which we did not ask for the ethical clearance. Data collection was done according to the Helsinki Declaration, and informed consent was obtained. Consent for publication Not applicable Competing interests The authors declare that they have no competing interests. Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Author details Department of Educational Development and Research and Graduate School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, PO Box 616, 6200, MD, Maastricht, the Netherlands. 2 3 Department of Nursing Science, Medical University of Graz, Universitaetsplatz 4, A-8010 Graz, Austria. Department of Preventive Medicine and Institute of Health Research at the Faculty of Public Health, Lithuanian University of Health Sciences, A.Mickeviciaus street 9, 44307 Kaunas, Lithuania. Centre for Leadership in Health and Social Care, Faculty of Health and Wellbeing, Sheffield Hallam University, Sheffield S10 2BP, UK. England Centre for Practice Development, Faculty of Health and Wellbeing, Canterbury Christ Church University, Canterbury, UK. Department of International Health, CAPHRI School for Public Health and Primary Care, Faculty of Health, Medicine and Life Sciences, Maastricht University, PO Box 616, 6200, MD, Maastricht, the Netherlands. Institute of Public Health, Faculty of Health Sciences, Medical College, Jagiellonian University, Krakow, Poland. Received: 10 July 2017 Accepted: 6 March 2018 References 1. Marmot M, Friel S, Bell R, Houweling TA, Taylor S. Health CoSDo. Closing the gap in a generation: health equity through action on the social determinants of health. Lancet. 2008;372(9650):1661–9. 2. Bjegovic-Mikanovic V, Vukovic D, Otok R, Czabanowska K, Laaser U. Education and training of public health professionals in the European region: variation and convergence. Int J Public Health. 2013;58(6):801–10. 3. Czabanowska K, Smith T, Stankunas M, Avery M, Otok R. Transforming public health specialists into public health leaders. The role of LEPHIE project. Lancet. 2013;381(9865):449–50. 4. Day M, Shickle D, Smith K, Zakariasen K, Oliver T, Moskol J. Time for heroes: public health leadership in the 21st century. Lancet. 2012;380(9849):1205–6. 5. Day M, Shickle D, Smith K, Zakariasen K, Oliver T, Moskol J. Transforming public health specialists into public health leaders—authors’ reply. Lancet. 2013;381(9865):450. 6. Czabanowska K, Rethmeier KA, Lueddeke G, et al. Public health in the 21st century: working differently means leading and learning differently. Eur J Public Health. 2014;24(6):1047–52. 7. Lueddeke GR, Dayrit MM. Transforming medical education for the 21st century: megatrends, priorities and change. Radcliffe; 2012. 8. Graham CR. Emerging practice and research in blended learning. Chapter 21. In: Moore MG, editor. Handbook of distance education, third edition. Routledge; p. 333-50. 9. Beckmann EA. Learners on the move: mobile modalities in development studies. Distance Education. 2010;31(2): 159–73. 10. Ginns P, Ellis R. Quality in blended learning: exploring the relationships between on-line and face-to-face teaching and learning. Internet High Educ. 2007;10(1):53–64. 11. Graham C. Blended learning systems: definition, current trends, and future directions, The handbook of blended learning: global perspectives, local design. San Francisco: John Wiley & Sons; 2006. p. 3–21. 12. Altbach PG, Knight J. The internationalization of higher education: motivations and realities. J Stud Int Educ. 2007; 11(3-4):290–305. Könings et al. Public Health Reviews (2018) 39:13 Page 12 of 12 13. Ziegler MF, Paulus T, Woodside M. Understanding informal group learning in online communities through discourse analysis. Adult Educ Q. 2014;64(1):60–78. 14. Bowman L. Book review: handbook of adult and continuing education. Adult Educ Q. 2012;62(4):413–5. 15. Coryell JE. Collaborative, comparative inquiry and transformative cross-cultural adult learning and teaching: a western educator metanarrative and inspiring a global vision. Adult Educ Q. 2013;62(4):299–320. 16. Suzuki Y, Niwa M, Shibata T, et al. Internet-based problem-based learning: international collaborative learning experiences, Problem-based learning in eLearning breakthroughs; 2007. p. 131–46. 17. Spanjers IA, Könings KD, Leppink J, et al. The promised land of blended learning: quizzes as a moderator. Educational Res Rev. 2015;15:59-74. 18. Wutoh R, Boren SA, Balas EA. ELearning: a review of Internet-based continuing medical education. J Contin Educ Health Prof. 2004;24(1):20–30. 19. Könings KD, Popa D, Gerken M, et al. Improving supervision for students at a distance: videoconferencing for group meetings, Innovations in education and teaching international; 2015(ahead-of-print). p. 1–12. 20. Mustard JF, Neufeld VR, Walsh WJ, Cochran J. New trends in health sciences education, research, and services: the McMaster experience. New York: Praeger Publishers; 1982. 21. Dolmans DH, De Grave W, Wolfhagen IH, Van Der Vleuten CP. Problem-based learning: future challenges for educational practice and research. Med Educ. 2005;39(7):732–41. 22. Dolmans D, Michaelsen L, Van Merrienboer J, van der Vleuten C. Should we choose between problem-based learning and team-based learning? No, combine the best of both worlds! Med Teach. 2015;37(4):354–9. 23. Jippes M, Majoor GD. Influence of national culture on the adoption of integrated and problem-based curricula in Europe. Med Educ. 2008;42(3):279–85. 24. Czabanowska K, Smith T, O'Connor S, Sumskas L, Otok R, Brand H. Identifying the leadership competencies for European public health professionals. Eur J Pub Health. 2012;22:91. 25. Birt CF, A.. European core competences for MPH education (ECCMPHE). Brussels: ASPHER; 2011. 26. World Health Organization. European action plan for strengthening public health capacities and services. Copenhagen: Regional Committee for Europe; 2012. http://www.euro.who.int/en/health-topics/Health-systems/public- health-services/publications/2012/european-action-plan-for-strengthening-public-health-capacities-and-services. 27. Czabanowska K, Smith T, Könings KD, et al. In search for a public health leadership competency framework to support leadership curriculum—a consensus study. Eur J Public Health. 2013:24(5):850–56. 28. Frick TW, Chadha R, Watson C, Zlatkovska E. Improving course evaluations to improve instruction and complex learning in higher education. Educ Technol Res Dev. 2010;58(2):115–36. 29. Dolmans DH, Wolfhagen IH, Van der Vleuten CP. Thinking about student thinking: motivational and cognitive processes influencing tutorial groups. Acad Med. 1998;73(10):S22–4. 30. Dolmans DH, Ginns P. A short questionnaire to evaluate the effectiveness of tutors in PBL: validity and reliability. Med Teach. 2005;27(6):534–8. 31. Dreyfus SE, Dreyfus, HL. A five-stage model of the mental activities involved in directed skill acquisition (No. ORC- 80-2). California Univ Berkeley Operations Research Center. 1980. 32. Keller C. User acceptance of virtual learning environments: a case study from three northern European universities. Commun Assoc Inf Syst. 2009;25(1):38. 33. Keller C, Hrastinski S, Carlsson S. "Students' Acceptance of E-Learning Environments: A Comparative Study in Sweden and Lithuania" (2007). ECIS 2007 Proceedings. 40. http://aisel.aisnet.org/ecis2007/40. 34. Könings KD, Brand-Gruwel S, van Merriënboer JJ, Broers NJ. Does a new learning environment come up to students’ expectations? A longitudinal study. J Educ Psychol. 2008;100(3):535–48. 35. Sitzmann T, Kraiger K, Stewart D, Wisher R. The comparative effectiveness of web-based and classroom instruction: a meta-analysis. Pers Psychol. 2006;59(3):623–64. 36. Health UGSoGP. Public health leadership program. https://sph.unc.edu/phlp/phlp/. Submit your next manuscript to BioMed Central and we will help you at every step: • We accept pre-submission inquiries � Our selector tool helps you to ﬁnd the most relevant journal � We provide round the clock customer support � Convenient online submission � Thorough peer review � Inclusion in PubMed and all major indexing services � Maximum visibility for your research Submit your manuscript at www.biomedcentral.com/submit
Public Health Reviews – Springer Journals
Published: Jun 1, 2018
It’s your single place to instantly
discover and read the research
that matters to you.
Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.
All for just $49/month
Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly
Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.
Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.
Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.
All the latest content is available, no embargo periods.
“Hi guys, I cannot tell you how much I love this resource. Incredible. I really believe you've hit the nail on the head with this site in regards to solving the research-purchase issue.”Daniel C.
“Whoa! It’s like Spotify but for academic articles.”@Phil_Robichaud
“I must say, @deepdyve is a fabulous solution to the independent researcher's problem of #access to #information.”@deepthiw
“My last article couldn't be possible without the platform @deepdyve that makes journal papers cheaper.”@JoseServera