TY - JOUR
AU - Miettinen,, Reijo
AB - Abstract This article focuses on the social impact of educational sciences. It introduces a framework for studying the social impact of research and uses it to analyze the social impact of the work of a research group studying learning difficulties at the University of Jyväskylä, Finland. The framework suggests that the social impact of research can be understood in terms of the context of impact. This context is composed of four interacting dimensions: epistemic, artefactual, social-institutional, and spatial-geographical dimensions. In addition, the paper suggests that the understanding of the phenomena to be studied and mediating artefacts based on this understanding play a key role in the expansion of the social impact in the educational sciences. The article provides a means of analyzing the narratives of longer-term impact of research and suggests that even in education—a classical area of advancing public good—the distribution of mediating artefacts is increasingly likely to be realized through markets. Introduction It has been widely recognized that understanding social impact of research in economic terms is too narrow an approach (Bozemann and Sarewitz 2011). This is evident in the arts, humanities, and social sciences. Benneworth (2015) argues that the impacts go untraced because these fields have not yet been able to reframe ideas of impact in terms of social benefit and cultural value. The diversity of the arts and humanities and social sciences challenges the evaluation methods. Thus far, they have failed to characterize the ways in which these sciences generate public value (Bozemann and Sarewitz 2011; Molas-Gallart 2015; Reale et al. 2018). Indicator systems developed for the measurement of social impact have been criticized for being based on a linear concept of innovation, being too insensitive to the specificities of different disciplines, and tending to grow too complex and costly (Martin 2011). Consequently, approaches of evaluation have moved towards frameworks that focus on processes of impact creation and interactions between researchers and stakeholders (Bozemann and Sarewitz 2011; Donovan and Hanney 2011; van Drooge and Spaapen 2017). Molas-Gallart (2015: 122) has proposed that evaluation needs to be aware of the context where knowledge is generated and applied. The interactional approaches and impact pathway models also call for contextual sensitivity and focus on user engagement in collaboration between researchers and stakeholders (De Jong et al. 2011). In this article, we focus on the social impact of educational sciences, which thus far has not been studied in the evaluation literature. Their contribution to public good is evident. Educational sciences are institutionally context-specific in terms of having the education and school systems as its object of study. The role of education is paramount for democracy and the promotion of equality in the society (Dewey 1916/1985). In addition, education and human capital are increasingly important for the economy because of the knowledge-society development. The interactional models of impact evaluation provide valuable knowledge on the collaborative relationships between researchers and stakeholders. In disciplines connected to public services, such as education and social work, the role of professions is important (Morton and Flemming 2013; Morton 2015a). The role and autonomy of professions is strong and they have a range of organizations, which are important in disseminating and utilizing research knowledge (Morton 2015b). In most countries, the school system and educational policies are expected to provide equal opportunities for every child. This calls for an early prevention policy and special education system, which is maintained by professional groups, such as school psychologists, special education teachers, teachers, pupil counselors, and public health nurses. The role of beneficiaries of research varies depending on the research field and its institutional connections to the surrounding society. The terminology of referring to stakeholders is diverse: research users, citizens, beneficiaries, sponsors, knowledge transformers, end-users, and co-producers (Rowe and Frewer 2005; Jörgensen and Bozeman 2007; Molas-Gallart and Tang 2011; Morton 2015b; Benneworth and Olmos-Peñuela 2018). These stakeholders and beneficiaries’ role varies depending on their position in the networks of knowledge production and research utilization. For instance, the notion of the knowledge transformer is related to entrepreneurship (Benneworth and Olmos-Peñuela 2018: 765) while the notion of the citizen opens the debate towards rights and responsibilities of an individual (Bozemann and Sarewitz 2011: 4). The degree to which any stakeholder can have an effect on interaction processes varies too. As Reale et al. (2018: 304) point out research evaluation has not paid attention on studying how partnerships include the most vulnerable end-users throughout the research process. This article studies the social impact of a Finnish research group that studies learning difficulties at the University of Jyväskylä. Our goal is to introduce a framework to analyze the development of the social impact of a research group. We follow the history of the group’s research and its applications over three decades, starting from the 1990s. Our research questions are: What is the specificity of mechanisms of impact in educational sciences? How social impact and its mechanisms changed over time? How new research results influence the scope and direction of impact? What possibilities the information and communication technology (ICT)-based means provide for social impact? We provide a narrative of the development of research and its social impact. Because productive interactions (Spaapen and van Drooge 2011) and contributions to social change (Morton 2015b) are transformed over the course of research process, we find it important to draw a distinction between qualitatively diverse phases in the development of social impact. In each phase the understanding of the relevant phenomena, the mediating role of artefacts, as well as the social networks through which the impact is realized differ. We start by presenting our theoretical approach and describing the research group under study. Then we explain the data collection and analysis, and finally, present three consecutive contexts of impact, which emerged along the decades of the research activity of the group. We conclude by answering the research questions and discussing in which way our framework contributes to social impact studies. Theoretical framework The interaction between science, technology, and social practices is a historically changing relationship. A widely accepted account of their relationship is that starting from the late 19th century, the scientification of technology took place (Böhme, van den Daele, and Krohn 1983). The same process also has been characterized as the transformation of science into a force of production (e.g. von Wright 1992). Solutions to complex social and environmental problems (such as climate change) require scientific understanding of the phenomena related to these problems. The specific contribution of scientific work in society is the production of such an understanding, a task that is mostly unattainable by other stakeholders. The breakthroughs in research and the deepening of understanding of relevant phenomena, often in unexpected ways, contribute to substantial improvements of social practices (Balconi, Brusoni, and Orsenigo 2010). This is why we find it important to include an epistemological point of view in an account of social impact of research, that is, the articulation of the increased understanding of phenomena that allow for new applications and solutions for social practices. The relation between research and its social impact has been analyzed in terms of context transitions (Raiski 1991, 1993). The realities encountered in a practical context can be moved to a laboratory or to a theoretical environment. There they are transformed into scientifically analyzable phenomena using conceptual resources and the methods of science. Theoretical accounts of a phenomenon permit the application of new knowledge to a wider range of contexts than before. Bruno Latour’s analysis of the impact of Louis Pasteur’s work on French society (Latour 1988) is a case in point. Pasteur studied the problems of contamination in breweries and wineries (practical contexts) in his laboratory (or a theoretical context). The explanation of problems in terms of microbes and the emergent microbiology helped eradicate the contamination problems and resulted in major changes in many social practices. The impact evaluation literature has used the term context in diverse ways. Mostly it has been used to refer to a particular evaluation context, such as the performance of an organization, a research program or the evaluation of individuals’ performance (Darling-Hammond 1990; Geuna and Martin 2003; Hicks 2012). Used in this sense, the context then depends on the aims of that particular evaluation. A recent paper that aims to provide ideal types of social impact in social sciences and humanities (Muhonen, Benneworth, and Olmos-Peñuela 2019: 3) uses the notion of a broader context interchangeably with the notion of impact pathway. Kearnes and Wienroth (2011: 167–168) suggest that pathways of impact mark a shift form the definition of final impact to socio-technical devices that structure and articulate potential research impacts. They (ibid. 172, see also Guston 1999) speak of ‘standardized packages’, which take a number of forms: metrics of research expenditure and case studies of research impact. These packages work as an alternative theorization of the relationship between research and real-world impacts. Pathways are therefore a way of typifying the diverse impact patterns, which are detached from a particular context allowing for a uniform comparative framework for the purposes of funding and evaluation. We use the concept of context of impact to refer to qualitatively different phases in the development of the social impact in the history of the research group. In our account, the context consists of four dimensions. The first dimension is epistemological and objectual, which is composed of the object and motive of research. In cultural-historical activity theory, any activity is defined according to its object, that is, according to what activity is focused on and what it aims to achieve (Miettinen 1998; Engeström, Puonti, and Seppänen 2003a; Engeström, Engeström, and Kerosuo 2003b). In science studies, Rheinberger (1997) and Knorr-Cetina (1997) have studied the open-ended nature of epistemic objects characteristic of research work: they are open to redefinition and capture something that does not exist yet. The breakthroughs in scientific understanding of the relevant phenomena are likely to help in redesigning the ways in which practical problems are dealt with and resolved. The second dimension of the context of impact is the social and institutional dimension, composed of networks of collaboration through which the impact is realized. It includes the direct personal contacts as characterized in terms of ‘productive interactions’ (Spaapen and van Drooge 2011). The public value mapping framework refers to formative interaction between research and stakeholders resulting in the construction of social value of research (Bozemann and Sarewitz 2011). It underlines the significance of the users’ role in the process in which results are adopted for the benefit of the society. The third, artefactual dimension, covers the mediating artefacts (such as instruments, methods, research-based guidelines, and training materials) through which the impact is realized. Derek de Solla Price (1984) suggested that technological artefacts and instruments constitute a key bridge between science, technology, and industry. Procedures, tools and methods developed in research can often be used as such or transformed into tools in industry. This is the case also in non-industrial practices. For instance, researchers and clinicians share tools in medicine, and software developed for research purposes may be utilized in other sectors of society. Artefacts are recognized by the SIAMPI approach as a material carrier or mediator of indirect contact (Spaapen and van Drooge 2011: 213). They are included in the Payback Framework in the form of secondary output realized in the interface of dissemination. We find the production of secondary outcomes or mediating artefacts to be essential in the understanding of social impact of research. The fourth dimension of the context of impact is the spatial and geographical dimension. It expresses the spatial extension of the impact. By analyzing how the four dimensions are interconnected and constitute a context of impact in different phases of the development of a research community we provide a framework for case narratives of social impact of research groups and local communities. We find it difficult to have a separate funded project as a unit of analysis because a local research program is funded from several sources and constitutes an agglomeration of projects and funding for individual researchers. The breakthroughs in research and deeper understanding of a phenomenon cannot be attributed to a single project. This makes the evaluation of impact in terms of the input-output relationship difficult. The Jyväskylä research group on learning difficulties The research group under study works within the Department of Psychology at the University of Jyväskylä, in central Finland. Its work has contributed to the development of central institutions in society: kindergartens, comprehensive school, and secondary education. There is close collaboration between the research group and the Niilo Mäki Insitute (NMI), which is a center for multidisciplinary research on learning difficulties based on the fields of neuropsychology and special pedagogy.1 The Department of Psychology at the University of Jyväskylä and the City of Jyväskylä family counselling unit established a Child Research Clinic in 1985. After the foundation of the Niilo Mäki Institute in 1990, the clinic moved its premises towards contributing to research on learning difficulties and to the development of diagnostic tests and remediation materials. It provided a place to study the whole spectrum of learning difficulties. The combination of the clinic, the university, and the institute formed the basis for the first generation of researchers, who learned to combine academic knowledge with practical work with children. The Academy of Finland twice supported the group through its Centre of Excellence program. First, it was nominated as the Centre of Excellence of Human Development and Its Risk Factors (1997–2005), and again as the Centre of Excellence of Learning and Motivation (2006–11). The latter period integrated two research groups related to learning difficulties: the neuropsychological research group (led by Heikki Lyytinen) and the research group on motivational aspects of learning and learning difficulties (led by Jari Nurmi). The project gathered together researchers from cognitive psychology, special pedagogy, educational sciences, teacher education, genetics, social psychology, and statistics. The most visible research effort of the group was the Jyväskylä Longitudinal Study of Dyslexia (JLD) 1993–2015. Data and analysis The data comprise interviews and documentary material. We conducted thematic and narrative interviews with members of the research group, the Niilo Mäki Institute and representatives outside academia responsible for the dissemination and commercialization of the research group’s results. The interviews were gathered from 2011 to 2018 in three rounds. We interviewed our key informant, the head and scientific director of the research group three times: in 2011, in 2014, and again in 2018. The interviews of the first round focused on the early phases of research work and its institutional context, the development of the research field, and the development of diagnostics and training methods. The second round of interviews had a stronger focus on the dissemination efforts of research results, training methods and artefacts. The key researchers were also asked to characterize the significance of the research results for the development of training methods. The third round of interviews broadened the focus towards dissemination of results, especially of GraphoGame, a research-based training tool. The second set of data comprise four types of documentary material. The first included the scholarly articles, doctoral dissertations and masters’ theses of the research group. The second documentary source was the NMI bulletin (1992–2015) published by the Niilo Mäki Institute. The NMI bulletin publishes peer-reviewed scholarly articles, reports on the current research, and accounts of new diagnostic tests and rehabilitation materials. Third, the annual reports of the Niilo Mäki Institute (1991–2016) helped us specify activities related to the children’s clinic, training, and publications as well as the use of GraphoGame. Fourth, we included in the data 15 articles published in professional publications (Psykologia, the journal of the Finnish Psychological Association and Kielikukko, a publication of the Finnish Reading Association). Table 1 summarizes the data sources of the study. Table 1. Data sources of the study Data source Amount of data Interviews First round in 2011: Four researchers in total Second round in 2013/2014: Six in total: three researchers, two intermediary organization’s representatives, and one project coordinator (university innovation services) Third round in 2018: Four in total: three researchers, one stakeholder manager (business partner) Documentary material Scientific and scholarly articles: 55 articles in total 28 scientific journal articles, years 1995–2015 16 Masters’ theses, years 2003–2010 12 doctoral dissertations, years 2007–2015 NMI Bulletin: 23 articles in total, years 1991–2015 1992–1999: seven articles on diagnostics, training methods, and context of research 2000–2008: seven articles on LukiMat, longitudinal study, and GraphoGame 2009–2015: nine articles on GraphoGame, LukiMat, and research results Annual reports of Niilo Mäki Institute: 26 in total, years 1991–2016 Other professional publications: 15 articles in total Data source Amount of data Interviews First round in 2011: Four researchers in total Second round in 2013/2014: Six in total: three researchers, two intermediary organization’s representatives, and one project coordinator (university innovation services) Third round in 2018: Four in total: three researchers, one stakeholder manager (business partner) Documentary material Scientific and scholarly articles: 55 articles in total 28 scientific journal articles, years 1995–2015 16 Masters’ theses, years 2003–2010 12 doctoral dissertations, years 2007–2015 NMI Bulletin: 23 articles in total, years 1991–2015 1992–1999: seven articles on diagnostics, training methods, and context of research 2000–2008: seven articles on LukiMat, longitudinal study, and GraphoGame 2009–2015: nine articles on GraphoGame, LukiMat, and research results Annual reports of Niilo Mäki Institute: 26 in total, years 1991–2016 Other professional publications: 15 articles in total Open in new tab Table 1. Data sources of the study Data source Amount of data Interviews First round in 2011: Four researchers in total Second round in 2013/2014: Six in total: three researchers, two intermediary organization’s representatives, and one project coordinator (university innovation services) Third round in 2018: Four in total: three researchers, one stakeholder manager (business partner) Documentary material Scientific and scholarly articles: 55 articles in total 28 scientific journal articles, years 1995–2015 16 Masters’ theses, years 2003–2010 12 doctoral dissertations, years 2007–2015 NMI Bulletin: 23 articles in total, years 1991–2015 1992–1999: seven articles on diagnostics, training methods, and context of research 2000–2008: seven articles on LukiMat, longitudinal study, and GraphoGame 2009–2015: nine articles on GraphoGame, LukiMat, and research results Annual reports of Niilo Mäki Institute: 26 in total, years 1991–2016 Other professional publications: 15 articles in total Data source Amount of data Interviews First round in 2011: Four researchers in total Second round in 2013/2014: Six in total: three researchers, two intermediary organization’s representatives, and one project coordinator (university innovation services) Third round in 2018: Four in total: three researchers, one stakeholder manager (business partner) Documentary material Scientific and scholarly articles: 55 articles in total 28 scientific journal articles, years 1995–2015 16 Masters’ theses, years 2003–2010 12 doctoral dissertations, years 2007–2015 NMI Bulletin: 23 articles in total, years 1991–2015 1992–1999: seven articles on diagnostics, training methods, and context of research 2000–2008: seven articles on LukiMat, longitudinal study, and GraphoGame 2009–2015: nine articles on GraphoGame, LukiMat, and research results Annual reports of Niilo Mäki Institute: 26 in total, years 1991–2016 Other professional publications: 15 articles in total Open in new tab The analysis was divided into three parts, which were linked to each other accumulatively. Figure 1 shows the three steps and the data used in each step. The longitudinal data collection and analysis enabled the follow up of dissemination efforts of research-based materials and a retrospective tracking of research groups’ interactions with other stakeholders. Figure 1. Open in new tabDownload slide Phases of analyses and data used. Figure 1. Open in new tabDownload slide Phases of analyses and data used. First, we created a chronological timeline of the main events and turning points in the development of research and its applications from the beginning of the formation of the research group. This was done by using the interviews conducted in 2011 and analyzing the contribution of scholarly documents based on their contribution to the understanding of reading and writing difficulties and to the methods of intervention. As a result, we drew a distinction between three phases of the research and its social impact. The second part of the analysis was carried out after the second round of interviews in 2014. We continued the analysis in an iterative way in order to link a range of data sources together. After distinguishing three phases of research work, we used our framework to analyze the data and characterize the four dimensions of the contexts of impact. In this phase, new data was collected to specify the quantitative indicators of impacts. The annual reports of NMI provided us with numbers of courses, participants in training and an overall description of the work done in the NMI over the years. In the third part of the analysis, we constructed a complete narrative based on our framework. Narrative analysis synthesizes the data by relating events and actions to one another and requires recursive movement between the data (Polkinghorne 1995: 15–16). For us this meant that we read the interviews and the documents side by side to provide an account of the development of the four dimensions of social impact and their interdepencies. In each of the phases, the understanding of learning difficulties, artefacts used, social networks and their regional scope were intertwined in a characteristic way allowing the definition of the context of impact. Local-national context: child research clinic and the provision of tools for prevention and rehabilitation The universal 9-year comprehensive school system was established in Finland during the first half of the 1970s. The whole generation of pupils were to meet the goals of the curriculum. Consequently, the new school system made many learning problems visible and created a strong need to overcome them. In the 1980s and early 1990s, a psycho-medical approach dominated the development of tests and the recognition of learning difficulties. The neuropsychological symptoms of pupils were typically diagnosed as disabilities listed in the international disease classifications such as International Classification of Diseases (ICD) of and the DSM-III-R of the American Psychiatric Association. The researchers started to collect and develop extensive batteries of tests to be able to recognize specific learning difficulties and to define diagnostic categories for them (Ahonen & al. 1991). However, in this work the limitations of the psycho-medical framework and classifications became evident. They did not consider the teaching and learning environment sufficiently nor did they suggest the pedagogic solutions or methods to overcome the problems: The concept of “learning difficulties” is funny in the sense that they were included in the ICD (International Classification of diseases) classifications, along with influenza and cancer. The definition [of learning difficulties] states that these cannot be diagnosed if a person has not had sufficient teaching and support (…). Therefore, there is a disease whose precondition is that you need to be in school. You have to have a school system in order to diagnose dyslexia. (Researcher 3, Vice-Executive Manager NMI, 19 May 2011) The research started to move its focus from classification of developmental disturbances to learning difficulties as they appeared in kindergarten and school environments with an orientation to find the remedial measures and materials that could be used by school psychologists and teachers. The researchers designed various assessment and screening methods to diagnose learning difficulties: FonOrto for reading difficulties, NEPSY for the adhd-type of neuropsychological problems, ESIKKO for the preverbal communication of children at the age of 6–24 months. The diagnostics that were developed provided a language and a shared understanding of the learning difficulties for the schools psychologists, nurses, special education teachers, teachers and parents. The tests and methods were first pilot-tested in schools and kindergartens in the Jyväskylä region: “In order to do any research, we need to collaborate with kindergartens, preschools and schools. A big part of data collection takes place in schools and it’s done by teachers, meaning that teachers lead group testing or we need facilities from schools. So, our work is fully based on trust, teachers invest their time in our projects without remuneration. There is a mutual respect for each other’s work.” (Vice-Executive Manager NMI, 19 May 2011) A new area of research emerged, the study of the effectiveness of intervention methods and materials. Both the tests and materials were tested by the ‘end users’ to assure the ecological validity of the results. The researchers provided the instructions and materials, the teachers used them and provided the results for the researchers. The researchers also observed and videotaped the intervention events to improve the instructions. For example, Nallematikka (TeddybearMath) materials for improving the ability of young children (from 3 to 5 years) to learn mathematics were developed in this way. Nallematikka is an example of play pedagogy: mathematical thinking is practiced by participating to the life of a teddy bear family and by playing games that the bears teach them. The major projects for developing the tests and rehabilitation methods and materials were done in collaboration with the organizations of the third sector, such as The Finnish Association for Intellectual and Developmental Disabilities and Rehabilitation Foundation. To disseminate research findings, diagnostic tests, and remediation materials, the Niilo Mäki Institute provided training courses mainly for teachers. In 1998, there were eight courses and the number of participants was 758. By 2005, the number of people attending the NMI’s training courses was 5500. The participants remained at around 5500 in 2005–08 but online courses have broadened the scope of training. The institute sold tests and materials and training courses to municipalities and schools, which used their publicly funded budgets to buy them. Expanded national context and the discovery of the key precursor of dyslexia The most important scientific enterprise of the group was the JLD. During 1993–2012, researchers followed 200 children, of which 108 were from dyslexic families and a control group of 92 children from birth to young adulthood. The central goal of the study was to understand dyslexia in terms of its antecedents, that is, the cognitive features of the children that anticipate the onset of dyslexia in preschool and school. The recognition of precursors would allow the prediction of dyslexia and would help to find methods for its prevention. During the study, the accumulation of research results strongly indicated that dyslexia is a genetically determined specific problem. The person suffering from dyslexia may otherwise be visually, mathematically or verbally talented. It was estimated that 5-10% of the age group suffers from dyslexia. The major proportion of studies of dyslexia focused on learners of the English language. The researchers stated that the relevance of specific phonological abilities for reading skills depended on the features of the writing system of the language (Aro et al. 1999; Holopainen et al. 2000; Lyytinen et al. 2001). They questioned the generalizability of the research results based on the studies of English speakers. In contrast to the English language, Finnish, like many other languages, has a transparent writing system (orthography) with a consistent connection between a letter and a sound. In transparent languages, it is easier to master this connection than in nontransparent orthographies such as English, in which none of the letters represents the same sound (phoneme) in all contexts of writing. The central concept of the JLD study was precursor. A precursor establishes a bridge between an explanation of dyslexia—in terms of its antecedents—and orientation to its prevention and remediation: A precursor is a predictor … a variable that can be explained theoretically showing the way the connection is established. A correlate can be anything, which by coincidence correlates without us knowing beforehand that it has any conceptually interpretable connection to the criterion. This is the distinction I have used. Many [studies] just look at correlations empirically and argue that this is a good predictor [of dyslexia]. However, they may be poor in predicting the connection in the right way. In some circumstances, they do not [predict the connection]. (Head of the research group, 20 March 2014) Among the precursors found, a child’s ability to connect a letter with its corresponding sound proved to be the strongest predictor of dyslexia (Lyytinen et al. 2004). The finding of the letter-sound connection as the key precursor of dyslexia was a starting point for the development of methods and means of cultivating this ability. The need to develop such a method became clear when researchers were presenting their results in public: “After a couple of presentations [of our findings] in which we explained that we are able to predict who will have learning difficulties the audience asked, “so what?” It made us think about how we answer this question. After that we had to start developing a method to actually do something.” (Head of the research group, 20 March 2014) The new understanding of the letter-sound connection helped the researchers to recognize children at risk, but the question remained about how to help children. In 2003, this concern led to the development of GraphoGame (Ekapeli in Finnish), a game in which a child is asked to recognize and select a letter among moving alternatives on a computer screen on hearing a sound. The game was developed in a way that would be motivating and pleasurable to children. A student in computer science coded the first version, and later, the game was piloted in a master’s thesis (Lehtonen 2003) in teaching children letter-sound correspondences and relationships between written and pronounced syllables. The Jyväskylä research group and the NMI received funding from the Ministry of Education for the years 2006–15 to collect, organize and disseminate already existing knowledge about learning difficulties through a LukiMat Internet platform. LukiMat included three sections: learning difficulties in reading and writing, learning difficulties in mathematics, and a section for pedagogical evaluation. The Niilo Mäki Foundation retained the intellectual property rights of the Finnish version of GraphoGame. This was a conscious policy to optimize the game’s social impact: The GraphoGame project has disseminated the research results exceptionally well. Because the game has been developed based on the longitudinal study, it is an application. There has been funding that has enabled development along with research.… First of all, it was Heikki Lyytinen’s idea that it has to be free. The other point is that it has been deliberately developed in a user-oriented way so that it has been disseminated to schools and children. (Coordinator of LukiMat, 18 March 2014) The Finnish version of GraphoGame (Ekapeli) was first published on the LukiMat service in November 2007. By the end of 2008, there were almost 40,000 registered users of the game and over 100,000 visitors in the LukiMat service. A child can play the game freely but in order to have this right, a parent or a teacher has to register on the server as an instructor user and verify the permission for the child to use the game. The number of players and instructor users increased year by year. The researchers themselves explained the impact of the game as follows: We believe that children with familial risk and/or low letter knowledge during the few months preceding school entry benefit from preventive playing in terms of avoiding unwanted failure experiences during the early months of school instruction. Therefore, we have recommended to kindergartens where all children in Finland have their pre-school year just before school that the game should be used during the last two months (April–May) and preferably with massed practice. This means short 5–15 minute periods several times per day for as long as children require to learn the letter-sound connections (…). Today, more than 50, 000 children in Finland have tried the game and very few have failed to benefit. (Lyytinen et al. 2009, 672) Table 2 shows the increase of registered players, total number of players and visits to LukiMat platform during 2008–12. In 2013, the researchers started an anonymized follow-up of players and in 2014 it showed that all versions of GraphoGame had been played for 2 billion seconds, which equals to 23,000 consecutive days. Replacing the same amount with individual special education would require the yearly work effort of 400 teachers (Annual report of NMI 2014: 16). In 2015, the game was played by over 250, 000 children (Rantanen 2015) and by 2016, there were 485,000 visitors to the LukiMat service. Table 2. Number of players and visits to the LukiMat platform 2008–2012 Year Number of new players per year Total number of players Visits to LukiMat platform 2008 40,000 40,000 100,000 2009 21,500 61,500 123,000 2010 30,937 92,437 173,000 2011 36,759 129,196 220,000 2012 40,804 170,000 – Year Number of new players per year Total number of players Visits to LukiMat platform 2008 40,000 40,000 100,000 2009 21,500 61,500 123,000 2010 30,937 92,437 173,000 2011 36,759 129,196 220,000 2012 40,804 170,000 – Open in new tab Table 2. Number of players and visits to the LukiMat platform 2008–2012 Year Number of new players per year Total number of players Visits to LukiMat platform 2008 40,000 40,000 100,000 2009 21,500 61,500 123,000 2010 30,937 92,437 173,000 2011 36,759 129,196 220,000 2012 40,804 170,000 – Year Number of new players per year Total number of players Visits to LukiMat platform 2008 40,000 40,000 100,000 2009 21,500 61,500 123,000 2010 30,937 92,437 173,000 2011 36,759 129,196 220,000 2012 40,804 170,000 – Open in new tab In 2015, the Ministry of Education stopped funding the LukiMat project but required the ownership of parts of the online material. Researchers found that this limited the social impact of their work because they could no longer record new knowledge, methods, and materials on the platform. Global context: the study and dissemination of GraphoGame After the research group had established its findings in the Finnish language and developed versions of GraphoGame for different age-groups and for diverse purposes (such as reading fluency, recognition of syllables, and sounds) in Finland, they started to expand their research into other languages. The Niilo Mäki Institute had been organizing training for teachers and psychologists in Zambia and Kenya since the early 1990s. Zambia and Kenya were also the first African countries in which local versions of GraphoGame for Bantu languages were developed and piloted at the end of the 2000s. Over the years, this collaboration has established a broad network to promote special education and to tackle the problem of illiteracy with people who work in schools, ministries and academic research institutes in Africa. In 2012, Tanzania and Namibia became part of this collaboration. Since language orthography and cultural environment affect the nature of learning difficulties, the GraphoGame has been adjusted to the local school systems. Printed reading materials in original languages are often lacking in Africa. Therefore, the researchers have used other types of content for GraphoGame that would interest children, such as lyrics of popular songs. The social-institutional dimension expanded when the GraphoWorld Network of Excellence was established in 2010. It comprises a network of researchers who have established contact with the NMI over the years. Key participants come from the UK, the USA, Europe, and Africa.2 The aim of the network is to provide technology-enhanced support globally, with a special emphasis on countries where access to literacy education is limited. New research collaborations in each country follow an established procedure. First, local linguistic experts help to build an accurate sound content to the game. Second, the game is used in the research setting to see whether it works in the new language under controlled conditions. Third, it is tested in natural conditions in home and school use. After that, the stakeholders of the national network consider how to distribute the game. Corresponding to the double motive of research, there are separate licenses for research and development of GraphoGame: “We have a separate developer license and a researcher license. The core idea of the developer license is that when a citizen or an expert from a country helps us in development, they of course own everything that they create but they give us the right to use it. Then the researcher license is a license that the researchers can use for free and in most cases we provide help to them.” (Head of the research group, 20 March 2014) The international research network has expanded greatly. Recently, the game has been studied in India (Patel et al. 2018) and the Pinyin language version has been implemented and validated in China (Li, Richardson and Lyytinen 2016). In Pinyin, the special aim is to reduce the burden children have in acquiring the reading skill of the writing of Chinese language. In September 2017, the University of Jyväskylä and the Niilo Mäki Institute announced that they had signed a contract with the Learning Intelligence Group, a consortium for developing educational technology businesses in the Nordic region. It quickly became evident that research and commercial activities had to be separated, and an affiliate company, Graphogroup, was established to take care of GraphoGame’s commercial distribution. In the negotiations between the University of Jyväskylä, the Niilo Mäki Institute, and Graphogroup the parties agreed on the shared ownership of intellectual property rights. However, Graphogroup has exclusive commercial rights to distribute and to sell the game outside Finland. The contract furthermore specifies that researchers will use the term GraphoLearn when they refer to research activities and game versions that are under validation. After validation, the rights of GraphoGame’s different language versions are transferred to Graphogroup. The game distribution follows three separate paths according to regional and socio-economic conditions: free availability in Finland, non-profit distribution in Africa, and commercial distribution in European and North-American countries. In Europe, there are validated commercial versions of GraphoGame in Portuguese, Norwegian, French and Dutch. The British English version was launched in September 2018. The contract states that Graphogroup cannot profit from countries that are poor according to UNESCO categorization. In Africa, the distribution must be based on public and donor funding. In the USA, the validation of GraphoGame is advancing strongly, and the distribution will take place with the help of local business partners. According to the business representatives, the advantage of GraphoGame compared to other commercial products in the market is that it is strongly research-based: “This is the most researched game in the world. There is no other digital game that has been studied scientifically as extensively as GraphoGame. It is evidence-based, which will be one of the most important characteristics in EdTech [educational technologies]. A parent knows that their child is not going to learn anything wrong, and that local researchers have validated the game. Teachers know exactly how the game works and helps children. And most importantly, no state will buy anything that is not research-based.” (Stakeholder manager, Graphogroup, 14 August 2018) Conclusions In this article, we studied the development of social impact of research on learning difficulties. The mechanisms, social networks and intermediaries of impact changed from one stage to another, as the summary of the development presented in the Table 3 shows. This may provide a methodological challenge for social impact studies. It might be that the development of impact over a longer time gives a more realistic conception of the social impact than study of the payback from a specific funded program or overall impact of an activity of a research organization within certain well-defined period. If this is the case, the framework presented in this paper may provide a means of analyzing the narratives of longer-term impact of research. Table 3 presents the framework and the phases of research process in detail. Table 3. Contexts of impact in the three phases of the research on learning difficulties Context of impact Research object and development of understanding of relevant phenomena Mediating artefacts Social-institutional dimension Spatial-geographical dimension Local-national context 1990–2003 Unspecified nature of learning difficulties. Development of evaluation methods and diagnostic categories to allow preventive and remedial measures Diagnostic tests, special education materials Local research clinic and collaboration with pilot schools, kindergartens, and organizations of third sector Jyväskylä region and national distribution of materials and training courses Expanded national context 2004–2010 The discovery of the letter-sound connection as the key precursor of dyslexia made dyslexia the primary research object Ekapeli (GraphoGame), Lukimat platform, computer games for math learning Pupils, parents, teachers through the Internet Anyone in Finland and Finnish-speaking populations in Scandinavia and other countries Global context since 2010-> Specificities of letter-sound connection in different languages Versions of GraphoGame for different languages adapted to local cultures Contents sensitive to local culture Bilateral collaboration with African countries GraphoWorld Network of Excellence Contract with LIG Ltd for the commercial distribution of GraphoGame Several countries in Africa, Europe, South and North America, and Asia Context of impact Research object and development of understanding of relevant phenomena Mediating artefacts Social-institutional dimension Spatial-geographical dimension Local-national context 1990–2003 Unspecified nature of learning difficulties. Development of evaluation methods and diagnostic categories to allow preventive and remedial measures Diagnostic tests, special education materials Local research clinic and collaboration with pilot schools, kindergartens, and organizations of third sector Jyväskylä region and national distribution of materials and training courses Expanded national context 2004–2010 The discovery of the letter-sound connection as the key precursor of dyslexia made dyslexia the primary research object Ekapeli (GraphoGame), Lukimat platform, computer games for math learning Pupils, parents, teachers through the Internet Anyone in Finland and Finnish-speaking populations in Scandinavia and other countries Global context since 2010-> Specificities of letter-sound connection in different languages Versions of GraphoGame for different languages adapted to local cultures Contents sensitive to local culture Bilateral collaboration with African countries GraphoWorld Network of Excellence Contract with LIG Ltd for the commercial distribution of GraphoGame Several countries in Africa, Europe, South and North America, and Asia Open in new tab Table 3. Contexts of impact in the three phases of the research on learning difficulties Context of impact Research object and development of understanding of relevant phenomena Mediating artefacts Social-institutional dimension Spatial-geographical dimension Local-national context 1990–2003 Unspecified nature of learning difficulties. Development of evaluation methods and diagnostic categories to allow preventive and remedial measures Diagnostic tests, special education materials Local research clinic and collaboration with pilot schools, kindergartens, and organizations of third sector Jyväskylä region and national distribution of materials and training courses Expanded national context 2004–2010 The discovery of the letter-sound connection as the key precursor of dyslexia made dyslexia the primary research object Ekapeli (GraphoGame), Lukimat platform, computer games for math learning Pupils, parents, teachers through the Internet Anyone in Finland and Finnish-speaking populations in Scandinavia and other countries Global context since 2010-> Specificities of letter-sound connection in different languages Versions of GraphoGame for different languages adapted to local cultures Contents sensitive to local culture Bilateral collaboration with African countries GraphoWorld Network of Excellence Contract with LIG Ltd for the commercial distribution of GraphoGame Several countries in Africa, Europe, South and North America, and Asia Context of impact Research object and development of understanding of relevant phenomena Mediating artefacts Social-institutional dimension Spatial-geographical dimension Local-national context 1990–2003 Unspecified nature of learning difficulties. Development of evaluation methods and diagnostic categories to allow preventive and remedial measures Diagnostic tests, special education materials Local research clinic and collaboration with pilot schools, kindergartens, and organizations of third sector Jyväskylä region and national distribution of materials and training courses Expanded national context 2004–2010 The discovery of the letter-sound connection as the key precursor of dyslexia made dyslexia the primary research object Ekapeli (GraphoGame), Lukimat platform, computer games for math learning Pupils, parents, teachers through the Internet Anyone in Finland and Finnish-speaking populations in Scandinavia and other countries Global context since 2010-> Specificities of letter-sound connection in different languages Versions of GraphoGame for different languages adapted to local cultures Contents sensitive to local culture Bilateral collaboration with African countries GraphoWorld Network of Excellence Contract with LIG Ltd for the commercial distribution of GraphoGame Several countries in Africa, Europe, South and North America, and Asia Open in new tab Our first research question scrutinized the specificity of impact in educational sciences. An evident starting point are the connections to education, learning and the school system. The articulation of public value of education has a long history. For instance, philosopher of education John Dewey (1916/1985) found public education a central basis of democracy in the society, which included the idea that every citizen must have equal opportunities to develop their natural capabilities. The study of learning difficulties directly contributes to this aim by increasing the understanding of learning difficulties and providing means of helping children who have difficulties of reaching the goals defined in the curriculum. Second, in recent decades education and development of human capital has been regarded an increasingly important input to the economic development (e.g. OECD 1989). This impact can be interpreted as a contribution to the good of a vulnerable group (Reale et al. 2018). From the point of view of productive interactions, educational sciences typically have specific chain of beneficiaries: state and municipal educational authorities, schools psychologists, special education teachers and teachers, and finally, pupils and their parents. In our account, in the first context of impact, productive interactions in the development and use of tests and remedial materials took place with the professionals. The development of the LukiMat platform and GraphoGame opened a way of helping directly the end beneficiaries. In the third, international context, an international epistemic community was formed and the beneficiaries in African countries were hybrids composed of local researchers, ministries, development organizations and local school systems. The changes in the composition of the stakeholder chain or network might be used as a one of the criteria in the evaluation of the development of social impact. The second research question was in which way the social impact and its mechanism changed during the history of research. As Table 3 shows, the impact expanded socially and spatially from local networks with school and municipally to national impact through Internet and in the third phase simultaneously into a global epistemic community, into bilateral collaborative contracts especially with African countries and finally into a globally operating business activity. This suggests that spatial extension of the social impact might be used as one of the criteria of the evaluation. In addition, a contribution of research into sustainable development (defined in the Agenda 2030 of the United Nations) might be included in the criteria of social impact in the globalizing world. Behind the social and spatial expansion of impact the key mediating artefact, GraphoGame played a decisive role. The third research question asked in which way new research results influence the scope and direction of impact. The discovery of the precursor of dyslexia in the longitudinal study was a turning point in the group’s research. Its results contributed to the understanding of dyslexia and made the differences between transparent and nontransparent languages visible calling for further studies of dyslexia in different languages. In addition, it led to the development of GraphoGame, an artefact around which an international network or research (GraphoLearn) was established. It also provided a new practical approach for dealing with dyslexia and reading and writing difficulties. It undeniably led to the expansion of social impact in the prevention of reading difficulties. However, being strongly based on the neuropsychological research tradition, the discovery does not deal with other reasons for reading difficulties, such as lack of adequate motivation and self-regulation, which are connected to functional illiteracy in adulthood. For these reason, we can agree with Bornmann and Marx (2014) that review summaries of the state of research in certain areas transcend the limitations of a single case study. In our case, it would be an evaluation of the status of research and research-based intervention methods in reading and writing difficulties. Such an account would allow for the evaluation of the contribution of different research groups as part of a more holistic view. Our fourth research question asked what possibilities the ICT-based means provide for social impact. In Jyväskylä studies on learning difficulties, GraphoGame became the central artefact of impact. It was freely distributed through the LukiMat Internet platform. Compared with the impact mediated by distribution of tests, rehabilitation materials used by the professionals, GraphoGame could be used directly by the end-beneficiaries. The number of people using the artefacts produced by the group increased from thousands to hundreds of thousands. The regional impact expanded from the local level to the national level also reaching Finnish-speaking people living outside the country. In Africa, where in many cases the indigenous languages do not have a literature of their own, GraphoGame provided a way to advance literacy using mobile phones and new types of content. Consequently, we think that the expansive potential for impact of the ICT- and Internet-based intermediary artefacts is so promising, that also their use might be used as a criteria of evaluating the social impact at least in social sciences, educational sciences and jurisprudence. Acknowledgement Conflict of interest statement. None declared. Footnotes 1 The Institute is administered by the Niilo Mäki Foundation, which was established by the Jyväskylä University Foundation, the City of Jyväskylä, the Haukkala Foundation and the heirs of Niilo Mäki in 21.6.1990. Niilo Mäki (1902–1968) was a neuropsychologist and the “founding father” of Finnish special education. 2 See GraphoWorld Network of Excellence declaration: http://info.grapholearn.com/wp-uploads/2015/02/GraphoWORLD-Declaration_2010_2015.pdf References Annual Report of NMI ( 2014 ). Niilo Mäki Instituutti. Vuosikertomus 2014. Hyväksytty valtuuskunnan kokouksessa 14.4.2015. https://www.nmi.fi/niilo-maki-instituutti/vuosikertomukset, retrieved 17.7.2017. Aro M. , Aro T. , Ahonen T. , Räsänen T. , Hietala A. , Lyytinen H. ( 1999 ) ‘ The Development of Phonological Abilities and Their Relation to Reading Acquisition: Case Studies of Six Finnish Children ’, Journal of Learning Disabilities , 32 / 5 : 457 – 63 . Google Scholar Crossref Search ADS PubMed WorldCat Balconi M. , Brusoni S. , Orsenigo L. ( 2010 ) ‘ In Defense of the Linear Model: An Essay ’, Research Policy , 39 / 1 : 1 – 13 . Google Scholar Crossref Search ADS WorldCat Benneworth P. ( 2015 ) ‘ Tracing How Arts and Humanities Research Translates, Circulates and Consolidates in Society. How Have Scholars Been Reacting to Diverse Impact and Public Value Agendas ’, Arts & Humanities in Higher Education , 14 / 1 : 45 – 60 . Google Scholar Crossref Search ADS WorldCat Benneworth P. , Olmos-Peñuela ( 2018 ) ‘ Reflecting on the Tensions of Research Utilization: Understanding the Coupling of Academic and User Knowledge ’, Science and Public Policy , 45 / 6 : 764 – 74 . WorldCat Bornmann L. , Marx W. ( 2014 ) ‘ How Should the Societal Impact of Research Be Generated and Measured? A Proposal for a Simple and Practicable Approach to Allow Interdisciplinary Comparisons ’, Scientometrics , 98 / 1 : 211 – 19 . Google Scholar Crossref Search ADS WorldCat Böhme G. , van den Daele W. , Krohn W. ( 1983 ) ‘The Scientification of Technology’, in Böhme Gernot , van den Daele Wolfgang , Hohlfeld Rainer , Krohn Wolfgang , Wolf Schäfer (eds.) Finalization in Science: The Social Orientation of Scientific Progress. Boston Studies in the Philosophy of Science , vol. 77 , 173 – 206 . Dordrecht : D. Reidel . Google Preview WorldCat COPAC Bozemann B. , Sarewitz D. ( 2011 ) ‘ Public Value Mapping and Science Evaluation ’, Minerva , 49 : 1 – 23 . Google Scholar Crossref Search ADS WorldCat Darling-Hammond L. ( 1990 ) ‘Teacher Evaluation in the Organizational Context’, in Mitchell James Y. Jr. , Wise Steven L. , Plake Barbara S. (eds.) Assessment of Teaching: Purposes, Practices, and Implications for the Profession , Digital Edition. Hillsdale, NJ; Hove and London: Lawrence Erlbaum Associates, Inc . Google Preview WorldCat COPAC De Jong S. , van Arensbergen P. , Daemen F. , van der Meulen B. , van den Besselaar P. ( 2011 ) ‘ Evaluation of research in context: an approach and two cases’, Research Evaluation , 20 / 1 , 61 – 72 . Google Scholar Crossref Search ADS WorldCat de Solla Price J. D. ( 1984 ) ‘Notes towards a Philosophy of the Science/Technology Interaction', in Laudan Rachel (ed.) The Technological Knowledge. Are the Models of Scientific Change Relevant? Part of the Sociology of the Sciences Monographs book series SOSM, volume 4 , 105 – 14 , Dordrecht : Reidel . Google Preview WorldCat COPAC Dewey J. ( 1916 /1985) ‘Democracy and Education’, in Boydston Jo Ann (ed.) The Middle Works of John Dewey , vol. 9 . Carbondale & Edwardsville : Southern Illinois University Press . Google Preview WorldCat COPAC Donovan C. , Hanney S. ( 2011 ) ‘ The “Payback Framework” Explained ’, Research Evaluation , 20/3 : 181 – 3 . Google Scholar Crossref Search ADS WorldCat Engeström Y. , Puonti A. , Seppänen L. ( 2003a ) ‘Spatial and Temporal Expansion of the Object as a Challenge for Reorganizing Work’, in Nicolini D , Gherardi S , Danow D (eds.) Knowing in Organizations. A Practice-Based Approach . Armonk, NY : M.E. Sharpe . Google Preview WorldCat COPAC Engeström Y. , Engeström R. , Kerosuo H. ( 2003b ) ‘ The Discursive Construction of Care ’, Applied Linguistics , 24 / 3 : 286 – 315 . Google Scholar Crossref Search ADS WorldCat Geuna A. , Martin B. ( 2003 ) ‘ University Research Evaluation and Funding: An International Comparison ’, Minerva , 41 : 277 – 304 . Google Scholar Crossref Search ADS WorldCat Guston D. ( 1999 ) ‘ Stabilizing the Boundary between US Politics and Science ’, Social Studies of Science , 29 / 1 : 87 – 111 . Google Scholar Crossref Search ADS PubMed WorldCat Hicks D. ( 2012 ) ‘ Performance-Based University Research Funding Systems ’, Research Policy , 41 / 2 : 251 – 61 . Google Scholar Crossref Search ADS WorldCat Holopainen L. et al. ( 2000 ) ‘ Two Alternative Ways to Model the Relation between Reading Accuracy and Phonological Awareness at Preschool Age ’, Scientific Studies of Reading , 4 / /2 : 77 – 100 . Google Scholar Crossref Search ADS WorldCat Jörgensen T.B. , Bozeman B. ( 2007 ) ‘ Public Values: An Inventory’, Administration & Society , 39 / 3 , 354 – 81 . Google Scholar Crossref Search ADS WorldCat Kearnes M. , Wienroth M. ( 2011 ) ‘ Tools of the Trade: UK Research Intermediaries and the Politics of Impacts ’, Minerva , 49 : 153 – 74 . Google Scholar Crossref Search ADS WorldCat Knorr-Cetina K. ( 1997 ) ‘ Sociality with Objects. Social Relation in Postsocial Knowledge Societies ’, Theory, Culture and Society , 14 / 4 : 1 – 31 . Google Scholar Crossref Search ADS WorldCat Latour B. ( 1988 ) The Pasteurization of France . Cambridge, Massachusetts, and London, England : Harvard University Press. Originally Published as Les Microbes: guerre et paix suivi de irréductions (1984). Google Preview WorldCat COPAC Lehtonen I. ( 2003 ) ‘ Kirjain-Äännevastaavuuksien ja Tavujen Oppiminen Tietokonepelin Avulla. Tapaustutkimus Kuudesta Kuusivuotiaasta Lapsesta ’, Pro Gradu – Tutkielma. Jyväskylän Yliopisto. Psykologian Laitos , 1–60. WorldCat Li Y. , Richardson U. , Lyytinen H. ( 2016 ) ‘ Early Identification of Poor Readers and Preventive Training in Chinese Using Pinyin Graphogame: Results from Disadvantaged Children in Mainland ’, International Journal of Psychology , 51 (S1): 831. WorldCat Lyytinen H. et al. ( 2001 ) ‘ Developmental Pathways of Children with and without Familial Risk for Dyslexia during the First Years of Life ’, Developmental Neuropsychology , 20 / 2 : 535 – 54 . Google Scholar Crossref Search ADS PubMed WorldCat Lyytinen H. et al. ( 2004 ) ‘ Early Development of Children at Familial Risk for Dyslexia – Follow-up Form Birth to School Age ’, Dyslexia :146–78, DOI: 10.1002/dys.274 WorldCat Lyytinen H. et al. ( 2009 ) ‘ In Search of a Science-Based Application: A Learning Tool for Reading Acquisition ’, Scandinavian Journal of Psychology , 50 : 668 – 75 . Google Scholar Crossref Search ADS PubMed WorldCat Martin B.R. ( 2011 ) ‘ The Research Excellence Framework and the ‘Impact Agenda’: are we Creating a Frankenstein Monster? ’, Research Evaluation , 20/3 : 247 – 54 . Google Scholar Crossref Search ADS WorldCat Miettinen R. ( 1998 ) ‘ Object Construction and Networks in Research Work: The Case of Research on Cellulose Degrading Enzymes ’, Social Studies of Science , 28 / 3 : 423 – 63 . Google Scholar Crossref Search ADS WorldCat Molas-Gallart J. , Tang P. ( 2011 ) ‘ Tracing ‘productive interactions' to identify social impacts: an example of the social sciences', Research Evaluation , 20 / 3 , 219 – 26 . Google Scholar Crossref Search ADS WorldCat Molas-Gallart J. ( 2015 ) ‘ Research Evaluation and the Assessment of Public Value ’, Arts & Humanities in Higher Education , 14 / 1 : 111 – 26 . Google Scholar Crossref Search ADS WorldCat Morton S. ( 2015a ) ‘ Progressing Research Impact Assessment. A Contributions Approach ’, Research Evaluation , 24 / 4 : 405 – 19 . Google Scholar Crossref Search ADS WorldCat Morton S. ( 2015b ) ‘ Creating Research Impact. The Roles of Research Users in Interactive Research Mobilisation ’, Evidence and Policy: A Journal of Research, Debate and Practice , 11 / 1 : 35 – 55 . Google Scholar Crossref Search ADS WorldCat Morton S. , Flemming J. ( 2013 ) Assessing Research Impact: A Case Study of Participatory Research , p. 66 . Edinburgh : Centre for Research on Families and Relationships: Research Briefing . Google Preview WorldCat COPAC Muhonen R. , Benneworth P. , Olmos-Peñuela M. ( 2019 ) ‘ From Productive Interactions to Impact Pathways: Understanding the Key Dimensions in Developing SSH Research Societal Impact ’, Research Evaluation , 1 – 14 . https://doi.org/10.1093/reseval/rvz003. WorldCat OECD ( 1989 ) Education and the Economy in a Changing Society . Paris : OECD . WorldCat COPAC Patel P. et al. ( 2018 ) ‘ GraphoLearn India: The Effectiveness of a Computer-Assisted Reading Intervention in Supporting Struggling Readers of English ’, Frontiers in Psychology , 9 : 1045. Google Scholar Crossref Search ADS PubMed WorldCat Polkinghorne D. E. ( 1995 ) ‘ Narrative Configuration in Qualitative Analysis ’, International Journal of Qualitative Studies in Education , 8 / 1 : 5 – 23 . Google Scholar Crossref Search ADS WorldCat Raiski S. ( 1991 ) ‘ Latourian Constructivism Unfolded into Epistemic History ’, Science Studies , 4 / 1 : 33 – 52 . WorldCat Raiski S. ( 1993 ) ‘Käytännöt, teknologinen varanto ja tutkimus’. Tampereen yliopisto. Yhteiskuntatieteiden tutkimuslaitos . Julkaisuja 4 /1993. WorldCat Rantanen V. ( 2015 ) ‘ Pelattavuus ja Tutkittavuus Samassa Oppimispelissä’. Kieli, Koulutus ja Yhteiskunta - Lokakuu 2015 . [Player Enjoyment and Researchability in One Learning Game.] , 1–6, Retrieved from http://www.kieliverkosto.fi/journals/kieli-koulutus-ja-yhteiskunta-lokakuu-2015/. WorldCat Reale E. , Avramov D. , Canhial K. , Donovan C. , Flecha R. , Holm P. , Larkin C. , Lepori B. , Mosoni-Fried J. , Oliver E. , Primeri E. , Puigvert L. , Scharnhorst A. , Schubert A. , Soler M. , Soòs S. , Sordé T. , Travis C. , Van Horik R. ( 2018 ) ‘ A Review of Literature on Evaluating the Scientific, Social and Political Impact of Social Sciences and Humanities Research’, Research Evaluation , 27/ 4 : 298 – 308 . Google Scholar Crossref Search ADS WorldCat Rheinberger H.-J. ( 1997 ) Toward a History of Epistemic Things: Synthesizing Proteins in the Test Tube . Stanford, CA : Stanford University Press . Google Preview WorldCat COPAC Rowe G. , Frewer L. ( 2005 ) ‘ A Typology of Public Engagement Mechanisms ’, Science, Technology & Human Values , 30 / 2 : 251 – 90 . Google Scholar Crossref Search ADS WorldCat Spaapen J., , van Drooge L. ( 2011 ) ‘ Introducing “productive interactions” in Social Impact Assessment’, Research Evaluation , 20 / 3 : 211 – 18 . Google Scholar Crossref Search ADS WorldCat van Drooge L. , Spaapen J. ( 2017 ) ‘Evaluation and monitoring of transdisciplinary collaborations’ , Journal of Technology Transfer : 1 – 15 . https://doi.org/10.1007/s10961-017-9607-7. WorldCat Von Wright G. H. ( 1992 ) Minervan Pöllö (the Owl of Minerva) . Keuruu : Otava . Google Preview WorldCat COPAC © The Author(s) 2019. Published by Oxford University Press. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - Scholarly understanding, mediating artefacts and the social impact of research in the educational sciences
JF - Research Evaluation
DO - 10.1093/reseval/rvz018
DA - 2019-10-01
UR - https://www.deepdyve.com/lp/oxford-university-press/scholarly-understanding-mediating-artefacts-and-the-social-impact-of-R4K7Kvm3ob
SP - 295
VL - 28
IS - 4
DP - DeepDyve
ER -