Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Automatic detection of image manipulations in the biomedical literature

Automatic detection of image manipulations in the biomedical literature Images in scientific papers are used to support the experimental description and the discussion of the findings since several centuries. In the field of biomedical sciences, in particular, the use of images to depict laboratory results is widely diffused, at such a level that one would not err in saying that there is barely any experimental paper devoid of images to document the attained results. With the advent of software for digital image manipulation, however, even photographic reproductions of experimental results may be easily altered by researchers, leading to an increasingly high rate of scientific papers containing unreliable images. In this paper I introduce a software pipeline to detect some of the most diffuse misbehaviours, running two independent tests on a random set of papers and on the full publishing record of a single journal. The results obtained by these two tests support the feasibility of the software approach and imply an alarming level of image manipulation in the published record. Introduction experiment became a method accessible to anyone; a In a set of drawings dating 13th March 1610 published method perceived to be as objective as any other experi- on the “Sidereus Nuncius”, Galileo represented the mental set-up, so that in many cases images produced by uneven curve of the sun’s light over the moon disc, as seen dedicated apparatuses became the results to be analyzed, only in January of the same year using his telescope . The qualitatively and quantitatively, to prove a given hypoth- intent was to prove that the moon surface was rough, with esis. This fact led to a proliferation of images published in several differences in elevation, in contrast to the idea the biomedical literature, where photographs are used to prevalent at the time of a smooth, perfect sphere. This is a document experimental results, as opposed to abstract good example of the conscious usage of a series of images graphs and graphical arts used mostly to summarize to document a scientific observation and to prove a sci- mathematical quantities or to represent an experimental entific hypothesis, a common practice in several domains set-up or a theoretical model. The status of relative of science. Given the complexity of the subjects to be “objectivity” attributed to photographic documents was represented, however, in the field of life sciences only few however severely challenged in the transition from clas- scientists with excellent drawing skills (or having access to sical photography to digital imaging, because the same gifted artists) could successfully and universally propagate software used for producing and analyzing digital images their findings using images— think for example of was very early used to retouch the images to be published. Haeckel’s embryos or of Darwin’s orchids. It was only While this can be acceptable in principle—for example, after “objective” photographic reproduction of experi- intensity calibration of a digital image can be required for mental outcomes was routinely available, that using a quantitative analysis—it is also true that image manip- images to represent the outcome of a biological ulation aiming to deceive the readers of a scientific paper became extremely easy. The once difficult photographic retouching is today technically available to anyone; thus, Correspondence: Enrico M. Bucci (bucci@shro.org) an easy prediction would be that illicit manipulation of Temple University, Philadelphia, PA, USA scientific images should be highly prevalent. In particular, Sbarro Health Research Organization, Philadelphia, PA, USA Edited by G. Melino. © The Author(s) 2018 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to theCreativeCommons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. Official journal of the Cell Death Differentiation Association 1234567890():,; 1234567890():,; Bucci Cell Death and Disease (2018) 9:400 Page 2 of 9 once the original obstacle (i.e., technical feasibility) has rotation, linear stretching, cropping, contrast, or been lifted, there are certain conditions that would lead to brightness (in any combination); a higher number of misconduct cases connected to image 3. Reusing an image or part of it from a previous paper, manipulation, namely: including reusing a “slightly modified” version of a 1. the manipulation confers some strong advantages to previously published image (in the same sense as for the person committing it; point 2). 2. the probability of being discovered is low; This list is intentionally restricted to a fraction of what 3. even after an actual fraud is discovered, the is technically possible to detect, because proving any of consequences for the offender are mild if any. the above-mentioned image manipulation strongly Indirect evidence for the hypothesis that fraudulent implies a scientific misconduct case. image manipulations are indeed increasingly common Specifically, point 1 corresponds to data fabrication—no comes from the US Office for Research Integrity (ORI) original experiment for the published image exists. database. In fact, since the introduction of Photoshop in Points 2 and 3 include cases that range from image 1988, the number of ORI cases with questioned images plagiarism, if the involved images are presented in the has been growing exponentially. However, image same way (e.g., they are labelled in the same way and they manipulations that surfaced in ORI cases are by definition are referred to the same experiments, discussed in the originating from a tiny selection of research groups—only same way), to falsification, if they are presented as refer- cases involving US Federal Funding are reported to and ring to completely different experiments (e.g., they are considered by ORI—and, even for the population con- labelled differently and refer to physical objects which are sidered, ORI cases are suspected to be only the tip of the not the same). iceberg. In recognition of this problem, we thus decided to measure the actual extent of suspect image manipula- Investigation of a random set of Open Access tion in the biomedical literature by performing an papers unbiased, automated analysis of a large image sample In a first experiment, we considered the open source obtained from recent scientific publications, supple- papers released by PubMed Central (PMC) in January mented by expert analysis for verification of the findings. 2014. Assuming a global population of 30,000,000 of To this aim, we tweaked some home-made software with papers, to ensure that the results were representative available open-source and commercial tools, to get an (error level ±5% with 99% confidence), we included in this efficient pipeline for the extraction and processing of sample a number of papers equal to more than twice the images from the scientific literature on a bulk scale. minimum requested sample size (which would be 664). In this way, we could balance for the presence of up to 50% Type of image manipulations considered and of irrelevant papers (review, letters, image-free papers instruments selected for the analysis etc.). The sample thus included 1364 papers randomly One of the most debated questions in the field of sci- selected from PMC, from 451 journals. After automated entific misconduct involving images is the necessarily extraction and filtering, this set gave 4778 images anno- arbitrary definition of what is acceptable and what is not. tated by the software pipeline. The processing time on a Beside the general idea that manipulations that aim at Xenon E5 exacore equipped with a 30 Gb RAM was about deceiving the reader, concealing data features, or fabri- 30 min. cating whole or parts of an image are all examples of Out of the 1364 examined papers, we discovered 78 misconduct, there are few if any clear-cut guidelines. We papers (5.7% of the total) from 46 different journals started from the ORI guidelines as reported in the ORI (10.2% of the total, average IF = 4.00, ranging from 0.11 to website at the time of writing this manuscript. 9.13) containing at least one instance of suspected image In particular, we considered the following evidence of manipulation. To see whether any of the retrieved papers potential misconduct: was known to contain any problem, we checked twice on 1. Cloning objects into an image, to add features that the anonymous post-publication peer review site PubPeer were not present in the first place, taking the cloned (www.pubpeer.com): once at the time of the first analysis object from the same or a different image; (March 2014) and once at the time of preparation of this 2. Reusing an image or a “slightly modified” version of manuscript. None of the identified papers was found the same image in the same paper without an among those discussed by PubPeer. It is to be seen whe- explicit mention of it. Two or more images are ther the site community will detect problems in the considered as a “slightly modified” version of a single identified papers in the next future. image if their difference is restricted to a small, As for the type of manipulated images, the vast majority discrete region (not larger than 5% of the total area of the identified papers contain manipulations of gel expressed in pixels), or if they differ only in scale, electrophoresis images (n = 65, i.e., 83% of flagged papers Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 3 of 9 contain at least one manipulated gel image). Given the Table 1 Analysis of a random sample of PMC journals. For each journal, the number of examined papers, the fact that part of our pipeline was specifically designed to number of potentially manipulated papers and the identify cloning of bands and lanes in gel images, this corresponding ratio of manipulated papers is reported. result is hardly surprising. However, if we refer this Journals with only 1 examined paper are not reported in number to papers containing at least one image of a gel the table electrophoresis experiment (n = 299), we obtain that 21.7% of this subset do contain a potential ORI policy Examined Suspicious Ratio Journal violating manipulation involving gel images—which papers papers appears to be a high incidence per se. This particular 127 18 0.14 PLoS One finding comes as an experimental verification of suspi- cions raised on the extensive manipulation of gel- 59 5 0.08 Nucleic Acids Res electrophoresis images by Marcus and Oransky among 55 3 0.05 Exp Ther Med others, and it is consistent with the easiness with which 55 1 0.02 Oncol Lett such manipulations can be produced and can escape 32 2 0.06 Sci Rep human visual inspection. The affected journals that yielded more than one paper 32 1 0.03 Yonsei Med J for the analysis are reported in the Table 1, sorted by 24 1 0.04 J Radiat Res (Tokyo) number of papers included in the sample (examined 22 2 0.09 J Exp Bot papers). The absolute number of potentially manipulated 18 3 0.17 J Biol Chem papers and the corresponding ratio over the total is reported for each journal. 15 1 0.07 PLoS Genet Of note, we checked whether there is any correlation 14 3 0.21 Oncol Rep between the ratio of manipulated papers and the IF of the 14 1 0.07 Int J Mol Med affected journal (2012 values), but we could find no evi- 12 1 0.08 Dis Model Mech dence for it. In this respect, at least in the examined sample, we could neither find that higher IF guarantee 9 1 0.11 Mol Cell Proteomics more stringent checking procedures, nor that journals 8 1 0.13 Br J Cancer having higher IF are target of more manipulations. 7 2 0.29 PLoS Pathog We then tested whether the amount of image manip- 5 1 0.20 Clin Ophthalmol ulations found in each journal correlates with the number of retractions already published by that journal. This 4 1 0.25 Mol Cell Biochem possibility follows from assuming that image manipula- 3 2 0.67 Biochem J tion is highly prevalent among scientific misconduct cases 3 1 0.33 J Chromatogr —which is indeed true for claims examined by ORI —and 3 1 0.33 Mol Cancer that (when discovered) it results in a retraction, so that journals were image manipulations are highly prevalent 2 2 1.00 Theranostics should also retract more papers than others. To test 2 1 0.50 Indian J Lepr whether this correlation exists, at least in the limited 2 1 0.50 BMC Complement sample examined in this paper, we isolated from our set Altern Med those journals which: 2 1 0.50 J Exp Clin Cancer Res 1. were represented by at least 10 papers included in our initial set; 2 1 0.50 Mol Vis 2. were found to have at least 1 manipulated paper 2 1 0.50 J Clin Biochem Nutr included in our set; 2 1 0.50 Mol Biol Rep 3. had at least 1 paper dubbed as retracted by PubMed. 2 1 0.50 BMC Neurosci Seven journals satisfied all the above-mentioned con- ditions. We thus compared the ratio of manipulated 2 1 0.50 J Biol Inorg Chem images found in the sample to the ratio of retracted 2 1 0.50 BMC Cancer papers (number of retracted over total published papers). 2 1 0.50 Plant Mol Biol The result is exemplified in Fig. 1. 2 1 0.50 PLoS Negl Trop Dis A strong linear correlation is observed between the Image Manipulation Ratio and the Retraction Ratio for all journals but the Yonsei Medical Journal, which appears to have by far more retractions than expected. An Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 4 of 9 Fig. 1 Linear correlation between the retraction rate and the rate of manipulated images found in published manuscripts, as obtained in the examined random sample of journals considered Fig. 2 Number of papers containing manipulated images as a in this paper function of the overall number of papers examined for each nation included in the analyzed random sample. A paper is attributed to a given nation according to the nationality of affiliation of the institution of the corresponding author Table 2 Analysis of paper retracted for the journals reported in figure 1. See figure 1 for further details shows the number of retractions which were at least in Journal Retracted Retractions Retractions Unknown part caused by image manipulations (as of May 2015). papers due to image due to other reason On average, image problems are reported in about 40% manipulations problems of the retraction notes detailing the reasons for paper Nucleic Acid Res 8 2 6 0 withdrawal. Therefore, it appears that the discrepancy between retraction rates and manipulation rates is mainly J Biol Chem 26 15 0 11 due to a detection problem, not to the dismissal of claims PLOS One 37 10 27 0 by the editorial boards. PLOS Genet 2 1 1 0 We next examined the distribution of manipulated Int J Mol Med 2 0 1 1 papers by country. We assigned each paper to a country Sci Rep 4 0 2 2 according to the location of the corresponding author’s institutions. The original sample contained papers from Yonsei Med J 6 0 5 1 69 countries, with an average of 21 papers per country (standard deviation = 49, range = 1–256). Figure 2 reports the number of problematic papers as a function examination of the 6 retractions retrieved for this journal of the total number of examined papers for each (at the time of preparation of this paper) may clarify why: country. 4 retractions were due to text plagiarism, 1 to intellectual Considering the first three countries sorted by number property issues, and 1 is due to undisclosed reasons. It of examined papers, groups from China (and to a lower appears that, for this journal, retractions do not correlate extent USA) produced more manipulated papers than with image manipulations; whether this fact is due to lack expected, while UK groups produced less. of detection or to dismissal of the corresponding manip- Eventually, in an attempt to evaluate the potential ulation claims by the editorial board remains to be economic impact of the 78 papers containing some pro- ascertained. However, it holds true that, for 6 out of the 7 blematic images, we retrieved the funding information journals examined, the image manipulation rate appears provided by PubMed for each of the included papers. to correlate with the retraction rate. As shown in the While this information is only partial—there is a variety of previous graph, for the journals considered, retractions obligations in disclosing funding sources, which depends totaled a mere 0.38% of papers containing potentially on National legislations—we could nonetheless assess the manipulated images. However, for the same journals, an minimal economic impact by examining the disclosed examination of retractions or corrections reveals that, for information. The number of problematic papers per those cases where enough information is disclosed, a funding source according to PubMed is reported in the substantial amount of retractions is indeed due to image upper pie graph in Fig. 3. The bottom pie graph repre- manipulations of the kind discussed in this paper. Table 2 sents the distribution of funding sources for the overall Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 5 of 9 Fig. 3 Disclosed source of funding for the paper containing manipulated images (upper pie) and for the overall examined random sample (bottom pie) test sample (information available for 926 papers out of published. It should be noted that, while detecting 1364). manipulations in papers cannot prevent the loss of money First, by comparing the funding source distribution in invested in the corresponding projects (since it already the two pie charts, we may notice that there is no specific happened), it can however prevent these papers to be used enrichment in the set of manipulated papers. This means in further grant requests, and, if used to screen at that that there is not a “preferred” funding source for level, can be used to assess the quality of all data— manipulated papers. including unpublished one. As for the 78 papers containing manipulated images, the total number of papers, which disclosed some funding Uncovering time trends: investigation of the information, is 53, with some of the papers having more journal Cell Death and Disease than one funding source (69 grants reported as a To further test our pipeline, we would approach the funding source). While the identified paper analysis of a single journal, examining all its published manipulations are not necessarily connected to mis- papers, instead of a sample of different journals like in the conduct or fraud in a scientific project, the corresponding previous example. grant values allow the estimate of a lower bound for the This allows to follow the temporal spreading of the money potentially lost in bad science. For example, if we manipulations, to see whether there is a growing trend or consider that the average value of extramural NIH a nearly constant rate of manipulation. Moreover, if the research projects is above $400,000 , then the overall yearly acceptance rate for a journal is known, it is possible value for the 12 NIH projects which produced 12 to see whether a direct correlation exists between pub- problematic papers (red portion of the pie in the pre- lished manipulated data and easiness for a paper to get ceding graph) is greater than $4,800,000, in agreement published. As a last point, by looking at the entire dataset with an independent estimate that was recently of images published by a journal, image reusing may be Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 6 of 9 Another interesting result, which is not caught by the preceding figures and tables, is that of image reusing in different papers published by the same journal. By looking to papers published by CDDIS in 2014, we discovered that 20 contained images previously published by the same group of authors. While self-plagiarism could have easily been spotted by an automated procedure relying on a database including all images published by a journal, referees of a paper submitted in 2014 might have never seen the reused images before or might have dedicated less time to the revision process, due to the aforementioned increase in the number of submitted manuscripts. To draw any conclusions, one should com- pare my analysis with the analysis of a journal of similar level, that in the same period has not increased the number of articles. Fig. 4 Yearly rate of published papers containing at list one manipulated image (orange line) for the journal Cell Death and Conclusions Disease. The yearly number of retrieved manipulated images and the We run a scientific literature analysis to check for image number of papers containing them is also reported (blue and red bars, manipulations. While our approach suffers from a few respectively) limitations in scope, being restricted to the detection of only a tiny amount of possible data manipulations, we easily spotted, adding a further layer to detectable discovered that about 6% of published papers contain misconduct. manipulated images, and that about 22% of papers We selected as a representative target the journal Cell reporting gel electrophoresis experiments are published Death and Disease (CDDIS), published by the Nature with unacceptable images. This last figure might be Publishing Group (NPG), and focused on the 1546 papers compared to the result of recently published independent published in the period 2010–2014. Overall, we found study , which examined a set of randomly selected 8.6% of papers to contain manipulated images, which is papers in basic oncology and found 25% of them con- well in the range found for the PubMed Central sample. taining manipulated gel images. However, if one looks to the temporal evolution of the On a larger dataset, using flagging criteria that appear to yearly percentage of manipulated papers, a growing trend be quite like those assumed in this paper, Bik and cow- is immediately evident, with a percentage of manipula- orkers found 3.8% of manually examined papers to tions exceeding the PMC range in the last 2 years include at least one figure containing an inappropriate examined (Fig. 4). image manipulation. Looking at single journals in the While for CDDIS we observed a growing trend in image same sample, this percentage ranged from 0.3% in Journal manipulation, from published data it appears that during of Cell Biology to 12.4% in International Journal of the same period the acceptance rate slightly decreased Oncology, albeit the examined samples for different jour- 9 11 (from above 50% to about 40%). nals were very different in size . Again, the manipulation However, the overall number of papers submitted rate found independently on a different sample of papers increased substantially, from about 100 papers in 2010 to appear compatible to what has been automatically more than 1000 in 2014. The I.F. of the journal during the detected using our procedure in a different sample. same period has been stable or it slightly increased; Moreover, our study allowed for the first time, albeit in however, the submission quality was apparently affected a limited sample, to establish a correlation between the by an increase in potentially manipulated images, which number of manipulated images in a journal and the again confirms that these two parameters are not number of retractions issued by that journal: with some correlated. notable exception, the measured ratio of manipulations is While from a publisher perspective I.F. and submission a proxy for the retraction rate. This is also confirmed by growth are the hallmark of a successful journal, our data the fact that the retraction notes refer in a substantial point to the fact that, without some extra checking of the number of cases to image manipulations; however, the manuscript quality, it is not possible to ensure that a number of published papers containing manipulated highly successful journal (in terms of its reception by the images exceeds by orders of magnitude the number of public) is free of a substantial number of manipulated retracted papers, pointing to a detection problem on the images. journal side. Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 7 of 9 Fig. 5 Workflow to extract image panels from a pdf file, corresponding to the automatic procedure adopted by the Java software ImageCutter. JPEG versions of each page in the pdf file are changed to 8-bit, gray level images. Assuming a white background, page images are then inverted, smoothed and used for a segmentation step adopting the rolling ball procedure. To avoid over-segmenting, a region growing step increasing each segmented region is performed; if two segmented regions are joined after moderate area growing, they are considered as a single object to segment. After that, borders of the segmented object are refined by identifying abrupt changes in gray level intensity, assuming all objects to segment are rectangular. After segmentation, those objects with predefined properties (e.g., uniform color and small size) are discarded, while all the others are passed as image panels to the following routines As for the details of our analysis, in contrast with pre- be allowed anymore, nor can it be justified by pretending vious proposed hypotheses and some published results , the analysis is too complex or long to run. we could not find any correlation between a journal At the same time, academic and scientific institutions impact factor and the ratio of manipulations detected in should implement procedures to properly handle allega- our sample, but we do identified China as a country pro- tions of image manipulations, using software tools as a ducing more problematic papers than average (in agree- source of unbiased flagging and screening, before human ment with statistics on different kinds of misconduct, such assessment leads to conclusion about any potential mis- as plagiarism) and UK as a country producing less. This conduct; this process indeed is initiated in some large last result might be related to the limited period con- scale research institution . sidered by our analysis (a single month in 2014), and needs to be confirmed before trusted; both the higher prevalence Materials and methods measured for China and the lower measured for UK do We designed an automated pipeline able to extract all however agree with recently published data . images from a set of papers and perform several tests on Eventually, by focusing on a single journal from the the images aiming to assess any evidence of the type Nature Publishing Group, we could unearth a temporal described in the previous section. This pipeline includes growing trend for potential misconduct connected to the use of the following open source, commercial and in- image manipulation, which sounds as an alarm bell for house pieces of software: any journal. 1. A pdf converter, to extract single pages from papers In conclusion, we want to stress here the fact that, being and save them as jpg files (we used a home-made such a prevalent form of misconduct, image manipulation software, but there are several open source tools that should and could be faced by journals, and no delay can can be used for the same scope). Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 8 of 9 2. An in-house developed software, dubbed image manipulation by looking at all open-source ImageCutter, to perform the extraction of image retracted publication in the PMC collection. In agree- panels from each page. ment with previous findings , we realized that very often 3. A specific gel-checking routine, named ImageCheck, illicit cloning of portion images happens in figure to uncover cloning of image portions elsewhere in depicting fictitious western blotting experiments (or the same or in different image panels. other sorts of gel-electrophoresis experiments). An image 4. An image duplication tracking software, to check for documenting the result of a gel-electrophoresis image panel duplication in the same or in different experiment (including western blots) consists in a rec- papers. tangular area, which should present a noisy background The first step in the procedure starts from a set of pdf (either dark or light) and some prominent elliptical or files corresponding to a collection of papers to be ana- rectangular spots (called bands, dark or light in lyzed (pdf conversion step). From each pdf file, an ordered opposition to the background), arranged in several col- set of jpg files is generated, each jpg file corresponding to umns (the gel lanes). The relative dimensions and the an entire page of the original pdf file. intensity of some bands in specific positions represent the The second step consists in the extraction of image expected outcome of an enormous variety of biomedical panels from each page of any paper included in the ori- experiments, which is one of the reason why the ginal collection (panel segmentation step). Starting from technique is so popular among researchers (the other the jpg files representing all the pages of the target papers, being its relative inexpensiveness). Fabrication of gel- image panels are automatically identified and cropped electrophoretic images, on the other hand, is quite a using our in-house software ImageCutter. Ideally, an simple process, and usually consists in the addition of image panel is any portion of a page, which corresponds to some bands to a realistic background, to simulate the a single graphical element—e.g. the photo of a single outcome of a given experiment. Given the large diffusion Petri dish or a single western blotting membrane. of the technique, as well as its prevalence in alleged cases Small graphical arts—including, e.g., mathematical for- of fabrication, we decided to tailor our routine toward the mulas, logos, or other small graphical objects, are filtered detection of fabricated gel-electrophoresis images. In based on size of the generated image (images not larger particular, images corresponding to the jpg versions of than 10 Kb are eliminated). The automated workflow each page from pdf files (the above-mentioned set 1) were corresponding to this step is schematically represented in used to feed a routine aimed to detect gel features cloned Fig. 5. into a single gel or among different gels reported in a At this point, we have two sets of images: paper page. To achieve this, our software ImageCheck was 1. The page set (jpg files corresponding each to a single set up to look for typical gel features in image panels, i.e., page from a paper). rectangular images, preferably in grey scale (or with a 2. The image panel set (jpg files corresponding to relatively low number of colored pixels), containing either single graphical elements incorporated into the a dark or a clear background and several internal spots. figures of a paper). After a segmentation step, if their area did not differ by The first set is used to check for cloning of specific more than 10%, the software compared each possible image portions from the original location elsewhere in the couple of spots by reciprocal alignment (using their image, as previously described—i.e., to detect a specific respective center-of-mass and allowing a minimal shifting type of data fabrication. The second set is used for in every direction). A couple of spots were flagged if after detecting image panel reusing—i.e., for finding out a type alignment (checking also for 180° rotation and/or a mirror of plagiarism and a specific type falsification. transformation) a pixel-by-pixel intensity subtraction The third step consists in looking for cloned image por- resulted in a difference lower than 2% of the normalized tions. While changes in background, or intensity dis- average area of the two spots (i.e., the sum of their area continuity or other subtle evidence can reveal the splicing of divided by 2). The 2% threshold was selected according to objects into images, to reach the absolute certainty of a the R.O.C. analysis performed as described in the sup- cloning event one must find thesourceofthe cloned plementary section. object. This is an easy task when the object is present twice Once all the gel images contained in the papers under or more in the same or in different image panels—i.e., when investigation had been checked for cloned features, all one or more copies of the same graphical feature are image panels contained in set 2—representing a gel or any detected in a figure, which is not expected to contain self- other type of scientific image—were used to investigate similar regions. image reuse. To this aim, we originally used the com- What kind of images are more often manipulated in this mercial software “Visual Similarity Duplicate Image Fin- way? To address this question, we performed an der Pro” (MindGems Inc), setting a threshold of 95% for overview of the biomedical publications retracted for similarity (the default value by the software producer) and Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 9 of 9 Received: 11 January 2018 Revised: 17 February 2018 Accepted: 19 allowing for intensity differences, vertical and horizontal February 2018 stretching, mirroring, and 180° rotation. Of note, during the preparation of this paper, several free alternatives emerged to the commercial solution we used; however, we could not (yet) find anything as quick or effective as the References originally selected tool, which was apparently developed 1. Galilei, G. & Van Helden, A. Sidereus Nuncius, or, The Sidereal Messenger (Uni- versity of Chicago Press, Chicago & London, 1989). for helping professional photographer to find duplicates 2. Bucci, E. M., Adamo, G., Frandi, A. & Caporale, C. Introducing an unbiased in their large collections of images. software procedure for image checking in a large research institution. In Extended Abstracts from the 5th World Conference on Research Integrity (Elsevier, Acknowledgements Amsterdam, 2017). I wish to thank here Professor G. Melino for his critical reading of this 3. Titus, S. L.,Wells,J.A.&Rhoades, L. J. Repairingresearchintegrity. Nature 453, manuscript and Professor D. Vaux for the fruitful discussions on the problems 980–982 (2008). of scientific image manipulation. 4. Questionable Practices. Available at http://ori.hhs.gov/education/products/ RIandImages/practices/default.html (Accessed 18 May 2014). 5. Home - PMC - NCBI. Available at http://www.ncbi.nlm.nih.gov/pmc/ (Acces- Author details 1 2 sed 18 May 2014). Temple University, Philadelphia, PA, USA. Sbarro Health Research 6. Marcus,A.& Oransky, I. Canwe trust western blots? Lab Times 2, 41 (2012). Organization, Philadelphia, PA, USA 7. FY2013 By The Numbers: Research Applications, Funding, and Awards, NIH Extramural Nexus. Available at http://nexus.od.nih.gov/all/2014/01/10/fy2013- Conflict of interest by-the-numbers/ (Accessed 13 July 2014). E.M.B. is the founder and owner of Resis Srl (www.resis-srl.com), a company 8. Stern, A. M., Casadevall, A.,Steen, R.G.& Fang,F.C.Financial costs and dedicated to improving scientific publishing, promoting research integrity, and personal consequences of research misconduct resulting in retracted pub- fighting academic misconduct. lications. eLife 3, e02956 (2014). 9. Melino, G. 1000 successes as CDDIS reaches 1000 published papers! Cell Death Dis. 5, e1041 (2014). Publisher's note 10. Oksvold,M.P.Incidence of data duplications in a randomly selected pool of Springer Nature remains neutral with regard to jurisdictional claims in life science publications. Sci. Eng. Ethics 22,487–496 (2015). published maps and institutional affiliations. 11. Bik, E. M., Casadevall, A. & Fang, F. C. The prevalence of inappropriate image duplication in biomedical research publications. mBio 7,e00809–e00816 Supplementary Information accompanies this paper at https://doi.org/ (2016). 10.1038/s41419-018-0430-3. Official journal of the Cell Death Differentiation Association http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Cell Death & Disease Springer Journals

Automatic detection of image manipulations in the biomedical literature

Cell Death & Disease , Volume 9 (3) – Mar 14, 2018

Loading next page...
 
/lp/springer_journal/automatic-detection-of-image-manipulations-in-the-biomedical-BG7G3Fot0b

References (6)

Publisher
Springer Journals
Copyright
Copyright © 2018 by The Author(s)
Subject
Life Sciences; Life Sciences, general; Biochemistry, general; Cell Biology; Immunology; Cell Culture; Antibodies
eISSN
2041-4889
DOI
10.1038/s41419-018-0430-3
Publisher site
See Article on Publisher Site

Abstract

Images in scientific papers are used to support the experimental description and the discussion of the findings since several centuries. In the field of biomedical sciences, in particular, the use of images to depict laboratory results is widely diffused, at such a level that one would not err in saying that there is barely any experimental paper devoid of images to document the attained results. With the advent of software for digital image manipulation, however, even photographic reproductions of experimental results may be easily altered by researchers, leading to an increasingly high rate of scientific papers containing unreliable images. In this paper I introduce a software pipeline to detect some of the most diffuse misbehaviours, running two independent tests on a random set of papers and on the full publishing record of a single journal. The results obtained by these two tests support the feasibility of the software approach and imply an alarming level of image manipulation in the published record. Introduction experiment became a method accessible to anyone; a In a set of drawings dating 13th March 1610 published method perceived to be as objective as any other experi- on the “Sidereus Nuncius”, Galileo represented the mental set-up, so that in many cases images produced by uneven curve of the sun’s light over the moon disc, as seen dedicated apparatuses became the results to be analyzed, only in January of the same year using his telescope . The qualitatively and quantitatively, to prove a given hypoth- intent was to prove that the moon surface was rough, with esis. This fact led to a proliferation of images published in several differences in elevation, in contrast to the idea the biomedical literature, where photographs are used to prevalent at the time of a smooth, perfect sphere. This is a document experimental results, as opposed to abstract good example of the conscious usage of a series of images graphs and graphical arts used mostly to summarize to document a scientific observation and to prove a sci- mathematical quantities or to represent an experimental entific hypothesis, a common practice in several domains set-up or a theoretical model. The status of relative of science. Given the complexity of the subjects to be “objectivity” attributed to photographic documents was represented, however, in the field of life sciences only few however severely challenged in the transition from clas- scientists with excellent drawing skills (or having access to sical photography to digital imaging, because the same gifted artists) could successfully and universally propagate software used for producing and analyzing digital images their findings using images— think for example of was very early used to retouch the images to be published. Haeckel’s embryos or of Darwin’s orchids. It was only While this can be acceptable in principle—for example, after “objective” photographic reproduction of experi- intensity calibration of a digital image can be required for mental outcomes was routinely available, that using a quantitative analysis—it is also true that image manip- images to represent the outcome of a biological ulation aiming to deceive the readers of a scientific paper became extremely easy. The once difficult photographic retouching is today technically available to anyone; thus, Correspondence: Enrico M. Bucci (bucci@shro.org) an easy prediction would be that illicit manipulation of Temple University, Philadelphia, PA, USA scientific images should be highly prevalent. In particular, Sbarro Health Research Organization, Philadelphia, PA, USA Edited by G. Melino. © The Author(s) 2018 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to theCreativeCommons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. Official journal of the Cell Death Differentiation Association 1234567890():,; 1234567890():,; Bucci Cell Death and Disease (2018) 9:400 Page 2 of 9 once the original obstacle (i.e., technical feasibility) has rotation, linear stretching, cropping, contrast, or been lifted, there are certain conditions that would lead to brightness (in any combination); a higher number of misconduct cases connected to image 3. Reusing an image or part of it from a previous paper, manipulation, namely: including reusing a “slightly modified” version of a 1. the manipulation confers some strong advantages to previously published image (in the same sense as for the person committing it; point 2). 2. the probability of being discovered is low; This list is intentionally restricted to a fraction of what 3. even after an actual fraud is discovered, the is technically possible to detect, because proving any of consequences for the offender are mild if any. the above-mentioned image manipulation strongly Indirect evidence for the hypothesis that fraudulent implies a scientific misconduct case. image manipulations are indeed increasingly common Specifically, point 1 corresponds to data fabrication—no comes from the US Office for Research Integrity (ORI) original experiment for the published image exists. database. In fact, since the introduction of Photoshop in Points 2 and 3 include cases that range from image 1988, the number of ORI cases with questioned images plagiarism, if the involved images are presented in the has been growing exponentially. However, image same way (e.g., they are labelled in the same way and they manipulations that surfaced in ORI cases are by definition are referred to the same experiments, discussed in the originating from a tiny selection of research groups—only same way), to falsification, if they are presented as refer- cases involving US Federal Funding are reported to and ring to completely different experiments (e.g., they are considered by ORI—and, even for the population con- labelled differently and refer to physical objects which are sidered, ORI cases are suspected to be only the tip of the not the same). iceberg. In recognition of this problem, we thus decided to measure the actual extent of suspect image manipula- Investigation of a random set of Open Access tion in the biomedical literature by performing an papers unbiased, automated analysis of a large image sample In a first experiment, we considered the open source obtained from recent scientific publications, supple- papers released by PubMed Central (PMC) in January mented by expert analysis for verification of the findings. 2014. Assuming a global population of 30,000,000 of To this aim, we tweaked some home-made software with papers, to ensure that the results were representative available open-source and commercial tools, to get an (error level ±5% with 99% confidence), we included in this efficient pipeline for the extraction and processing of sample a number of papers equal to more than twice the images from the scientific literature on a bulk scale. minimum requested sample size (which would be 664). In this way, we could balance for the presence of up to 50% Type of image manipulations considered and of irrelevant papers (review, letters, image-free papers instruments selected for the analysis etc.). The sample thus included 1364 papers randomly One of the most debated questions in the field of sci- selected from PMC, from 451 journals. After automated entific misconduct involving images is the necessarily extraction and filtering, this set gave 4778 images anno- arbitrary definition of what is acceptable and what is not. tated by the software pipeline. The processing time on a Beside the general idea that manipulations that aim at Xenon E5 exacore equipped with a 30 Gb RAM was about deceiving the reader, concealing data features, or fabri- 30 min. cating whole or parts of an image are all examples of Out of the 1364 examined papers, we discovered 78 misconduct, there are few if any clear-cut guidelines. We papers (5.7% of the total) from 46 different journals started from the ORI guidelines as reported in the ORI (10.2% of the total, average IF = 4.00, ranging from 0.11 to website at the time of writing this manuscript. 9.13) containing at least one instance of suspected image In particular, we considered the following evidence of manipulation. To see whether any of the retrieved papers potential misconduct: was known to contain any problem, we checked twice on 1. Cloning objects into an image, to add features that the anonymous post-publication peer review site PubPeer were not present in the first place, taking the cloned (www.pubpeer.com): once at the time of the first analysis object from the same or a different image; (March 2014) and once at the time of preparation of this 2. Reusing an image or a “slightly modified” version of manuscript. None of the identified papers was found the same image in the same paper without an among those discussed by PubPeer. It is to be seen whe- explicit mention of it. Two or more images are ther the site community will detect problems in the considered as a “slightly modified” version of a single identified papers in the next future. image if their difference is restricted to a small, As for the type of manipulated images, the vast majority discrete region (not larger than 5% of the total area of the identified papers contain manipulations of gel expressed in pixels), or if they differ only in scale, electrophoresis images (n = 65, i.e., 83% of flagged papers Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 3 of 9 contain at least one manipulated gel image). Given the Table 1 Analysis of a random sample of PMC journals. For each journal, the number of examined papers, the fact that part of our pipeline was specifically designed to number of potentially manipulated papers and the identify cloning of bands and lanes in gel images, this corresponding ratio of manipulated papers is reported. result is hardly surprising. However, if we refer this Journals with only 1 examined paper are not reported in number to papers containing at least one image of a gel the table electrophoresis experiment (n = 299), we obtain that 21.7% of this subset do contain a potential ORI policy Examined Suspicious Ratio Journal violating manipulation involving gel images—which papers papers appears to be a high incidence per se. This particular 127 18 0.14 PLoS One finding comes as an experimental verification of suspi- cions raised on the extensive manipulation of gel- 59 5 0.08 Nucleic Acids Res electrophoresis images by Marcus and Oransky among 55 3 0.05 Exp Ther Med others, and it is consistent with the easiness with which 55 1 0.02 Oncol Lett such manipulations can be produced and can escape 32 2 0.06 Sci Rep human visual inspection. The affected journals that yielded more than one paper 32 1 0.03 Yonsei Med J for the analysis are reported in the Table 1, sorted by 24 1 0.04 J Radiat Res (Tokyo) number of papers included in the sample (examined 22 2 0.09 J Exp Bot papers). The absolute number of potentially manipulated 18 3 0.17 J Biol Chem papers and the corresponding ratio over the total is reported for each journal. 15 1 0.07 PLoS Genet Of note, we checked whether there is any correlation 14 3 0.21 Oncol Rep between the ratio of manipulated papers and the IF of the 14 1 0.07 Int J Mol Med affected journal (2012 values), but we could find no evi- 12 1 0.08 Dis Model Mech dence for it. In this respect, at least in the examined sample, we could neither find that higher IF guarantee 9 1 0.11 Mol Cell Proteomics more stringent checking procedures, nor that journals 8 1 0.13 Br J Cancer having higher IF are target of more manipulations. 7 2 0.29 PLoS Pathog We then tested whether the amount of image manip- 5 1 0.20 Clin Ophthalmol ulations found in each journal correlates with the number of retractions already published by that journal. This 4 1 0.25 Mol Cell Biochem possibility follows from assuming that image manipula- 3 2 0.67 Biochem J tion is highly prevalent among scientific misconduct cases 3 1 0.33 J Chromatogr —which is indeed true for claims examined by ORI —and 3 1 0.33 Mol Cancer that (when discovered) it results in a retraction, so that journals were image manipulations are highly prevalent 2 2 1.00 Theranostics should also retract more papers than others. To test 2 1 0.50 Indian J Lepr whether this correlation exists, at least in the limited 2 1 0.50 BMC Complement sample examined in this paper, we isolated from our set Altern Med those journals which: 2 1 0.50 J Exp Clin Cancer Res 1. were represented by at least 10 papers included in our initial set; 2 1 0.50 Mol Vis 2. were found to have at least 1 manipulated paper 2 1 0.50 J Clin Biochem Nutr included in our set; 2 1 0.50 Mol Biol Rep 3. had at least 1 paper dubbed as retracted by PubMed. 2 1 0.50 BMC Neurosci Seven journals satisfied all the above-mentioned con- ditions. We thus compared the ratio of manipulated 2 1 0.50 J Biol Inorg Chem images found in the sample to the ratio of retracted 2 1 0.50 BMC Cancer papers (number of retracted over total published papers). 2 1 0.50 Plant Mol Biol The result is exemplified in Fig. 1. 2 1 0.50 PLoS Negl Trop Dis A strong linear correlation is observed between the Image Manipulation Ratio and the Retraction Ratio for all journals but the Yonsei Medical Journal, which appears to have by far more retractions than expected. An Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 4 of 9 Fig. 1 Linear correlation between the retraction rate and the rate of manipulated images found in published manuscripts, as obtained in the examined random sample of journals considered Fig. 2 Number of papers containing manipulated images as a in this paper function of the overall number of papers examined for each nation included in the analyzed random sample. A paper is attributed to a given nation according to the nationality of affiliation of the institution of the corresponding author Table 2 Analysis of paper retracted for the journals reported in figure 1. See figure 1 for further details shows the number of retractions which were at least in Journal Retracted Retractions Retractions Unknown part caused by image manipulations (as of May 2015). papers due to image due to other reason On average, image problems are reported in about 40% manipulations problems of the retraction notes detailing the reasons for paper Nucleic Acid Res 8 2 6 0 withdrawal. Therefore, it appears that the discrepancy between retraction rates and manipulation rates is mainly J Biol Chem 26 15 0 11 due to a detection problem, not to the dismissal of claims PLOS One 37 10 27 0 by the editorial boards. PLOS Genet 2 1 1 0 We next examined the distribution of manipulated Int J Mol Med 2 0 1 1 papers by country. We assigned each paper to a country Sci Rep 4 0 2 2 according to the location of the corresponding author’s institutions. The original sample contained papers from Yonsei Med J 6 0 5 1 69 countries, with an average of 21 papers per country (standard deviation = 49, range = 1–256). Figure 2 reports the number of problematic papers as a function examination of the 6 retractions retrieved for this journal of the total number of examined papers for each (at the time of preparation of this paper) may clarify why: country. 4 retractions were due to text plagiarism, 1 to intellectual Considering the first three countries sorted by number property issues, and 1 is due to undisclosed reasons. It of examined papers, groups from China (and to a lower appears that, for this journal, retractions do not correlate extent USA) produced more manipulated papers than with image manipulations; whether this fact is due to lack expected, while UK groups produced less. of detection or to dismissal of the corresponding manip- Eventually, in an attempt to evaluate the potential ulation claims by the editorial board remains to be economic impact of the 78 papers containing some pro- ascertained. However, it holds true that, for 6 out of the 7 blematic images, we retrieved the funding information journals examined, the image manipulation rate appears provided by PubMed for each of the included papers. to correlate with the retraction rate. As shown in the While this information is only partial—there is a variety of previous graph, for the journals considered, retractions obligations in disclosing funding sources, which depends totaled a mere 0.38% of papers containing potentially on National legislations—we could nonetheless assess the manipulated images. However, for the same journals, an minimal economic impact by examining the disclosed examination of retractions or corrections reveals that, for information. The number of problematic papers per those cases where enough information is disclosed, a funding source according to PubMed is reported in the substantial amount of retractions is indeed due to image upper pie graph in Fig. 3. The bottom pie graph repre- manipulations of the kind discussed in this paper. Table 2 sents the distribution of funding sources for the overall Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 5 of 9 Fig. 3 Disclosed source of funding for the paper containing manipulated images (upper pie) and for the overall examined random sample (bottom pie) test sample (information available for 926 papers out of published. It should be noted that, while detecting 1364). manipulations in papers cannot prevent the loss of money First, by comparing the funding source distribution in invested in the corresponding projects (since it already the two pie charts, we may notice that there is no specific happened), it can however prevent these papers to be used enrichment in the set of manipulated papers. This means in further grant requests, and, if used to screen at that that there is not a “preferred” funding source for level, can be used to assess the quality of all data— manipulated papers. including unpublished one. As for the 78 papers containing manipulated images, the total number of papers, which disclosed some funding Uncovering time trends: investigation of the information, is 53, with some of the papers having more journal Cell Death and Disease than one funding source (69 grants reported as a To further test our pipeline, we would approach the funding source). While the identified paper analysis of a single journal, examining all its published manipulations are not necessarily connected to mis- papers, instead of a sample of different journals like in the conduct or fraud in a scientific project, the corresponding previous example. grant values allow the estimate of a lower bound for the This allows to follow the temporal spreading of the money potentially lost in bad science. For example, if we manipulations, to see whether there is a growing trend or consider that the average value of extramural NIH a nearly constant rate of manipulation. Moreover, if the research projects is above $400,000 , then the overall yearly acceptance rate for a journal is known, it is possible value for the 12 NIH projects which produced 12 to see whether a direct correlation exists between pub- problematic papers (red portion of the pie in the pre- lished manipulated data and easiness for a paper to get ceding graph) is greater than $4,800,000, in agreement published. As a last point, by looking at the entire dataset with an independent estimate that was recently of images published by a journal, image reusing may be Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 6 of 9 Another interesting result, which is not caught by the preceding figures and tables, is that of image reusing in different papers published by the same journal. By looking to papers published by CDDIS in 2014, we discovered that 20 contained images previously published by the same group of authors. While self-plagiarism could have easily been spotted by an automated procedure relying on a database including all images published by a journal, referees of a paper submitted in 2014 might have never seen the reused images before or might have dedicated less time to the revision process, due to the aforementioned increase in the number of submitted manuscripts. To draw any conclusions, one should com- pare my analysis with the analysis of a journal of similar level, that in the same period has not increased the number of articles. Fig. 4 Yearly rate of published papers containing at list one manipulated image (orange line) for the journal Cell Death and Conclusions Disease. The yearly number of retrieved manipulated images and the We run a scientific literature analysis to check for image number of papers containing them is also reported (blue and red bars, manipulations. While our approach suffers from a few respectively) limitations in scope, being restricted to the detection of only a tiny amount of possible data manipulations, we easily spotted, adding a further layer to detectable discovered that about 6% of published papers contain misconduct. manipulated images, and that about 22% of papers We selected as a representative target the journal Cell reporting gel electrophoresis experiments are published Death and Disease (CDDIS), published by the Nature with unacceptable images. This last figure might be Publishing Group (NPG), and focused on the 1546 papers compared to the result of recently published independent published in the period 2010–2014. Overall, we found study , which examined a set of randomly selected 8.6% of papers to contain manipulated images, which is papers in basic oncology and found 25% of them con- well in the range found for the PubMed Central sample. taining manipulated gel images. However, if one looks to the temporal evolution of the On a larger dataset, using flagging criteria that appear to yearly percentage of manipulated papers, a growing trend be quite like those assumed in this paper, Bik and cow- is immediately evident, with a percentage of manipula- orkers found 3.8% of manually examined papers to tions exceeding the PMC range in the last 2 years include at least one figure containing an inappropriate examined (Fig. 4). image manipulation. Looking at single journals in the While for CDDIS we observed a growing trend in image same sample, this percentage ranged from 0.3% in Journal manipulation, from published data it appears that during of Cell Biology to 12.4% in International Journal of the same period the acceptance rate slightly decreased Oncology, albeit the examined samples for different jour- 9 11 (from above 50% to about 40%). nals were very different in size . Again, the manipulation However, the overall number of papers submitted rate found independently on a different sample of papers increased substantially, from about 100 papers in 2010 to appear compatible to what has been automatically more than 1000 in 2014. The I.F. of the journal during the detected using our procedure in a different sample. same period has been stable or it slightly increased; Moreover, our study allowed for the first time, albeit in however, the submission quality was apparently affected a limited sample, to establish a correlation between the by an increase in potentially manipulated images, which number of manipulated images in a journal and the again confirms that these two parameters are not number of retractions issued by that journal: with some correlated. notable exception, the measured ratio of manipulations is While from a publisher perspective I.F. and submission a proxy for the retraction rate. This is also confirmed by growth are the hallmark of a successful journal, our data the fact that the retraction notes refer in a substantial point to the fact that, without some extra checking of the number of cases to image manipulations; however, the manuscript quality, it is not possible to ensure that a number of published papers containing manipulated highly successful journal (in terms of its reception by the images exceeds by orders of magnitude the number of public) is free of a substantial number of manipulated retracted papers, pointing to a detection problem on the images. journal side. Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 7 of 9 Fig. 5 Workflow to extract image panels from a pdf file, corresponding to the automatic procedure adopted by the Java software ImageCutter. JPEG versions of each page in the pdf file are changed to 8-bit, gray level images. Assuming a white background, page images are then inverted, smoothed and used for a segmentation step adopting the rolling ball procedure. To avoid over-segmenting, a region growing step increasing each segmented region is performed; if two segmented regions are joined after moderate area growing, they are considered as a single object to segment. After that, borders of the segmented object are refined by identifying abrupt changes in gray level intensity, assuming all objects to segment are rectangular. After segmentation, those objects with predefined properties (e.g., uniform color and small size) are discarded, while all the others are passed as image panels to the following routines As for the details of our analysis, in contrast with pre- be allowed anymore, nor can it be justified by pretending vious proposed hypotheses and some published results , the analysis is too complex or long to run. we could not find any correlation between a journal At the same time, academic and scientific institutions impact factor and the ratio of manipulations detected in should implement procedures to properly handle allega- our sample, but we do identified China as a country pro- tions of image manipulations, using software tools as a ducing more problematic papers than average (in agree- source of unbiased flagging and screening, before human ment with statistics on different kinds of misconduct, such assessment leads to conclusion about any potential mis- as plagiarism) and UK as a country producing less. This conduct; this process indeed is initiated in some large last result might be related to the limited period con- scale research institution . sidered by our analysis (a single month in 2014), and needs to be confirmed before trusted; both the higher prevalence Materials and methods measured for China and the lower measured for UK do We designed an automated pipeline able to extract all however agree with recently published data . images from a set of papers and perform several tests on Eventually, by focusing on a single journal from the the images aiming to assess any evidence of the type Nature Publishing Group, we could unearth a temporal described in the previous section. This pipeline includes growing trend for potential misconduct connected to the use of the following open source, commercial and in- image manipulation, which sounds as an alarm bell for house pieces of software: any journal. 1. A pdf converter, to extract single pages from papers In conclusion, we want to stress here the fact that, being and save them as jpg files (we used a home-made such a prevalent form of misconduct, image manipulation software, but there are several open source tools that should and could be faced by journals, and no delay can can be used for the same scope). Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 8 of 9 2. An in-house developed software, dubbed image manipulation by looking at all open-source ImageCutter, to perform the extraction of image retracted publication in the PMC collection. In agree- panels from each page. ment with previous findings , we realized that very often 3. A specific gel-checking routine, named ImageCheck, illicit cloning of portion images happens in figure to uncover cloning of image portions elsewhere in depicting fictitious western blotting experiments (or the same or in different image panels. other sorts of gel-electrophoresis experiments). An image 4. An image duplication tracking software, to check for documenting the result of a gel-electrophoresis image panel duplication in the same or in different experiment (including western blots) consists in a rec- papers. tangular area, which should present a noisy background The first step in the procedure starts from a set of pdf (either dark or light) and some prominent elliptical or files corresponding to a collection of papers to be ana- rectangular spots (called bands, dark or light in lyzed (pdf conversion step). From each pdf file, an ordered opposition to the background), arranged in several col- set of jpg files is generated, each jpg file corresponding to umns (the gel lanes). The relative dimensions and the an entire page of the original pdf file. intensity of some bands in specific positions represent the The second step consists in the extraction of image expected outcome of an enormous variety of biomedical panels from each page of any paper included in the ori- experiments, which is one of the reason why the ginal collection (panel segmentation step). Starting from technique is so popular among researchers (the other the jpg files representing all the pages of the target papers, being its relative inexpensiveness). Fabrication of gel- image panels are automatically identified and cropped electrophoretic images, on the other hand, is quite a using our in-house software ImageCutter. Ideally, an simple process, and usually consists in the addition of image panel is any portion of a page, which corresponds to some bands to a realistic background, to simulate the a single graphical element—e.g. the photo of a single outcome of a given experiment. Given the large diffusion Petri dish or a single western blotting membrane. of the technique, as well as its prevalence in alleged cases Small graphical arts—including, e.g., mathematical for- of fabrication, we decided to tailor our routine toward the mulas, logos, or other small graphical objects, are filtered detection of fabricated gel-electrophoresis images. In based on size of the generated image (images not larger particular, images corresponding to the jpg versions of than 10 Kb are eliminated). The automated workflow each page from pdf files (the above-mentioned set 1) were corresponding to this step is schematically represented in used to feed a routine aimed to detect gel features cloned Fig. 5. into a single gel or among different gels reported in a At this point, we have two sets of images: paper page. To achieve this, our software ImageCheck was 1. The page set (jpg files corresponding each to a single set up to look for typical gel features in image panels, i.e., page from a paper). rectangular images, preferably in grey scale (or with a 2. The image panel set (jpg files corresponding to relatively low number of colored pixels), containing either single graphical elements incorporated into the a dark or a clear background and several internal spots. figures of a paper). After a segmentation step, if their area did not differ by The first set is used to check for cloning of specific more than 10%, the software compared each possible image portions from the original location elsewhere in the couple of spots by reciprocal alignment (using their image, as previously described—i.e., to detect a specific respective center-of-mass and allowing a minimal shifting type of data fabrication. The second set is used for in every direction). A couple of spots were flagged if after detecting image panel reusing—i.e., for finding out a type alignment (checking also for 180° rotation and/or a mirror of plagiarism and a specific type falsification. transformation) a pixel-by-pixel intensity subtraction The third step consists in looking for cloned image por- resulted in a difference lower than 2% of the normalized tions. While changes in background, or intensity dis- average area of the two spots (i.e., the sum of their area continuity or other subtle evidence can reveal the splicing of divided by 2). The 2% threshold was selected according to objects into images, to reach the absolute certainty of a the R.O.C. analysis performed as described in the sup- cloning event one must find thesourceofthe cloned plementary section. object. This is an easy task when the object is present twice Once all the gel images contained in the papers under or more in the same or in different image panels—i.e., when investigation had been checked for cloned features, all one or more copies of the same graphical feature are image panels contained in set 2—representing a gel or any detected in a figure, which is not expected to contain self- other type of scientific image—were used to investigate similar regions. image reuse. To this aim, we originally used the com- What kind of images are more often manipulated in this mercial software “Visual Similarity Duplicate Image Fin- way? To address this question, we performed an der Pro” (MindGems Inc), setting a threshold of 95% for overview of the biomedical publications retracted for similarity (the default value by the software producer) and Official journal of the Cell Death Differentiation Association Bucci Cell Death and Disease (2018) 9:400 Page 9 of 9 Received: 11 January 2018 Revised: 17 February 2018 Accepted: 19 allowing for intensity differences, vertical and horizontal February 2018 stretching, mirroring, and 180° rotation. Of note, during the preparation of this paper, several free alternatives emerged to the commercial solution we used; however, we could not (yet) find anything as quick or effective as the References originally selected tool, which was apparently developed 1. Galilei, G. & Van Helden, A. Sidereus Nuncius, or, The Sidereal Messenger (Uni- versity of Chicago Press, Chicago & London, 1989). for helping professional photographer to find duplicates 2. Bucci, E. M., Adamo, G., Frandi, A. & Caporale, C. Introducing an unbiased in their large collections of images. software procedure for image checking in a large research institution. In Extended Abstracts from the 5th World Conference on Research Integrity (Elsevier, Acknowledgements Amsterdam, 2017). I wish to thank here Professor G. Melino for his critical reading of this 3. Titus, S. L.,Wells,J.A.&Rhoades, L. J. Repairingresearchintegrity. Nature 453, manuscript and Professor D. Vaux for the fruitful discussions on the problems 980–982 (2008). of scientific image manipulation. 4. Questionable Practices. Available at http://ori.hhs.gov/education/products/ RIandImages/practices/default.html (Accessed 18 May 2014). 5. Home - PMC - NCBI. Available at http://www.ncbi.nlm.nih.gov/pmc/ (Acces- Author details 1 2 sed 18 May 2014). Temple University, Philadelphia, PA, USA. Sbarro Health Research 6. Marcus,A.& Oransky, I. Canwe trust western blots? Lab Times 2, 41 (2012). Organization, Philadelphia, PA, USA 7. FY2013 By The Numbers: Research Applications, Funding, and Awards, NIH Extramural Nexus. Available at http://nexus.od.nih.gov/all/2014/01/10/fy2013- Conflict of interest by-the-numbers/ (Accessed 13 July 2014). E.M.B. is the founder and owner of Resis Srl (www.resis-srl.com), a company 8. Stern, A. M., Casadevall, A.,Steen, R.G.& Fang,F.C.Financial costs and dedicated to improving scientific publishing, promoting research integrity, and personal consequences of research misconduct resulting in retracted pub- fighting academic misconduct. lications. eLife 3, e02956 (2014). 9. Melino, G. 1000 successes as CDDIS reaches 1000 published papers! Cell Death Dis. 5, e1041 (2014). Publisher's note 10. Oksvold,M.P.Incidence of data duplications in a randomly selected pool of Springer Nature remains neutral with regard to jurisdictional claims in life science publications. Sci. Eng. Ethics 22,487–496 (2015). published maps and institutional affiliations. 11. Bik, E. M., Casadevall, A. & Fang, F. C. The prevalence of inappropriate image duplication in biomedical research publications. mBio 7,e00809–e00816 Supplementary Information accompanies this paper at https://doi.org/ (2016). 10.1038/s41419-018-0430-3. Official journal of the Cell Death Differentiation Association

Journal

Cell Death & DiseaseSpringer Journals

Published: Mar 14, 2018

There are no references for this article.