TY - JOUR
AU - Feinstein, Alvan R.
AB - Abstract Context A genetic basis has been identified for many medical conditions and some molecular tests have been commercialized. However, little attention has been given to the quality of clinical epidemiology in molecular studies. Objective To examine the clinical epidemiological quality of recent publications on molecular genetic analysis. Design Cross-sectional study of original research articles published in 1995, identified by manually searching 4 general clinical journals. Of 83 articles identified, 40 were selected for analysis; these 40 discussed molecular genetic techniques, studied 10 or more patients, and had inferential conclusions. Main Outcome Measure Compliance of the selected articles with 7 methodological standards for clinical epidemiological science (reproducibility, objectivity, delineation of case group, adequacy of spectrum in case group, delineation of comparison group, adequacy of comparison group, and quantitative summary of results). Results Among the 40 inferential articles that studied 10 or more patients, only 5 (12.5%) complied with all 7 applicable standards, and 10 (25.0%) complied with all but 1 standard, whereas 25 articles (62.5%) failed to comply with 2 or more standards and 9 (22.5%) failed 4 or 5 standards. Most articles did not comply with standards for reproducibility (n=25, 62.5%) or objectivity (n=27, 67.5%); however, the majority of articles did comply with standards for adequacy of case group (n=35, 87.5%), adequacy of comparison group (n=35, 87.5%), and quantitative summary of results (n=36, 90%). Conclusions Despite major laboratory advances in molecular genetic analysis, our data suggest that reported applications in clinical journals often have troubling omissions, deficiencies, and lack of attention to the different, but necessary, principles of clinical epidemiological science. Without suitable attention to fundamental methodological standards, the expected benefits of molecular genetic testing may not be achieved. Rapid progress in molecular biologic techniques has revolutionized the field of genetics and pushed the frontier to complex, everyday phenomena such as hypertension,1 coronary artery disease,2-6 diabetes mellitus,7 and schizophrenia.8 Efforts to unlock the genetic code and to determine "the master blueprint"9 for individual persons have received added impetus from the Human Genome Project10,11 and from the potential for profit.12 The anticipated health benefits of genetic diagnosis and therapy are countered by new social, ethical, and legal concerns regarding discrimination in insurance and work,13,14 the possibility of a new eugenics,15-18 and the potential for personal and family distress.19 The recent report of the National Institutes of Health Task Force on Genetic Testing20 highlights both the challenges involved in developing and evaluating genetic tests and the perils of not doing so appropriately and thoroughly. The role of genetic factors in complex, multifactorial conditions has not been easy to establish.12 Examples of problems are the difficulty confirming21-26 the proposed associations between the DD allele of the angiotensin-converting enzyme and coronary artery disease,2,27,28 the subsequent doubts about initial findings of links between the D4 allele of the dopamine receptor and alcoholism,29,30 the flawed research designs8 and inability to replicate findings31 for genetic factors in schizophrenia, and the absence of confirmation, despite substantial media publicity,32 for the so-called novelty-seeking (or bungee-jumping) gene.33 Many of these problems can be traced at least in part to basic methodological flaws, including lack of consideration of potential confounders such as ethnicity,29,30 failure to delineate case and control groups adequately,29 and so-called definition drift in which subsequent studies substantially alter the definition for the condition of interest.34 Because evaluations of previous reports on new clinical technologies that do not involve molecular methods have often revealed major methodological flaws,35-44 we wondered about the quality of clinical epidemiology in molecular studies. Prominent examples of sophisticated new tests that were widely adopted before adequate evaluation and subsequently found to perform disappointingly include the carcinoembryonic antigen test,43,45 the dexamethasone suppression test for depression,44 the indirect immunofluorescence assay for Lyme disease,46 and the iodine 125–labeled fibrinogen scans for deep venous thrombosis.47 From earlier this century, the Wasserman test for syphilis is a striking example of the deleterious effects that false-positive results can have on both medical status and human life.48 Our aim was to examine the quality of published clinical literature on molecular genetic findings using a set of methodological standards, based on established principles of clinical epidemiology, which can be used for designing, reporting, or evaluating molecular genetic research. We assumed that certain principles of clinical epidemiological science are needed to produce reliable and valid clinical information35-44,49-56 and that the errors of inaccurate, irreproducible, or inadequately evaluated tests can have serious consequences.50,57 We decided to evaluate all publications on molecular genetic analysis in 4 prominent clinical journals, but to focus our evaluation on those articles reporting association studies that can be regarded as epidemiological. Although some of these articles would be early studies, collecting data several steps away from actual clinical application, we believed that basic principles of clinical epidemiology should not be ignored merely because an early or preliminary study reports a novel finding. Methods We performed a manual search of original research studies published during 1995 in 4 prominent general clinical journals: Lancet, New England Journal of Medicine, JAMA, and BMJ. We retrieved all articles in which human genetic material was analyzed at a molecular level, and we did not limit the search to any particular laboratory technique or clinical condition. Furthermore, since genetic is not synonymous with inherited, we included articles dealing with either inherited or somatic mutations. We also included articles evaluating the influence of normal genetic polymorphisms, referring collectively to mutations, polymorphisms, or any other genetic finding under study as genetic abnormalities. The search revealed 83 articles of which 36 were from Lancet,25,58-92 34 were from New England Journal of Medicine,24,93-125 8 were from JAMA,126-133 and 5 were from BMJ.134-138 Classification of Articles Our first task was to classify the 83 eligible articles into logical, easily interpretable categories to facilitate the application of clinical epidemiological principles to the appropriate articles. Several axes were considered for the classification, including the type of molecular biologic technique (eg, polymerase chain reaction vs Southern blotting), the study design (eg, case-control vs kindred for linkage analysis), and the type of genetic abnormality under study (eg, single or multiple genes, inherited or acquired abnormality). The most cogent attribute on which to base the classification, however, seemed to be the presence of an inferential conclusion about cause and effect. Articles that reach inferential conclusions, with statements pertaining to cause and risk, are generally structured as analytic contrasts between a study group and a control group, and follow either a cohort or a case-control design. Most of these articles are called association studies and report the association of a genetic abnormality with a clinical condition; some articles, however, report on genetic abnormalities as markers for clinical use in diagnosis, prognosis, or therapy. Articles that are not inferential are descriptive, and generally report on a study group alone, without a control group; descriptive articles include studies that describe the prevalence of a genetic lesion, that describe the distribution of clinical findings for a given genetic lesion, or that describe the distribution of genetic findings for a given clinical condition. For the purposes of this report, we include as descriptive those articles that are structured as linkage studies, in which kindreds are analyzed to localize a putative gene to a region on a chromosome. Although a particular study could have more than 1 focus, a primary classification was always assigned to what seemed to be the main intent of the study, based on claims made in the title and abstract (or text, when necessary). Methodological Standards The methodological standards for the conduct and reporting of studies were based on existing clinical epidemiological principles,35,139 which have been developed and used in previous evaluations of clinical technologies. The standards for the current report were developed with the intent of focusing on inferential studies; these studies can be regarded as epidemiological in nature and subject to the same kinds of problems that can prejudice the soundness of the conclusions in nongenetic studies of association that make causal inferences. Mindful of the complexity and variety of the articles under examination, we set the criteria for meeting the standards at a lenient level. The methodological standards include reproducibility, objectivity, delineation of cases, adequacy of spectrum in case group, delineation of comparison group, adequacy of comparison group, and quantitative summary. The 7 standards are described below. Reproducibility. Despite impressive molecular accomplishments, the results of DNA tests regularly appear in electrophoretic "bar codes" that require visual inspection and interpretation, with the attendant challenges of observer variability.139 A recent study140 demonstrated poor reliability for different laboratories in the molecular analysis of genetic material. We therefore required that reproducibility be considered and mentioned. Although requiring the authors to provide a direct comment that the molecular genetic method was either tested in the cited study or previously shown to be reproducible in the laboratory where the work was done, we did not insist on either a formal presentation of the results or a specific method for appraising reproducibility. We accepted either repetition of the test in a pertinent sample of specimens, or confirmation of the test with another procedure. In an example of a satisfactory study,24 each sample found to have the angiotensin-converting enzyme DD genotype was subjected to a second, independent polymerase chain reaction amplification. An unsatisfactory study129 of the prognostic significance of apolipoprotein E did not mention any attempt to verify or check the reliability of the DNA analysis. Objectivity. To ensure objectivity of interpretation, the person doing a measurement should be blinded (or masked) to pertinent characteristics of the patient or hypothesis being tested. When appropriate, the blinding may keep either the patient's clinical assessors from knowing the genetic results, or the genetic analysts from knowing the patients' clinical status. To satisfy this standard, an appropriate method of blinding should have been used and reported in each study. In a successful example,100 genetic susceptibility to asthma was studied with each gel being read by observers who were unaware of the corresponding clinical status. A study on the association of the angiotensinogen gene and coronary heart disease, on the other hand, contained no mention of any blinding.25 Delineation of Cases. The group chosen as cases of the condition under investigation should be described in enough detail so that other investigators could assemble a similar group in similar circumstances. This description should include the criteria for diagnosing the selected condition and a simple indication of the sampling frame (pertinent demographic details and source for the patients). For leniency in scoring, we did not demand that studies report the calendar time or time interval of sampling, or the sampling method (eg, random, consecutive, or other), and we accepted an easily obtainable reference if it seemed to offer an adequate description of the case group. An excellent article, however, would give an outline of the condition, place, and time of the cited study. Specific diagnostic criteria are seldom needed for histological or gross morphologic abnormalities, but should be cited if a particular diagnosis depends on a specifically demarcated value (such as weight in obesity) or on a combination of components (as in myotonic dystrophy). Again, for leniency, we did not require a statement of the rules for combining components. For example, a study of type 2 diabetes mellitus could cite the diagnostic criteria of the World Health Organization. An example passing the standard was a study of coagulation factor V and myocardial infarction, stroke, and venous thrombosis.108 The investigators provided clear criteria for the diagnosis of each case condition and described the origin of the cohort. An example failing the standard was a study of the β3-adrenergic receptor gene and type 2 diabetes mellitus, in which no criteria were stated for the diagnosis of diabetes (eg, published standards vs type of therapy used).113 Adequacy of Spectrum in Case Group. The spectrum of cases should be suitable to support the claim of the article. For example, results from a narrow portion of a condition's clinical spectrum (eg, cases of only mild severity, or with only metastatic or debilitating disease) do not usually apply to the full spectrum. Although citation of results for pertinent subgroups would be desirable, we were willing to accept undifferentiated results for a broad spectrum of patients. If the investigated condition is rare enough that a broad spectrum of patients is difficult to obtain, even a single case may sometimes suffice to demonstrate the spectrum. For our analyses, however, we excluded articles with fewer than 10 case subjects. In a successful example, we gave credit to an article25 that studied genetic associations with heart disease and included subjects with a wide range of disease severity. In an example of a study failing this standard, an article58 on spina bifida did not indicate the spectrum of examined disease despite a general conclusion about neural tube defects. Delineation of Comparison Group. The comparison group should also be described in enough detail to allow its recapitulation by other investigators. The description should include the main clinical criteria for inclusion in the comparison group and the sampling frame from which the group was chosen. For leniency, however, we did not insist that the time frame or sampling method be cited. A single description of the sampling frame could suffice if a single group was chosen and then divided into case and comparison groups, according to the presence of a genetic abnormality; but if the single description is inadequate, neither group will satisfy delineation of cases and delineation of comparison group standards. As with the delineation of the case group, we accepted an article that offered a satisfactory account of the comparison group, although a brief summary description should preferably be reported for the comparison group even if a reference is provided. A passing article136 provided a detailed description of the source and diagnostic criteria for the comparison groups in a study of apolipoprotein E and Alzheimer disease. An example failing this standard was an article64 on IgA nephropathy, which listed no source or criteria for the healthy controls. Adequacy of Comparison Group. The comparison group should include pertinent challenging conditions, such as a related or similar ailment that might plausibly share an etiology, or that should be suitably differentiated from the main condition. For leniency in scoring, this criterion could be satisfied by members from 1 or 2 challenging groups rather than from a full spectrum of challenges. In studies claiming to identify risk factors, we accepted healthy or community controls, but appropriate challenge groups were needed for studies of diagnosis or claims about etiology or pathogenesis. The standard was satisfied in a study94 of Gilbert syndrome, which included both healthy controls and patients with Crigler-Najjar syndrome type 2. The standard was not satisfied in a study of idiopathic short stature, which contained healthy controls but no other forms of short stature.97 Quantitative Summary of Results. The main claim made in comparisons of 2 or more groups should receive a quantitative summary of the results. The summary should indicate both the magnitude of the difference and a statistical expression of numerical stability (eg, a confidence interval or P value) for the possible role of chance in the results. If the compared numbers were provided, we did not insist on a specific index of contrast, such as a risk ratio. An example of successful citation contained odds ratios and 95% confidence intervals for a study of the angiotensinogen gene and risk of coronary artery disease.25 An unsatisfactory article reporting on the frequency of a gene potentially involved in the cause of Gilbert syndrome gave no statistical summary for the comparison of allele frequency in patients vs healthy controls.94 Evaluation Process To avoid excessively complicated evaluation rules, we applied the 7 standards only to articles that had inferential conclusions and that studied at least 10 patients. We excluded articles reporting on fewer than 10 patients because standards such as the adequacy of case spectrum and quantitative summary of results are more difficult to apply to individual case reports and small case series. In addition, although some standards appeared to be potentially applicable to some of the articles classified as descriptive, we do not report the results for those articles here. Finally, some studies (8/83) did not include a control group but made what seemed to be possibly overstated claims of association or causation; although these articles probably contained defective inferences, we classified them as descriptive for this report and did not apply the standards to them. Observer Variability To reduce and then measure interobserver variability in the application of the standards, 20 of the 83 articles were randomly selected and then independently scored for the methodological standards by each of the 3 authors. After disagreements were resolved in conference, the standards were revised and clarified. When 10 more articles were then randomly selected from the remaining 63 and independently evaluated by 2 of the authors (S.T.B., J.C.) the overall percentage of agreement was 92%, with κ=0.83, reflecting excellent agreement.141 Results Classification of Articles Table 1 shows the number of articles in each category of classification; Table 2 lists the clinical conditions examined in the 83 articles. The largest proportion of articles (43/83 [52%]) had inferential conclusions, with most being association studies aimed at determining the cause or risk for a particular condition. Six of the 43 articles were also inferential but aimed at clinical decisions, such as diagnosis, prognosis, or therapy. Of the 43 inferential articles, 40 studied 10 or more patients. The remaining articles (40/83 [48%]) were primarily descriptive but included some not classified as association studies that nevertheless made inferential claims (ie, 4 articles based on linkage analysis and 8 articles that made a causal claim but did not include a comparison group). Summary Results for Methodological Standards Table 3 shows the results for individual methodological standards applied to the 40 inferential articles that studied 10 or more patients, and Table 4 shows the overall results for the 7 standards in these articles. Most articles did not pass the standards for reproducibility (n=25, 62.5%) or objectivity (n=27, 67.5%), and substantial proportions also did not pass the standards for delineation of case group (n=9, 22.5%) or comparison group (n=12, 30%). Most articles, however, did pass the standards for adequacy of case group (n=35, 87.5%) and comparison group (n=35, 87.5%), and for quantitative summary of results (n=36, 90%). Of the 40 articles analyzed, 5 (12.5%) passed all 7 standards and 10 (25%) failed only 1 standard; the remaining 25 (62.5%), however, failed 2 or more standards, with 9 (22.5%) of these failing 4 or 5 standards. Comment The results indicate that a substantial proportion of articles involving molecular genetic research may contain flaws in basic elements of research design and reporting. The importance of this finding is emphasized by the recent report20 by the National Institutes of Health Task Force on Genetic Testing, which urged a more careful and systematic approach to the development of genetic tests. The methodological standards used in the current study are based on well-accepted fundamental scientific principles, applicable in any type of clinical investigation involving human subjects and diseases, and have the strong face validity of scientific "common sense." For example, anyone using a test would want to know its reproducibility in yielding the same result if applied twice to the same specimen, and objective interpretations are needed to avoid the potential for bias whenever data receive human interpretations, whether for clinical diagnoses or for reading a gel. Giving an adequate description of the groups under study and justifying the clinical spectra of the chosen conditions are perhaps the most fundamental steps in any act of clinical research. For example, differences in composition of groups may be crucial if a gene abnormality occurs more frequently in one ethnic group than another; and inadequate spectra may undermine conclusions about the relationship of a gene and a particular clinical condition. Similarly, certain inferences rest on demonstrations that an abnormality often present in one group is often absent in others, and that the difference between the groups is statistically stable. The neglect of fundamental methodological principles has important clinical consequences and can lead to contradictions and even retractions in published work.142,143 An example of conflicting results was found even in the limited number of studies we examined: 2 articles72,94 published in different journals reached different conclusions about the genetic basis of Gilbert syndrome. In addition, with the extraordinary pace of development in genetic research and the pressure to publish results quickly, quality may be sacrificed in favor of rapid dissemination. For example, some of the articles we evaluated can be described as preliminary reports, and lacked methodological rigor. Even early articles that hasten to report a new finding, however, can give suitable attention to the quality and reliability of the research. More careful scientific methods can spare other investigators the wasted effort of attempting to replicate spurious findings. We were deliberately lenient in the requirements for meeting the standards. For example, we did not set specifications for the particular manner in which reproducibility should be checked or quantitative summaries reported. One reason for the leniency was the extraordinary complexity of many of the articles, complexity that makes reliable classification and grading a challenging task. Nevertheless, despite the relatively "low height of the bar," many articles failed to meet the standards. A count of the number of standards failed, however, does not necessarily determine either the overall quality of the work or the validity of its results because 1 major flaw could potentially undermine all the conclusions, or, conversely, a series of minor flaws might not affect the conclusions. Since critiquing decisions of journal editors is not the aim of this study, we did not try to correlate the quality of the articles with the sites of publication. The 4 journals used for this study were chosen because they are the most widely read, influential general clinical journals in the English language, and are particularly likely to publish work that is carefully reviewed and clinically pertinent. We do not know what might have been found for articles published in other journals, which may place a greater emphasis on the publication of molecular genetic studies than the journals chosen for this report. We also did not try to determine whether the cited flaws were produced by research design or because of authors' omissions or editorial excisions. Although flaws such as an inadequate spectrum of cases or controls seem due to research design, the source is more difficult to assign for omitted details such as reproducibility or objectivity of tests, or delineation of groups. For example, journals that vigorously attempt to limit the length of articles may delete material that other journals would publish, and this may be 1 explanation for the high failure rate for the standards referring to reproducibility and objectivity. Regardless of where or how each problem arises, however, its unsatisfactory management will prevent other investigators from replicating or suitably interpreting the findings. We conclude that the 7 standards listed here are often not satisfied in published reports of molecular genetic findings. Although not everyone may agree with each of our decisions, we believe the standards themselves have strong face validity and are important to ensuring the reliability of future genetic research. The standards can be used by investigators in planning and reporting genetic research, and by reviewers, editors, and readers who evaluate the reports. Unless applied with analogous principles of clinical epidemiological science, the molecular science that has produced magnificent genetic advances may also lead to an epidemic of harm from spurious associations, unnecessary fears, and unfulfilled promises of benefit. References 1. Jamieson A. Dissecting hypertension: the role of the "new genetics." J R Coll Physicians Lond.1994;28:512-518.Google Scholar 2. Mattu RK, Needham EW, Galton DJ, Frangos E, Clark AJ, Caulfield M. A DNA variant at the angiotensin-converting enzyme locus associates with coronary artery disease in the Caerphilly Heart Study. Circulation.1995;91:270-274.Google Scholar 3. Gallagher PM, Meleady R, Shields DC. et al. Homocysteine and risk of premature coronary heart disease: evidence for a common gene mutation. Circulation.1996;94:2154-2158.Google Scholar 4. Jukema JW, Van Boven AJ, Groenemeijer B. et al. The Asp9Asn mutation in the lipoprotein lipase gene is associated with increased progression of coronary atherosclerosis. Circulation.1996;94:1913-1918.Google Scholar 5. Yu Q, Safavi F, Roberts R, Marian AJ. A variant of beta fibrinogen is a genetic risk factor for coronary artery disease and myocardial infarction. J Investig Med.1996;44:154-159.Google Scholar 6. Stengard JH, Zerba KE, Pekkanen J, Ehnholm C, Nissinen A, Sing CF. Apolipoprotein E polymorphism predicts death from coronary heart disease in a longitudinal study of elderly Finnish men. Circulation.1995;91:265-269.Google Scholar 7. Owerback D, Gabbay KH. The search for IDDM susceptibility genes: the next generation. Diabetes.1996;45:544-551.Google Scholar 8. Cloninger CR. Turning point in the design of linkage studies of schizophrenia. Am J Med Genet.1994;54:83-92.Google Scholar 9. US Department of Energy. Human Genome 1991-92 Program Report, Foreword. Washington, DC: US Dept of Energy, Office of Environmental Research; 1992:111. 10. Guyer MS, Collins FS. How is the Human Genome Project doing, and what have we learned so far? Proc Natl Acad Sci U S A.1995;92:10841-10848.Google Scholar 11. Collins FS. Ahead of schedule and under budget: the Genome Project passes its fifth birthday. Proc Natl Acad Sci U S A.1995;92:10821-10823.Google Scholar 12. Hubbard R, Lewontin RC. Pitfalls of genetic testing. N Engl J Med.1996;334:1192-1194.Google Scholar 13. Hudson KL, Rothenberg KH, Andrews LB, Kahn M, Collins FS. Genetic discrimination and health insurance: an urgent need for reform. Science.1995;270:391-393.Google Scholar 14. Preston J. Trenton votes to put strict limits on use of gene tests by insurers. The New York Times.June 18, 1996:A1.Google Scholar 15. Muller-Hill B. The shadow of genetic injustice. Nature.1993;362:491-492.Google Scholar 16. McLean SA. Mapping the human genome—friend or foe? Soc Sci Med.1994;39:1221-1227.Google Scholar 17. Genome research risks abuse warns panel Nature.1995;378:529.Google Scholar 18. Hodgkinson N. The master race. Sunday Times (London).January 2, 1994.Google Scholar 19. Sack C. Tropic of cancer. The New Republic.June 2, 1997:46.Google Scholar 20. Holtzman NA, Watson MS. Promoting Safe and Effective Genetic Testing in the United States: Final Report of the Task Force on Genetic Testing. Bethesda, Md: National Human Genome Research Institute/National Institutes of Health; 1997. 21. Bohn M, Berge KE, Bakken A, Erikssen J, Berg K. Insertion/deletion (I/D) polymorphism at the locus for the angiotensin 1-converting enzyme and myocardial infarction. Clin Genet.1993;44:292-297.Google Scholar 22. Lindpaintner K, Pfeffer MA, Kreutz R. et al. Angiotensin converting enzyme (ACE) genotype and risk of myocardial infarction (MI) [abstract]. Clin Res.1993;41:216.Google Scholar 23. Miettinen HE, Korpela K, Hamalainen L, Kontula K. Polymorphisms of the apolipoprotein and angiotensin converting enzyme genes in young North Karelian patients with coronary artery disease. Hum Genet.1994;94:189-192.Google Scholar 24. Lindpaintner K, Pfeffer MA, Kreutz R. et al. A prospective evaluation of an angiotensin-converting enzyme gene polymorphism and the risk of ischemic heart disease. N Engl J Med.1995;332:706-711.Google Scholar 25. Katsuya T, Koike G, Yee TW. et al. Association of angiotensinogen gene T235 variant with increased risk of coronary heart disease. Lancet.1995;345:1600-1603.Google Scholar 26. Winkelmann BR, Nauck M, Klein B. et al. Deletion polymorphism of the angiotensin I-converting enzyme gene is associated with increased plasma angiotensin-converting enzyme activity but not with increased risk for myocardial infarction and coronary artery disease. Ann Intern Med.1996;125:19-25.Google Scholar 27. Cambien F, Poirier O, Lecerf L. et al. Deletion polymorphism in the gene for angiotensin-converting enzyme is a potent risk factor for myocardial infarction. Nature.1992;359:641-644.Google Scholar 28. Evans A, Poirier O, Kee F. et al. Polymorphisms of the angiotensin-converting-enzyme gene in subjects who die from coronary artery disease. QJM.1994;87:211-214.Google Scholar 29. Gelernter J, Goldman D, Risch N. The A1 allele at the D2 dopamine receptor gene and alcoholism: a reappraisal. JAMA.1993;269:1673-1677.Google Scholar 30. Pato CN, Macciardi F, Pato MT, Verga M, Kennedy JL. Review of the putative association of dopamine D2 receptor and alcoholism: a meta-analysis. Am J Med Genet.1993;48:78-82.Google Scholar 31. Yang L, Li T, Wiese C. et al. No association between schizophrenia and homozygosity at the D3 dopamine receptor gene. Am J Med Genet.1993;48:83-86.Google Scholar 32. Angier N. Maybe it's not a gene behind a person's thrill-seeking ways. The New York Times.November 1, 1996:A22.Google Scholar 33. Malhotra AK, Virkkunen M, Rooney W, Eggert M, Linnoila M, Goldman D. The association between the dopamine D4 receptor (D4DR) 16 amino acid repeat polymorphism and novelty seeking. Mol Psychiatry.1996;1:388-391.Google Scholar 34. Kidd KK. Associations of disease with genetic markers: deja vu all over again. Am J Med Genet.1993;48:71-73.Google Scholar 35. Reid MC, Lachs MS, Feinstein AR. Use of methodological standards in diagnostic test research: getting better but still not good. JAMA.1995;274:645-651.Google Scholar 36. Becker DM, Philbrick JT, Abbitt PL. Real-time ultrasonography for the diagnosis of lower extremity deep venous thrombosis: the wave of the future? Arch Intern Med.1989;149:1731-1734.Google Scholar 37. Cooper LS, Chalmers TC, McCally M, Berrier J, Sacks HS. The poor quality of early evaluations of magnetic resonance imaging. JAMA.1988;259:3277-3280.Google Scholar 38. Kent DL, Haynor DR, Longstreth WT, Larson EB. The clinical efficacy of magnetic resonance imaging in neuroimaging. Ann Intern Med.1994;120:856-871.Google Scholar 39. Greenes RA, Begg CB. Assessment of diagnostic technologies: methodology for unbiased estimation from samples of selectively verified patients. Invest Radiol.1985;20:751-757.Google Scholar 40. Bates AS, Margolis PA, Evans AT. Verification bias in pediatric studies evaluating diagnostic tests. J Pediatr.1993;122:585-590.Google Scholar 41. Sheps SB, Schechter MT. The assessment of diagnostic tests: a survey of current medical research. JAMA.1984;252:2418-2422.Google Scholar 42. Arroll B, Schechter MT, Sheps SB. The assessment of diagnostic tests: a comparison of medical literature in 1982 and 1985. J Gen Intern Med.1988;3:443-447.Google Scholar 43. Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med.1978;299:926-930.Google Scholar 44. Nierenberg AA, Feinstein AR. How to evaluate a diagnostic marker test: lessons from the rise and fall of the dexamethasone suppression test. JAMA.1988;259:1699-1702.Google Scholar 45. Fletcher RH. Carcinoembryonic antigen. Ann Intern Med.1986;104:66-73.Google Scholar 46. Golightly MG. Laboratory considerations in the diagnosis and management of Lyme borreliosis. Am J Clin Pathol.1993;99:168-174.Google Scholar 47. Lensing AWA, Hirsh J. 125 I-fibrinogen leg scanning: reassessment of its role for the diagnosis of venous thrombosis in post-operative patients. Thromb Haemost.1993;69:2-7.Google Scholar 48. Roy B. The Julius Rosenwald Fund syphilis seroprevalence studies. J Natl Med Assoc.1996;88:315-322.Google Scholar 49. Feinstein AR, Horwitz RI. Choosing cases and controls: the clinical epidemiology of "clinical investigation." J Clin Invest.1988;81:1-5.Google Scholar 50. Fineberg HV, Hiatt HH. Evaluation of medical practices: the case for technology assessment. N Engl J Med.1979;301:1086-1091.Google Scholar 51. Jaeschke R, Guyatt GH, Sackett DL.for the Evidence-Based Medicine Working Group. Users' guides to the medical literature, III: how to use an article about a diagnostic test, A: are the results of the study valid? JAMA.1994;271:389-391.Google Scholar 52. Jaeschke R, Guyatt GH, Sackett DL.for the Evidence-Based Medicine Working Group. Users' guide to the medical literature, III: how to use an article about a diagnostic test, B: what are the results and will they help me in caring for my patient? JAMA.1994;271:703-707.Google Scholar 53. Guyatt GH, Tugwell PX, Feeny DH, Haynes RB, Drummond M. A framework for clinical evaluation of diagnostic technologies. CMAJ.1986;134:587-594.Google Scholar 54. Begg CB. Biases in the assessment of diagnostic tests. Stat Med.1987;6:411-423.Google Scholar 55. Kent DL, Larson EB. Disease, level of impact, and quality of research methods: three dimensions of clinical efficacy assessment applied to magnetic resonance imaging. Invest Radiol.1992;27:245-254.Google Scholar 56. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modifications of methodological standards. JAMA.1997;277:488-494.Google Scholar 57. Power EJ, Tunis SR, Wagner JL. Technology assessment and public health. Annu Rev Public Health.1994;15:561-579.Google Scholar 58. van der Put NMJ, Steegers-Theunissen RPM, Frosst P. et al. Mutated methylenetetrahydrofolate reductase as a risk factor for spina bifida. Lancet.1995;346:1070-1071.Google Scholar 59. Abrahamov A, Elstein D, Gross-Tsur V. et al. Gaucher's disease variant characterized by progressive calcification of heart valves and unique genotype. Lancet.1995;346:1000-1003.Google Scholar 60. Rees DC, Cox M, Clegg JB. World distribution of factor V Leiden. Lancet.1995;346:1133-1134.Google Scholar 61. Brahe C, Servidei S, Zappata S, Ricci E, Tonali P, Neri G. Genetic homogeneity between childhood-onset and adult-onset autosomal recessive spinal muscular atrophy. Lancet.1995;346:741-742.Google Scholar 62. Stern RC, Doershuk CF, Drumm ML. 3849+10 kb C→T mutation and disease severity in cystic fibrosis. Lancet.1995;346:274-276.Google Scholar 63. Arranz M, Collier D, Sodhi M. et al. Association between clozapine response and allelic variation in 5-HT2A receptor gene. Lancet.1995;346:281-282.Google Scholar 64. Harden PN, Geddes C, Rowe PA. et al. Polymorphisms in angiotensin-converting-enzyme gene and progression of IgA nephropathy. Lancet.1995;345:1540-1542.Google Scholar 65. Garred P, Madsen HO, Hofmann B, Svejgaard A. Increased frequency of homozygosity of abnormal mannan-binding-protein alleles in patients with suspected immunodeficiency. Lancet.1995;346:941-943.Google Scholar 66. Chabriat H, Vahedi K, Iba-Zizen MT. et al. Clinical spectrum of CADASIL: a study of 7 families. Lancet.1995;346:934-939.Google Scholar 67. Ruiz J, Blanche H, James RW. et al. Gln-Arg192 polymorphism of paraoxonase and coronary heart disease in type 2 diabetes. Lancet.1995;346:869-872.Google Scholar 68. Thorlacius S, Tryggvadottir L, Olafsdottir GH. et al. Linkage to BRCA2 region in hereditary male breast cancer. Lancet.1995;346:544-545.Google Scholar 69. Clausen JO, Hansen T, Bjorbaek C. et al. Insulin resistance: interactions between obesity and a common variant of insulin receptor substrate-1. Lancet.1995;346:397-402.Google Scholar 70. Hayashi N, Ito I, Yanagisawa A. et al. Genetic diagnosis of lymph-node metastasis in colorectal cancer. Lancet.1995;345:1257-1259.Google Scholar 71. Samani NJ, Martin DS, Brack M. et al. Insertion/deletion polymorphism in the angiotensin-converting enzyme gene and risk of restenosis after coronary angioplasty. Lancet.1995;345:1013-1016.Google Scholar 72. Aono S, Adachi Y, Uyama E. et al. Analysis of genes for bilirubin UDP-glucuronosyltransferase in Gilbert's syndrome. Lancet.1995;345:958-959.Google Scholar 73. Summerfield JA, Ryder S, Sumiya M. et al. Mannose binding protein gene mutations associated with unusual and severe infections in adults. Lancet.1995;345:886-889.Google Scholar 74. Agundez JAG, Ledesma MC, Benitez J. et al. CYP2D6 genes and risk of liver cancer. Lancet.1995;345:830-831.Google Scholar 75. Ferrari S, Rizzoli R, Chevalley T, Slosman D, Eisman JA, Bonjour JP. Vitamin D receptor-gene polymorphisms and change in lumbar-spine bone mineral density. Lancet.1995;345:423-424.Google Scholar 76. Jouet M, Kenwrick S. Gene analysis of L1 neural cell adhesion molecule in prenatal diagnosis of hydrocephalus. Lancet.1995;345:161-162.Google Scholar 77. Orscheschek K, Merz H, Hell J, Binder T, Bartels H, Feller AC. Large-cell anaplastic lymphoma-specific translocation in Hodgkin's disease: indication of a common pathogenesis? Lancet.1995;345:87-90.Google Scholar 78. Calvert R, Randerson J, Evans P. et al. Genetic abnormalities during transition from Helicobacter pylori-associated gastritis to low-grade maltoma. Lancet.1995;345:26-27.Google Scholar 79. Bloemenkamp KWM, Rosendaal FR, Helmerhorst FM, Buller HR, Vandenbroucke JP. Enhancement by factor V Leiden mutation of risk of deep-vein thrombosis associated with oral contraceptives containing a third-generation progestagen. Lancet.1995;346:1593-1596.Google Scholar 80. Barrett TG, Bundey SE, Macleod AF. Neurodegeneration and diabetes: UK nationwide study of Wolfram (DIDMOAD) syndrome. Lancet.1995;346:1458-1463.Google Scholar 81. Henderson AS, Easteal S, Jorm AF. et al. Apolipoprotein E allele ∊4, dementia, and cognitive decline in a population sample. Lancet.1995;346:1387-1390.Google Scholar 82. van Herwerden L, Harrap SB, Wong ZYH. et al. Linkage of high-affinity IgE receptor gene with bronchial hyperreactivity, even in absence of atopy. Lancet.1995;346:1262-1265.Google Scholar 83. Jass JR, Cottier DS, Jeevaratnam P. et al. Diagnostic use of microsatellite instability in hereditary non-polyposis colorectal cancer. Lancet.1995;346:1200-1201.Google Scholar 84. Whatley SD, Roberts AG, Elder GH. De-novo mutation and sporadic presentation of acute intermittent porphyria. Lancet.1995;346:1007-1008.Google Scholar 85. Kwak LW, Taub DD, Duffey PL. et al. Transfer of myeloma idiotype-specific immunity from an actively immunized marrow donor. Lancet.1995;345:1016-1020.Google Scholar 86. Hall IP, Wheatley A, Wilding P, Liggett SB. Association of Glu 27 β2-adrenoceptor polymorphism with lower airway reactivity in asthmatic subjects. Lancet.1995;345:1213-1214.Google Scholar 87. Smith JD, Rose ML, Pomerance A, Burke M, Yacoub MH. Reduction of cellular rejection and increase in longer-term survival after heart transplantation after HLA-DR matching. Lancet.1995;346:1318-1322.Google Scholar 88. Gjertsen MK, Bakka A, Breivik J. et al. Vaccination with mutant ras peptides and induction of T-cell responsiveness in pancreatic carcinoma patients carrying the corresponding RAS mutation. Lancet.1995;346:1399-1400.Google Scholar 89. Wagner JE, Kernan NA, Steinbuch M, Broxmeyer HE, Gluckman E. Allogeneic sibling umbilical cord blood transplantation in children with malignant and non-malignant disease. Lancet.1995;346:214-219.Google Scholar 90. Bayerdorffer E, Neubauer A, Rudolph B. et al. Regression of primary gastric lymphoma of mucosa-associated lymphoid tissue type after cure of Helicobacter pylori infection. Lancet.1995;345:1591-1594.Google Scholar 91. Kramer MH, Kluin PM, Wijburg ER, Fibbe WE, Kluin-Nelemans HC. Differentiation of follicular lymphoma cells after autologous bone marrow transplantation and haematopoietic growth factor treatment. Lancet.1995;345:488-490.Google Scholar 92. Rooney CM, Smith A, Ng CYC. et al. Use of gene-modified virus-specific T lymphocytes to control Epstein-Barr virus-related lymphoproliferation. Lancet.1995;345:9-13.Google Scholar 93. Polvikoski T, Sulkava R, Haltia M. et al. Apolipoprotein E, dementia, and cortical deposition of β-amyloid protein. N Engl J Med.1995;333:1242-1247.Google Scholar 94. Bosma PJ, Chowdhury JR, Bakker C. et al. The genetic basis of the reduced expression of bilirubin UDP-glucuronosyltransferase 1 in Gilbert's syndrome. N Engl J Med.1995;333:1171-1175.Google Scholar 95. Hummel M, Ziemann K, Lammert H, Pileri S, Sabattini E, Stein H. Hodgkin's disease with monoclonal and polyclonal populations of Reed-Sternberg cells. N Engl J Med.1995;333:901-906.Google Scholar 96. Gotoda T, Arita M, Arai H. et al. Adult-onset spinocerebellar dysfunction caused by a mutation in the gene for the α-tocopherol-transfer protein. N Engl J Med.1995;333:1313-1318.Google Scholar 97. Goddard AD, Covello R, Luoh SM. et al. Mutations of the growth hormone receptor in children with idiopathic short stature. N Engl J Med.1995;333:1093-1098.Google Scholar 98. Dong F, Brynes RK, Tidow N, Welte K, Lowenberg B, Touw IP. Mutations in the gene for the granulocyte colony-stimulating factor receptor in patients with acute myeloid leukemia preceded by severe congenital neutropenia. N Engl J Med.1995;333:487-493.Google Scholar 99. Goldstein AM, Fraser MC, Struewing JP. et al. Increased risk of pancreatic cancer in melanoma-prone kindreds with p16 mutations. N Engl J Med.1995;333:970-974.Google Scholar 100. Postma DS, Bleecker ER, Amelung PJ. et al. Genetic susceptibility to asthma: bronchial hyperresponsiveness coinherited with a major gene for atopy. N Engl J Med.1995;333:894-900.Google Scholar 101. Brennan JA, Mao L, Hruban RH. et al. Molecular assessment of histopathological staging in squamous-cell carcinoma of the head and neck. N Engl J Med.1995;332:429-435.Google Scholar 102. Candeliere GA, Glorieux FH, Prud'homme J, St Arnaud R. Increased expression of the c-fos proto-oncogene in bone from patients with fibrous dysplasia. N Engl J Med.1995;332:1546-1551.Google Scholar 103. Chillon M, Casals T, Mercier B. et al. Mutations in the cystic fibrosis gene in patients with congenital absence of the vas deferens. N Engl J Med.1995;332:1475-1480.Google Scholar 104. Taplin ME, Bubley GJ, Shuster TD. et al. Mutation of the androgen-receptor gene in metastatic androgen-independent prostate cancer. N Engl J Med.1995;332:1393-1398.Google Scholar 105. Britz-Cunningham SH, Shah MM, Zuppan CW, Fletcher WH. Mutations of the Connexin43 gap-junction gene in patients with heart malformations and defects of laterality. N Engl J Med.1995;332:1323-1329.Google Scholar 106. Thursz MR, Kwiatkowski D, Allsopp CEM, Greenwood BM, Thomas HC, Hill AVS. Association between an MHC class II allele and clearance of hepatitis B virus in the Gambia. N Engl J Med.1995;332:1065-1069.Google Scholar 107. Watkins H, McKenna WJ, Thierfelder L. et al. Mutations in the genes for cardiac troponin T and α-tropomyosin in hypertrophic cardiomyopathy. N Engl J Med.1995;332:1058-1064.Google Scholar 108. Ridker PM, Hennekens CH, Lindpaintner K, Stampfer MJ, Eisenberg PR, Miletich JP. Mutation in the gene coding for coagulation factor V and the risk of myocardial infarction, stroke, and venous thrombosis in apparently healthy men. N Engl J Med.1995;332:912-917.Google Scholar 109. Brennan JA, Boyle JO, Koch WM. et al. Association between cigarette smoking and mutation of the p53 gene in squamous-cell carcinoma of the head and neck. N Engl J Med.1995;332:712-717.Google Scholar 110. Hamilton SR, Liu B, Parsons RE. et al. The molecular basis of Turcot's syndrome. N Engl J Med.1995;332:839-847.Google Scholar 111. Whelan AJ, Bartsch D, Goodfellow PJ. Brief report: a familial syndrome of pancreatic cancer and melanoma with a mutation in the CDKN2 tumor-suppressor gene. N Engl J Med.1995;333:975-977.Google Scholar 112. Clement K, Vaisse C, Manning BSJ. et al. Genetic variation in the β3-adrenergic receptor and an increased capacity to gain weight in patients with morbid obesity. N Engl J Med.1995;333:352-354.Google Scholar 113. Widen E, Lehto M, Kanninen T, Walston J, Shuldiner AR, Groop LC. Association of a polymorphism in the β3-adrenergic-receptor gene with features of the insulin resistance syndrome in Finns. N Engl J Med.1995;333:348-351.Google Scholar 114. Walston J, Silver K, Bogardus C. et al. Time of onset of non-insulin-dependent diabetes mellitus and genetic variation in the β3-adrenergic-receptor gene. N Engl J Med.1995;333:343-347.Google Scholar 115. Nakao S, Takenaka T, Maeda M. et al. An atypical variant of Fabry's disease in men with left ventricular hypertrophy. N Engl J Med.1995;333:288-293.Google Scholar 116. Gan KH, Veeze HJ, Van Den Ouweland AMW. et al. A cystic fibrosis mutation associated with mild lung disease. N Engl J Med.1995;333:95-99.Google Scholar 117. Sunthornthepvarakul T, Gottschalk ME, Hayashi Y, Refetoff S. Brief report: resistance to thyrotropin caused by mutations in the thyrotropin-receptor gene. N Engl J Med.1995;332:155-160.Google Scholar 118. Kopp P, Van Sande J, Parma J. et al. Brief report: congenital hyperthyroidism caused by a mutation in the thyrotropin-receptor gene. N Engl J Med.1995;332:150-154.Google Scholar 119. Wolf HM, Hauber I, Gulle H. et al. Brief report: twin boys with major histocompatibility complex class II deficiency but inducible immune responses. N Engl J Med.1995;332:86-90.Google Scholar 120. O'Rahilly S, Gray H, Humphreys PJ. et al. Brief report: impaired processing of prohormones associated with abnormalities of glucose homeostasis and adrenal function. N Engl J Med.1995;333:1386-1390.Google Scholar 121. Knowles MR, Hohneker KW, Zhou Z. et al. A controlled study of adenoviral vector-mediated gene transfer in the nasal epithelium of patients with cystic fibrosis. N Engl J Med.1995;333:823-831.Google Scholar 122. Walter EA, Greenberg PD, Gilbert MJ. et al. Reconstitution of cellular immunity against cytomegalovirus in recipients of allogeneic bone marrow by transfer of T-cell clones from the donor. N Engl J Med.1995;333:1038-1044.Google Scholar 123. Thomas C, de Saint Basile G, Le Deist F. et al. Brief report: correction of X-linked hyper-IgM syndrome by allogeneic bone marrow transplantation. N Engl J Med.1995;333:426-429.Google Scholar 124. Kaufmann Y, Many A, Rechavi G. et al. Brief report: lymphoma with recurrent cycles of spontaneous remission and relapse—possible role of apoptosis. N Engl J Med.1995;332:507-510.Google Scholar 125. Issaragrisil S, Visuthisakchai S, Suvatte V. et al. Brief report: transplantation of cord-blood stem cells into a patient with severe thalassemia. N Engl J Med.1995;332:367-369.Google Scholar 126. Neumann HPH, Eng C, Mulligan LM. et al. Consequences of direct genetic testing for germline mutations in the clinical management of families with multiple endocrine neoplasia, type II. JAMA.1995;274:1149-1151.Google Scholar 127. Barr FG, Chatten J, D'Cruz CM. et al. Molecular assays for chromosomal translocations in the diagnosis of pediatric soft tissue sarcomas. JAMA.1995;273:553-557.Google Scholar 128. Tokgozoglu LS, Ashizawa T, Pacifico A, Armstrong RM, Epstein HF, Zoghbi WA. Cardiac involvement in a large kindred with myotonic dystrophy. JAMA.1995;274:813-819.Google Scholar 129. Petersen RC, Smith GE, Ivnik RJ. et al. Apolipoprotein E status as a predictor of the development of Alzheimer's disease in memory-impaired individuals. JAMA.1995;273:1274-1278.Google Scholar 130. Guo W, Mason JS, Stone Jr CG. et al. Diagnosis of X-linked adrenal hypoplasia congenita by mutation analysis of the DAX1 gene. JAMA.1995;274:324-330.Google Scholar 131. Hung J, Kishimoto Y, Sugio K. et al. Allele-specific chromosome 3p deletions occur at an early stage in the pathogenesis of lung carcinoma. JAMA.1995;273:558-563.Google Scholar 132. Small GW, Mazziotta JC, Collins MT. et al. Apolipoprotein E type 4 allele and cerebral glucose metabolism in relatives at risk for familial Alzheimer disease. JAMA.1995;273:942-947.Google Scholar 133. Shattuck-Eidens D, McClure M, Simard J. et al. A collaborative survey of 80 mutations in the BRCA1 breast and ovarian cancer susceptibility gene. JAMA.1995;273:535-541.Google Scholar 134. Spector TD, Keen RW, Arden NK. et al. Influence of vitamin D receptor genotype on bone mineral density in postmenopausal women: a twin study in Britain. BMJ.1995;310:1357-1360.Google Scholar 135. Cui KH, Haan EA, Wang LJ, Matthews CD. Optimal polymerase chain reaction amplification for preimplantation diagnosis in cystic fibrosis (Δ508). BMJ.1995;311:536-540.Google Scholar 136. van Duijn CM, Havekes LM, Van Broeckhoven C, de Knijff P, Hofman A. Apolipoprotein E genotype and association between smoking and early onset Alzheimer's disease. BMJ.1995;310:627-631.Google Scholar 137. Hill MR, James AL, Faux JA. et al. Fc∊RI-β polymorphism and risk of atopy in a general population sample. BMJ.1995;311:776-779.Google Scholar 138. Miedzybrodzka ZH, Hall MH, Mollison J. et al. Antenatal screening for carriers of cystic fibrosis: randomized trial of stepwise v couple screening. BMJ.1995;310:353-357.Google Scholar 139. Feinstein AR. Clinical Epidemiology: The Architecture of Clinical Research. Philadelphia, Pa: WB Saunders Co; 1985. 140. Zaaijer HL, Cuypers HT, Reesink HW, Winkel IN, Gerken G, Lelie PN. Reliability of polymerase chain reaction for detection of hepatitis C virus. Lancet.1993;341:722-724.Google Scholar 141. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics.1977;33:159-174.Google Scholar 142. Campbell DA, Field M, McArdle CS, Cooke TG, Gallagher G. Retraction: polymorphism at the tumor necrosis factor locus—a marker of genetic predisposition to colorectal cancer? Lancet.1996;347:1706.Google Scholar 143. Chapman J, Korczyn AD, Goldfarb LG. Retraction: familial Alzheimer's disease associated with S182 codon 286 mutation. Lancet.1996;348:206.Google Scholar
TI - Clinical Epidemiological Quality in Molecular Genetic Research: The Need for Methodological Standards
JO - JAMA
DO - 10.1001/jama.281.20.1919
DA - 1999-05-26
UR - https://www.deepdyve.com/lp/american-medical-association/clinical-epidemiological-quality-in-molecular-genetic-research-the-GO5201oIIY
SP - 1919
EP - 1926
VL - 281
IS - 20
DP - DeepDyve
ER -