Reply to Pearl: Algorithm of the truth vs real-world science

Reply to Pearl: Algorithm of the truth vs real-world science We thank Professor Pearl for his response1 to our papers on causal inference in epidemiology2,3 and appreciate his concern that, by failing to keeping up with ‘the dazzling speed with which epidemiology has modernized its tools’, we risk being left behind as epidemiology moves into an exclusively DAG-driven phase.1 To be clear, neither of our papers2,3 dismissed the use of directed acyclic graphs (DAGs) and formal causal representations, and we have used them in our own work.4–6 We recognize that DAGs are a powerful tool that can, for example, make the general structure of certain mechanisms generating biases clear, and thus identify when understandings of these can be rendered transportable across study settings.2,7 However, as we also pointed out,2,3 epidemiology and public health are not well served by allowing the ability to construct DAGs to constrain the content and range of substantive questions asked. We find it striking that Pearl did not address these points in his reply, particularly since much of the exchange following our initial paper focused on this issue. Pearl advances a viewpoint that there is only one approach to causal inference: the one he advocates. To Pearl, infallible science is within reach, because his methods are ‘based on a rigorous logic of causation and, therefore [are] as protected from leading to spurious causal inferences as mathematics itself’.1 He appears to have no time for triangulation, which he deems to be an ‘ostensibly unavailable methodology’,1 despite the examples we provided2,3 and many others in the literature.8,9 Similarly, inference to the best explanation (IBE) is dismissed as ‘a romantic aspiration’.1 Surprisingly, Pearl then appears to reframe his discussion by claiming that both triangulation and IBE are possible—but only through the approved application of methods he has developed.1 Being ‘unavailable’ and ‘romantic’ are, seemingly, damnations reserved for those foolish enough to attempt inference using approaches unlicensed by Pearl. Indeed, with regard to causal questions, Pearl goes so far as to claim that ‘until very recently science gave us no means even to articulate them, let alone answer them’.10 Presumably the answers we have been naïve enough to accept, which came from the pre-Pearlian period (with respect to cholera, smoking, LDL cholesterol, etc.) now need to be revisited. What about all that time you spend as an epidemiologist worrying about control of confounding in aetiological studies? Solved: ‘the task of selecting an appropriate set of covariates to control for confounding has been reduced to a simple “roadblocks” puzzle manageable by a simple algorithm’.10 Pearl considers that epidemiology ‘has been a pioneer in accepting the DAG-counterfactuals symbiosis as a ruling paradigm’.1 We remain unconvinced that this has radically transformed the reliability of causal claims in published epidemiology. We reviewed 200+ articles that had cited the most popular package that assists users in constructing DAGs, DAGitty, and found much to be desired in many of them, which sometimes reached highly implausible causal conclusions while following the principles to the letter.3 Since our review, more than 200 further papers have appeared sing this platform. We cordially invite Professor Pearl to review these more recent papers, and look forward to his conclusions regarding the plausibility of causal inference in, and general quality of, work that is apparently following his lead. Even the instigators of this useful tool have themselves expressed some reservations about the conduct of some studies in our pioneering DAG-adopting discipline.11 Pearl repeatedly affirms the infallibility of the structural framework of DAGs and counterfactuals.1,10 This stance, however, rests on the seemingly straightforward and yet, as we discussed,3 highly problematic assumption that existing subject-matter knowledge is what enables construction of reliable DAGs. The infinite regress quickly beckons, as of course that (causal) subject-matter knowledge must have come from somewhere.3 Further, Pearl ignores the fact that there are important questions that relate to the social production of health and disease, not to mention the social production of science, which cannot be usefully disciplined by a linear series of DAGs, even if such DAGs could be meaningfully constructed.2,3 Consider the case of skin colour, that hoary old allegedly obvious marker of ‘race’, used so prominently in the counterfactual disputes we described over whether ‘race’ can or cannot be treated as a ‘cause’ if it cannot be ‘modified’,2,3—with Pearl among those arguing that causality is possible for ‘race’, because ‘the essential ingredient of causation … is responsiveness, namely, the capacity of some variables to respond to variations in other variables, regardless of how those variations came about’.12 Clearly, more than logic is at issue in these debates, just as more than logic is needed to understand the fallacies, for analysing health inequities, of reducing ‘racism’ to ‘race’ to ‘skin colour’. Also at issue is the impact of racism on the social production of science, its ideas and its substantive findings, with this evidence informing the supposedly reliable ‘existing subject-matter knowledge’ that would be used for DAG construction. Underscoring this point is a recent paper in Cell with the cautionary title: ‘An unexpectedly complex architecture for skin pigmentation in Africans’.13 This study reported that genetic control of skin colour is not just by 15 genes, as previously identified by research overwhelmingly conducted (surprise, surprise) on populations of the Global North, but by at least 50 if not more loci, as revealed by the first-ever study of skin colour among the Southern African KhoeSan populations.13 The authors’ inference? – that ‘skin pigmentation is a far more complex trait than previously discussed, analogous to numerous other complex traits discussed in biomedical literature’.13 Inference to the best explanation, anyone? Seeing selection bias requires more than logic alone. We realise that our desire to retain some form of pluralism within causal inference in epidemiology may render us obsolete. However, our opinion is that epidemiology as a discipline requires familiarity with a wide range of techniques for interrogating the ways through which how the world is organised influences the health of those living on it. Certainly, DAGs and counterfactual approaches are an important element of contemporary training in epidemiology, but this should not be at the expense of recognizing the importance of biological, sociological and historical understanding, and questions should be prioritized by what is important, rather than what can be reduced to any particular form of representation. It would be unfortunate to confine questions only to those which can be reduced to algorithms, as the world of public health is much larger than this. There are many excellent presentations of the use of DAGs which recognize limitations as well as strengths.14 We feel these are more likely to help epidemiologists conduct balanced investigations of causes (at all levels) of disease, than inculcation of the belief that there are algorithms of the truth already out there, just waiting to be implemented on data that just happen to be available. References 1 Pearl J. Comment on: ‘The tale wagged by the DAG’ . Int J Epidemiol 2018 ; 47 : 1002 – 4 . 2 Krieger N , Davey Smith G. The tale wagged by the DAG: broadening the scope of causal inference and explanation for epidemiology . Int J Epidemiol 2016 ; 45 : 1787 – 808 . Google Scholar PubMed 3 Krieger N , Davey Smith G. Response: FACEing reality: productive tensions between our epidemiological questions, methods and mission . Int J Epidemiol 2016 ; 45 : 1852 – 65 . Google Scholar PubMed 4 Munafo M , Tilling K , Taylor AE , Evans DM , Davey Smith G. Collider scope: when selection bias can substantially influence observed associations . Int J Epidemiol 2018 ; 47 : 226 – 35 . Google Scholar CrossRef Search ADS PubMed 5 Paternoster L , Tilling KM , Davey Smith G. Genetic epidemiology and Mendelian randomization for informing disease therapeutics: conceptual and methodological challenges . PLoS Genet 2017 ; 13 : e1006944 . Google Scholar CrossRef Search ADS PubMed 6 Valeri L , Chen J , Garcia-Albeniz X , Krieger N , VanderWeele TJ , Coull B. The role of stage at diagnosis in colorectal cancer racial/ethnic survival disparities: a counterfactual causal inference approach. Cancer Epidemiol Biomarkers Prev 2016 ; 25 : 83 – 89 . Google Scholar CrossRef Search ADS 7 Swanson SD. Communicating causality . Eur J Epidemiol 2015 ; 30 : 1073 – 75 . Google Scholar CrossRef Search ADS PubMed 8 Lawlor DA , Tilling K , Davey Smith G. Triangulation in aetiological epidemiology . Int J Epidemiol 2016 ; 45 : 1866 – 86 . Google Scholar CrossRef Search ADS PubMed 9 Munafo M , Davey Smith G. Robust research needs many lines of evidence . Nature 2018 ; 553 : 399 – 401 . Google Scholar CrossRef Search ADS PubMed 10 Pearl J. Theoretical impediments to machine learning with seven sparks from the causal revolution. arXiv 2018 . https://arxiv.org/abs/1801.04016 (4 March 2018, date last accessed). 11 Tennant PWG , Textor J , Gilthorpe MS , Ellison GTH. DAGitty and directed acyclic graphs in observational research: a critical review . J Epidemiol Community Health 2017 ; 71 : A43 . Google Scholar CrossRef Search ADS 12 Bollen KA , Pearl J. Eight myths about causality and structural equation models. In: Morgan SL (ed). Handbook of Causal Analysis for Social Research . New York, NY : Springer , 2013 , pp. 301 – 28 . Google Scholar CrossRef Search ADS 13 Martin AR , Lin M , Granka JM et al. An unexpectedly complex architecture for skin pigmentation in Africans . Cell 2017 ; 171 : 1340 – 53 . Google Scholar CrossRef Search ADS PubMed 14 Rohrer JM. Thinking about correlations and causation: graphical causal models for observational data . Adv Methods Pract Psychol Sci 2018 ; 1 : 27 . Google Scholar CrossRef Search ADS © The Author(s) 2018; all rights reserved. Published by Oxford University Press on behalf of the International Epidemiological Association This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal of Epidemiology Oxford University Press

Reply to Pearl: Algorithm of the truth vs real-world science

Loading next page...
 
/lp/ou_press/reply-to-pearl-algorithm-of-the-truth-vs-real-world-science-letter-iO0abXwYy0
Publisher
Oxford University Press
Copyright
© The Author(s) 2018; all rights reserved. Published by Oxford University Press on behalf of the International Epidemiological Association
ISSN
0300-5771
eISSN
1464-3685
D.O.I.
10.1093/ije/dyy071
Publisher site
See Article on Publisher Site

Abstract

We thank Professor Pearl for his response1 to our papers on causal inference in epidemiology2,3 and appreciate his concern that, by failing to keeping up with ‘the dazzling speed with which epidemiology has modernized its tools’, we risk being left behind as epidemiology moves into an exclusively DAG-driven phase.1 To be clear, neither of our papers2,3 dismissed the use of directed acyclic graphs (DAGs) and formal causal representations, and we have used them in our own work.4–6 We recognize that DAGs are a powerful tool that can, for example, make the general structure of certain mechanisms generating biases clear, and thus identify when understandings of these can be rendered transportable across study settings.2,7 However, as we also pointed out,2,3 epidemiology and public health are not well served by allowing the ability to construct DAGs to constrain the content and range of substantive questions asked. We find it striking that Pearl did not address these points in his reply, particularly since much of the exchange following our initial paper focused on this issue. Pearl advances a viewpoint that there is only one approach to causal inference: the one he advocates. To Pearl, infallible science is within reach, because his methods are ‘based on a rigorous logic of causation and, therefore [are] as protected from leading to spurious causal inferences as mathematics itself’.1 He appears to have no time for triangulation, which he deems to be an ‘ostensibly unavailable methodology’,1 despite the examples we provided2,3 and many others in the literature.8,9 Similarly, inference to the best explanation (IBE) is dismissed as ‘a romantic aspiration’.1 Surprisingly, Pearl then appears to reframe his discussion by claiming that both triangulation and IBE are possible—but only through the approved application of methods he has developed.1 Being ‘unavailable’ and ‘romantic’ are, seemingly, damnations reserved for those foolish enough to attempt inference using approaches unlicensed by Pearl. Indeed, with regard to causal questions, Pearl goes so far as to claim that ‘until very recently science gave us no means even to articulate them, let alone answer them’.10 Presumably the answers we have been naïve enough to accept, which came from the pre-Pearlian period (with respect to cholera, smoking, LDL cholesterol, etc.) now need to be revisited. What about all that time you spend as an epidemiologist worrying about control of confounding in aetiological studies? Solved: ‘the task of selecting an appropriate set of covariates to control for confounding has been reduced to a simple “roadblocks” puzzle manageable by a simple algorithm’.10 Pearl considers that epidemiology ‘has been a pioneer in accepting the DAG-counterfactuals symbiosis as a ruling paradigm’.1 We remain unconvinced that this has radically transformed the reliability of causal claims in published epidemiology. We reviewed 200+ articles that had cited the most popular package that assists users in constructing DAGs, DAGitty, and found much to be desired in many of them, which sometimes reached highly implausible causal conclusions while following the principles to the letter.3 Since our review, more than 200 further papers have appeared sing this platform. We cordially invite Professor Pearl to review these more recent papers, and look forward to his conclusions regarding the plausibility of causal inference in, and general quality of, work that is apparently following his lead. Even the instigators of this useful tool have themselves expressed some reservations about the conduct of some studies in our pioneering DAG-adopting discipline.11 Pearl repeatedly affirms the infallibility of the structural framework of DAGs and counterfactuals.1,10 This stance, however, rests on the seemingly straightforward and yet, as we discussed,3 highly problematic assumption that existing subject-matter knowledge is what enables construction of reliable DAGs. The infinite regress quickly beckons, as of course that (causal) subject-matter knowledge must have come from somewhere.3 Further, Pearl ignores the fact that there are important questions that relate to the social production of health and disease, not to mention the social production of science, which cannot be usefully disciplined by a linear series of DAGs, even if such DAGs could be meaningfully constructed.2,3 Consider the case of skin colour, that hoary old allegedly obvious marker of ‘race’, used so prominently in the counterfactual disputes we described over whether ‘race’ can or cannot be treated as a ‘cause’ if it cannot be ‘modified’,2,3—with Pearl among those arguing that causality is possible for ‘race’, because ‘the essential ingredient of causation … is responsiveness, namely, the capacity of some variables to respond to variations in other variables, regardless of how those variations came about’.12 Clearly, more than logic is at issue in these debates, just as more than logic is needed to understand the fallacies, for analysing health inequities, of reducing ‘racism’ to ‘race’ to ‘skin colour’. Also at issue is the impact of racism on the social production of science, its ideas and its substantive findings, with this evidence informing the supposedly reliable ‘existing subject-matter knowledge’ that would be used for DAG construction. Underscoring this point is a recent paper in Cell with the cautionary title: ‘An unexpectedly complex architecture for skin pigmentation in Africans’.13 This study reported that genetic control of skin colour is not just by 15 genes, as previously identified by research overwhelmingly conducted (surprise, surprise) on populations of the Global North, but by at least 50 if not more loci, as revealed by the first-ever study of skin colour among the Southern African KhoeSan populations.13 The authors’ inference? – that ‘skin pigmentation is a far more complex trait than previously discussed, analogous to numerous other complex traits discussed in biomedical literature’.13 Inference to the best explanation, anyone? Seeing selection bias requires more than logic alone. We realise that our desire to retain some form of pluralism within causal inference in epidemiology may render us obsolete. However, our opinion is that epidemiology as a discipline requires familiarity with a wide range of techniques for interrogating the ways through which how the world is organised influences the health of those living on it. Certainly, DAGs and counterfactual approaches are an important element of contemporary training in epidemiology, but this should not be at the expense of recognizing the importance of biological, sociological and historical understanding, and questions should be prioritized by what is important, rather than what can be reduced to any particular form of representation. It would be unfortunate to confine questions only to those which can be reduced to algorithms, as the world of public health is much larger than this. There are many excellent presentations of the use of DAGs which recognize limitations as well as strengths.14 We feel these are more likely to help epidemiologists conduct balanced investigations of causes (at all levels) of disease, than inculcation of the belief that there are algorithms of the truth already out there, just waiting to be implemented on data that just happen to be available. References 1 Pearl J. Comment on: ‘The tale wagged by the DAG’ . Int J Epidemiol 2018 ; 47 : 1002 – 4 . 2 Krieger N , Davey Smith G. The tale wagged by the DAG: broadening the scope of causal inference and explanation for epidemiology . Int J Epidemiol 2016 ; 45 : 1787 – 808 . Google Scholar PubMed 3 Krieger N , Davey Smith G. Response: FACEing reality: productive tensions between our epidemiological questions, methods and mission . Int J Epidemiol 2016 ; 45 : 1852 – 65 . Google Scholar PubMed 4 Munafo M , Tilling K , Taylor AE , Evans DM , Davey Smith G. Collider scope: when selection bias can substantially influence observed associations . Int J Epidemiol 2018 ; 47 : 226 – 35 . Google Scholar CrossRef Search ADS PubMed 5 Paternoster L , Tilling KM , Davey Smith G. Genetic epidemiology and Mendelian randomization for informing disease therapeutics: conceptual and methodological challenges . PLoS Genet 2017 ; 13 : e1006944 . Google Scholar CrossRef Search ADS PubMed 6 Valeri L , Chen J , Garcia-Albeniz X , Krieger N , VanderWeele TJ , Coull B. The role of stage at diagnosis in colorectal cancer racial/ethnic survival disparities: a counterfactual causal inference approach. Cancer Epidemiol Biomarkers Prev 2016 ; 25 : 83 – 89 . Google Scholar CrossRef Search ADS 7 Swanson SD. Communicating causality . Eur J Epidemiol 2015 ; 30 : 1073 – 75 . Google Scholar CrossRef Search ADS PubMed 8 Lawlor DA , Tilling K , Davey Smith G. Triangulation in aetiological epidemiology . Int J Epidemiol 2016 ; 45 : 1866 – 86 . Google Scholar CrossRef Search ADS PubMed 9 Munafo M , Davey Smith G. Robust research needs many lines of evidence . Nature 2018 ; 553 : 399 – 401 . Google Scholar CrossRef Search ADS PubMed 10 Pearl J. Theoretical impediments to machine learning with seven sparks from the causal revolution. arXiv 2018 . https://arxiv.org/abs/1801.04016 (4 March 2018, date last accessed). 11 Tennant PWG , Textor J , Gilthorpe MS , Ellison GTH. DAGitty and directed acyclic graphs in observational research: a critical review . J Epidemiol Community Health 2017 ; 71 : A43 . Google Scholar CrossRef Search ADS 12 Bollen KA , Pearl J. Eight myths about causality and structural equation models. In: Morgan SL (ed). Handbook of Causal Analysis for Social Research . New York, NY : Springer , 2013 , pp. 301 – 28 . Google Scholar CrossRef Search ADS 13 Martin AR , Lin M , Granka JM et al. An unexpectedly complex architecture for skin pigmentation in Africans . Cell 2017 ; 171 : 1340 – 53 . Google Scholar CrossRef Search ADS PubMed 14 Rohrer JM. Thinking about correlations and causation: graphical causal models for observational data . Adv Methods Pract Psychol Sci 2018 ; 1 : 27 . Google Scholar CrossRef Search ADS © The Author(s) 2018; all rights reserved. Published by Oxford University Press on behalf of the International Epidemiological Association This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

International Journal of EpidemiologyOxford University Press

Published: Apr 24, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off