TY - JOUR
AU1 - Engelman, Karl
AU2 - Mattes, Richard D.
AB - You cannot use insignificant numbers to draw statistically significant conclusions. He and colleagues1 claim to have shown a positive relationship between salt intake and the later development (over a 19-year time frame) of congestive heart failure in overweight (P = .02 for trend) but not normal weight men and women. Their major investigative variable, reported 1-day salt intake extrapolated to presumed lifelong salt intake, is based on the First National Health and Nutrition Examination Survey (NHANES I) numbers derived from each individual's recall of a single day's food and salt intake. Even if it were valid to extrapolate a lifetime of salt intake from a single day's estimated value (which is highly questionable), it is amazing that the authors give credence to, and base their conclusions on, "estimates" of salt intake that are so obviously wrong, both in their absolute numbers and in the population distribution curve. Somehow, the NHANES I investigators found a random population of over 10 000 American adults in which they estimated that close to 75% consumed less than 100 mEq of salt daily, and the quartile with the lowest reported salt intake consumed only a mean of 35 mEq/d, with mean 24-hour sodium intakes of about 90 mEq/d in the entire population. To put these numbers in perspective for readers with a limited nutritional background, what would a diet containing only 35 mEq/d of sodium look like? If that 25% of the adult American population ate only 4 to 5 slices of commercial bread, they would have fulfilled their entire daily quota of salt intake. Of course, this would have precluded the use of any table salt, condiments containing salt, or any other commercially processed foods. Even beyond such unusual dietary restrictions, these same individuals would have been unable to eat any foods naturally containing sodium such as a glass (240 mL) of milk, 0.5 oz (15 g) of butter, or 8 oz (240 g) of unflavored ground beef, each of which would have introduced an excess of 6 to 9 mEq of sodium per day. Thus, this quarter of the US population would have been able to eat only raw or cooked unsalted fruits and vegetables in addition to their slices of plain bread. Now, extrapolate these numbers to the second quartile (25%-50%) of the population who reported consuming only a mean of 63 mEq/d of sodium. Add the aforementioned glass of milk, pat of butter, and unsalted hamburger (but no roll or ketchup!), and you have essentially used up all the additional allowable salt intake for up to 50% of US adults. Alternatively, these same individuals might have ingested just one third of a "Big Mac" to move up to the mean salt intake of the second quartile. How can anyone possibly give credence to these NHANES I salt intake data, which contaminate the literature and are accorded such reverence? Who is kidding whom? Anyone who has ever measured random daily salt intake by measuring the 24-hour urinary excretion of sodium knows that it is rare, indeed, to find anyone excreting less than 100 mEq/d, and the usual mean excretion data are in the range of 150 to 200 mEq/d when complete urine collections are obtained or when sodium excretion is normalized to urinary creatinine. Furthermore, the quartile population distribution of measured sodium excretion is skewed far to the upper values rather than to the very low values as reported by NHANES I. In a balance state, urinary sodium excretion almost completely reflects salt intake provided there are not large extrarenal losses in vomitus, excessive sweat, or diarrhea. Estimating dietary intake from dietary recall data is notoriously inaccurate and should not be used to define critical experimental facts. For example, in a double-blind placebo controlled study2 in which we used a highly motivated and intelligent volunteer adult subject population, we found that long-term ingestion of the diuretic hydrochlorothiazide, but not placebo or amiloride, significantly increased mean dietary salt intake by 30% to 50% (urinary sodium–creatinine ratio increased from a control baseline of about 100 mEq of sodium to about 150 mEq of sodium per gram of creatinine in multiple 24-hour urine samples collected and analyzed over a 4- to 6-week period of hydrochlorothiazide use). These data show that at baseline this was a relatively modest salt-eating group because their control mean total sodium excretion was in the range of 130 to 150 mEq/d. In a subexperiment of this study, using 39 college student volunteers, no change in contemporaneously reported dietary salt intake was found despite the fact that the subjects kept written records during each meal documenting the nature and amount of food they consumed and salt shaker use over a 14-day period. Of course, multiple 24-hour urine collections were collected both during the predrug and postdrug placebo and active hydrochlorothiazide periods. Despite the 30% to 50% increase in measured sodium excretion, the written food diaries did not document any increased intake of salt or salty foods, and when questioned after the study was completed, the subjects (who were blinded to the purpose of the study or any relationship to salt) were totally unaware of their change in dietary habits. So much for determining salt intake from questionnaires and recall. He and colleagues1 state that a possible limitation of their study "may result in misclassification of usual sodium intake . . . and the dietary recall method used in NHANES I may have underestimated sodium intake because it estimated, but did not quantitate discretionary salt use" (salt shaker). But they acknowledge that this constitutes only a small fraction of daily salt intake in our society where we obtain most (85%-95%) of our sodium intake from salt added in preparation of processed foods, which constitute such an important part of our diet. Yet, they surely must know that the estimated mean daily sodium intakes (about 90 mEq/d) that they report from NHANES I represent only about 50% of the usual urinary sodium excretion when actually measured in similar American adult populations. Nonetheless, they go on to conclude that a "strength" of their study is that the relationship they report "can be established with confidence because daily sodium intake was ‘measured at' baseline . . . ." We would be the first to admit that doing valid research in clinical populations is very difficult, but that doesn't give investigators the license to cut corners and extrapolate "conclusions" from obviously invalid numbers, even when they may be "the best available." To anyone who has rigorously applied critical analysis in research, there is a vast chasm between "numbers" and "research data," but the computer and statistical formulas cannot tell the difference. It is not appropriate to publish "truths" derived from unvalidated estimates or best guesses, notwithstanding the best intentions of a group of experienced well-known investigators. It is also intellectually invalid to use a computer to massage guesstimates and to reach a scientific conclusion. For clinical research to be credible and useful, it must adhere to the rigorous criteria of the scientific method. To do less is to denigrate the field. He and colleagues1 are experienced investigators, and they well may be correct in claiming that increased salt intake in overweight (and hypertensive) people may eventually predispose to the development of congestive heart failure, but their reliance on these ludicrous salt intake numbers from NHANES I detracts from the credibility of their conclusions. It surely doesn't give one confidence that they have proved their point. Unfortunately, they are not the first to draw and publish questionable epidemiological conclusions citing these NHANES I salt intake numbers. Alderman et al3 reached exactly the opposite conclusion using the same data. They concluded that, after adjustment for age and sex, reported salt intake numbers showed an inverse relationship between salt intake and all-cause mortality (P<.0001) and cardiovascular mortality (P<.002), especially among the quartile of male and female cohorts reporting the lowest salt intake. Admittedly, the development of congestive heart failure1 and death from cardiovascular disease3 are not exactly the same, but the obvious common pathophysiologic relationships would suggest a commonality between these 2 end points. Unlike He et al,1 Alderman et al3 did not separate normal weight from obese individuals in their analysis. However, most participants in the NHANES I study were overweight with mean body mass index greater than 25.0 (calculated as weight in kilograms divided by the square of height in meters) for the entire group. Furthermore, the quartile with the lowest reported salt intake was also the heaviest (as a group) of the NHANES I participants, especially among the women (mean body mass index of 25.7 for men and 26.6 for women), and also had higher measured blood pressures than the others.3 This same, most overweight, group also reported consuming the lowest food intake (mean daily caloric intakes of 1473 kcal in men and 988 kcal in women). Again, this reported food intake represents only about 50% of the caloric content necessary to maintain body weight in a 60-kg woman or an 80-kg man with only low physical activity. Continuity of these caloric intakes would have resulted in emaciation or death by starvation well before the 19-year end point of the study. The publication by Alderman et al3 elicited a strong international reader response with some of the very critical letters published in later issues of The Lancet.4-9 These investigator-critics also decried the validity of the database that was purported to support the authors' conclusions. Frankly, it looks as though the most hypertensive and most obese NHANES I participants may have reported what they thought they should have been eating, rather than what they actually ingested. While others have also used these NHANES I nutritional data to support epidemiological conclusions, few authors have questioned the validity and applicability of these reported salt (or other nutrient) intake numbers or the general applicability of faulty 1-day recall data to extrapolation of long-term dietary habits. Furthermore, to argue that these are the best data available or that the numbers "tend to lean in the right direction" provides neither solace nor intellectual support to the argument or to the conclusions drawn from the use of these data. Unfortunately, such a post hoc critical analysis of a publication (as represented by this letter) gets lost in cyber netherland and rarely finds its way into the thoughts and bibliographies of future authors to counterbalance the better retained impact and interpretation of the erroneous original article. This is one of the hazards and shortcomings of computerized "literature research" as all too commonly performed these days. Hopefully, this note will serve as an object lesson to young investigators. Remember, it is impermissible to use insignificant numbers to determine statistically significant conclusions. References 1. He JOgden LGBazzano LAVupputuri SLoria CWelton PK Dietary sodium intake and incidence of congestive failure in overweight US men and women. Arch Intern Med. 2002;1621619- 1624Google ScholarCrossref 2. Mattes RDChristensen CMEngelman K Effects of hydrochlorothiazide and amiloride on salt taste and excretion (intake). Am J Hypertens. 1990;3436- 443Google ScholarCrossref 3. Alderman MHCohen HMadhavan S Dietary sodium intake and mortality: the National Health and Nutrition Examination Survey (NHANES I). Lancet. 1998;351781- 785Google ScholarCrossref 4. de Wardener HMacGregor GA Sodium intake and mortality. Lancet. 1998;3511508; author reply 1509- 10Google ScholarCrossref 5. Engelman K Sodium intake and mortality. Lancet. 1998;3511508- 1509; author reply 1509-10Google ScholarCrossref 6. Karppanen HMervaala E Sodium intake and mortality [letter]. Lancet. 1998;3511509Google ScholarCrossref 7. Alderman MHCohen HMadhaven S Sodium intake and mortality [letter]. Lancet. 1998;3511509- 1510Google ScholarCrossref 8. Poulter NRfor The Faculty 31st International Society and Federation of Cardiology 10-day Teaching Seminar in Cardiovascular Disease, Epidemiology and Prevention, Dietary sodium intake and mortality [letter]. Lancet. 1998;352987- 988Google ScholarCrossref 9. Alderman MHCohen HWMadhaven S Dietary sodium intake and mortality [letter]. Lancet. 1998;352988Google ScholarCrossref
TI - Insignificant Data Cannot Yield Statistically Significant Conclusions
JF - Archives of Internal Medicine
DO - 10.1001/archinte.163.7.851
DA - 2003-04-14
UR - https://www.deepdyve.com/lp/american-medical-association/insignificant-data-cannot-yield-statistically-significant-conclusions-gcXN5t6VFn
SP - 851
EP - 855
VL - 163
IS - 7
DP - DeepDyve
ER -