A Cancer Epidemiologist refutes Denise Mingers China Study Claims due to incorrect data analysis

From Denise's blog...

"An exhaustive analysis of the raw data from the China Project by

educator and freelance writer Denise Minger [41] shows that Campbell
failed to take into account other disease-causing variables (increased
Hepatitis B and schistosomiasis infection and rates, industrial work
hazards, etc.) that tend to cluster in higher-cholesterol counties in
the China Study. Campbell also omitted data showing a higher
correlation between wheat flour intake and many diseases (notably
coronary heart disease, cervical cancer, hypertension and stroke) than
with animal protein intake."

Read more...

http://rawfoodsos.com/2010/07/07/the-china-study-fact-or-fallac/

The following is from an Epidemiologist that refutes Denise Mingers China Study Claims due to incorrect data analysis...

OH MY. By request of beautiful Freelee, I've taken a look at Denise's analysis. I'm an epidemiologist, and on top of that my research focuses on cancer (not that this makes me completely infallible, but at least I feel equipped to provide an informed critique of her statistical capability). Dr. Campbell was certainly gracious in his response to criticism, but I cannot be so kind. Denise is incredibly naive in her crude analysis of the raw data. She uses correlations and ecologic comparisons to draw conclusions about relationships between diet and outcome (cancer, cardiovascular disease, etc.). WRONG WRONG WRONG!!

A correlation does not an association make.

And, as epidemiologists, our studies are intended to determine associations between exposures and disease. (Yes, there are special methods to determine actual causes of disease, but for most of us, associations will do.) See point 1 below for more on this.

Denise, while meticulous, went through a series of exercises only to:

1) Provide a series of correlations, which honestly, is just the FIRST STEP of any good statistical analysis. Let me explain in a nutshell - a correlation is a linear (assumes a "straight-line" relationship - but not all things are related in this manner), unadjusted (does not account for multiple factors that could potentially confound the relationship between an exposure, like diet, and outcome, like cancer), and non-directional (it does not say if one caused the other or the other way around). An association, on the other hand, is generally adjusted for potential confounding factors and - if a study is properly conducted - gives us an idea of temporality or direction. While we certainly look at correlations between all factors (i.e. between the exposure, potential confounding factors, and the outcome), typically more complex modeling of the data ensues so that multiple factors can be accounted for when investigating the relationship between an exposure and the outcome.

2) Much of her conclusions are drawn from purely ecologic data - that is, data that is in aggregate - such as evaluating total cholesterol and colorectal cancer (as Denise does). Sure, it can be informative, but it doesn't tell us anything about some of the other factors that might be related to cholesterol and colorectal cancer. And while she does perform a stratified analysis (stratifying on schistomiasis), which is a form of "adjustment for confounding"), it still does not take into account other possible confounders and still only tells us about general patterns, but nothing of individual-level associations. Furthermore, she doesn't present results for regions with schistomiasis. What if there was also little correlation between cholesterol and colorectal cancer in these regions? There might be other factors unaccounted for.

Ecologic studies are considered to be at the bottom of the "epidemiologic study totem pole." And we can NOT draw individual-level conclusions from them, i.e. we cannot say that an individual who consumes less fat will, on average, be protected from breast cancer (even if that's true, we cannot draw this conclusion from an ecologic study - there's even a term for it: "ecologic fallacy").

OK, my disclaimer: I'm an epidemiologist, and yes, scientists are NOT objective (I'll say it: I'm an ardent veggie with a happy veggie family). Hell, science is not objective. I mean, you could be given a blob of numbers that mean nothing. It takes some context, interpretation, and data processing to make anything meaningful out of those numbers. Yes, scientists can be biased, and so can the studies they conduct, and the analysis of those studies. But good scientists do the best they can, are open about their methods, and fair when discussing their results. I applaud Dr. Campbell for making his raw data available - very few scientists do this. I will be totally honest and say I have not read "The China Study" (I guess I feel it'd just be preaching to the choir, but I think I will read it now...). But I know enough to know that Denise's analysis was crude at best and completely wrong at worst. No card-carrying epidemiologist would EVER be able to publish her results and draw the conclusions that she does.

I've posted the following comment on Denise's blog (which, was there for a few minutes, and now when I go back to the site, it is mysteriously not there anymore...):
Your analysis is completely OVER-SIMPLIFIED. Every good epidemiologist/statistician will tell you that a correlation does NOT equal an association. By running a series of correlations, you’ve merely pointed out linear, non-directional, and unadjusted relationships between two factors. I suggest you pick up a basic biostatistics book, download a free copy of “R” (an open-source statistical software program), and learn how to analyze data properly. I’m a PhD cancer epidemiologist, and would be happy to help you do this properly. While I’m impressed by your crude, and – at best – preliminary analyses, it is quite irresponsible of you to draw conclusions based on these results alone. At the very least, you need to model the data using regression analyses so that you can account for multiple factors at one time.

** Updated to include an example from Denise's analysis rather than my original example of fat consumption and breast cancer.

I just realized that there's still some trail left about "fat consumption and breast cancer". I should clarify. Denise looked at cholesterol level in each Chinese county and the corresponding incidence rates of colorectal cancer in that region (this is what makes it "ecologic" - each dot represents a county). But the statement still stands - we can't make individual-level conclusions about cholesterol, colorectal cancer, and schistomiasis.

I also just want to add that when she refers to "statistical significance", all that's being tested is the "null hypothesis" that there is no correlation (i.e. correlation = 0). it is not testing whether an exposure is or is not arisk factor for the outcome, even though Denise uses this term loosely.

Tags: Denise china study debunk, The china study critics debunked, epidemiologist refutes china study critic

0 members like this

Replies to This Discussion

Permalink Reply by pradtf on July 22, 2010 at 1:42pm

simply because she knows how to use a simple statistics software program

yes and let's not forget that this software is no doubt msexcel produced by micro$oft makers of windoze the bearer of the blue screen of death!

golly the folly of it all!

in fiendship,
prad

Permalink Reply by The Humane Hominid on July 22, 2010 at 2:47pm

Three, her tone in her analysis was irreverent, in a field in which she is not an authority and has no expertise...yet she refers to her "BS-o-meter", as if she is the one who knows more about what is going on than an expert in field. What?!? Even if I found a scientific authority to be wrong, I would never be so disrespectful or presumptuous! This to me suggests either than she has an axe to grind and is trying to prove a certain point and is not open-minded, or that her ego has gotten in the way simply because she knows how to use a simple statistics software program.

This is, sadly, an all-too-common occurrence in our society, especially in the blogosphere.

(Insert-deity-here) knows how many times I've had to suffer the same argument-from-rebellion when talking to creationists, global warming deniers, 9/11 Truthers, Birthers, and various and sundry other people with weird beliefs.

Now, for the record, I don't think Minger is as far-out as some of those nincompoops, but the thought process seems to be the same: "my ego trumps all expertise; aren't I impressive?"

I find the argument put forth (implicitly by Minger and explicitly by many of her defenders) that one does not need years of expertise or training to do science properly utterly fatuous and self-aggrandizing. I used to argue it myself, until I went back to university and found out how much I didn't know about the things I'd been studying all my life.

The recent US Science and Engineering Indicators show that 55 percent of the population doesn't accept conclusive scientific facts as the truth, while nearly 80 percent of them don't understand how the scientific method actually works.

And sadly, every one of them probably thinks they're the exception to those stats.

Permalink Reply by pradtf on July 22, 2010 at 2:57pm

great commentary on courtney's excellent post, robert!

there is a saying that the more one knows, the more one finds out how little one knows.

on the other hand, the less one knows, the more one seems to need to make it sound like knowledge and expertise amount to zilch.

the internet has an 'equalizing' effect - which, while is generally a good thing, tends unfortunately to be misapplied to make things appear equal, when they are often not - and scientific evidence so often is made to take a backseat to personal belief.

in such situations, it is good to remind the 'believer' what yogananda's guru once told him:
"the universe doesn't wait for the sanction of our beliefs"

in friendship,
prad

Permalink Reply by B on July 22, 2010 at 2:55pm

the following excerpt from a post by Denise on her blog I think is revealing.

Hi Mark,

That’s not quite what I said. “I eat animal products for health reasons” means I reintroduced them into my diet and discovered the health problems I experienced as a vegan vanished. In repeated experiments of going back to veganism and then back to non-veganism, I’ve found that, for me, the animal products are the factor keeping those problems from cropping up again. I prefer to eat whole foods than to supplement, so non-vegan is the route I’ve gone rather than stuffing my cupboard with pills.

doesn't sound like she changed her diet as a result of what she found in the china study. I think all who've written critiques on the china study (none of whom have been epidemiologists) are on some level trying to vindicate their dietary choices. I agree that it does seem like a lame way of doing so. No doubt all the slander that has been thrown at Campbell as a result of these attempts at discrediting him, whether you agree with everything he says or question his methods, I think is disgraceful.

I appreciate reading your posts on these matters courtney (and in general) btw.

Permalink Reply by veganmama18 on July 23, 2010 at 7:12am

Denise has posted a link to a multi-variable analysis performed by Ned. Hmmm... didn't she say she already did this but decided not to post the results for fear of getting her audience off track? ANYWAY. Here's a response regarding Ned's analysis and Denise's claim that she used data from the 1990 monograph rather than the data that is freely available on Oxford's site:

*******
Impressive. Nice analysis and DAGs. However, I’d like to know what kind of regression model he used – linear regression? If so, it would be important to check that your outcome variable is normally distributed (this is separate from using a nonparametric method to determine the variance and p-values for the beta coefficients).

Are there other factors that could be included in the DAG? Probably.

This is just one analysis for one disease, and points to the strong confounding effect of schistosomiasis. However, is schistosomiasis really a big issue in the North America?? Not really: http://www.cdc.gov/ncidod/dpd/parasites/schistosomiasis/factsht_sch...

So, in a model that is generalizable to the US/Canadian population, the DAG shouldn’t include schistosomiasis. This is precisely one of the points Campbell makes in his response to Denise’s original analysis – context is important!

BTW, which data was used? Oxford data or hand-keyed data from the monograph? If Ms. Minger is to remain open about her methods, then she should probably start using the data that is freely available to all, so that comparable analyses can be repeated by anyone.

Permalink Reply by pradtf on July 23, 2010 at 8:15am

This is precisely one of the points Campbell makes in his response to Denise’s original analysis – context is important!
this concept seems to repeatedly fall on deaf ears.

BTW, which data was used? Oxford data or hand-keyed data from the monograph?
ok so that's part of the confusion then - denise has been using data that is not the same as what we have access to? "hand-keyed"??? that's about as raw as raw data can get and would certainly require scrutinizing and likely considerable correcting of input errors. interesting that these matters are starting to surface after the big brohahas have died down.

in friendship,
prad

Permalink Reply by veganmama18 on July 23, 2010 at 8:48am

precisely! someone commented back questioning the term 'generalizability.' here is healthycritique's response (emphasis my own):

------
That was part of my point, sorry it wasn’t clearer: we might not be able to generalize to the US/Canadian population based on the China survey data alone. But “The China Study” wasn’t just based on the survey data.

You’re right, we absolutely can’t ignore schistosomiasis, especially in this dataset, as Ned’s analysis made evident.

I do think the analysis excluded a potentially important confounder – fiber. Perhaps one might also want to include wheat based on Ms. Minger’s findings. So, while the analysis was quite well done, and the proposed causal pathways nicely laid out in DAG form, I think the model can still be improved upon (as with many things in epidemiologic research).

BTW, I thought Ms. Minger had already performed multiple regression analyses and chosen not to post them? So why is someone else repeating her analyses and posting them instead? It would be best to see these results from Ms. Minger herself.

Permalink Reply by TheBananaGirl on July 24, 2010 at 7:58pm

wow that "healthy critique" is onto it! :)

Permalink Reply by pradtf on July 25, 2010 at 4:54am

well since you missed out on getting maxwell because B got him first, may be you should go after healthycritique.

i haven't really looked at ned's stuff in detail yet, but i did read his comment about campbell's data being good (though he doesn't provide explanation as to why he thinks so). his argument may be a good one for our group to examine more closely than healthycritique has done.

in friendship,
prad

Permalink Reply by TheBananaGirl on July 25, 2010 at 10:31am

Yes I've already contacted Healthy Critique, nothing back yet :( but they would be a great addition !

Permalink Reply by veganmama18 on July 25, 2010 at 3:10pm

ned seems quite knowledgeable regarding statistical models and perhaps computation biology. his approach is somewhat different, and i respect that he seems quite scientific about the whole analysis and conclusion, as well as open to suggestions for improvement. i also like that the comments on his blog remain incredibly civil.

Permalink Reply by James bananaman Bailey on March 3, 2012 at 4:32am