From Denise's blog...
"An exhaustive analysis of the raw data from the China Project by
educator and freelance writer Denise Minger  shows that Campbell
failed to take into account other disease-causing variables (increased
Hepatitis B and schistosomiasis infection and rates, industrial work
hazards, etc.) that tend to cluster in higher-cholesterol counties in
the China Study. Campbell also omitted data showing a higher
correlation between wheat flour intake and many diseases (notably
coronary heart disease, cervical cancer, hypertension and stroke) than
with animal protein intake."
The following is from an Epidemiologist that refutes Denise Mingers China Study Claims due to incorrect data analysis...
OH MY. By request of beautiful Freelee, I've taken a look at Denise's analysis. I'm an epidemiologist, and on top of that my research focuses on cancer (not that this makes me completely infallible, but at least I feel equipped to provide an informed critique of her statistical capability). Dr. Campbell was certainly gracious in his response to criticism, but I cannot be so kind. Denise is incredibly naive in her crude analysis of the raw data. She uses correlations and ecologic comparisons to draw conclusions about relationships between diet and outcome (cancer, cardiovascular disease, etc.). WRONG WRONG WRONG!!
A correlation does not an association make.
And, as epidemiologists, our studies are intended to determine associations between exposures and disease. (Yes, there are special methods to determine actual causes of disease, but for most of us, associations will do.) See point 1 below for more on this.
Denise, while meticulous, went through a series of exercises only to:
1) Provide a series of correlations, which honestly, is just the FIRST STEP of any good statistical analysis. Let me explain in a nutshell - a correlation is a linear (assumes a "straight-line" relationship - but not all things are related in this manner), unadjusted (does not account for multiple factors that could potentially confound the relationship between an exposure, like diet, and outcome, like cancer), and non-directional (it does not say if one caused the other or the other way around). An association, on the other hand, is generally adjusted for potential confounding factors and - if a study is properly conducted - gives us an idea of temporality or direction. While we certainly look at correlations between all factors (i.e. between the exposure, potential confounding factors, and the outcome), typically more complex modeling of the data ensues so that multiple factors can be accounted for when investigating the relationship between an exposure and the outcome.
2) Much of her conclusions are drawn from purely ecologic data - that is, data that is in aggregate - such as evaluating total cholesterol and colorectal cancer (as Denise does). Sure, it can be informative, but it doesn't tell us anything about some of the other factors that might be related to cholesterol and colorectal cancer. And while she does perform a stratified analysis (stratifying on schistomiasis), which is a form of "adjustment for confounding"), it still does not take into account other possible confounders and still only tells us about general patterns, but nothing of individual-level associations. Furthermore, she doesn't present results for regions with schistomiasis. What if there was also little correlation between cholesterol and colorectal cancer in these regions? There might be other factors unaccounted for.
Ecologic studies are considered to be at the bottom of the "epidemiologic study totem pole." And we can NOT draw individual-level conclusions from them, i.e. we cannot say that an individual who consumes less fat will, on average, be protected from breast cancer (even if that's true, we cannot draw this conclusion from an ecologic study - there's even a term for it: "ecologic fallacy").
OK, my disclaimer: I'm an epidemiologist, and yes, scientists are NOT objective (I'll say it: I'm an ardent veggie with a happy veggie family). Hell, science is not objective. I mean, you could be given a blob of numbers that mean nothing. It takes some context, interpretation, and data processing to make anything meaningful out of those numbers. Yes, scientists can be biased, and so can the studies they conduct, and the analysis of those studies. But good scientists do the best they can, are open about their methods, and fair when discussing their results. I applaud Dr. Campbell for making his raw data available - very few scientists do this. I will be totally honest and say I have not read "The China Study" (I guess I feel it'd just be preaching to the choir, but I think I will read it now...). But I know enough to know that Denise's analysis was crude at best and completely wrong at worst. No card-carrying epidemiologist would EVER be able to publish her results and draw the conclusions that she does.
I've posted the following comment on Denise's blog (which, was there for a few minutes, and now when I go back to the site, it is mysteriously not there anymore...):
Your analysis is completely OVER-SIMPLIFIED. Every good epidemiologist/statistician will tell you that a correlation does NOT equal an association. By running a series of correlations, you’ve merely pointed out linear, non-directional, and unadjusted relationships between two factors. I suggest you pick up a basic biostatistics book, download a free copy of “R” (an open-source statistical software program), and learn how to analyze data properly. I’m a PhD cancer epidemiologist, and would be happy to help you do this properly. While I’m impressed by your crude, and – at best – preliminary analyses, it is quite irresponsible of you to draw conclusions based on these results alone. At the very least, you need to model the data using regression analyses so that you can account for multiple factors at one time.
** Updated to include an example from Denise's analysis rather than my original example of fat consumption and breast cancer.
I just realized that there's still some trail left about "fat consumption and breast cancer". I should clarify. Denise looked at cholesterol level in each Chinese county and the corresponding incidence rates of colorectal cancer in that region (this is what makes it "ecologic" - each dot represents a county). But the statement still stands - we can't make individual-level conclusions about cholesterol, colorectal cancer, and schistomiasis.
I also just want to add that when she refers to "statistical significance", all that's being tested is the "null hypothesis" that there is no correlation (i.e. correlation = 0). it is not testing whether an exposure is or is not arisk factor for the outcome, even though Denise uses this term loosely.