What is a statistical meta-analysis?

This term refers to the statistical analysis of a group of individual studies.

The authors of a meta-analysis re-analyze the results of many smaller studies (with only the limited information available in each published report of a study, not the original data from each) pooled as if a larger study had been done with all of the patients in the smaller studies.

Thus a meta-analysis will take results from 10 studies of 10 patients each and claim to have valid results from 100 patients, but without having access to the original data for any of the 100 patients.

Thus a meta-analysis, in itself, is not an actual scientific study, but a type of averaging of averages from many different studies.

Doing a meta-analysis has become very popular in medicine, much to the dismay of mathematicians.

It has been frequently done to determine the "effectiveness" of a treatment compared to a placebo, by mixing the information from many studies that have different requirements for choosing subjects for the studies, different lengths of times of treatment, different drop-out rates, different numbers of participants (usually small numbers), etc.

Sometimes such an analysis is done by comparing studies that compare a single treatment to a placebo, but the studies used different treatments. (Sort of like mixing studies that compare the use of aspirin to a placebo with those that compare ibuprofen to a placebo. Then one claims to be able to deduce whether aspirin or ibuprofen was more effective, when none of the studies compared the two drugs directly.)

Therefore, from a mathematical and also a medical standpoint, trying to get "effectiveness" data comparing drugs from varying studies that only compared a single drug to placebo is not really valid. There are too many factors that can throw one's conclusions off such as variations in study length, severity of the condition in patients of different studies, numbers of dropouts in a study due to severity of side-effects, etc.

So a meta-analysis done in this way is similar to comparing apples to oranges (placebo), persimmons to oranges and redwood trees to oranges, and then coming up with conclusions about how apples, persimmons and redwood trees compare to each other.

To put things even more into perspective, some time ago a meta-analysis of treatments for CFS found that Cognitive Behavioral Therapy (CBT) was the "best" treatment for CFS.

The facts that most of the studies analyzed used the Oxford definition of CFS and compared CBT to placebo, and only a few of the analyzed studies looked at other treatments and patients identified by the 1988 or 1994 definitions of CFS, were ignored.

Thus, the conclusion of a meta-analysis can be heavily influenced by the number of studies of a certain treatment outweighing the number of studies looked at in the analysis that used a different treatment. Somes studies may have been conducted poorly, as well, and in the case of ME/CFS, the patient populations under examination could be very different.

So the take-away message is to be very dubious about the results of a meta-analysis.