Read enough of this library and the same few labels keep turning up. Tai chi for knee arthritis is recommended by name in a guideline. Acupuncture’s effect on chronic pain is contested. Yoga’s benefit for back pain falls below the threshold patients notice. Qigong could not be graded at all. The reflexology foot map was tested and not supported.
Those labels are deliberate. Together they form a five-step scale, and this article explains it once, with the numbers you need to read it fairly.
Start with the number that reframes everything else. In 2007 a team at the Brazilian Cochrane Centre drew 1,016 completed Cochrane reviews at random, from every corner of medicine, and sorted their conclusions. In 49 percent, the evidence supported neither benefit nor harm. The sample spanned all 50 of Cochrane’s review groups, so it covered conventional treatments of every kind, not just herbs and needles.
So insufficient evidence is not a verdict reserved for traditional practices. It is close to the typical result of careful research on anything. That cuts two ways, and both matter: an inconclusive review of a complementary practice is not a scandal by itself, and the practices that do clear the bar have cleared something real.
How often any treatment clears the bar
The Cochrane Library is the largest collection of systematic reviews, the studies that pool every eligible trial on one question. Three independent audits of it point the same way.
The 2007 audit, by Regina El Dib and colleagues in the Journal of Evaluation in Clinical Practice, found that 44 percent of reviews judged a treatment likely beneficial and 7 percent likely harmful. In 96 percent the authors asked for more research. A follow-up by a team that included El Dib, drawn from 1,128 reviews in the 2011 library, found the inconclusive share had barely moved, at 45 percent.
A stricter test arrived in 2022. Jeremy Howick and an international team sampled 2,428 Cochrane reviews published from 2008 to 2021 and looked only at interventions compared against a placebo or no treatment whose outcomes the reviewers had rated with GRADE, the standard system for grading how certain evidence is. To count as proven, an intervention needed a top GRADE rating on at least one main outcome, a statistically significant result, and authors who judged it effective.
Of 1,567 eligible interventions, 87 qualified. That is 5.6 percent. Harms had been measured for only 577 of the 1,567, about 37 percent.
None of this means most of medicine is useless. These audits measure how often a review reaches a firm answer, which depends on how many good trials exist and how high the bar is set. They do not measure how much of what clinicians do every day rests on evidence, which is a different question with different numbers.
The audits also show that the answer depends partly on who is reading. In a 2001 study, a University of Maryland team led by Ezzo had methodologists classify the conclusions of 160 Cochrane reviews and then asked each review’s own authors to classify them the same way. Agreement between readers and authors was low, and the authors “tended to be more optimistic in their conclusions than the readers.”
The same arithmetic for traditional practices
Run the same kind of audit on complementary practices alone and the inconclusive share climbs.
A 2022 overview in the journal Phytomedicine, led by Zeqi Dai of the China Academy of Chinese Medical Sciences with Nicola Robinson as senior author, examined 104 Cochrane reviews of traditional Chinese medicine therapies, with searches running through May 2021. They covered herbal medicine, acupuncture, moxibustion and exercises such as tai chi. Five reached a positive overall conclusion, three about acupuncture and two about herbal medicine. Forty-two found the evidence insufficient, 54 drew no firm conclusion, and 3 found the results did not support the therapy.
Where reviews rated certainty, the picture held. Of 353 rated outcomes, 6 earned a high rating, and all six came from one review of a Chinese herbal capsule given alongside Western drugs for coronary heart disease. Most were low or very low. One more detail matters: of the 1,642 trials inside those reviews, 980 tested a combination of therapies, and in 955 of those the partner was a non-traditional treatment. Much of the evidence here is evidence about add-ons.
Children’s medicine looks similar. Sascha Meyer and colleagues in Germany examined 135 Cochrane reviews of complementary therapies in children, published from 1995 to 2012, in the Swiss Medical Weekly in 2013. Five favored an intervention outright, 26 were conditionally positive, 9 concluded an intervention should not be used, and 95, or 70.3 percent, were inconclusive.
The inconclusive share rose over the years, from about 56 percent in the earliest period to roughly three in four later. More reviews did not produce more answers.
Each audit sorts conclusions a little differently, so the comparison is rough. The direction is not. Across all Cochrane reviews, about half are inconclusive. For pediatric complementary therapies the figure is about 70 percent, and in the Chinese medicine set more than 90 percent ended inconclusive.
An article that reports only the second set of numbers implies a double standard. One that reports only the first implies there is no difference. This library reports both.
The five terms, with one example each
The scale is not exotic. In 2011 Edzard Ernst sorted complementary medicine evidence into five directions, from clearly positive through uncertain to clearly negative. The Society for Integrative Oncology grades therapies with letters adapted from the US Preventive Services Task Force. The Dai team used four bins. This library uses five, and it grades a claim, never a whole practice.
1. Recommended by name in a clinical guideline
A named professional body names the practice, for a named condition, in a published guideline. The label always travels with three companions: how strongly the body recommends it, how good it judged the evidence for that particular practice, and the date. The model case is the joint 2019 guideline on osteoarthritis issued by the American College of Rheumatology with the Arthritis Foundation, which strongly recommends tai chi, in the same top tier as exercise, while yoga and acupuncture receive only conditional recommendations in that document. The tai chi report in this collection also explains why a famous federal endorsement often credited to the practice does not exist.
Strength and evidence quality are separate dials. The American College of Physicians’ 2017 back pain guideline strongly recommends a list of non-drug options, yoga among them, while labeling the evidence for yoga low quality. Panels weigh cost, safety and the alternatives, not just trial results. Recommended means a reasonable choice in a panel’s judgment. It does not mean proven.
2. Positive trial evidence, contested interpretation
The effect is measured reliably, and competent experts disagree about what it means. Acupuncture for chronic pain is the clearest case. The leading pooled analysis and its sharpest critics agree on an advantage over sham needling of about 0.2 standard deviations and disagree entirely about whether that matters, an argument laid out number by number in our acupuncture coverage.
A second case shows two expert bodies reading overlapping trials and landing in different places. The Society for Integrative Oncology’s 2017 breast cancer guideline gave acupressure, added to anti-nausea drugs during chemotherapy, a grade B. The American Society of Clinical Oncology’s own anti-nausea guideline that year called the evidence for such therapies insufficient to recommend for or against.
When ASCO endorsed the integrative guideline in 2018, its expert panel said grade C would fit better, noting that several trials predated current anti-nausea drugs and that two of the three acupressure trials enrolled fewer than 40 patients. The endorsement’s own summary compresses all of that into a single word: recommended.
3. Measured, but below the threshold that matters
An effect exists, but it is smaller than the change patients would notice, or it disappears against a fair comparison. Yoga for chronic low back pain is the textbook example. In Cochrane’s 2022 review of 21 trials and 2,223 people, yoga improved pain by about 4.5 points on a 100-point scale compared with no exercise. The reviewers had set 15 points in advance as the smallest difference that matters to a patient.
Compared with standard back exercise, the gap probably shrank to little or nothing. The yoga piece in this series sets that result beside the strong guideline recommendation, and both are accurate.
4. Insufficient evidence to assess
This bucket holds two different situations, and telling them apart is the most useful skill in this article.
Sometimes the evidence is missing or weak. The Dai overview excluded 17 Chinese medicine reviews that were empty: reviewers searched and found no trial good enough to include. Some gaps are about reviewing rather than testing. In a paper published in 2011, Cochrane’s complementary medicine field reported 241 controlled trials of yoga in Cochrane’s trials database against only 5 Cochrane reviews on the subject.
Other gaps come from trials that exist in bulk but cannot be trusted. A 2024 evaluation for the Australian government analyzed more than a hundred reflexology trials and still concluded that nothing could be said with confidence about any condition, because every included study raised concerns about bias. Cupping’s record has the same problem, with 4 of 135 trials reporting any blinding at all, as the cupping report counted.
Sometimes the practice is too loosely defined to test. When the rheumatology panel reached qigong, it declined to grade it, citing thin evidence and a lack of standard definitions, the sentence that opens our qigong article. Chinese herbal trials hit the same wall, because formulas are adjusted for each patient through what the Dai team called additions and subtractions, so two trials of the same named formula may not test the same mixture. The 2023 guideline on anxiety and depression from the Society for Integrative Oncology and ASCO lists nonstandardized therapies among the limits of its own evidence.
The first problem is solved by better trials. The second is solved only by deciding what the intervention is.
5. Tested and not supported
A specific, testable claim was tested properly and failed. Reflexology’s diagnostic map is the cleanest example. In a 1998 Norwegian study, three reflexologists examining the same 76 patients’ feet agreed with one another barely above chance, and a later blinded British test found their readings did not track which conditions patients had. Our reflexology write-up gives the full account.
Notice that reflexology now sits in two buckets at once. The claim that feet reveal organ disease is in bucket five. The claim that a session eases symptoms is in bucket four. That is normal, and it is why the grade attaches to the claim.
Why these practices are harder to test
Some of the uncertainty is structural, and it helps to know which parts.
Nobody can fake a yoga class. Cochrane’s yoga reviewers found no trial comparing yoga with a sham version, and every trial was at high risk of bias because participants knew what they were doing and rated their own pain. Hands-on and movement therapies share the problem.
A fake can be active. Sham needling is meant to be inert. A 2023 network analysis in JAMA Network Open, of 10 trials with 4,379 back pain patients led largely by researchers at Korean medicine institutes, found that sham needles placed at the real acupuncture points did about as well as true acupuncture, while shams at other points did not. A 2009 BMJ review of three-armed pain trials found no link between sham type and results. Which sham a trial chooses can decide which bucket a claim lands in.
Whether unblinded trials exaggerate is itself disputed. In 2014 Asbjørn Hróbjartsson and colleagues at the Nordic Cochrane Centre examined 12 trials that had randomized patients into blinded and unblinded versions of the same comparison. Unblinded patients reported effects larger by an average of 0.56 standard deviations, and the bias was concentrated in the 10 acupuncture trials among them.
In 2020 a far larger BMJ study called MetaBLIND, covering 142 meta-analyses and 1,153 trials across medicine, found no average difference between trials with and without blinding. Hróbjartsson co-wrote both. The 2020 authors said their result might reflect the limits of their method and that blinding should remain a safeguard. Both findings stand, and neither settles the question for any one practice.
One trial can produce two true headlines. The German GERAC trial, published in 2007, randomized 1,162 people with chronic back pain across 340 practices. At six months, 47.6 percent responded to real acupuncture, 44.2 percent to sham and 27.4 percent to guideline-based conventional care. Real versus sham was not a significant difference. Either needle group versus usual care was, by a wide margin.
Ask whether point placement matters and the trial says no. Ask whether referral beat usual care and it says yes. Researchers call the first an explanatory question and the second a pragmatic one. In 2006 a group led by Harald Walach argued in BMC Medical Research Methodology that the standard ranking of study designs, built on a drug model, fits complex interventions such as physiotherapy, surgery and complementary medicine poorly.
The argument about plausibility
One more dispute shapes how people read this evidence, and it deserves both sides.
In 2014 the surgeon David Gorski and the neurologist Steven Novella argued in Trends in Molecular Medicine that trials of treatments with very low prior probability of success have multiplied because evidence-based medicine underweights whether a treatment fits established science. They proposed what they called science-based medicine. The same year, two researchers based in Sweden and Italy made a related statistical case in the European Journal of Internal Medicine: a significant result means less when the hypothesis was unlikely to begin with. Three homeopathy researchers answered Gorski and Novella in the same journal with a letter titled Making sense of prior probabilities in research, and the pair replied that weighing plausibility does not mean ignoring evidence.
The formal grading systems sit to one side of this fight. A 2014 Johns Hopkins review of meditation programs, for instance, graded its evidence on four domains: risk of bias, precision, directness and consistency. Plausibility was not among them, which is precisely what the critics object to.
This library takes a position on only part of the argument. A measured benefit does not confirm a tradition’s explanation of why it works, and an implausible explanation does not by itself make a well-run trial wrong. Those are separate questions, and the articles here keep them apart.
Not shown is different from shown not to work
The statisticians Altman and Bland gave this distinction its slogan in a 1995 BMJ note titled Absence of evidence is not evidence of absence. GERAC shows why it matters. The advantage of real over sham needles was 3.4 percentage points, with a plausible range running from 3.7 points worse to 10.3 points better. That result does not show point placement is useless. It fails to show it matters, and the range still includes a benefit some patients would value.
That is the line between bucket four and bucket five. A claim earns the tested-and-not-supported label when a fair test with enough precision came back empty, as the reflexology map did. It stays in the insufficient bucket when the test was too small, too biased or too vague to tell.
A 2006 BMJ paper by Polly Brown and colleagues opened with a line every reader of reviews will recognize: “‘More research is needed’ is a conclusion that fits most systematic reviews.” The authors argued that reviewers should spell out exactly what kind of study is still needed. When this library calls evidence insufficient, it tries to say which kind.
Five questions for any evidence claim
These work on anything you read, here or elsewhere.
- Which claim? A practice is not a claim. Tai chi works cannot be graded. Tai chi eases knee osteoarthritis pain can.
- Compared with what? Sham, no treatment, usual care and an active alternative give different answers, and all four can be true at once.
- How big, against what threshold? A statistically significant result smaller than the change patients notice belongs in bucket three.
- Who said it, how strongly, and when? A guideline label needs the body, the strength, the evidence grade and the year.
- If insufficient, why? Evidence that is missing or unreliable is a different problem from a practice too vague to test, and only the first is fixed by more trials.
None of these grades says anything about your own symptoms. Bleeding you cannot explain, a new lump, weight loss you did not intend, chest pain, sudden weakness or numbness, or any symptom that keeps getting worse needs a diagnosis from a clinician, whatever label a therapy you are considering carries. If anyone cites a grade, whether recommended, promising or unproven, as a reason to change, delay or drop care you already receive, that belongs in a conversation with the clinician who manages the condition.
The Takeaway
Inconclusive is the most common answer careful research gives about any treatment, conventional or not. The useful follow-up is always the same three questions: which claim, against what comparison, and insufficient for which reason. A single practice can be strongly recommended for one condition, contested for another and disproven for a third, and the labels in this library exist to keep those findings from blurring into one verdict.
Sources
- Journal of Evaluation in Clinical Practice. “El Dib RP, Atallah AN, Andriolo RB. "Mapping the Cochrane evidence for decision making in health care." J Eval Clin Pract, 2007. PubMed. DOI: 10.1111/j.1365-2753.2007.00886.x.” pubmed.ncbi.nlm.nih.gov
- Journal of Evaluation in Clinical Practice. “Villas Boas PJF et al. "Systematic reviews showed insufficient evidence for clinical practice in 2004: what about in 2011? The next appeal for the evidence-based medicine age." J Eval Clin Pract, 2013. PubMed. DOI: 10.1111/j.1365-2753.2012.01877.x.” pubmed.ncbi.nlm.nih.gov
- Journal of Clinical Epidemiology. “Howick J, Koletsi D, Ioannidis JPA et al. "Most healthcare interventions tested in Cochrane Reviews are not effective according to high quality evidence: a systematic review and meta-analysis." J Clin Epidemiol, 2022. PubMed. DOI: 10.1016/j.jclinepi.2022.04.017.” pubmed.ncbi.nlm.nih.gov
- International Journal of Technology Assessment in Health Care. “Ezzo J, Bausell B, Moerman DE, Berman B, Hadhazy V. "Reviewing the reviews. How strong is the evidence? How clear are the conclusions?" Int J Technol Assess Health Care, 2001. PubMed. PMID: 11758290.” pubmed.ncbi.nlm.nih.gov
- Phytomedicine. “Dai Z, Liao X, Wieland LS, Hu J, Wang Y, Kim TH, Liu JP, Zhan S, Robinson N. "Cochrane systematic reviews on traditional Chinese medicine: What matters, the quantity or quality of evidence?" Phytomedicine, 2022. PubMed. DOI: 10.1016/j.phymed.2021.153921.” pubmed.ncbi.nlm.nih.gov
- Swiss Medical Weekly. “Meyer S, Gortner L, Larsen A et al. "Complementary and alternative medicine in paediatrics: a systematic overview/synthesis of Cochrane Collaboration reviews." Swiss Med Wkly, 2013. PubMed. DOI: 10.4414/smw.2013.13794.” pubmed.ncbi.nlm.nih.gov
- Evidence-Based Complementary and Alternative Medicine. “Ernst E. "How Much of CAM Is Based on Research Evidence?" Evid Based Complement Alternat Med, 2011. PubMed. DOI: 10.1093/ecam/nep044.” pubmed.ncbi.nlm.nih.gov
- Alternative Therapies in Health and Medicine. “Wieland LS, Manheimer E, Berman BM. "Development and classification of an operational definition of complementary and alternative medicine for the Cochrane Collaboration." Altern Ther Health Med, 2011. PubMed. PMID: 21717826.” pubmed.ncbi.nlm.nih.gov
- Arthritis Care & Research (American College of Rheumatology). “Kolasinski SL, Neogi T, Hochberg MC et al. "2019 American College of Rheumatology/Arthritis Foundation Guideline for the Management of Osteoarthritis of the Hand, Hip, and Knee." Arthritis Care Res, 2020. PubMed. DOI: 10.1002/acr.24131.” pubmed.ncbi.nlm.nih.gov
- Annals of Internal Medicine (American College of Physicians). “Qaseem A, Wilt TJ, McLean RM, Forciea MA et al. "Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline From the American College of Physicians." Ann Intern Med, 2017. PubMed. DOI: 10.7326/M16-2367.” pubmed.ncbi.nlm.nih.gov
- Journal of Clinical Oncology (American Society of Clinical Oncology). “Lyman GH, Greenlee H, Bohlke K et al. "Integrative Therapies During and After Breast Cancer Treatment: ASCO Endorsement of the SIO Clinical Practice Guideline." J Clin Oncol, 2018. PubMed. DOI: 10.1200/JCO.2018.79.2721.” pubmed.ncbi.nlm.nih.gov
- Journal of Clinical Oncology (American Society of Clinical Oncology). “Carlson LE, Ismaila N, Addington EL et al. "Integrative Oncology Care of Symptoms of Anxiety and Depression in Adults With Cancer: Society for Integrative Oncology-ASCO Guideline." J Clin Oncol, 2023. PubMed. DOI: 10.1200/JCO.23.00857.” pubmed.ncbi.nlm.nih.gov
- Cochrane Database of Systematic Reviews. “Wieland LS, Skoetz N, Pilkington K, Harbin S, Vempati R, Berman BM. "Yoga for chronic non-specific low back pain." Cochrane Database Syst Rev, 2022. PubMed. DOI: 10.1002/14651858.CD010671.pub3.” pubmed.ncbi.nlm.nih.gov
- JAMA Network Open. “Lee B, Kwon CY, Lee HW et al. "Needling Point Location Used in Sham Acupuncture for Chronic Nonspecific Low Back Pain: A Systematic Review and Network Meta-Analysis." JAMA Netw Open, 2023. PubMed. DOI: 10.1001/jamanetworkopen.2023.32452.” pubmed.ncbi.nlm.nih.gov
- BMJ. “Madsen MV, Gotzsche PC, Hrobjartsson A. "Acupuncture treatment for pain: systematic review of randomised clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups." BMJ, 2009. PubMed. DOI: 10.1136/bmj.a3115.” pubmed.ncbi.nlm.nih.gov
- International Journal of Epidemiology. “Hrobjartsson A, Emanuelsson F, Skou Thomsen AS, Hilden J, Brorson S. "Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies." Int J Epidemiol, 2014. PubMed. DOI: 10.1093/ije/dyu115.” pubmed.ncbi.nlm.nih.gov
- BMJ. “Moustgaard H, Clayton GL, Jones HE et al. "Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study." BMJ, 2020. PubMed. DOI: 10.1136/bmj.l6802.” pubmed.ncbi.nlm.nih.gov
- Archives of Internal Medicine. “Haake M, Muller HH, Schade-Brittinger C et al. "German Acupuncture Trials (GERAC) for chronic low back pain: randomized, multicenter, blinded, parallel-group trial with 3 groups." Arch Intern Med, 2007. PubMed. DOI: 10.1001/archinte.167.17.1892.” pubmed.ncbi.nlm.nih.gov
- BMC Medical Research Methodology. “Walach H, Falkenberg T, Fonnebo V, Lewith G, Jonas WB. "Circular instead of hierarchical: methodological principles for the evaluation of complex interventions." BMC Med Res Methodol, 2006. PubMed. DOI: 10.1186/1471-2288-6-29.” pubmed.ncbi.nlm.nih.gov
- Trends in Molecular Medicine. “Gorski DH, Novella SP. "Clinical trials of integrative medicine: testing whether magic works?" Trends Mol Med, 2014. PubMed. DOI: 10.1016/j.molmed.2014.06.007.” pubmed.ncbi.nlm.nih.gov
- Trends in Molecular Medicine. “Rutten LALB, Mathie RT, Manchanda RK. "Making sense of prior probabilities in research." Trends Mol Med, 2014. PubMed. DOI: 10.1016/j.molmed.2014.09.007.” pubmed.ncbi.nlm.nih.gov
- Trends in Molecular Medicine. “Gorski DH, Novella SP. "Considering prior plausibility in clinical trials does not mean ignoring scientific evidence." Trends Mol Med, 2014. PubMed. DOI: 10.1016/j.molmed.2014.10.001.” pubmed.ncbi.nlm.nih.gov
- European Journal of Internal Medicine. “Pandolfi M, Carreras G. "The faulty statistics of complementary alternative medicine (CAM)." Eur J Intern Med, 2014. PubMed. DOI: 10.1016/j.ejim.2014.05.014.” pubmed.ncbi.nlm.nih.gov
- JAMA Internal Medicine. “Goyal M, Singh S, Sibinga EMS et al. "Meditation programs for psychological stress and well-being: a systematic review and meta-analysis." JAMA Intern Med, 2014. PubMed. DOI: 10.1001/jamainternmed.2013.13018.” pubmed.ncbi.nlm.nih.gov
- BMJ. “Altman DG, Bland JM. "Absence of evidence is not evidence of absence." BMJ, 1995. PubMed. DOI: 10.1136/bmj.311.7003.485.” pubmed.ncbi.nlm.nih.gov
- BMJ. “Brown P, Brunnhuber K, Chalkidou K, Chalmers I et al. "How to formulate research recommendations." BMJ, 2006. PubMed. DOI: 10.1136/bmj.38987.492014.94.” pubmed.ncbi.nlm.nih.gov