In the image above you can see the weakest to the strongest information categories with the strongest, most logically reliable type of study at the top (called Meta Analysis) and the weakest at the bottom (called Expert Opinion / Background Information). Of course meta analysis articles like the one linked here brought a swift response from the powerful gatekeepers who tried to push meta studies down the trustworthiness pyramid and raise the individual randomized controlled trials up to the top.
It’s true that a meta-study is only as good as the original studies that comprise it, but that principle of quality also applies to any single study, so neither category can claim an innate higher quality without the same level of critical, logical evaluation. Perhaps more time is required to go through a meta analysis, but once that job is done, you’ve got a more valuable source of information than a single randomized, controlled trial. The same careful, critical analysis is required at both levels.
Yet some will use quality claims as a reason to push single trials above the meta analysis of many single trials on the value pyramid above. This claim is pushed as Expert Opinion (the bottom level of the information quality pyramid) here in Nature, where we are warned of the supposed inherent dangers of meta studies, as if there were something inherently deceptive about the statistical analyses of multiple blinded, controlled, randomized, and statistically evaluated studies that is somehow inherently absent from a single trial.
It’s not logical to me, but I see their argument and the hard work that went into making it. Kudos for that.
“Expert opinion” rightly belongs at the bottom of the scientific strength pyramid, largely because breakthrough science must always fight an uphill battle against entrenched experts who “know beyond a shadow of doubt” in their unbasted wisdom, that any “new-fangled idea” must be wrong and should be zealously squelched.
Scientists are only human.
Another reason for keeping “expert opinion” at the bottom of the scientific evidence pyramid is the ever-changing decrees of Anthony Fauci throughout the COVID-19 pandemic.
Since the bean-counting lawyers and administrators who run clinical medicine and the US government don’t know just how horribly unreliable “expert opinion” has always been in science, they went ahead and reversed the entire process of scientific medical discovery.
It has always involved hundreds and thousands of MD’s and PhD’s arguing in open literature and meetings. Instead, medical science was replaced by the tyrannical dictates of an 80-something-year-old MD who avoids treating any category of patients, let alone the COVID patients over whom he wielded life-and-death decrees in harmony with Big Pharma’s financial motives.
Inertia against any potential scientific breakthrough happens in every field of science, modern medicine being typical throughout its brief history.
Many medical people and mainstream reporters now believe that a single randomized controlled study (third from the top) is the strongest form of evidence. These good folk are extremely busy doing stressful, difficult work and can’t help it that they often seem brain-dead. They have barely the time to skim through an abstract of a peer-reviewed scientific paper. They look only for a one-sentence synopsis of the conclusion while scanning for the holy words: “blinded, controlled, randomized.”
When they see these words and note that a few thousand patients were involved in reaching a “significant” p-value of 0.05 or less, they “know” they’ve got “infallible” information, about the way a fundamentalist believer of any Western religion feels confident they’ve got the truth when reading an ancient text from a holy book.
But what’s really going on here?
A p-value of 0.05 means that someone wearing thick glasses who can crunch statistical odds in a way that hardly anyone else can has determined from naked numerical data alone that the mathematical odds show a 95% probability that the study’s conclusion is valid (i.e. NOT due to random chance). Which is to say, there’s a 5% chance that the study’s conclusion is due to random chance alone, not due to the drug being effective, but this 5% chance is probably small enough to ignore. (It’s an arbitrary cutoff point, not a natural phenomenon.)
When the stars align and these nice words and numbers appear in the abstract (the only part of the paper that’s freely available to the public who funded the research) these busy medical professionals and the public’s busy mainstream reporters who have no medical education whatsoever rush off and spread yet another “final medical truth” to the patients and public respectively.
It’s useful, however, to realize that a failure to reach a significant p-value can come entirely from having too few patients in the study. (The fewer patients involved in a trial, the more the results look like anecdotal stories to a statistician. The effectiveness of the drug cannot be measured without a large number of patients in the trial. The more the merrier. )
For example, you canNOT do a truly scientific study to determine whether or not a cheap generic “antibiotic X” cures bacterial pneumonia if you only have 30 patients in your trial. Every clinician using this “antibiotic X” may swear that it’s worked well on thousands of their own patients (anecdotally). But the “scientific” study with too few subjects will necessarily fail to show statistical significance no matter how good the drug is.
In our hypothetical example, the p-value isn’t small enough for significance. Let’s say it’s “p = 0.09” (meaning that there’s only a 90% probability that “antibiotic X” really saves lives).
Since the details are a bit subtle, the ridiculously stressed and busy reporters run a literal footrace to become the first to publish a story with a headline like, “New Study Proves Antibiotic X Ineffective Against Bacterial Pneumonia.”
OK, p-values are complex to calculate, have an arbitrary cut-off point, and are steeped in the sort of simple binary thinking that appeals to busy medical doctors in the cook-book practices forced upon them by dollar bean-counters, insurance companies, and ambulance-chasing lawyers. But understanding p-values is not beyond a reporter’s ability, at least in binary terms and a tad beyond. Let’s go there now…
It would be downright life-saving if the reporters who decide medical truth for the public nowadays would try to understood a little about the connection between treating infections early and p-values.
If you suffer recurring viral “fever blisters,” for example, you know to take your acyclovir (or whatever) as soon as possible after the first symptoms appear, or else you’ll have a big ugly sore on you lip for a week. “No it’s not Herpes, I was mugged again.”
Or if you have a migraine headache coming on, you know you’ve got to do your Wim Hof breath holding (to get your adrenalin and your heart rate up) and/or take whatever medication works for you as soon as possible to avoid a painful, nauseating misery that could last for days.
It’s the same with any viral infection, with any type of cancer, and with many other harmful biological phenomena.
The later you treat a disease, the less likely the treatment will work, no matter how great it is when used early.
There’s a natural cut-off deadline, or tipping point where time has run out, you’ve waited too long and the treatment that would have worked will no longer have much effect.
So in our example of an inexpensive generic “antibiotic X,” lets say there were 3,000 patients (n=3,000) in the trial. We should expect a significant p-value, right?
Well, not if “antibiotic X” is given (on average) too late in the course of infection.
Suppose the study was deliberately set up to allow many of the patients into the study who had been sick with bacterial pneumonia for a week before getting “antibiotic X.”
Your study would have a mix of patients who were treated early enough to be saved along with a large number whose pneumonia was treated after the condition was too advanced and couldn’t be stopped by anything short of a miracle.
Let’s say the study came out with a p-value that was too high for the typical binary, arbitrary interpretation of statistical significance. The p-value crunched out at “p = 0.09” (meaning there is only a 91 % likelihood that the antibiotic was effective, rather than the arbitrary cut-off of 95%).
Would you think that MDs and the media would be totally convinced that “antibiotic X” is worthless?
Yes they would.
We know this from a real-world example coming to us from a study of Ivermectin reported in JAMA, (Journal of the American Medical Association), a widely respected medical journal despite accusations of an “anti-Ivermectin for COVID” bias fueled by Big Pharma shenanigans.
The average time from first COVID symptoms to Ivermectin treatment was 5.1 days in this deliberately botched clinical trial reported in JAMA. The reported “confidence interval” for the 5.1 days was 1.3. This tells us that few patients were treatment within 3 days of their first COVID symptoms. This is a huge design error that appears deliberate.
Those docs who have treated thousands of COVID patients with Ivermectin will tell you that it’s crucial to begin the drug within 3 days or less of the patient’s first flu-like symptoms: runny nose, chills, fever, loss of smell, headache, weakness, sore throat, etc.
The gatekeepers at JAMA know this full well. They are extreme outliers in intelligence (IQ) and in their personal reading time of the medical literature. They understand the pathophysiology of early treatment of infectious diseases. They’re likely all “scientific” materialists with a worldview that excludes the existence of anything approaching non-relative morality. If so, they believe that dishonesty and cheating are fine if you “win” for some greater cause, such as avoiding the spread of “vaccine hesitancy” around the globe.
So IF Big Pharma scratches the backs of the JAMA editors, or perhaps threatens their careers, they might tend to do what they’re told and believe what they’ve been taught to believe.
IF Big Pharma advised them to discredit a cheap generic drug like Ivermectin and push a brand-new expensive drug with fresh patents, they might go along for the ride, hoping to retire early and keep their jobs, while doing the “right thing” for humanity.
But even the slightest degree of dishonesty and cheating stops genuine science in its tracks. This is the strongest secular air-tight reason for total honesty, at least in science if not in everything else humans do.
As you’ve probably noticed, corporations tend to behave like “scientific” materialists and tyrants such as Putin who believe that “survival of the fittest” is true morality, “natural selection” is virtuous, and there is no objective good or evil, only changeable notions of right and wrong with no rock-solid reason for honesty in a laboratory.
So it might be expected that JAMA’s gatekeepers and Big Pharma would publish an Ivermectin study where most of the patients received Ivermectin long after the first 3 days of symptom onset. And that’s exactly what they did.
Another thing that’s helpful in avoiding p-value deceptions is this: a study’s measured outcomes (like death) can be selected in a way that’s destined to fail the p-value analysis.
For example, if you’re studying a treatment for a disease like COVID that kills roughly four people out of 1000 these days (the approximate current COVID death rate in Mississippi now, as I understand it), you would probably need several hundred thousand people in the study to “achieve significance” no matter how good your drug is.
Any such study with only a thousand patients would be expected to have about four deaths total in the controls. If the drug worked well and there were only one death in the treated patient cohort, the number crunchers would say there are not enough instances of death to give a significant p-value to the avoidance of death in the drug cohort.
But the headlines would say the drug is worthless…
Unless, of course, the drug is an expensive new one with patents. Then Big Pharma would send out reps to help the journalists’ and MD’s understand the subtleties of p-values. Plus there would be a big section in the published paper explaining how this wonderful is likely going to save lives because it achieved “near statistical efficacy.”
Like a study with too few patients overall, a study that measures too rare of an outcome will fail to achieve p-value significance. Intelligent Designers of a study would know this in the planning stage and avoid it if they were being honest.
This is what went wrong in the study that “proved” the ineffectiveness of Ivermectin to the public. The study only measured two outcomes, death and being placed on a ventilator.
But despite that, try to imagine how JAMA hid this glaring revelation about Ivermectin, forcing people to dig it out of the paper if they have a few hours and know what to look for…
Even with these dishonest biases baked into the trial ahead of time, the study in JAMA that supposedly “proved” Ivermectin was ineffective, actually showed that the patients who were not treated with Ivermectin (the controls) were about 300% (3 times) more likely to die of COVID-19 than the patients who were treated with Ivermectin. And the p-value for this was 0.09 which means that the number crunchers of naked statistics showed that the odds are 91% that this study’s death-defying outcome was not due to random chance, but was almost certainly due to the generic, cheap drug, Ivermectin alone. Which is to say that the odds are only 9 out of 100 (9%) that the life-saving outcomes in this deliberately flawed study of Ivermectin were due to chance alone.
Medical science is like learning a complex computer app for trading the financial markets, it’s easy to understand, but it takes patience, a lot of persistence, and above all, repetition of super-boring information to get things burned into long-term memory. From there you can step back and make a logical, informed analysis.
Hope I didn’t bore you with this article.
So far, it seems that Omicron is providing humanity with herd immunity as hoped. The new Omicron subvariant BA.2 is definitely more easily spread from person to person than the original Omicron. And BA.2 might also be somewhat more dangerous, but I think the jury is still out on this question. Time will tell fairly soon.
Anyway, ask yourself this: if and when you get COVID-19 (experts say everyone will get it), will you take Ivermectin? It’s a medication that’s cheap, has a long track record of safety in humans, and has a 90% chance of actually being the cause-and-effect agent that kept three times as many patients alive compared to controls in a clinical trial that appears to have been obviously designed to fail at the arbitrary p-value cutoff level, missing by only 4%.
Or is it more logical to go along with mainstream headlines and refuse Ivermectin treatment? After all, it has been emotionally associated with the “wrong” political party, with cancelled “anti-vaxxer” physicians, and in my humble case, with a retired surgical pathologist and cytopathologist who thinks UFOs are unquestionably real and the Ancient Astronaut Theory is not as nuts as Giorgio’s hair.
Whatever you decide, especially if you’re a person of color, please make sure your vitamin D levels are well up into the upper “normal” reference range. If not, ask your doc if you can safely take over-the-counter D3 supplements. The science on adequate vitamin D levels helping to prevent COVID deaths is rock-solid. And yet people of color around the world don’t seem to be getting enough of it, as best I can infer from global COVID death stats.
Morrill Talmage Moorehead, MD