Evaluating Online Health Claims

Mindset · §677

Most of the health content in your feed is built to be shared, not to be true. Across systematic reviews covering dozens of studies and platforms — vaccines, diet, chronic disease, mental health — large fractions of the most-viewed posts are misleading by expert review Suarez-Lledo & Alvarez-Galvez 2021. The skill is recognising the shape on sight: who profits, what was actually measured, what the cited paper actually said. Thirty seconds of careful reading per claim worth acting on — and the supplements you wouldn't have needed stop ending up in your cart, the symptoms you don't have stop describing you, the next ten years of wellness churn stop being the years you live.

Do · As-needed Evidence Emerging Chapter Mindset

No new behaviour to start, no purchase, no protocol — just a filter you run before any of those. The payoff is what stops happening: the four supplements you don't buy this year, the cyberchondria spiral that doesn't open at 11pm, the influencer you stop trusting before they cost you more than the last one did. The catch is that the work is invisible — nothing dopaminergic about not being scammed. The skill ports out of health into every other claim domain you live inside.

A health post that goes viral isn't being rewarded for being right. It's being rewarded for being shared, and the things that get shared are confident, novel, and emotionally charged — calibrated uncertainty performs badly in every engagement metric the platform has. One large MIT study tracked ~126,000 stories across Twitter and found false stories reached people roughly six times faster than true ones, and went further, deeper, and wider into the network Vosoughi et al. 2018. Health content sits inside that selection pressure. By the time a claim is in your feed, it has already been filtered for what the algorithm wants, which is rarely what you want.

Stack three more forces on top. A face you've watched for ten hours feels like a friend, and a friend's claim gets believed without the work — that's parasocial trust doing the lifting, transferring credibility into a domain the person was never trained in. Production polish — clean white-coat backdrop, lab-shot B-roll, a Latinate term every sentence — reads as scientific without carrying any actual scientific weight. And the default state of scrolling is sharing without evaluating: when readers are simply prompted to consider whether a headline is accurate before they share, sharing of misinformation drops Pennycook et al. 2021. Almost everyone has the equipment to spot bad claims. They just rarely turn it on.

How widespread the bad stuff is

This isn't a hunch about a few bad apples. Two large systematic reviews — one of 69 studies, one of 57 — both reach the same headline: across vaccines, diet and eating disorders, drugs, infectious disease, and chronic conditions, a substantial fraction of the most-viewed social-media health content is classified as misleading when experts in the relevant field grade it Suarez-Lledo & Alvarez-Galvez 2021 Wang et al. 2019. Topic-specific assays sharpen the picture.

The problem also starts upstream of the influencer. Sumner et al. examined 462 academic press releases and the 668 news stories that came from them and found exaggeration in the news tracked exaggeration in the press release rather than the underlying paper — a large fraction of what readers call "media misinformation" was already there when the university communications office wrote the press release Sumner et al. 2014. The post in your feed is the third or fourth hop in a chain that has been getting louder at every step.

On whether reading more carefully helps: small RCTs say yes, modestly and reliably. Roozenbeek et al. ran 90-second "inoculation" videos that teach common manipulation techniques rather than debunk specific claims; across roughly 30,000 participants — including a field test on YouTube — viewers got measurably better at spotting manipulated content Roozenbeek et al. 2022. Pennycook et al.'s accuracy-prompt RCT found that a brief nudge to consider whether a headline is accurate cut subsequent misinformation sharing Pennycook et al. 2021. The effect sizes are not transformative — call it a tenth to a fifth of a step in discernment — but they're real, they replicate, and they come from interventions that take less than two minutes.

The four moves

The whole protocol is four questions you ask of any health claim before you act on it. Each takes about thirty seconds. Don't apply this to every post — that's exhausting and unnecessary. Apply it to anything that would cost you money, time, or risk: a supplement you're about to buy, a protocol you're about to start, a diagnosis you're starting to believe describes you.

1. Follow the citation

If a claim mentions a study, the claim has to actually match the study. Not the topic of the study — the population, the dose, the endpoint, and the effect size. The usual failure is stretch: an in-vitro result in cells becomes "X reverses Y in humans," a six-week trial in twenty subjects becomes "X causes Y long-term," a 15% effect becomes "dramatic," a result in one demographic becomes a universal claim. Pull up the actual paper (the title is usually enough for a PubMed or Google Scholar search). If you can't find it, that's the answer. If you can find it but it doesn't say what the post said, that's also the answer.

2. Map the money

Who profits if you act on this? Sometimes the answer is on the post — a link in bio, a code, a "use INFLUENCER20." Often it isn't, because disclosure rules get routinely ignored despite the FTC requiring sponsored content to be marked FTC 2023. Look at the broader pattern: does the person have a supplement line, a clinic, a course, an affiliate relationship, an ambassador role? Are they pushing a product the maker also sponsored their trip with? None of this makes them wrong. It changes how much you discount.

This matters because money bends what gets reported. A Cochrane methodology review pooled 75 studies on the question and found industry-sponsored trials report more favourable efficacy and conclusions than non-industry-sponsored trials, with relative risk around 1.27 for favourable conclusions — large enough to flip what the headline of a trial says Lundh et al. 2017. DeJong et al. showed that doctors who received a single industry-sponsored meal worth under twenty dollars prescribed the sponsor's brand at measurably higher rates DeJong et al. 2016. If a sandwich moves a physician's prescription, an affiliate deal moves an influencer's recommendation.

3. Treat before/after as marketing, not measurement

A before-and-after photo is not a controlled comparison. Lighting, camera angle, posture, water retention, hydration, tan, makeup, hair, time of day, and the crop of the frame all move the image more than most interventions do over the time scale a wellness post claims. The photographer chose which photo to show. The two photos were not taken under the same conditions. If they had been — same room, same camera, same time, same pose — the post would say so, because that's the harder claim to fake.

The same logic applies to anecdote chains: "this changed my life" from a person who is selling the thing is a sales pitch, not evidence. It can be a true sales pitch (the person sincerely felt better) and still not be evidence — placebo, regression to the mean, concurrent behaviour change, and selection bias all produce the same testimonial.

4. Ask who isn't selling this

The asymmetric signal in health is who recommends something when they have nothing to gain. Independent guidelines bodies — USPSTF, AASM, AHA/ACC, NICE — and Cochrane reviews are the closest thing to a no-skin-in-the-game lane. They publish, they have no products. If the claim is real and important, it usually shows up there eventually. If a claim has been in social media for years and the independent bodies have not picked it up, that's information — sometimes the bodies are slow, but more often the claim didn't survive the evidence. Goldacre's wider critique of how commercial pressures bend formal medicine is a useful reminder that this signal is asymmetric, not absolute — even institutional sources warrant the same questions, just at a different baseline Goldacre 2012.

What still trips smart readers

"It's peer-reviewed, so it's true." Peer review is a filter against the worst methodological failures, not a guarantee that a finding will hold up. Ioannidis's PLoS Medicine paper modelled why the majority of published findings likely fail to replicate — small sample sizes, flexible analysis choices, publication bias against null results, and the prior probability of the hypothesis itself all conspire Ioannidis 2005. A single peer-reviewed paper is a starting signal, not a conclusion.

"They cited a study, so the claim is sourced." A citation is a pointer, not a verification. The chain from claim to citation to original paper breaks at the first hop more often than most readers expect — the citation is real, but it does not say what the post says it says. The whole point of move #1 is that checking the cite is the work; the cite being present is the table stakes, not the proof.

"FDA-approved means safe." For prescription drugs and medical devices, the FDA does pre-market review (with its own well-documented incentive pressures Lurie et al. 2022). For dietary supplements, it largely doesn't — the Dietary Supplement Health and Education Act of 1994 carved supplements out of pre-market efficacy and safety review, leaving the FDA mostly in a reactive role after products are already on shelves Cohen 2014. Geller et al. estimated about 23,000 US emergency-department visits per year are attributable to supplement adverse events Geller et al. 2015. "It's a supplement, so it's safe" is the inverse of true.

"Credentialed expert, so correct." Credentials protect against grosser failure modes — they don't immunise against incentive capture or scope creep. A real MD or PhD earned in one domain can be transferred to a different domain where the person has no training but full social licence, and the same person inside their actual scope can be moved by commercial pressure: a single sponsored meal worth under twenty dollars shifts physicians' prescribing toward the sponsor's brand DeJong et al. 2016. Credentials are necessary, not sufficient.

Where the trained eye still gets caught

The protocol is a filter, not a force field. Six patterns slip past readers who think of themselves as skeptical.

The "they don't want you to know" frame. The absence of mainstream endorsement gets reframed as evidence of suppression rather than as the expected outcome of a weak claim. This is move #4 weaponised — turning the asymmetric signal upside down. The honest reading of "no Cochrane review, no guidelines body has touched it, no independent clinician recommends it" is that the claim has not survived the evidence. The conspiracy frame asks you to read that exact pattern as proof the claim is too dangerous to acknowledge. Almost always false.
The credential edit. A real degree in cardiology used to recommend hormonal protocols. A real PhD in physics used to opine on vaccine immunology. The credential is real; the scope it was earned in is not the scope it is being spent in. Check what the person was specifically trained in, not what they have letters for.
The "I almost died" hook. A vivid origin story — undiagnosed condition, conventional medicine failed me, then I found this — grants the speaker authority to recommend things that don't follow from the story. The story may be true. The interventions sold on the back of it are a separate question.
Parasocial override. After enough hours with someone's voice in your ear, their wrong claim feels like a friend's claim, and friend-claims get believed without the work. The fix is mechanical: when the recommendation comes from a face you "know," run the protocol harder, not lighter.
Citation theatre. Dense footnotes, journal-style layout, "47 sources cited." The visual pattern of rigour without the citations actually supporting the load-bearing claims when checked. Test one or two cites at random. If they don't hold, none of them do.
The symptom-mirror. A list of common, non-specific symptoms ("tired in the afternoon, brain fog, can't lose weight") framed as evidence of a specific named condition the post is selling a fix for. The list is engineered to describe almost everyone over thirty. Recognition is not diagnosis.

What it costs to keep scrolling without the filter

The damage is rarely a single bad decision. It's the accumulation across years of small ones — and a few rare large ones.

This month. The cart fills up. A liver supplement someone in your feed swore by, a magnesium spray, a patch for sleep, an adaptogen blend whose label lists fourteen plant extracts at undisclosed doses. Most of it does nothing. Some of it interacts with something you already take and you don't notice the interaction because the supplement bottle didn't have to warn you. Geller et al. estimate about 23,000 ED visits a year in the US are attributable to supplement adverse events — that's a measured floor, not a ceiling Geller et al. 2015.

This year. You spend somewhere between several hundred and a few thousand dollars on interventions a literate eye would have filtered out at first read. More expensive than the money: you spend the willingness. Every fad you tried that didn't work makes you slightly less ready to try the next thing — and the next thing might have been real. Cynicism that follows a bad wellness purchase is itself a cost; it lands on the real interventions later.

This decade. You internalise four named deficiencies you don't have. You read a symptom list, you recognise yourself in it, you spend a year half-convinced you have the named condition behind it, you cycle through the protocols sold for it. Friends ask if you're doing okay. The version of you that was going to spend the 2030s doing the boring fundamentals — sleeping, lifting, eating, seeing sunlight — instead spends them cycling through whatever was loud on the platform that quarter. The opportunity cost is the life you would have had with that attention spent elsewhere.

Once in a while, larger. Wakefield's fraudulent 1998 paper, retracted only in 2010, contributed to measurable declines in MMR vaccination and the measles resurgences that followed across multiple countries — direct mortality from a single bad citation that lived for over a decade The Lancet 2010. The COVID-era pseudoscience wave produced documented poisonings from people taking the recommended "remedies" Caulfield 2020. Most readers will never be the casualty. Some are. The protocol exists for the days you might be.

What changes when you start running it

The payoff is mostly negative space — things that don't happen. That makes it hard to feel in the moment and easy to underestimate. Read across a year, it's large.

Within a week. Three claims that would have caught you on Sunday don't. You spot the script — the white-coat backdrop, the symptom list that "describes you exactly," the cited paper you spent thirty seconds on that did not say what the post said it said. You keep scrolling. No purchase, no rabbit hole, no 11pm tab you'll close at midnight.

Within a month. The cart that was filling up doesn't. Two or three hundred dollars stays in the account. The mental load of half-believing you have four named deficiencies starts to lift; the body you live in stops being a problem someone in your feed has identified for you. Friends start asking what you think before they buy the next thing.

Within a year. You have stopped trying things. Not in a cynical way — you have a working filter, and the things that pass the filter (sleep regularity, strength training, sunlight, the boring fundamentals the catalogue keeps coming back to) are the things you actually do, with attention left over to do them well. The thousand-dollar wellness budget that was going to be next year's, isn't. Your peers cycle; you compound.

Over a decade. The skill ports. The same moves you apply to a health reel apply to a financial reel, a political reel, a "this is the future of work" reel. You become the person friends and family text "is this real?" — and the answer you give is usually right, because the protocol is the same. Your kids watch you do it. The version of you that was a soft target for whoever was selling the next thing stops existing.

Adjacent threads worth knowing exist. How to read a research paper end-to-end — what a p-value means, what a confidence interval is telling you, what risk-of-bias scoring catches — is a separate skill that lives one level deeper than the four moves above. The supplement industry specifically warrants its own treatment: why the 1994 carve-out exists, what the labels are allowed to claim, how third-party testing works (USP, NSF, Informed Sport). Cyberchondria and health anxiety — the mental-health pattern that builds when you treat your feed as a diagnostic tool — overlaps with this entry but is a distinct condition worth recognising in yourself. And the broader version of this skill — media literacy outside health, applied to financial, political, and technology claims — is the same four moves in a different costume.

Substance + claimed effects

This entry covers a critical-evaluation skill applied to a specific corpus: health content on social media, influencer accounts, wellness blogs, and the long tail of health-adjacent websites. The "substance" is a set of heuristics — citation chain checking, financial-incentive mapping, evidence-grade recognition, and the structural markers that separate sourced claims from selling — applied at the moment of encounter, before any action is taken on the claim. Claimed effects: reduced exposure to harmful interventions (sketchy supplements, predatory clinics, ineffective devices), reduced wasted spend and time, reduced health anxiety driven by content optimised for engagement, better calibration between effort spent and effect obtained, and incrementally better health-decision quality across the life course. Scope is the evaluative skill itself — not a directory of debunked claims, not a how-to-read-a-paper course (those are adjacent entries).

Evidence by addressing question

mechanism

Three forces stack to make online health claims persuasive beyond their evidentiary weight. Algorithmic selection: platforms surface what holds attention, and certainty plus novelty plus emotional charge outperform calibrated uncertainty in nearly every engagement metric. Vosoughi et al. tracked ~126,000 stories on Twitter and found false stories diffused significantly farther, faster, deeper, and more broadly than true ones — false political stories reached 1,500 people about six times faster than the truth, and the gap held across categories including health Vosoughi et al. 2018. Parasocial trust: repeated exposure to a single face produces felt familiarity that the brain hard-codes as relational trust, transferring credibility to claims the person makes even outside their actual competence. Production polish: high-resolution video, clean white-coat backdrop, lab-shot B-roll, and a Latinate term per sentence all read as "scientific" without any of those signals carrying epistemic weight. Pennycook et al. show that when readers are simply prompted to consider whether a headline is accurate before sharing, sharing of misinformation drops — i.e., the default state is sharing without evaluation, and a small attention nudge corrects it Pennycook et al. 2021. The implication is that almost everyone has the cognitive equipment to detect bad claims; they just rarely engage it on scroll.

evidence

The prevalence literature is consistent across platforms and disease areas. Suarez-Lledo and Alvarez-Galvez systematically reviewed 69 studies of health misinformation on social media and found high rates across vaccines, drugs and substances, diet and eating disorders, and chronic disease — multiple subdomains had majorities or near-majorities of content classified as misleading by domain experts Suarez-Lledo & Alvarez-Galvez 2021. Wang et al. earlier reached the same headline conclusion across 57 studies Wang et al. 2019. Topic-specific assays are stark: of the top-100 most-popular TikTok videos tagged #ADHD, Yeung et al. found roughly half were classified as misleading by board-certified psychiatrists Yeung et al. 2022; a YouTube assay of Ebola information during the 2014 outbreak found about a quarter of high-view videos contained misleading content Pathak et al. 2015. Even one step upstream of social media — academic press releases — Sumner et al. analysed 462 releases and 668 derived news stories and found that exaggeration in news tracked exaggeration in the press release rather than the underlying paper, locating a large part of "misinformation" inside university and journal communications offices themselves Sumner et al. 2014. Caulfield documented the pseudoscience surge through the early COVID period and argued the pattern was structural, not aberrational Caulfield 2020.

On interventions: Roozenbeek et al. ran inoculation-theory videos (~90 seconds, exposing manipulation techniques rather than specific claims) on YouTube and found improved discernment of manipulation tactics across ~30,000 participants, including in a real-platform field test Roozenbeek et al. 2022. Van der Linden's earlier climate-misinformation inoculation work showed the same mechanism transfers across topic domains van der Linden et al. 2017. Pennycook et al.'s accuracy-prompt RCT (~5,000 US participants) showed a brief nudge to consider accuracy reduced misinformation sharing in subsequent feed-like environments Pennycook et al. 2021. The intervention effect sizes are modest but real and consistent across replications — call it a 10–20% improvement in discernment from very light-touch interventions, with larger effects when applied as a sustained habit.

protocol

The actionable checklist clusters into four moves. Citation chain. When a claim cites a study, the claim has to actually match the study's population, dose, endpoint, and effect size. The default failure is a real paper being stretched two or three steps: an in-vitro result becomes "X reverses Y"; a 6-week trial in a small sample becomes "X causes Y long-term." The Wakefield case is the canonical demonstration that a single fraudulent paper can drive years of policy and public-health damage even after retraction The Lancet 2010; Ioannidis's broader framework explains why a single positive paper — even an honest one — is a weak signal compared to a body of replicated trials Ioannidis 2005. Money map. Who profits if you act? The FTC requires sponsored content to be disclosed, and disclosure failures are routinely settled but rarely deter — disclosure does not appear in the content most people see FTC 2023. Industry sponsorship of trials biases reported outcomes toward the sponsor's product, with effect sizes large enough to flip the headline conclusion of a study — Lundh et al.'s Cochrane methodology review pooled 75 studies and found industry-sponsored studies reported more favourable efficacy and conclusions than non-industry studies, RR ≈ 1.27 for favourable conclusions Lundh et al. 2017. DeJong et al. linked industry-sponsored meals to physician prescribing of the sponsor's brand, showing a single meal <$20 was associated with a measurable shift in prescribing DeJong et al. 2016. The same incentive shape — small reward, real behaviour change — applies in the influencer economy, where the reward is access to product, free trips, affiliate commissions, and brand deals. Before/after as evidence claim. A photo pair is not a controlled comparison: lighting, camera angle, posture, pump, hydration, tan, makeup, hair, and crop all move outcome more than most interventions do over the time scale a wellness post claims. A before/after is a marketing surface, not a measurement; treating it as evidence is a category error. Who isn't selling this. Independent guidelines bodies (USPSTF, AASM, AHA/ACC, NICE), Cochrane reviews, and clinicians without product to push are the asymmetric signal — if nobody in the no-skin-in-the-game lane has the recommendation, the claim is almost always overstated or false. Goldacre's structural critique laid out how this shape persists even inside formal medicine Goldacre 2012.

contraindications

Modest. The two risk modes are over-application — readers turning into corrosive cynics who reject reasonable medical care because of pattern-matching to commercial framing — and analysis paralysis on claims that are well-evidenced. The protocol exists to filter signal in, not just filter noise out. Greenhalgh et al.'s critique of EBM excess is relevant here: rigour applied without judgement becomes a different failure mode Greenhalgh et al. 2014. The remedy is keeping the protocol asymmetric — apply it harder to the loud and the selling, lighter to the quiet and the consensus-aligned.

misconceptions

Four common errors. "It's peer-reviewed, so it's true." Peer review is a filter against grossly broken methodology, not a guarantor of replicability — Ioannidis's PLoS Medicine paper estimated, with explicit modelling assumptions, that the majority of published findings fail to replicate Ioannidis 2005. "They cited a study, so the claim is sourced." A citation is a pointer, not a verification; the chain from the citation back to the original source often breaks at the first hop. "Credentialed expert = correct." Credentialing protects against grosser failure modes but does not immunise against incentive capture; physicians with industry meals prescribe differently DeJong et al. 2016, and regulators carry their own incentive structures Lurie et al. 2022. "FDA-approved means safe." For supplements specifically, this is a category misunderstanding — the Dietary Supplement Health and Education Act of 1994 carved supplements out of the FDA's pre-market approval regime; products reach shelves without the safety review most consumers assume Cohen 2014, and Geller et al. estimated ~23,000 emergency-department visits per year in the US attributable to supplement adverse events Geller et al. 2015.

failure-modes

Where the trained reader still gets caught. The "they don't want you to know" frame reframes the absence of mainstream endorsement as evidence of suppression rather than as the expected outcome of weak claims; it is the structural inverse of the "who isn't selling" check, weaponised. The credential-edit: an MD or PhD legitimately earned in one domain transferred to a different domain where the holder has no training but full social licence. The "I almost died" hook: a vivid origin story that grants the speaker authority to recommend interventions that do not derive from the story. Parasocial override: after enough hours of a person's voice in your ear, their wrong claim feels like a friend's claim, which downgrades skepticism in the moment of decision. Citation theatre: dense footnotes and a journal-style layout that signals rigour without the citations actually supporting the load-bearing claims when checked. Symptom-mirror content: lists of common, non-specific symptoms framed as evidence of a specific named condition, sold against the recognition reflex.

practicalities

The skill takes ~30 seconds of mental work per claim — enough to be sustainable if reserved for claims you might act on, prohibitive if applied to every scroll. The leverage move is preselection: spend the budget on claims that would cost money, time, or risk to act on, and skim past purely informational claims unless they nag. Tools: PubMed lookup for the cited paper (does it exist, does it say what the post says); reverse image search on before/after photos (often surfaces the same image used by competing brands); Open Payments database (US) for physician industry payments; FTC's disclosures guidance for the ad-vs-non-ad distinction FTC 2023; Cochrane Library for consensus on whether the claim has been formally reviewed. None requires expertise; all are free.

stakes

Personal: wasted money on supplements that do nothing or harm, time spent on protocols that don't work, anxiety from convinced exposure to invented or inflated conditions, occasional direct harm — Geller et al.'s ~23,000 supplement-attributable ED visits per year is a measured floor, not a ceiling Geller et al. 2015. Compounding: each scammed protocol consumes the willingness to try the next thing, even when that next thing is real — the cynicism that follows a bad wellness purchase is itself a cost. Population: Wakefield's fraudulent 1998 paper, retracted only in 2010, contributed to measurable vaccination-rate declines and measles resurgence years after the underlying science was known to be false The Lancet 2010; the COVID-era pseudoscience wave drove documented harms including ingestion of unproven and dangerous "remedies" Caulfield 2020.

payoff

The benefit is mostly negative space — bad protocols not adopted, bad supplements not bought, scams not fallen for, anxieties not internalised. Read at scale, this is large: the typical heavy-wellness-content consumer spends hundreds to thousands of dollars annually on interventions a trained eye would have filtered out at first read, plus the harder-to-cost cognitive load of believing themselves to have conditions they do not have. Positive payoffs: better calibrated baseline expectations for what real interventions do (since most of them don't transform anyone in a week), and the meta-skill transfer to other claim domains — financial, political, technological — where the same evaluative moves apply.

out-of-scope

How to read a paper end-to-end is its own entry (statistical literacy, p-values, effect sizes, confidence intervals, Cochrane risk-of-bias). Specific debunkings of individual wellness fads belong in their own entries. The mental-health side of constant health-content exposure — orthorexia, cyberchondria, illness-anxiety amplification — overlaps but is a distinct condition entry. Media literacy as a general skill across non-health domains is broader.

Credibility range

Optimist case

This entry should land high. Health misinformation is measurably widespread, demonstrably harmful, and addressable with light-touch interventions whose effects replicate. The skill is free, the marginal time cost is low, and the negative-space payoff (harm avoided, scam avoided, anxiety avoided) compounds. The downside is small — overcorrection produces excessive skepticism in some readers, but that failure mode is bounded and self-correcting (it does not kill people). The optimist case for elevating evaluative literacy as a near-universal recommendation is strong.

Skeptic case

Three lines of pushback. The evidence on interventions is not yet at the RCT-many-trials level for "sustained personal evaluation habit reduces real-world harm." Inoculation and accuracy-prompt studies show short-term effect on a proximal measure (discernment, sharing) under controlled conditions Pennycook et al. 2021 Roozenbeek et al. 2022; the inferential leap to "reader applies checklist for years and avoids real harm" is plausible but not directly trialed. The skill itself can be applied uncharitably — readers can use evaluative posture to reject good medicine that has commercial backing, since nearly all useful pharmaceuticals are sold by entities that profit from them. Most readers will not actually apply this — meta-literacy interventions show modest effect sizes; expecting a single article to materially shift behaviour is optimistic. The honest read is that the skill is real, the prevalence problem is real, but the rated effect on the individual reader's downstream health is mediated through their willingness to actually apply it.

Author's call

Land mid-evidence (3) with the article framed practically — give the reader specific moves rather than abstract literacy. Score impact dimensions modestly because effect is mediated through behaviour change, but score applicability at the ceiling because the problem is universal — every adult with a phone is exposed. Tone toward concrete relief rather than aspiration: the payoff is what gets clawed back, not what gets added. Controversy is low — neither the prevalence finding nor the protocol items are seriously disputed inside the relevant literatures, though specific debunkings (e.g., of named figures) would be.

Stakeholder + incentive map

Pushing the substance (sourced claims and skeptical reading): academic medicine, public-health communicators, independent journalists, science educators, regulators in their enforcement roles (FTC for ad disclosures, FDA for false claims). Incentive: institutional credibility, public health outcomes, audience trust.
Pushing against (loud underqualified claims): the supplement industry (DSHEA-protected, ~$50B US market, structurally permitted to sell without pre-market efficacy review), wellness influencers (revenue from affiliate, brand deals, and proprietary product), platform algorithms (revenue from engagement, which selects for emotional charge), some legacy media (clickable health headlines outperform calibrated ones), some physicians monetising on social platforms outside their scope of practice.
Counter-counter pressure: debunkers and science communicators (Caulfield, Goldacre, Offit, Gorski) producing critique content; consumer-protection lawsuits; investigative journalism on specific actors. Incentive: audience, books, academic positioning.
Inside formal medicine: Goldacre's critique Goldacre 2012 and Lurie et al.'s FDA conflict critique Lurie et al. 2022 document the same commercial-capture dynamic operating inside the institutions readers default-trust; the evaluative skill applies to "official" claims too, not just to TikTok.

Population variability

Universal exposure: any adult with internet access encounters this content, and rates of social-media-as-health-information-source are high in every adult age band, with Pew documenting strong reliance especially among younger adults who get news predominantly through social platforms Pew Research 2020. Variability in vulnerability tracks: low baseline science literacy (less ability to check the citation); social isolation (more parasocial loading onto the influencer); active health complaint without satisfying clinical answer (motivated reasoning toward alternative claims); and demographics underserved by mainstream medicine (rational distrust transferring to less-warranted distrust of evidence-based claims). Variability in cost: readers with discretionary income to spend on bad supplements lose more money; readers without it lose proportionally more of their available resource; readers managing serious conditions risk the largest direct harm if they substitute online claims for treatment. The skill is roughly universally applicable; the people who would benefit most from it are often those least primed by formal education to use it.

Knowledge gaps

Several. (1) Long-term real-world behavioural outcomes from media-literacy interventions: short-term proximal effects are well-evidenced, durable change on health-decision behaviour is not. (2) Effect heterogeneity by demographic and education level — most intervention trials run in convenience samples that under-represent the populations most exposed to bad content. (3) The specific contribution of algorithmic amplification vs. content production: we know both contribute, the marginal effect of each is not crisp. (4) Whether the protocol generalises from "evaluating a TikTok claim" to "evaluating a clinician recommendation" — same moves apply but the social cost of skepticism differs. (5) Evidence that would shift the author's call: a large RCT showing minimal real-world behaviour change from sustained reader-level evaluation training would push the score down on impact dimensions; replicated evidence of measured reductions in wasted spend or supplement-attributable harm would push it up. Pollock and Lancaster's analysis of asymptomatic-transmission evidence handling Pollock & Lancaster 2020 is a useful reminder that even high-stakes, institution-led evidence handling can drift — the protocol's relevance does not stop at the boundary of formal medicine.

Scope call. The brief named four hooks: citation chains, financial incentives, before-and-after evidence, and the markers separating sourced claims from selling. All four are covered as the explicit moves in the protocol section. Resisted the temptation to balloon into adjacent meta-skills (how to read a paper, statistics literacy, the supplement carve-out specifically) — those would each warrant their own entry and are flagged in out-of-scope and below as separate-entry candidates.

Rating call on impact dimensions. Scored health-short-term, longevity, and mood at 2 — real but indirect, mediated through behaviour change. The temptation was higher scores because the topic feels important; the discipline was that the per-reader effect is contingent on the reader actually applying the protocol, and the RCT evidence on sustained real-world behaviour change from media-literacy training is not yet at a level that justifies 3+. The applicability-5 ceiling carries the catalogue weight here, correctly.

On evidence at 3 specifically. Prevalence side is well-evidenced (two large systematic reviews plus topic-specific assays). Intervention side is replicated but proximal (Pennycook accuracy prompt, Roozenbeek inoculation videos) — discernment under controlled conditions, not long-term real-world behaviour. The honest call was 3, not 4: the inferential leap from "people discern slightly better after a 90-second video" to "reader avoids years of bad wellness purchases" is plausible but not directly trialed. Flagged as a knowledge gap.

Dream narrative written despite score ~33 (below 40 obligatory floor) because the entry's honest hook is relief / not-being-conned, and a short relief-lever narrative sharpens the dek and tagline without overpromising. The narrative anchored every link in a cited mechanism (prevalence reviews, sponsorship-bias review, accuracy-prompt and inoculation RCTs, the DSHEA carve-out, supplement-attributable ED visits) per the §1 hinge rule.

Excluded: specific debunkings of named figures (date faster than the meta-skill they illustrate; would invite arguments-by-update and dilute the structural point), detailed statistics tutorial (separate entry), supplements regulation deep-dive (separate entry), cyberchondria and orthorexia as conditions (overlap but distinct condition entries).

Separate-entry candidates surfaced during the write:

How to read a research paper. P-values, confidence intervals, risk-of-bias, effect size vs significance. Adjacent to but separate from this entry's evaluative-posture frame.
The supplements regulatory carve-out. DSHEA 1994 specifically — how a $50B industry came to be exempt from pre-market efficacy review, what third-party certifications (USP, NSF, Informed Sport) actually mean.
Cyberchondria. The mental-health pattern of using a feed as a diagnostic tool; overlaps with this entry's stakes section but is a distinct condition.
Media literacy outside health. Same four moves applied to financial, political, technology claims. Probably belongs in mindset alongside this one.

Future-link candidates: once those exist, the out-of-scope section should be re-pointed to them as proper cross-links via meta.related.

Voice: leaned on the "thirty seconds per claim worth acting on" framing repeatedly to keep effort_burden honest — the protocol is not free, and overselling its ease would lose the trust the rest of the entry needs. Stakes and payoff written with explicit timescale rungs (week / month / year / decade) per §5c; anchored on the typical reader (the everyday scroller with discretionary spending), not the conspiracy-theorist or the wellness-influencer-victim extreme.