Blogpost-effort-level pseudo-meta-meta analyses

Didn't bother double checking the title but I'm sure it's wright

Welling with rage from this screenshot

from this substack post: Our Preoccupation With Protein Intake by Eric Topol on Ground Truths.

I decided to enter some thoughts here.

Firstly, as I said*, bro-influencers are not swayable by the most careful meta analyses. I *AM* aware of at least one such individual who reports scientific papers (Mr. Nippard, I am aware of you), but also have looked into the screaming vortex of the influencer-short-form-media machine, and seen it’s lack.

But Mr. Topol might be focusing on others, e.g. people who would respond to and understand such Microsoft Excel** compiled figures such as the following, which is also the subject of this rant:

Creating this approximately takes as much effort as looking up and being sure it’s the whole list every current meta analysis on protein supplementation —*** wait, the title of this Table says “Supplementation” but e.g. in the abstract of Tagawa it says “The aim of this systematic review was to evaluate the dose–response relationship of the effects of protein intake on lean body mass”! But wait again, Table one of Tagawa lists the PICOS (Population, Intervention, Comparator, Outcome, Study Design)‘s Intervention as “Supplementary protein intake for ≥ 2 weeks”, so it is supplementation, not total protein intake? Keep waiting, since “Table S5 Summary of conditions of the interventions in the included studies” has as its first few rows

, so these are supplementations or meals, with different protein sources?

These details are lost in Topol’s blog-level effort at a meta meta analysis. The issue is, there IS a need for a firm conclusion: does > 1.6 g/kg/day help or not?

The issue is, that question is only answerable by handwavy bullshit, for instance, does that help (how?) for whom? controlling for what? is this with or without resistance training for young people?

Topol includes the figure in Morton and quotes the relentless phonecian’s bullshit (𐤀)‎ calling of the 1.6 threshold, but omits the Tagawa’s figure 2:

Copypasta: “Dose–response relationship between total protein intake and change in lean body mass in each group. Spline curves illustrating the associations between total protein intake and change in lean body mass in each group in an unadjusted model (a, b, and c for all trials, trials with resistance training, and trials without resistance training, respectively), in multivariate-adjusted model 1 (d, e, and f for all trials, trials with resistance training, and trials without resistance training, respectively), or multivariate-adjusted model 2 (g, h, and i for all trials, trials with resistance training, and trials without resistance training, respectively). The solid line and dashed line represent the mean change in LBM and 95%CIs, respectively. Covariates of multivariate-adjusted model 1 are age, sex, intervention period, and resistance training. Covariates of multivariate-adjusted model 2 are weight change in addition to the covariates of multivariate-adjusted model 1. Abbreviations: BW, body weight; FFM, fat-free mass; LBM, lean body mass.”

That section of the tirade went over how the blogpost effort level meta meta analysis glosses over and mushes the details of the individual much more careful meta analyses it pretends to encapsulate. Next, let’s talk about what a real meta meta analysis, not just a table, would help with:

• PICOS, and like, a real understanding of those. Like what are the mediators and moderators? (I’m pedantic enough where’ there’s a difference in those two)?
• Study quality mixing! In meta analyses, they collect and tables of what the authors deem are important variables. Tagawa’s S1-S5 tables is over 100 pages**mk2, and the value “unclear” appears 151 times in describing e.g. “Frequency of exercise before intervention” in Supplementary Table S2 Summary of characteristics of included studies
• (AI thought of this one) are there overlaps? between those meta analyses?? Do they have some of the same studies being double counted??? Maybe????
• Finally, in the other direction, meta analyses put together very opinionated models (like the splines in Fig 2. of Tagawa) that aim to say something strong aggregating data, not just throw their hands up in defeat and say there’s contradictions and missingness so the science is unclear! They try to synthesize and list the assumptions in doing so!

Astute readers of this and the post which this smells like is refuting might note that Topol’s post, firstly, was much more interesting and well written, secondly, covered more ground, and thirdly, at least mentions more than just one meta analysis, where this blog does just one. Focusing down, however, on the non-older group studies (Nunes, “Nunes study no benefit for older group”, Tagawa, not mentioned in his post besides in the table, Morton, primary subject of this sort of nebulous “healthy adults with resistance training”), he really focuses on just refuting the Morton studies specific threshold of 1.6.

Here’s the thing: me and him and everyone I’ve met has this thing I like to call refutation-parity: we like to follow the chain of refutations, stopping when it reaches an even number of steps from what our beliefs are originally, unless we’re really smart and great as human beings.

E.g. he feels that the supplement industry is a bunch of bastards that lobbied a while ago and are now underregulated and they’re using influencers to peddle snake oil to impressionable harmable young and old people that might be causing harm, and he is allergic to bullshit statistics making overstrong opaque claims, and he laments the impatience even our good era public had for actually looking at empirical works backing mass public health interventions (like specific threshold recommendations on big websites or twitter accounts).

And so he does an odd number of refutations (original claim approximant: 2.2 g/kg/day! → odd numbered refutation: no evidence beyond 1.6 for this population for this exercise amount!), and here in this blog post I attempt an even numbered refutation (… → 1.6 isn’t supported by Morton but might be supported by a real and not blogeffort level meta analysis! Tagawa’s Fig 2 b, e, h seem to imply that additional protein still helps, maybe beyond 1.6!). Naturally, another refutation of that node in the DAG can follow, and I invite it heartily. (e.g. … → I’m cherrypicking Tagawa! I’m intuitively doing the same thing as Topol!)

Here’s the main difference I’d like to state: the people who made these meta analyses worked quite hard to really define what’s going on in those underlying studies, and mine and Topol’s blogpost don’t reach that same level of rigor, but me, the hero of this story, is willing put forth an even numbered refutation (I also hate supplements and social media) for the purpose of flagging POTENTIALLY sloppy aggregation of science.

~Opinions~

To be really really right, that is not necessarily to grab onto and squeeze the Truth out of other people’s observations and reporting, but to most completely squeeze out meaning that follows from the many many experiments, stats, interpretations, intuition, rationality, Bayesian-type biases**** and others, there’s a level of effort needed that confounds and befuddles and can be not at all worthwhile to those that seek to cultivate a large audience and thereby achieve influence. Mr. Topol, I would bet, definitely knows this, and has made an implicit decision to report existing literature via what could be achieved in a blogpost level of effort.

And for me, also, I also am not going to do a “real” meta meta analysis, since this alas is also a blogpost.

But I’m a big fan of using evidence to guide public health recommendations, and actually an ENJOYER of the complexity involved in really understanding how science is done and synthesized into insight, and so, IMHO**5†, we shouldn’t shy away from admitting “a shortcut was taken to blogeffort†† this.”

To quote someone whom I don’t necessarily agree with in his radicalness, “IMO the Mortons and the Tagawas are the silent heroes, doing the drudge work and being closer to correctness while the Topols get rich sitting on his twitter (and bluesky) drones”. Maybe too fringe, but I feel where this friend of mine is coming from.

*In a dream

**as a proud LibreOffice Suite user I have no idea which proprietary spreadsheet software this came from and am making this specific fact up.

***again, - - was typed, not AI written, — is a useful punctuation

****Priors, in British English

**5† In my haughty opinion

†Note that this is an asterix, followed by the use of an asterix as a “multiply” symbol, followed by a five, i.e. *****. I’ve since resorted to using † to avoid further confusion amongst my many readers.

**mk2 I really don’t like that software. I had to use a proprietary alternative to easily get this count.

††Kennings, I read recently read, are a form of language development w.r.t. English, etc. I had introduced “blogeffort” in this blog in an attempt to improve the Language with a novel contribution, but alas, https://carolinapublican.blogspot.com/2007/05/?m=1 evidences that Tom S.’s contribution predates mine by almost two decades (although can’t find anything since), with “I have put a great deal of blogeffort into bitching about mass, music, etc. And I am quite sincere in my bitchiness.”

𐤀‎Edit: To understand this reference you have to have read the original post, and also many of Taleb’s works, and formed a similar opinion of Taleb. This refers to Taleb.

EDITS AND ERRATA

which I will always include since every document I make tries to be a living document,

1. Upon posting, I got the option for this

Which opens an interesting dilemma. Do I? I don’t dislike the guy, I don’t even really disagree with him in this specific case. In fact, I am kind of spending a long while writing this where I could have done exactly what I called him out for not doing.

But I feel he has enough clout and the fact that this website automatically prompted me to do this implies that this website gets money from me doing this, and although I happily return shopping carts I’m not helping out this website in its current form.

2. One of my many fans exhorts me to simply answer:

“Do you dispute Eric’s conclusions”? “I just need to know how much brotein [sic] I should eat”

To which I reply: have you even READ the above? (evidently not, since he didn’t have the link at the time). But do I? Here’s my actual belief, conviction, “STRONG OPINION LOOSELY HELD”, etc. and how it’s changed. I was aware of the kidney risks, loosely. I had a phase in which I did protein powder, but now am off that phase. I have intermittent bursts of motivation in which I do resistance training, and in those, was nervous about not hitting a somewhere quoted 1g/lb/day, which is arguably more than 1g/kg/day. Reading the meta analyses, I am less nervous about not hitting those. Topol’s post led me to those. I will also fully disclose that one of the AI vendor’s products reaffirmed the need for lots and lots of protein when asked casually, and I believed that number more than I should have given that I could have just looked up meta analyses. But I wanted a quick answer, and that was one of very many I got in that chat session.

So no, don’t dispute the conclusions, mostly just take issue with the massive problem of science communication balancing being really effortful and honest and self examining vs. actually reaching the masses. And I eat as much protein as I feel like and will be less nervous about it now given what I’ve read.

3. ALSO I FORGOT THE ABSOLUTE MOST IMPORTANT PART OF THIS ENTIRE DEBATE, FOR ME:

I love food. I love thinking about food, I love cooking food, I love buying food, I love smelling food, I love food from here and food from there, I love different sorts of food — according to actuarial tables I’ve got like 50k meals left until I die. I would like to not have to have too many of them be BLENDED CHICKEN and protein powder. As another friend of mine said, ”[protein powder] takes the sweetness and light out of smoothies in a clinical way”.

4. Edit 4: I KNEW I wasn’t hallucinating blended chicken. LOOK. PUT YOUR GLASSES BACK ON AND SEE THE TRUTH.

5. Final (?) Edit. Some of my readers find my work difficult to parse, claiming this very work “does have a way of encouraging your eyes to slide off of it”. Perhaps some additional punctuation would help. I put enough here and you can pepper and salt them as you please.

U+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000AU+000A.

6. After careful deliberation I feel the need to explain the above little joke. It’s a reference to the following.

Alas, like a frog, dissecting a joke helps elucidate but, as a result, the joke is dead.