ProPublica: A Revealing Email Exchange [Full Version]
ProPublica: A Revealing Email Exchange [Full Version]
Part 2 in a series about what we see when journalism's gold standards go under the microscope.
People often ask me which news outlets I feel are still trustworthy, to which I’m never able to give a very satisfying reply. As readers here will have a sense of by now, all prestige outlets are capable of deeply flawed and irresponsible work, and even the most vaunted editorial regimes don’t seem to be nearly curious or critical enough to consistently establish the substance of that which they give their stamp to.
But as caveat of sorts I do often list the outlets that I feel are trying, who if nothing else are at least committed to chasing stories that actually matter. And ProPublica tops that list for me. They’re rightly lauded for chasing unsexy stories of genuine civic importance that require a lot of shoe leather. They routinely aim high.
Even so, they’re ultimately subject to the same forces as the rest. Putting them in first doesn’t mean that I don’t have serious reservations with some of their work. It just happens less often, and to a somewhat lesser degree.
But a claim like this demands justification. Hence this story, which I’m going to tell in the form of a series of emails between myself and the lead author of a half-dozen ProPublica articles about a hospital in New Mexico. My hope is that the exchange will be instructive in demonstrating the difficulty of getting journalists and editors to revisit their work, and of the gap between the best and what we really need.
Perhaps after reading you’ll come to different conclusions than I did. That’s ok! I’m happy to hear about it! But at the very least you’ll see in a fuller light.
We reward corrections. See something wrong, misleading, or unfair? Say something. It helps. New readers can also learn more about our mission here.
Also, as both trigger warning and story note, this article gets into extended (though not graphic) commentary about premature infant deaths. While there are few stories more weighty or personally tragic to those affected, I’ve gone for a clinical tone throughout. Use discretion.
This is the full version of this story. If you’re looking for a TLDR summary, see here.
Context
ProPublica — in partnership with New Mexico in Depth, a local affiliate — published a set of four articles about Lovelace Women’s Hospital in March/April, where the gist was that premature and extremely low-weight babies seemed to be dying at meaningfully higher rates vs. a comparable hospital. The thrust of my questions was towards establishing whether ProPublica was justified in how anti-Lovelace their framing was relative to their findings (especially in context of their prior history).
The meta-concern here is the degree to which journalism is being weaponized by aggrieved workers to unreasonably drag their employers. While we’d all (hopefully) agree that the press has a vital function in holding power to account, we’d all (hopefully) also agree they can’t be a mediator of first resort. The time for journalists to use their enormous critical influence is when appropriate prior steps haven’t borne fruit, at which point the power of the press becomes a necessary corrective.
But even if you don’t agree with that philosophically (perhaps feeling that it’s fine for newspapers to write about whatever is leaked to them regardless of moral hazard), there still needs to be a minimum discernible sense of fairness. And even if the employer isn’t cooperative, there’s still a basic duty to ensure that all parties get a fair shake. Because the point of journalism is light-shedding, not prosecution.
Email #1:
[I addressed these emails to the lead reporter and a contributing colleague. All replies are from the former. While you can trivially find his identity, I’m not using his name here just to avoid this ever becoming a result when someone Googles him. While I stand by all my concerns here, this really isn’t personal.]
From me, April 13th, 4:58pm [all times Pacific]
Hey —
I run a journalism review Substack, and I’m currently working on a piece that touches on your Lovelace v. Presbyterian story.
While I appreciate the reporting (and love that this kind of story is being seriously investigated), my concern is that you say...
“The cause of the disparity in Lovelace’s and Presbyterian’s extremely preterm neonatal death rates is not clear. Possibilities include one hospital having a sicker patient population, differences in patient care, or both.”
...without really ever giving any oxygen to that sicker population thesis in terms of routing biases. I read through each of the main story and the “How We Investigated” supplement twice, and I see nothing in there about what causes these patients to be brought to one hospital vs. the other. Is it influenced by insurance coverage? Geographic proximity? OB/GYN preference/privileges? Ambulance discretion? Costs? Expected viability?
*If* I’m parsing your data correctly, Lovelace lost a much higher % of these babies within the first hour post-delivery. Assuming that this is less an indication of care quality than it is non-viability, I’m a bit stumped as to why there’s no commentary about what might be driving the overall disparity from a distribution POV?
I have a list of other questions/concerns (each minor, though seemingly biased in the same direction). But wanted to start with the big one.
(These checks are adversarial by nature. I’m always sorry about that on a personal level. But I’m highly interested in accuracy and fairness. As y’all have good answers, I’m eager to weigh them. And in this particular case — which is about reviewing gold-standard journalism — I have a lot of good to say as well.)
Jeremy
The Save Journalism Committee
[Just to reinforce the obvious here: if there are two possible causes of a bad result, and only one of them is within the hospital’s control, it’s important to dig into both. If Lovelace was getting riskier pregnancies for some reason, we’d expect them to have a higher fatality rate. But ProPublica basically admitted “ah yes sure this all could be a routing imbalance” without ever looking into routing!]
Email #2:
To me, April 14th, 3:28pm
Hi Jeremy,
Thank you for reaching out. We are working on a reply but won't have it done today.
Best,
Email #3:
From me, April 14th, 3:30pm
That’s fine. Happy to hold so y’all can give it a fair shake, etc.
Email #4:
[I then realized in re-reading my original email to them that something funky had happened, maybe because I’d drafted partly on my phone.]
From me, April 14th, 3:39pm
As an aside, when looking at the thread here I see a weird formatting issue where the last three paragraphs of my original email are only showing to me if I highlight them. No idea how that happened, or if it looks the same on your end. But seems consistent no matter which device I look at, so copying/pasting the original below just in case.
Email #5:
To me, April 14th, 5:18pm
Hi Jeremy,
Thanks for pointing out the white-text paragraphs.
Can you please send the list of your other questions/concerns, as well?
Best,
Email #6:
[This is the long one. Sorry. I’ve added a few notes in square brackets and italics where I’m clarifying something here that would have been obvious to the reporters. Anything in parentheses — i.e., curved brackets — was in the original email. If you’re a skimmer, the first part is the most important.]
From me, April 15th, 1:33pm
Yep. See below.
[The articles I’m going to reference were written by the same reporter about the same hospital last year, where anonymous clinicians had reported that Lovelace used racial profiling in assigning COVID tests to pregnant woman upon admission, leading some mothers to be separated from their newborns when results didn’t come back pre-delivery. While the state’s findings suggest administrative confusion around a well-intended policy that was applied roughly in places, the anonymous claims were later proven hyperbolic and the hospital’s actual sins weren’t in proportion to “A Hospital’s Secret Coronavirus Policy Separated Native American Mothers From Their Newborns”. It’s the difference between institutional racism that comes from failing to fully consider the POV of the disadvantaged and racism that’s actively callous. Both are bad, but they’re very different types of bad.]
While many of these [points] would individually be quibbles in a normal context, my concern is that there are a lot of them and they all seem to be prejudicial against Lovelace. There's also just a lot of anonymous speculation. While that also can be fine, in reviewing [your] story about Lovelace's zip code policy, one such anonymous Lovelace clinician retracted their original 6-to-1 over-estimate after being contradicted by the CEO's memo, with the memo being backed up by NMDHS's findings. Given that neither the retraction nor the memo were ever edited into the original story, nor into the August follow-up, it kinda feels like the deck is being stacked a bit against Lovelace. Same for using the plural in describing maternal separations in those articles when apparently there was only one that had anything to do with zip codes. So while I don't question that either story was worth pursuing, or that Lovelace has been less than forthright, I'm a bit stuck here on fairness.
[The original story was corrected in a vague sense. But the replacement text was still misleading/wrong being plural, the correction wasn’t noted in-line, and the edit note they left was placed at the bottom after a contact form where it was unlikely to be seen. Said note also failed to explain what led to the retraction.]
Anyway, in order (excerpts in bold; my questions in plaintext):
Primary Story[Meaning I’m excerpting from the main article.]
Another is a culture of silence at the hospital when things go wrong: “We don't even talk about it within the NICU, but especially to the parents.”
Were other Lovelace staff asked about this description? Did they agree? It's a strong claim to source to a single anon.
They have only rudimentary policies in place for micropreemies, but not nearly as comprehensive as things that I've seen at other hospitals,” the second Lovelace clinician said.
Were the clinicians asked what they'd done to flag this apparent gap with management? If so, how did Lovelace's administration respond?[I want to call attention to how important these questions are, not just as a general journalistic practice, but in context of this specific subject. If these clinicians had flagged that newborns were dying and the administration just waived it off, that’s a really big deal. But ProPublica has never confirmed if they even asked.]
The clinicians were two of eight current and former Lovelace care providers who spoke to New Mexico In Depth and ProPublica about newborn care at the facility on the condition that they remain anonymous because speaking publicly could hurt their employment within New Mexico’s small medical community. Not all of them were critical of the hospital.
Why not number how many didn't share the criticisms, or get more granular about where they disagreed? You could be describing a 7-to-1 situation, a 4-to-4, etc.
For example, New Mexico In Depth and ProPublica identified a discrepancy in Lovelace’s reporting of cases of NEC, the dangerous intestinal condition.
Was this also true at Presbyterian?
The number of extremely preterm twins and triplets, who often fare poorly, also did not explain the death-rate disparity. Nor did differences in maternal race or ethnicity, prenatal therapies or other potential risk factors for extremely preterm babies, including the proportion of boy births, teen mothers, mothers who underwent infertility treatment or induced labor, or mothers who had cesarean-section deliveries.
"Did not explain" is a little vague. Did the gap close at all? Can you share the summary that the health department provided you?
Extremely preterm babies who survive delivery should always eventually be admitted to NICUs, experts said, but the news organizations found infants for whom there was no record of a NICU admission or a death certificate.
Did Presbyterian have fewer or more of these? (And is the assumption that 100% of them survived / were sent home healthy without need for NICU admission?)
Lovelace is sometimes too slow to send babies in crisis to UNM Hospital, where surgery can be performed if needed, four clinicians from both Lovelace and UNM said.
"Both" is unclear here. Four each? Four combined? If combined, what was the split between hospitals? And were the clinicians in relevant units? And what did the 5+ other Lovelace NICU clinicians interviewed have to say about this? Did some disagree?
Hospital price sheets suggest care for these babies may bring Lovelace more than $1.2 million per baby from insurers.[The implication here is that Lovelace was aggressively pushing to receive more high-risk deliveries because of the money involved despite their negligence in establishing proper practice and staffing to support those deliveries, causing unnecessary deaths to newborns.]
Feels like there should be a caveat here of the traditional gap between price sheets and actual reimbursements. (A quick look through the literature suggests actuals were maybe 1/4 of that in the mid to late 2000s. While there's surely been some price inflation, and surely stay-length is longer for the very worst cases, there's just no way that these babies are bringing in anywhere near $1m on average, much less $1.2m+.)
“It is no secret that the NICU is Lovelace Women’s Hospital’s golden goose..."
While I get that you're quoting here, including speculation is an editorial judgment, and you yourself say "brought a lot of money to" in the paragraph prior without putting that ~$20m/year (which is what % of their NICU revenues?) in context of overall hospital revenues (or margin contribution per department, etc). A hospital of ~750 employees is likely to be bringing in significant income. Is their NICU a real golden goose? Could be, I guess. But that's an assertion in need of support.[To be clear, I don’t know the answers here myself. It very well could be their cash cow. And again, if Lovelace’s administrators really were/are just trying to goose revenue at the cost of the most vulnerable lives entrusted to their care, they deserve severe condemnation. But the extraordinary moral weight of that insinuation demands extraordinary care.]
That could put it in competition with UNM, where extremely preterm babies can bring in more than $2 million per infant.[UNM is the local university hospital, a level 4 unit to which level 3 units like Lovelace and Presbyterian send on babies they’re unable to adequately care for. Lovelace had apparently tried to become a level 4 themselves. But this is a bit misleading, as a change in level wouldn’t in itself change which babies they feel unequipped to care for.]
The implied argument here is that getting bumped to Level 4 would allow Lovelace to bill at a higher rate. If true, what's that premium specifically? (This is important because UNM, to the degree that they are making more per patient, could be doing so because of longer stays, higher incidents of surgeries, and so on — none of which would accrete to Lovelace just by a level change in itself.)
New Mexico In Depth and ProPublica asked eight current and former clinicians who worked at the Lovelace NICU over the past decade if they had participated in M&M case reviews at Lovelace. None had.
Was Lovelace doing any other sort of post-mortem review (i.e., something short of a formal M&M), or just nothing at all?
Supplement[Meaning I’m now excerpting from here, again in bold.]
The CDC’s national death rate might be higher than a rate for only level 3 hospitals would be, because it includes hospitals with level 1 and 2 neonatal designations, which are less equipped to care for these babies.
I could say something like "Level 3s ought to have higher rates than 1s/2s because they're more likely to see high-risk deliveries routed their way", and it would be just as facially plausible. Why include speculation?
Excluding all babies who died within an hour of delivery, regardless of gestational age or resuscitation reports, Lovelace’s death rate was 26%, compared to 18% at Presbyterian and a CDC-provided national rate of 22%.
I found this part a bit confusing, in that "regardless of gestational age" made the reference point a bit unclear (to me, at least). Are we still looking narrowly at just 21-23 weeks here still, or are we back to the 84 premies? If the former, no issue. If the latter, it would imply that Lovelace saw a much higher % die more or less upon arrival, which would imply a sicker patient population.
Presbyterian officials did not object to a hospital-wide comparison ... Presbyterian refused to disclose its NICU-only death rate.
This is why I think the major point about routing/distribution matters. If Presbyterian is dealing with a healthier population (i.e., fewer non-viables / likely delivery-room deaths), that would skew things considerably. And it would explain why Presbyterian declined to provide a stat that would contextualize Lovelace's higher rate. (From Lovelace's numbers, it seems 11 of their 30 lost low-weight babies were lost outside the NICU. If that's because they were non-viables, that might explain much of the gap.)
Goodman rejected Lovelace’s objection to hospital-wide comparisons, noting that NICU staff should be present at the delivery of extremely preterm babies...
Sure. But this seems to have been a problem at both hospitals, and it's an aside away. If a baby is non-viable, admission or non-admission to the NICU won't matter either way.
But we found that the distributions of births by week of gestation at the two hospitals were very similar.
With such small data samples, any gap here could be meaningful. What was the gap precisely?
[Ok, you’re over the hump now. Home stretch.]
Email #7:
[A while goes by without a reply, so I check in.]
From me, April 27th, 6:56am
Hey folks. I've been happy to sit on this to make sure y'all have sufficient time to give this a fair shake, etc. Do you have a sense of if/when you'll be replying?
Email #8:
To me, April 27th, 8:30am
(Thank you, Jeremy — my apologies if this didn't reach you previously. Below is our response.)
[An email was pasted in that I’m bumping to the next section for reasons you’ll see in a second. But I do take this to mean that they’d meant to reply earlier.]
Email #9:
[Bits in italics and brackets are again my comments here now.]
To me, April 27th, 8:48am
Jeremy,
Resending below. There was a copy/paste typo in my previous reply.
Best,
--
Hi Jeremy,
Thank you for your questions.Can you please disclose which, if any, of your clients have a direct interest in this reporting?
[I imagine this was their legal team speaking. But the insinuation here is outrageously offensive. And if I’m disqualifyingly dishonest enough to have not disclosed this prior, why would I be honest now? It’s just such a bizarre question to ask.]
We carefully sourced the story and the clinicians we spoke to had firsthand knowledge of the things they described.
[I’m sure they did/do! But it doesn’t mean they didn’t exaggerate in the same way they did in the last story! It could even be the same primary source(s) for all we know!]
We had hoped the available data would allow us to assess more factors/hypotheses. Incomplete data and inaccessible data posed considerable challenges to exploring possible causes of the two hospitals’ death rates disparity. (We describe those challenges here, here and here.)
[Those are the other links I pasted above. None are new. They describe difficulties getting data, especially from Lovelace. But given the reporter’s history there I’m not enormously surprised. Also note though that this elides my central concern about routing. The factors by which certain pregnancies are routed to certain hospitals is something they could have looked into without Lovelace’s help. It does’t seem like they ever did.]
As we stated in our report, using the available data we were unable to determine whether patient population differences, or clinical practices, or both contributed to outcomes. However, the disparity in death rates of extremely preterm babies persisted when, using the data that was available to us, we ran our analysis in different ways trying to account for possible differences in patient population.[Again eliding my actual questions, which were about how/whether the disparities changed at all in different conditions — not whether a gap still remained. With such small data samples, closing the gap even halfway would make this a much less compelling story.]
We asked Lovelace to respond to their clinicians’ comments, to answer our questions about M&M reviews, etc., and they declined. The hospitals knew our findings and had opportunities to respond well ahead of publication.
Because Presbyterian has on-site medical and surgical subspecialists, babies with high-risk conditions, including heart defects, are frequently transferred there for care.
To clarify a few things from your questions:
Our death rates included only babies who died at their birth hospital. They did not include babies who were born elsewhere and were transferred to Lovelace or Presbyterian for care. Our death rates included deaths within 28 days of delivery, not within an hour of uu [sic] delivery.
We did a sensitivity analysis that excluded babies who died within an hour, and found that even when these perhaps less-viable babies were excluded, Lovelace had a higher death rate.
[The first bullet here confuses me, as I hadn’t asked about it. And the second again fails to tell us anything. Assuming that the gap narrowed by a meaningful amount, this gives us a clue that patient health is playing a role, and thus that routing is a culprit here.]
Email #10:
[Their response honestly flabbergasted me, so I took a few hours to consider how I wanted to reply. It occurred to me that the fairest thing might be to just speak candidly and make the exchange itself into the story.]
From me, April 27th, 10:59am
Wow, ok. To answer your question: no, I don't have any clients with any kind of interest here. (Also zero of my writing for my Substack has any kind of undisclosed relationship with my client work, because, well, that would be a pretty wild and obvious conflict of interest / ethics violation!) That you'd ask that is amazing tbh, and will absolutely be part of my story now.
When you say that "babies" with high-risk conditions are transferred to Presbyterian, that seems an aside, as the delivery has already happened?
Anyway, nothing else in your email really answers any of my questions. Not sure if it was written by legal or what, but I think showing it in the piece next to my questions more or less is the story. Because what I'm left with is that y'all basically did a truly impressive amount of classic reporting (for which I genuinely applaud you), but then filed a prosecutor's brief without seeming to show anything close to an equal interest in what the other side might look like. And when I brought those concerns to you, your reply was to ask me who I may be secretly working for?! But I'll do you the courtesy of not responding in kind by asking whether Presbyterian paid you, because obviously there's no reason for me to assume that you're being malicious to Lovelace in some unethical way! My point is just that your presentation reads to me as unfair, where fairness is a reasonable thing to want from journalism. And as I said, I don't question that Lovelace was withholding in their answers. But, well, that's also true of you! They have their reasons for why. You have yours. I'm happy to let readers judge.
But I am interested in fairness. So I'm happy to give y'all an early look at the piece so you can rebut/comment. The thrust will be something like "per their reputation, ProPublica is a gold standard org that takes on unsexy, civically-important stories and chases them with real effort; even so, here are teardowns of two stories to give a sense of where they fall on the neutrality spectrum, make of this gold standard what you will".
[This is where I stopped in my draft, which I then sent them to review.]
Email #11:
[I sent this in a fresh thread to both reporters and their editor, as the latter had emailed me in the intervening time. See note on that at the end.]
From me, May 17th, 11:34am
Hey folks -
Here's a private draft link to the story as it stands today. I may yet adjust some wording, or offload bits to an appendix to make for a lighter read. But the gist is there.
While I'd like to run it Wednesday morning, I want to ensure you have the time you need to reply thoughtfully. So if that's not feasible, let me know what is.
The easiest way forward for continuity is just sending me back an email with your thoughts, which I can pop into the story in sequence. But I'm open-minded if you perceive a better way.
Again, I know this is all both a nuisance and something of an affront. I don't imagine anyone enjoys having their work put under the microscope. And it's unfair that it's you. I was looking for a ProPublica story to write about, and y'all had the bad luck of showing up on my Twitter feed.I've at least tried to be fair. As I can improve there, I'm open to that feedback too.
[Note that I did very lightly revise this after sending it to them, just for readability. But I kept a backup of the original, which I’m happy to share. No funny business.]
Email #12:
From me, May 18th, 4:05pm
Hey folks - It's EOB Tuesday and no reply here yet. Are you still working on it, or am I to understand that no statement is forthcoming?
Email #13:
[This reply came from their editor. The primary journalist hadn’t/hasn’t engaged since his April 27th email about who I might be working for.]
To me, May 18th, 4:09pm
I think we’ve said what we’re going to say. Thanks for reaching out.
Email #14:
From me, May 18th, 4:20pm
This really saddens me. But I'll respect your wishes. And I again applaud you all for the work invested here. While I obviously disagree about whether that work crossed the finish line, that isn't to diminish the many, many hours that you spent looking into a very important question. As I do my final edits, I'll do what I can to stress that point.
[And it does really sadden me. I’d hoped when I hadn’t heard for a day that they were relitigating internally and that there was a chance that they’d revisit or at least opt to answer some of my questions. What they went with was akin to standing firm in draw poker with a pair of deuces when their opponent has a made flush face-up on the table. I don’t get it. I’ll probably never get it. But as I say, I’m left balancing two wildly conflicting emotions. I find their overall stance ethically untenable while also genuinely admiring the work that they did do. It’s a weird mix.]
Afterword
Quoting from a prior writeup about another ProPublica story (about Congress's purported attempt to “ban the IRS from offering free online tax filing”).
Some believe that every journalist’s responsibility is to the effect of “collect some facts, avoid outright mistakes, and work with an editor to make your story marketable”. But this is the equivalent of requiring them to “tell the truth and nothing but the truth” while leaving out the bit about “the whole truth” as either unimportant or impractical.
Collecting the whole truth here would have been hard. The reporters wanted data they couldn't easily get or reconstruct. Even so, there was no discernible legwork at all in one obvious area. They could have asked how pregnant mothers get routed between competing hospitals. If the answer was “randomly”, fair enough! But not only do we have no reason to assume that this is true, the single relevant data point presented (along with Presbyterian's convenient silence) suggests that it isn't!
More to the point, they’re still refusing to look into this, despite this exchange, and despite their prior history with the hospital in question (which is never disclosed in these later articles, and where their “correction” was underwhelming in the extreme).
That this is the gold standard ought to give us pause.
PS - Their editor reached out to me on April 29th. I guess the reporter(s) forwarded my thread with them. Which is fine. You can read that separate exchange here if you like. There’s some extra commentary in there about public vs. in-house editors.