ProPublica: A Revealing Email Exchange [TLDR]

Part 2 in a series about what we see when journalism's gold standards go under the microscope.

I’ve just published a 5,000 word story about ProPublica. It’s about a 20-minute read. This is the ~800 word version in which I make a case for why the full version may be worth investing into for some, and to supply the gist to everyone else.

As context, there’s a line from the Old Testament / Tanakh that I think about a lot:

The first to plead his cause seems right, and then his neighbor comes and examines him. - Proverbs 18:17

The moral here is so obvious as to almost be unworthy of comment. The first side of a story can sound really compelling while also being really misleading and/or wrong.

That idea was bouncing around my head as I thought about how to present this story. While my corrections policy penalizes me for unfair framing, it’s still true that I’m generally only giving my side in my posts. I make a case for where a journalist went wrong. They might disagree but feel disinclined to engage or comment.

So for this story I tried something new: making my email exchange with a lead reporter (and to a lesser extent his editor) into the bulk of the post. That way you can read takes from both sides in sequence and make up your own mind.

The basics of the story are:

• Two local hospitals seem to have different outcomes so far as premature / extremely low-weight newborn mortality — one good and one bad

• This variance could be explained by differences in care and/or differences in patient health upon arrival (in terms of expected viability)

• To investigate the latter, we'd want to look at how the mothers-to-be are being routed to a given hospital to see if there's any bias there

• ProPublica acknowledged this patient-health-bias potential, but elected to not look into routing dynamics at all (both before my exchange and after, despite the data suggesting it was relevant here)

• Their resulting story was extremely negative regarding the “bad” hospital, strongly positioning them as putting profits above newborn lives

But there are subplots:

• ProPublica (the same lead journalist actually) had written about the “bad” hospital last year

• That prior reporting was driven by anonymous clinician testimony, where the most potent charge ended up being wrong and needing to be retracted (leaving a concerning but much less sensational story)

• (I found ProPublica’s resulting correction bad/underwhelming, and think they should have disclosed this whole episode in their second story)

• This second story was also based on anonymous clinician reporting (perhaps the same one(s); we’re not told)

• Some of the interviewed clinicians disagreed with the reporting, but we’re not told how many or about what (and ProPublica ignored my questions there)

• ProPublica wouldn’t confirm if those clinicians ever raised their concerns internally prior to leaking them (or if their journalist had ever asked)

This isn’t to say that anything nefarious was going on. ProPublica’s work on both stories was high-effort (compared to industry baselines) and I genuinely believe that their intentions were to fairly investigate two consequential stories.

I just feel they came up short, and not by a little bit. You can both put in a tremendous amount of work and come up only partway. It happens. I think their structure and standards failed them, which is an industry-wide problem.

To give a taste of how our exchange went, one comment from their editor:

Their answers to you were a good-faith attempt to [answer your questions].

But in reality:

• They declined to share their data

• They didn’t meaningfully acknowledge a single concern

• They didn’t meaningfully answer a single question

• They declined to revisit their reporting

• They instead asked who I was working for

• When I sent them the full story draft, their entire comment was “I think we’ve said what we’re going to say. Thanks for reaching out.”

Anyway, I don’t want to narrate this more. You shouldn’t take my word for any of it. The long version exists because I think showing the full exchanges gives a good sense of how difficult it is to get journalists to engage with critical questions about their work, and also because I think it gives them the fairest hearing here.

You might read it and come to different conclusions. I welcome all discussion to that end. And the door is always open for ProPublica to comment. I’ve given them avenues and incentives to do so — and I hold out hope that they’ll eventually use them.

The full version can be read here

[EDIT 05/20/21 - I edited one of the bullets to make it clear that the data presented — specifically on effective DOAs — suggests that routing imbalances were in play. Hence why it’s so concerning that ProPublica didn’t pursue that line of inquiry further.]

---

For part 1 of this series (about The New Yorker), see here. I have no immediate plans for a part 3, but will add one as the spirit leads me. My current sense is that adding more may be duplicative. These really are the gold standards, and my stance is that this pair of stories give a representative sense of what the best really look like.

To the best of my memory, I’ve never been able to get a journalist to run a single correction. And you can see my approach in full detail in this story.