How to Summarize Research News Responsibly

To summarize a research story responsibly, write from the study, not the headline or the press release, and keep three facts in your post: who or what was studied, what kind of study it was, and how big the effect was. Those three facts are usually short, and they're exactly what gets lost when science news is compressed into a post.

Research summaries matter more than most because people act on them. A post that says a supplement "prevents dementia" when the study found a weak association in older men can change what someone buys or stops taking.

Exaggeration usually starts upstream

It's tempting to blame headlines, but a lot of overstatement starts before journalists touch the story. A 2014 study in The BMJ looked at 462 health-related press releases from 20 UK universities and the news stories based on them. It found that 40% of the press releases contained exaggerated advice, 33% made causal claims that went beyond the research, and 36% stretched findings from animals to humans.

The exaggerations flowed through. When a press release overstated a causal claim, 81% of the related news stories did too. When the press release didn't, only 18% did.

The lesson for anyone summarizing: the press release is a marketing document. Use it to find the paper, then read the paper's abstract yourself.

Find the paper

A good news article names the journal, the lead author and the publication date, and ideally links to the study. If it doesn't, search the journal name with a distinctive phrase from the article. Look for a DOI, a code that begins with "10." and identifies the paper permanently.

You don't need to read the whole paper. The abstract and the limitations section, often a paragraph near the end of the discussion, tell you most of what you need. If the paper is paywalled, the abstract is almost always free.

If the "study" turns out to be a survey run by a company selling the product it's about, say so in your post, or don't share it.

Note who was studied

Write down the population in a few words: "mice", "1,200 nurses in Denmark", "40 college students", "patients with type 2 diabetes". This single detail prevents most scope errors. A finding in mice is a reason for more research, not a health tip. A study of college students may not apply to retirees.

"In mice" costs 8 characters. It's the best value in any science post.

Note what kind of study it was

Different designs support different claims. In rough order from weakest to strongest evidence for cause and effect:

The most common error in research summaries is using the verbs from the bottom rows for the studies in the top rows. Observational research almost never justifies "causes", "prevents" or "boosts".

Report the size, not just the direction

"Lowered risk" tells readers the direction of an effect, not whether it matters. Look for the absolute numbers. A drug that cuts the risk of something from 2 in 1,000 to 1 in 1,000 has "halved the risk", which sounds dramatic, but it helped one person in a thousand.

If the article gives only relative figures ("30% lower risk"), say that, and avoid adjectives like "dramatic" that the numbers can't back up. If the effect was small, "slightly" is an honest and short word.

Keep the status of the work

Mark where the research is in its life. A preprint has been posted publicly but not yet peer reviewed, so say "preprint" or "not yet peer reviewed" in the post. A conference abstract is often based on early data and is short on detail. A peer-reviewed paper has been checked by other researchers before publication, which catches some problems but not all. A replicated finding has been confirmed by other teams, and that is the point where "studies show" becomes fair.

A single new study, however good, is one data point. "A new study suggests" is more accurate than "science says".

A worked example

Suppose a news story with the headline "Daily walk cuts dementia risk by a third" reports on an observational study of 5,000 adults over 60, followed for eight years, in which people who walked daily were about a third less likely to be diagnosed with dementia.

A careless post:

Walking every day cuts your dementia risk by a third, new study shows! [link]

A responsible one, at about 220 characters with the link:

In an 8-year study of 5,000 adults over 60, those who walked daily were about a third less likely to be diagnosed with dementia. Observational, so it shows a link, not cause. (via Example News) [link]

The second version has the population, the design, the size of the effect and the source. None of it is jargon.

Using a summarizer on research news

An extractive summarizer like TweetSum is handy for finding the sentence in an article that states the main result, especially with the "keep numbers" option on. Watch for two problems. It may pick the headline-style sentence from the opening, which is often the least careful line in the piece. And it can't tell an observational study from a trial, so it won't add the design for you.

Use the summary as a starting point. Then add the population, the study type and the source in your own words, and check the post against the abstract before you share it.

Questions