Sorry, we could not find the combination you entered »
Please enter your email and we will send you an email where you can pick a new password.
Reset password:


By Thomas Baekdal - November 2019

More studies that don't mean what we think they mean

One of the ongoing things that I often talk about is how often we see studies about the media industry that don't actually mean what people think.

In particular, I have noticed a number of these over the past couple of weeks. Let me just mention a few of them that I see as particularly important.

The first study was reported by, which told us that:

Users of The Daily Telegraph's WhatsApp audio briefings are 12 times more likely to convert to paid subscribers than an average homepage reader, according to the briefings' editor Danny Boyle.

So, what do you think this means?

Does this mean that you should pivot to WhatsApp? Does this mean that WhatsApp is an amazing channel? Does it mean that you shouldn't focus on your website as much?

No, it means none of that because this number doesn't really mean anything by itself at all.

Let me explain.

Anyone who has worked with conversions knows that there is a funnel. At every step of this funnel, you lose more and more of the audience, until eventually, maybe about 2% decides to convert (but that really depends on how you measure this).

Like this:

What we also know is that this funnel is incredibly important, especially for publishers.

Unlike a web shop where people can just come across a random product and then decide to buy it almost on impulse, subscribing to a newspaper is something you need to build momentum for.

You don't just come across a random newspaper that you have never heard about and then, right there, decide to subscribe to it.

In fact, if we look at a newspaper like the Daily Telegraph, I can guarantee you that every single person who has ever subscribed to it, has known about it for years, and will have come across it, for most of their lives.

They have probably already read 100s of articles via the website over the years. They have seen links to it on social channels. They have seen and maybe even read the print newspaper on several occasions. And yes, a few of them have now also followed them on WhatsApp.

And every one of these steps has been critical to the conversion.

I'm reminded of something I tweeted a while back about the launch of Disney+. As you may have heard, Disney+ managed to gain 10 million subscribers on the day that they launched, which was spectacular (but not really surprising), and have continued to grow quite strongly ever since.

But there are six ways of thinking about the Disney+ launch:

It wasn't just the launch on the first day that caused this, every one of these contributed immensely to the launch of Disney+. Take any one of these away, and they wouldn't have gotten 10 million subscribers in a single day.

It's the momentum that made this such a big success, not any one specific thing or channel.

So when I see publishers go out and say "OMG WhatsApp has a 12 times higher conversion rate than our website!", this is simply not true.

Their website was part of that funnel, and your website is likely leading people to several other funnels that also cause conversions.

And, on top of that, it's just a really crappy way to do math. They are talking about a much smaller part of the overall audience, and comparing that to the conversions overall.

This doesn't mean that WhatsApp isn't important. Every channel that engages and gets you closer to your audience is important.

The next study I want to talk about is one from Reuters Institute.

They measured the proportion of people who participate around news online. And the result was this:

The study itself is fine. We can see that 48% participate overall, but there is quite a big difference between how this is done.

But here again is the question, what do you think this study means? Does it mean that you should focus more on 'online polls' and less on 'sharing via email'? ... or do we even need this at all?

Well, let me challenge your perception of this for a moment. Think about this in relation to print and how massively higher this engagement is compared to traditional media.

For instance, how many people participated in a poll in a print newspaper? Well, none.

How many shared a print article? Well, some people do, but the rate of this is extremely low.

How many people liked a print article? Well, you can't really do that.

How many people posted a comment on a print article? Well... none.

My point is, think about what participation means. What does it really mean to be engaged with something?

Notice how I phrase this slightly differently. To be engaged with something is different than liking something on Facebook.

Let me give you an example. Here you see a kid playing with LEGO (or Duplo in this case).

She is engaging with LEGO. In other words, she is enjoying spending time with it, using it, finding an interest in it, etc.

This is what it means to engage with something. It's a dedication of time, use, and value.

Compare this to if she just went onto Facebook and liked a random post from LEGO. Sure, that is also a form of engagement, but it's a much weaker form.

This is something that is very important for publishers to understand. Over the past several years, we have seen many publishers talk about audience participation and audience engagement, but you aren't really talking actual engagement. You are talking about things like how many likes you get, or whether people posted a random comment because they were outraged by the news story.

Actual engagement is something very different. It's more than likes. It's about what people do when they use your publication/product/service.

The study itself is not bad, but it's kind of a distraction.

The final study I want to mention highlights the problem of journalistic bias. I don't mean political bias, but bias towards focusing on the negatives.

The study itself is actually from last year, but I noticed several media people tweeting about it again last week.

It all started with an article in the Economist with the very scary title:

How heavy use of social media is linked to mental illness: Youngsters report problems with anxiety, depression, sleep and FOMO (the fear of missing out).

This certainly sounds very scary, but then you look at the article, and you see that it is based on a study that looks like this:

You see the problem here?

What the Economist has done (and many other journalists often do) is mostly ignore all the positive sides of things, and then try to make it sound like everything is terrible. Essentially, what they did was to just cut out all the positive, and present an article focusing on this:

This is a problem because look at what the study actually says. For instance, it reports that social channels might increase the feeling of loneliness. But then on the positive side, people also say that it helps foster real-world relationships.

It tells people that social channels have a negative impact on people's body image, but on the positive side it also encourages people's self-expression.

It says that it causes problems with bullying, but then on the positive side it also helps people with emotional support.

It says that it causes problems around sleep, but then it also creates positive awareness about people's health.

You see what is happening here?

This study didn't find what the journalist reported. Instead, it was discovered that social channels are very complex and diverse networks, where everything is happening all at once, both good and bad.

So, this is another one of those studies where it doesn't actually show what we think it shows, and where the journalistic bias towards only reporting the negatives actually misinforms people and, worse, makes people more depressed about things.

Is it a problem that social media causes some people to feel depressed? Yes, that is absolutely a problem. But if social channels can then also be used to help people get out of depression and better connect and define their identity and self-worth, then we should focus on that instead.

Sure, we need to fix the bad things. But you don't fix that by also removing the good things. In the media, we don't even talk about the good things.

This is not the right focus.

I could go on and show you many other studies because I come across similar examples multiple times every week. The problem is not that the studies themselves are wrong, but more that the way we look at them is wrong.

So, whenever you see a study, take that step back.


The Baekdal Plus Newsletter is the best way to be notified about the latest media reports, but it also comes with extra insights.

Get the newsletter

Thomas Baekdal

Founder, media analyst, author, and publisher. Follow on Twitter

"Thomas Baekdal is one of Scandinavia's most sought-after experts in the digitization of media companies. He has made ​​himself known for his analysis of how digitization has changed the way we consume media."
Swedish business magazine, Resumé


—   thoughts   —


What do I mean when I talk about privacy and tracking?


Let's talk about Google's 'cookie-less' future and why it's bad


I'm not impressed by the Guardian's OpenAI GPT-3 article


Should media be tax exempt?


More studies that don't mean what we think they mean


The thing about Facebook and political advertising