How accurate data was turned into misleading articles by the government ...and the press

Earlier this week, I came across a news story that frustrated me to no end. It was one of those stories where the data we were being given didn't match what was being reported.

As a media analyst, this is an instant red flag. If the data doesn't match the journalism, then one of them must be wrong.

So, I decided to look into it, and it turned into a full day of media analysis, illustrating how we in the press are being used to report stories in a misleading way. But it's happening because we didn't do the work we were supposed to as journalists.

The story is also not unique to Denmark, or to the Danish press, but it serves as a good illustration for a problem we see in the industry as a whole.

But let me tell you the story.

It was a Tuesday...

This story started on a Tuesday when I was checking the news like I usually do, and I came across a front page story on one of the largest news sites in Denmark, with a rather shocking headline.

It said this (translated via Google Translate):

This sounds pretty bad, and in the article we are told that, in 2019, the number of traffic fatalities in Denmark was 205 people, compared to only 171 the year before.

In other words, an increase of about 20%.

And the article is filled with data about this, and they are trying to do a good job reporting it. They tell you what type of people were killed (drivers, cyclists, etc), what the specific increase or decrease has been in different parts of the country, and so forth.

From the perspective of data reporting, they did a good job telling us what the data is right now. Including telling people that while the number of people killed was higher now, the number of people getting hurt (non-fatalities) is historically low.

However, as I was reading it, something was wrong about the 'framing' of this story. They tried to make it sound like an increasing problem, they interviewed people providing 'statements of shock', but when I looked at the data, there seemed to be no worsening effect at all.

In fact, almost at the end of the article, the journalist wrote this:

The number of people killed in traffic has fallen since the early 1970s, but this decline has stagnated in 2012. Subsequently there have been minor fluctuations, but no progress.

Wait a minute. You just told me it had gotten 20% worse? What's going on here? Is there a problem, or isn't there a problem?

Now, because I am an analyst, I couldn't let that slide, and so I looked up the data myself from the National Bureau of Statistics. I opened it up in Excel and turned it into a graph.

This graph very clearly shows that nothing is going on. As you can see, the numbers were a lot higher back in the 1990s. They then dropped until 2012, and since then they have hovered around the same level, slightly fluctuating from year to year.

But from a trend perspective, nothing is happening.

The article is trying to convince us that the world is getting worse, but it clearly isn't.

But "wait a minute", the journalist might say, "look at the difference between 2018 and 2019. It has gone up".

Yes, but that's normal fluctuations. It also went up in 2016 and in 2013. The fact that it has gone up one year doesn't tell you anything.

You don't have a story!

As a media analyst, I absolutely hate this. I consider it to be a crime against journalism. You are taking accurate data and twisting it into something that isn't true.

So, I got quite angry about this, and I tweeted about it (as I usually do). And then I moved on to other things. This was yet another example of 'outrage journalism'.

Anyway, a few hours later, I was reading the news again, this time in another newspaper. And here I also came across an article about this very thing.

To translate it, the first headline says:

See the negative development: Here are all the accidents per region and type of transport.

The second headline reads:

There has been a dramatic increase in traffic fatalities.

Really, newspapers?

Are you all going to run with this story in a wildly misleading way? What is this? Outrage day??!

*sigh*

Anyway, this also came with a twist, it included this sentence in the article:

At the same time, it is a 12 percent increase over the years 2014-2018, with an average of 183 deaths per year.

WHAT?!?!

You can't compare averages like this. This is a completely invalid and misleading way to compare these numbers.

It's like if you take the graph from before, and then just make all the previous years the same. Like this:

This is misleading in the extreme.

If you actually want to compare averages, you need to compare one average with another average. Like this:

Now, the average of the five years leading up to 2018 was 183.3, while the average today is 188. Meaning that there has only been a 2.5% increase in fatalities. Not a 12% increase as reported.

But even this is massively misleading, because you can make averages tell you anything you want. All you need to do is to change the scale a bit.

For instance, if you measure over the past three years instead of five, you get this:

Now the average number of traffic fatalities has dropped by 1%. And if you instead make the scale 10 years, we get this:

Now the drop from "before" compared to "now" is almost 9%.

This is the problem with averages. By adjusting the scale a bit, I can make it give me whatever result I want.

As a journalist, you should know this. Never report averages without checking if they match the overall trend.

How do you do this? Well, you take the raw data, turn it into a graph, and then you map out a trendline as a moving average.

Like this:

This trendline is using the scale as the average in the article (five years), and as you can see, it's flat. The fluctuation in 2019 does not change the trendline, because it is still just a fluctuation.

So, when I saw this in the second news article, I got a bit pissed off, because this is a rookie mistake. This journalist didn't do any math. They just took a number from somewhere and reported it uncritically and without checking the data.

This got me curious, and so I decided to look up all the other Danish national newspapers, and sure enough, they all reported the same story, with their own weird little flavors of misinformation.

Here is another newspaper with an even more suspicious headline.

If you remember from the graph above, this is not even remotely true. The numbers were higher in 2016, so the 2019 figure is not the highest in the past five years.

This is a lie!

I don't know if they did it on purpose (probably not), or if they were just so stressed they didn't take the time to actually look at the data correctly, but it's still a lie.

And this was happening across the board. All the national newspapers did this in some variation.

As a media analyst, this frustrates me to no end. How did this happen? How did all the Danish national newspapers collectively mislead the public?

Aren't we supposed to be better than this?

Well... this is where things get really bad.

The police, the government, and the lobbyists

But there is a much bigger story happening around all of this.

First of all, what I haven't told you yet is that many of the newspapers above talk about a press release issued by the Danish Road Directorate (a directorate under the Ministry of Transport).

And much of the data they reported seems to have come from this press release.

I will come back to why this is important in a second.

Instead, we are going to start with the government.

The newly elected Danish government has had a tendency to chase what you might call populistic agendas. We have seen this repeatedly over the past several months.

The government would find something everyone was outraged by and would introduce a kind of "tough on ..." type of legislation to appear to be doing something about it.

They have done it with crime, even though the crime level is actually in decline, and with the safety of children, even though the data here is also in question.

And the newspapers have kind of just 'run with the story', repeating the outrage the government was promoting, finding single examples to illustrate how bad it is, and making the public believe these things were really big problems, even though the data either says it isn't, or that we don't have enough data to really know.

Anyway, all the way back in 2013, the Danish Commision for Traffic set a goal that Denmark needed to reduce the number of traffic fatalities to only 120 per year in 2020, and they did this based on past years' decline.

If you look at the graph again, this was not an unreasonable goal. Up until 2013, things were going very steadily in that direction.

Of course, it didn't turn out that way. In 2013 and until today, things just went flat.

We can have a long discussion about why this is (it's complicated), but I won't go into that here. The bottom line is that newspapers have been pressuring the politicians to live up to their goal, asking them questions like "why aren't we there yet?"

Then something else happened. A couple of months ago, a police officer was run down by a reckless driver, and it created quite a public discussion in my country about what to do about this.

And the government has been questioned by the press to answer "what do they plan to do about this"?

So, this is the run up to this story. The government, which has a very populistic tendency, feels pressured to show action.

Fast forward to February 4, 2020.

First of all, the Danish Road Directorate issued its annual numbers for traffic fatalities, like they always do. And they issued a press release.

Overall, it's just a normal press release. They provide the data, they have a quote from the Minister of Transport, the directors and someone from the police.

However, this press release is also a political tool. Over the years press releases have been used very specifically to promote certain initiatives or goals.

In the press release, there is also a quote from the Society for Safe Traffic. This is a private lobbyist organization comprised of members of the government, and private organizations like the teachers' unions, the society of cyclists, a motorist organization and others.

Their goal is to make traffic safer. This is a very admirable goal, but it still makes them a lobbyist. They have an interest in furthering a certain outcome.

They said this:

It is deeply worrying that there have been 20% more deaths in a single year. This poor development comes after several years of stagnation, and it means we have not achieved the desired decrease in the number of traffic fatalities. We can and must succeed in reducing these casualties for both people killed and injured, but it requires an even more focused effort from politicians, authorities and organizations.

This last part was weird. Why is a government agency issuing a press release with a quote from a lobbyist group where they include a call for 'more focused efforts' by the politicians?

Well, the answer is that this was what the government wanted. Because the government is ready to push out their own legislation.

They want to put people who drive too fast in jail!

Specifically, this legislation wants to send people to jail if they drive faster than 200 KM per hour, or more than 100% over the posted speed limit.

But the interesting thing here is that, in Denmark, the government doesn't have enough votes to introduce legislation by itself. They need support from several other political parties.

Therefore the government needs to create a sense of outrage that would encourage other parties to support them.

And that's exactly what is happening here. The press release was designed to drive government support. It made it sound like there was a bigger problem than what we really see, and they brought in an outside organization to call upon the other politicians to 'do more'.

The police also joined in and issued their own numbers. They issued the number of cases where people driving this fast had been caught on camera, which the press reported like everything else.

They even added a 'fact box' like this to their articles, showing that the number has gone up from 90 in 2015, to 178 in 2019.

But, again, these numbers are also misleading. What you see here are the raw numbers, as recorded from the automatic police cameras around the country.

However, since 2015, the police have put up many more cameras. So we don't know if this increase is because more people drive recklessly ... or if it's just because there are more cameras to capture it.

So this is another form of misleading data. The actual data is accurate, but reporting it without context is misleading.

In other words, all of this plays into the same focus.

So, let me present this to you as a timeline:

1. The government wants to be seen to 'do something'.
2. They planned legislation to be tough on speeding.
3. They coordinated it with the annual release of traffic fatalities, which they knew was higher than the year earlier, which is then issued as a press release and includes a 'call for action'.
4. The press takes this data, and the framing of "OMG things are getting worse" (they are not), and adds their own flavor of outrage and misinformation to it. They write "You will be shocked how bad it is" and "It's the worst data in five years".
5. The press then starts to call up politicians, like they always do to ask "what are you going to do", thereby doing the exact thing the Government wanted them to do.
6. When the press asked the government, they were ready for them. They said: "Thank you for calling. This is a great question, and ....Surprise! ...we have legislation ready for you to write about. Here is a teaser for what we have planned, and then on Thursday, we will announce the whole thing (after the press has turned it into a frenzy)".
7. The press is then really happy to hear that, because it supports the outrage that they themselves reported earlier. So they report about this legislation, and turn the whole thing into a 'theme'.
8. The police then give the press the latest numbers, showing that the exact group of offenders the government is talking about is up by almost 90%.
9. The press, still in outrage mode, report about this too, telling their readers: "OMG Look how much the numbers are up!" ... and here, watch a video of a police chase so that you can feel even more outraged by the whole thing.

All of this went on for an entire day. It wasn't until the day after, when the government also said they wanted the right to confiscate cars if they didn't belong to the person driving them, that the press actually started asking questions.

But at this point, it's already too late. You have already convinced everyone that "OMG we must act NOW!"

The government spin doctors must be laughing right now. Not only did we in the press do everything they wanted us to do, we also took it to the next level, causing so much outrage and completely distracting people from any other focus so everyone is now convinced that what the government wants to do is the right thing.

Meanwhile, the questions that are not being asked are:

• What do the numbers actually say?
• What share of the people who drive more than 200 KM/hour actually caused any of the 205 traffic fatalities per year?
• What are the calculations behind this legislation? How many people does the government think would stop driving too fast because of it, what numbers do they base this assumption on ... and how does this compare to other crime-related data?
• Why would this even work since we know that other 'send them to prison' legislations rarely work, or have even backfired?
• Is this focus even where the problem is? Are we even having the right discussion here? What other factors are there? Speeding only accounts for 40% of the traffic fatalities, so why is that the main focus?
• How is any of this going to get us any closer to the 120 fatalities per year that is the goal for 2020?
• Is the 120 goal even the right goal? Or was that also created on a false assumption at the time?
• What is the right goal? And are there other areas of society where we could make changes to reduce fatalities even more?
• What about public transport? Is a part of the problem that the roads are too congested? 60% of fatalities were caused by people either being distracted or unaware of something around them, so would it be better to focus on that?
• What about the quality of the cars? We know that modern cars are much safer, and now include active protection that will stop the car if danger is detected. But how many cars in Denmark actually have that? And how many of those were involved in fatal accidents? Would it be better to do something to upgrade what cars people drive?
• How do we compare to other countries?

Nobody is asking these questions ... you are just focusing on the one part that people can feel the most outraged by.

This is not journalism. This is not an informed public. And it's not a sound basis for democratic decision making.

Don't get me wrong. I also think that people who drive recklessly should be stopped. In fact, my sister's husband is a police officer and works at the same precinct as the officer who was killed by one such driver.

But that is my personal opinion. My opinion doesn't matter here. What I'm talking about here is the role of the press, and how we need to be the public's watchdog and to help drive an intelligent and informed debate.

This, however, is not what we do. We are just helping push a very populistic legislation, based on people's feelings of outrage and anger instead of data and facts.

This should not be how we discuss politics.

Just one story, but a global problem

This was just one (long) story, and it had a lot of twists and turns.

I could excuse all of this on journalists being too busy and too stressed, or that newspapers believe that they shouldn't be involved in the story and instead just 'report the news'.

But these are really bad excuses.

Today, just reporting very often ends up driving specific political agendas.

That's what is happening. This story was defined by the government. They told us what to write, they created the framing, they provided the talking points and misleading numbers to use ... and we just reported it all.

And then there are the lies. When you as a newspaper write "you will be shocked by this" or "it's the worst in five years" (even though it isn't), you are lying to your readers.

And when we see this happen across newspapers all the time, it indicates a culture problem within the press industry. A problem that we need to fix.

But here is my problem.

I have now told you this story. I outlined how it happened, and why it's so bad. But this is not the first time I have done this. I have written about similar problems over the past 10 years ... and nothing is changing.

So, I'm at a loss.

I need your help because I don't know what to do anymore.

How do we convince national newspapers to stop doing journalism this way? How do we change the culture that exists in the newsroom that causes this to happen again and again? How do we stop editors and journalists feeling excited when they can add a misleading headline to their stories? How do we make newspapers understand that they need to do a very different type of journalism? That it's not our job to just wait for the politicians to say something, and then report about it?

So many good things are being undermined by stories like these.

I have tried everything I can think of to help create this change, but nothing ever changes.

More from Baekdal

How scared should we be of AIs taking our jobs?

30 pages