Facebook Promoted Posts; Another Test, Another Puzzle

Something doesn't add up when you test Facebook 'Promoted Posts'. The numbers are inflated, and the effect is sketchy...

I did another small test of Facebook's Promoted Posts, this time for my 'baekdal' Facebook page. I promoted one post, for $5 to my fans, just to see what would happen. One again, the results are interesting, and somewhat confusing.

I detailed my first experiment (with another page) in greater detail in 'Initial Test of Facebook's Promoted Posts', so I'm just going to sum up my findings. Let's play with the numbers.

Note: The page I tested was the 'Baekdal.com' Facebook page, which only has 875 likes. It's a very low number, but I don't spend that much time on it. I use it to give people a way to follow my articles on Facebook (I focus more on driving engagement on Twitter and G+, and via my personal Facebook page).

First of all, by paying Facebook $5, they estimated that I would reach 1,200 people. Like I discovered in my first test, this number is hugely overestimated, and I only reached 495 people. But, that's not that bad considering that I only have 875 likes. It's 56% of my total fan base - for $5.

Also, the promoted post generated 52 clicks, which is an astonishing click-through rate of 10.5%. I was really excited by this when I looked at these numbers, until I checked my stats. Here is what it reported for this particular link:

Then I checked Facebook Insights, for the same post, and it told me that this post generated a total of 9 clicks (which makes a lot more sense compared to my internal stats).

Where did those 52 clicks reported by 'promoted posts' come from? What's going on here Facebook? You are heavily overestimating reach and inflating post-click counts. 9 clicks is only a click-through rate of 1.8%. It's still much higher than Facebook ads (0.05%), but it's nowhere near 10.5%.

But there is more:

The total amount of engagement for this post was only 4 likes, all generated through promoted posts. That is a tiny number, but there are two interesting things about that.

First of all, it indicates once again that the people who Facebook is filtering out with EdgeRank (the fans who wouldn't see this post without promoting it), are actually more likely to click on a post then the people who engage. I found the same thing in my first test, and I have seen the same patterns in pretty much any other test I have done for brands in the past (testing other aspects).

The listeners, the people who don't engage, are extremely valuable to you. Facebook is not helping either our fans or us by filtering them out. I wrote much more about this in 'Social Conversion Rates: Where The Real Value Comes From'.

Also, because promoted posts makes it more likely that your see 'friend of fans' activity, each like generated an additional reach of 49 non-fans (149 viral reach divided by 4 likes).

In comparison, it is a significantly higher effect than for any of my other non-promoted posts which averages around 16 non-fans reached per like/comment.

It also proves another point I made back in April, in 'Facebook Insights: Debunking Friends of Fans' where I demonstrated that the huge 'friends of fans' number Facebook reports is complete bogus.

On my Facebook page, I have 875 fans and 417,094 friends of fans. And many people then say this number is 'your potential reach' (which is not true). Even if all my fans where too like a post, the viral effect for me would only be 14,000 people under normal circumstances (16 people reached per like), and 42,000 if everything was promoted (49 people per like). That's nowhere near 417,094 friends of fans. It's just a useless number.

But there is another interesting aspect. Take a look at this graph. As you can see, while my viral effect is trending upward, it actually dropped during the days the post where promoted. This might just be a coincidence, but it caught me by surprise.

I don't really know what to make of this, so I decided to check the same thing over at 42Concepts, and here is the graph for that page:

I didn't check this when I first tested it, and it is quite surprising. Promoting a post (the green line) made absolutely no difference. In fact, total reach dropped during the promotion. But since it has also dropped several times before, it seems there is no correlation between promoting a post and actually reaching people.

It might increase the reach for that particular post that is being promoted, but it looks to me as if it's just cannibalizing everything else. Might it be that during a promotion your other posts are devaluated by EdgeRank? ...or is the effect just so insignificant that it makes no difference? I don't really know.

We need a bigger sample size

The problem with both this test and the test before it, is that the sample size is simply too small. I do see some patterns. For instance that the listeners (the people Facebook filters out) are actually more likely to click on a link than those who engage. I see huge irregularities in the data, which is a problem. And I see patterns indicating that promoted posts aren't correlating to anything else (like total traffic), I just don't know why.

What we need is a much bigger sample size, so here is what I propose:

If you are working for a brand with a web shop and a Facebook page with about 10,000 fans (+/-), I will help you setup your own test.

Note: If you have 100,000 fans, that would be even better...

I will help you define a 4-week social media release plan. We will divide it into two. The first two weeks will be just normal postings. The last two weeks we will do the same, but also promoting all the posts that make sense promoting (particular those designed to drive a call-to-action).

I will then help you crunch the numbers, and help you see exactly what the effect is.

And I will do all that for free. In exchange, you give me permission to write a Plus report about it for the rest of my readers, so that we can all learn something from it.

If you are interested in testing this out with your own brand, with my help. Write to me at thomas@baekdal.com.

Note: Of course, if you want my help but without giving me permission to write about it, we can do that too. Head over to my consulting page to see my prices.

More from Baekdal