Why Pretty Much All 360 VR Videos Get It Wrong

I finally got around to buying one of those Google Cardboard thingies that allow me to get an immersive 3D experience using just my Android phone. I bought mine from Knoxlabs, and it's ...eh ...simple, cheap and not really as good as I had hoped.

It's not really because it's made of cardboard, nor are the lenses that bad. It's much more to do with how everyone seems to completely miss the point of what an 'immersive' field of view is.

Okay, so granted, I also have the problem that I wear glasses. And while you can technically use it with glasses (notice how you can bend out the side flaps), it moves your head away from the lenses and thus gives you a more binocular type of effect. On top of that, every single time I try some 3D tech, I end up with the a splitting headache because eye strain.

But let me explain the problem with the lack of immersive effect.

Immersion means that you feel like you are in the world that you are looking at, and in order to achieve that you need to give people a really high-quality, highly responsive, high-frame rate experience, with a wide field of view. The first part is important in order to make it feel real, but the last part is the most important element of all.

It's the field of view that makes the difference between feeling that you are looking at a screen through a pair of binoculars or the feeling of really being there.

So, how does field of view work in the real world? Well, stand up and walk over to your windows so that you are standing about 3 meters behind it. What do you see?

It could be something like this:

Okay, now walk two meters closer to your window, so that you are standing only one meter from it. Now what do you see?

The window is now a lot closer to your eyes, but the view outside isn't. The result being that you field of view has changed, allowing you to see much more of the view outside.

Like this:

And if you walk right smack up to the window, you can now see even more of everything. Like so:

Now we have the immersive experience, and notice how big of a difference that makes of what you see.

In the same sense, the opposite of an immersive experience, is the binocular effect. Now, instead of having a wider field of view, we get this:

It's doesn't matter that within these binoculars that you can shift the direction of where you are looking. It's not the direction or 3D effect that creates immersion. It's the field of view itself.

This is such an important thing to keep in mind when thinking about virtual reality, but sadly, most examples we see today completely miss this simple point, and this is especially true for when you use Google Cardboard.

In order to create an immersive virtual reality experience, the closer we move the screen to our eyes, the wider a field of view we need to have, just like with the window above. Meaning that when you put on Google Cardboard, it must give you a field of view of about 160 degrees. But instead, every single 360 VR Cardboard video I have seen so far gets this completely wrong.

Here is a screenshot from the Tomorrowland 360 video in normal desktop mode.

So, when we move our screen closer towards our eyes, the field of view should expand to reveal more of the picture. Right? Like this:

But this is not how YouTube 360 videos works. Instead, it simply divides the image in two and then show you this:

Notice how little of the image you are now able to see. And notice how incredibly zoomed in this looks. This is what 360 degree VR videos look like with Google Cardboard.

It's the opposite of immersion. Everything is now way too close, and you have a hard time just trying to focus on things. True, you can move your head around and see more of the scene (in fact, all 360 degrees of it), but that's not immersion.

What you are seeing here is a field of view of about 25 degrees, when it should be at least 125 degrees if not more. It's absolutely horrendous.

We see the same thing with the photosphere scenes, like this one. It looks very nice, but something is off about the view. Everything looks too far away in the distant, but at the same time your brain thinks everything much closer to you than it should be.

It's the same with the gaming stuff. The Tuscany Villa tour, for instance, also suffer from field of view being too narrow and too close.

Although the Roller Coaster VR game is the one that came closest to what I would call immersive. Not great, but... okay-ish.

Not all demos are this bad though. Google's own tour app looks kind of nice.

For most of the scenes the field of view looks 'real' and you don't get the zoomed in tunnel vision

There was, however, one app that did impress me. And it's the Cmoar Virtual Cinema. Here you can watch movies (either from your device or via YouTube) as if you are sitting in a real cinema. And it's really well made.

Notice how the lighting inside the cinema changes depending on the intensity of the light on the screen. That is a cool effect.

More to the point, this app allows me to tweak all the view settings, from the field of view, to the barrel distortion levels, to the distance between my eyes. Very, very nice (all VR apps should have this).

Because of this, it was the first app I have tried today where I didn't feel like I had intense tunnel vision. Part of the reason was also that I'm looking at a 2D plane at a distance, in a 3D space.

It has its flaws too, of course. It keeps drifting to one side, so I end up looking at an odd angle just to keep the screen centered in front of my eyes. And even though mobile phones are pretty cool these days, the same video looks a lot better on my iPad.

But this was the first experience where I could kind of see how Google Cardboard could work. And it was the first experience where my annoyance with how my glasses added to the tunnel-vision effect wasn't as noticeable.

With every other app, I never stopped feeling like as if I was looking through a pair of binoculars. The field of view is so incredibly important to get right, along with distance to the edge of the screen and how responsive it is.

While some of these Google Cardboard compatible apps kind of get this, most of them don't. And YouTube 360 is just horrendous. Every single video is doing it wrong. They are all narrowing the field of view when they should be expanding it.

Is this the future?

So... Is this merely a gimmick, or does this have a real use for the future?

Obviously, Google Cardboard is just a gimmick. It's not that comfortable to use because, well, it's made of cardboard and not soft foam. And it doesn't come with a strap, so you have to hold it in front of your eyes with your hand. You might be able to do that for five minutes, but try watching anything at length with it.

But, while I was disappointed by the low quality of all the cardboard apps, and also rather annoyed by my headache after using it, it's a fun little gimmick. And, it's so cheap that I really cannot come up with any reason why you shouldn't buy one. Why not?

It's not the future, however.

But what about the more high-end models, and specifically the Oculus Rift or maybe a future version of the very cool looking Cinemizer OLED from Zeiss?

Now if these headsets could be made with a high enough resolution, a high enough response rate, and lenses that were designed for each individual person (keep in mind 6 in 10 people in the western world has poor eyesight), then the answer is yes.

We are getting very close to this point. One might even say that the Oculus Rift is at this point, if they just started producing individual prescription lenses. And if we could then get all the developers to realize how field of view works, we could have something that is pretty cool.

There are many extremely interesting uses for these. We have the whole gaming element, especially first person games, which we could even expand into our real world, like what Microsoft is doing with Minecraft and the Hololens.

This is huge!

There are many practical applications for this as well. From being able to remote control your drone from its perspective, to design applications and so many other things. And yes, we might even watch movies or Netflix this way, thus no longer have a need to buy a regular TV. Keep in mind that singles make up 30-50% of the population in western countries.

But there is one area of virtual reality and 3D that people, brands and the media continually get wrong. That is that 3D isn't going to be successful because it's 3D.

The problem with 3D is that it only works if you move your head. Yes, yes... I know, that's the point. But, it's not a very good user experience. For instance. GoPro has uploaded a 360 degree experience of a car drifting.

In order to see it, you have to move your head around in a 360 degree turn. I'm sorry, I can't move my head like that. I'm not an owl. So, the only way to see this is by standing up, with a box in front of your eyes, turning your entire body around. Sure, that's fun for about 2 minutes... but after that, it gets really annoying.

And, most of the time, you will lose track of where the car is and you end up looking at absolutely nothing.

Compare that to one of the epic Ken Block drift videos, which are not in 3D, and you realize just how bad 3D really is.

 

This was also the problem that we had with 3D sites like Second Life. Many brands thought this was the future, but people quickly realized it was sub-optimized way of looking at things. You would walk into a virtual store, but in order to see anything, you would have to constantly navigate your view screen around to align yourself with each product.

It looked cool, but was a seriously crappy way to do things efficiently.

Second Life failed because it (and the many brands engaging with it) never realized just how much friction a 3D environment creates in terms of needless navigation.

It's the same reason why I do not even for a second understand why Facebook bought Oculus Rift. Sure, Oculus Right by itself is pretty cool. But trying to use Oculus for social interactions... that's just second life all over again.

Keep this in mind every time people talk about virtual reality and 3D. The reason why it works is because you are constantly navigating your head and your body to shift your point of view. This works great for games where navigating the scenes is an essential part of what makes that game enjoyable and exciting.

But in most other settings, navigating is something we are trying to prevent. You don't want people to spend 90% of their time trying to navigate your virtual store. You want people to spend 90% of their time enjoying your products. And you do this by eliminating as much of the navigation as possible.

So, companies like Tesco might think this looks pretty cool, but apparently haven't realized that if I'm interested in one of the product at the back of the store, I would first have to navigate all they back there... in 3D.

 

Not to mention that they have introduced the problem with limited shelf space (imagine trying to talk into an Amazon virtual store with millions of shelfs, you would never be able to find anything). ...It's Second Life all over again.

This is the part of 3D and virtual reality that will never make it, simply because you are adding a limitation when you should be eliminating it.

But in those cases where navigating is an integrated part of what makes that experience better, like with games, devices such as the Oculus Rift are very exciting.

Another very interesting trend is the whole concept of presence. The idea that we can extend the things we are already doing with apps, services, and devices that make it so much better. This is the trend we are already seeing with mobiles and wearables.

The Oculus Rift, however, isn't part of that because it shuts you out from the world around you, rather than extending it.

However, before you start to consider how virtual reality that can be added to that mix, you need to remember how people behave. The key element of mobile is how it has changed our behavior from something we plan in advance to momentum gained through a large number of micro-moments. I wrote a much longer article about that here.

The problem with 3D and VR, however, is that they are designed for the opposite of that. Today's VR is something you decide to do in advance. You don't put on an Oculus Rift just to quickly check something.

So, we are still very far from that particular future.

More from Baekdal

How scared should we be of AIs taking our jobs?

30 pages