Random Thoughts – Randocity!

So, I’ve recently gotten interested in 3D technology. Well, not recently exactly, 3D technologies have always fascinated me even back in the blue-red glasses days. However, since there are new technologies that better take advantage of 3D imagery, I’ve recently taken an interest again. My interest was additionally sparked by the purchase of a Nintendo 3DS. With the 3DS, you don’t need glasses as the technology uses small louvers to block out the image to each eye.  This is similar to lenticular technologies, but it doesn’t use prisms for this.  Instead, small louvers block light to each eye.  Not to get into too many technical details, the technology works reasonably well, but requires viewing the screen at a very specific angle or the effect breaks down.  For portable gaming, it works ok, but because of the very specific viewing angle, it breaks down further when the action in the game gets heated and you start moving the unit around.  So, I find that I’m constantly shifting the unit to get it back into the proper position which is, of course, very distracting when you’re trying to concentrate on the game itself.

3D Gaming

On the other hand, I’ve found that with the Nintendo 3DS, the games appear truly 3D.  That is, the objects in the 3D space appear geometrically correct.  Boxes appear square.  Spheres appear round.  Characters appear to have the proper volumes and shapes and move around in the space properly (depth perception wise).  All appears to work well with 3D games.  In fact, the marriage of 3D technology works very well with 3D games. Although, because of the specific viewing angle, the jury is still out whether it actually enhances the game play enough to justify it.  However, since you can turn it off or adjust 3D effect to be stronger or weaker, you can do some things to reduce the specific viewing angle problem.

3D Live Action and Films

On the other hand, I’ve tried viewing 3D shorts filmed with actual cameras.  For whatever reason, the whole filmed 3D technology part doesn’t work at all.  I’ve come to realize that while the 3D gaming calculates the vectors exactly in space, with a camera you’re capturing two 2D images only slightly apart.  So, you’re not really sampling enough points in space, but just marrying two flat images taken a specified distance.  As a result, this 3D doesn’t truly appear to be 3D.  In fact, what I find is that this type of filmed 3D ends up looking like flat parallax planes moving in space.  That is, people and objects end up looking like flat cardboard cutouts.  These cutouts appear to be placed in space at a specified distance from the camera.  It kind of reminds me of a moving shadowbox.  I don’t know why this is, but it makes filmed 3D quite less than impressive and appears fake and unnatural.

At first, I thought this to be a problem with the size of the 3DS screen.  In fact, I visited Best Buy and viewed a 3D film on both a large Samsung and Sony monitor.  To my surprise, the filmed action still appeared as flat cutouts in space.  I believe this is the reason why 3D film is failing (and will continue to fail) with the general public.  Flat cutouts that move in parallax through perceived space just doesn’t cut it. We don’t perceive 3D in this way.  We perceive 3D in full 3D, not as flat cutouts.  For this reason, this triggers an Uncanny Valley response from many people.  Basically, it appears just fake enough that we dismiss it as being slightly off and are, in many cases, repulsed or, in some cases, physically sickened (headaches, nausea, etc).

Filmed 3D translated to 3D vector

To resolve this flat cutout problem, film producers will need to add an extra step in their film process to make 3D films actually appear 3D when using 3D glasses.  Instead of just filming two flat images and combining them, the entire filming and post processing step needs to be reworked.  The 2D images will need to be mapped onto a 3D surface in a computer.  Then, these 3D environments are then ‘re-filmed’ into left and right information from the computer’s vector information.  Basically, the film will be turned into 3D models and filmed as a 3D animation within the computer. This will effectively turn the film into a 3D vector video game cinematic. Once mapped into a computer 3D space, this should immediately resolve the flat cutout problem as now the scene is described by points in space and can then be captured properly, much the way the video game works.  So, the characters and objects now appear to have volume along with depth in space.  There will need to be some care taken for the conversion from 2D to 3D as it could look bad if done wrong.  But, done correctly, this will completely enhance the film’s 3D experience and reduce the Uncanny Valley problem.  It might even resolve some of the issues causing people to get sick.

In fact, it might even be better to store the film into a format that can be replayed by the computer using live 3D vector information rather than baking the computer’s 3D information down to 2D flat frames to be reassembled later. Using film today is a bit obsolete anyway.  Since we now have powerful computers, we can do much of this in real-time today. So, replaying 3D vector information overlaid with live motion filmed information should be possible.  Again, it has the possibility of looking really bad if done incorrectly.  So, care must be taken to do this properly.

Rethinking Film

Clearly, to create a 3D film properly, as a filmmaker you’ll need to film the entire scene with not just 2 cameras, but at least 6-8 either in a full 360 degree rotation or at least 180 degrees.  You’ll need this much information to have the computer translate to a believable model on the computer.  A model that can be rotated around using cameras placed in this 3D space so it can be ‘re-filmed’ properly.  Once the original filmed information is placed onto the extruded 3D surface and the film is then animated onto these surfaces, the 3D will come alive and will really appear to occupy space.  So, when translated to a 3D version of the film, it no longer appears like flat cutouts and now appears to have true 3D volumes.

In fact, it would be best to have a computer translate the scene you’re filming into 3D information as you are filming.  This way, you have the vector information from the actual live scene rather than trying to extrapolate this 3D information from 6-8 cameras of information later.  Extrapolation introduces errors that can be substantially reduced by getting the vector information from the scene directly.

Of course, this isn’t without cost because now you need more cameras and a filming computer to get the images to translate the filmed scene into a 3D scene in the computer.  Additionally, this adds the processing work to convert the film into a 3D surface in the computer and then basically recreate the film a second time with the extruded 3D surfaces and cameras within the 3D environment.  But, a properly created end result will speak for itself and end the flat cutout problem.

When thinking about 3D, we really must think truly in 3D, not just as flat images combined to create stereo.  Clearly, the eyes aren’t tricked that easily and more information is necessary to avoid the flat cutout problem.