This is one of those things I'm betting that after a few years and some simplifications, every new high-end smartphone will have as a standard feature.
So this is a topic I have thought about for awhile -- I released a 3D scanning app for iPhone nearly 3 years ago (uses the screen as the light source for scanning, called Trimensional) and have had many conversations about when Kinect-style sensing would become standard.
If we just take Apple as an example, for them to include it standard, it would have to really justify its existence. (I realize many people don't care about Apple, but I choose Apple because, for example, there have been stereo cameras on various Android models over the years, but since developers can't really count on the presence of a stereo camera for a large number of users, nothing game-changing ever really came of it, even though stereo cameras enable 3D sensing.)
Though user interaction and 3D acquisition are amazingly cool, I don't think Apple would ever include a depth sensor just for those reasons. However, Apple has shown it cares pretty deeply about the iPhone as a camera and is even willing to add new hardware if it helps people take better photos, the dual LED flash being the latest example of this.
So, as crazy as it sounds, I think Apple could add a back-facing depth sensor in 2-3 years mostly in order to allow for automatic depth-based post-processing of photos, mostly for better-looking lighting, all without the user having to care that there was 3D sensing involved. All the other stuff will then become possible, and widespread, as a side-effect.
EDIT: On the Android front, the PrimeSense folks had a booth set up at CVPR this year and mentioned that at least one manufacturer was planning to include their Capri sensor (a miniature Kinect) in an Android tablet some time soon.
I think 2 things need to happen for depth sensing to go mainstream (and I do believe they will happen sooner or later):
- Games. Look at the tennis ball example on the original article. Remember Google's IRL photo-based game (Ingress), now just imagine that with fine-grained 3D (Google will use it to crowd-source a centimeter-model of the earth). Games like this remain a bit of a niche, but just imagine if someone makes a massive social game out it on FB (a cross between Farmville, Sims, and Minecraft, projected onto your real world). Of course, someone could also create a shocking IRL FPS game (imagine your kids pointing this out the window in traffic and "shooting" at people and cars to watch them blow up). Finally something to use the processing power in theses little phones and tablets.
- 3D photography. I think this is the future of photography. Take a picture of something, extract the spacial data from the image, modify it/change the p.o.v. Recall the recent image/object maniupulation video (the SIGGRAPH one that used the PatchMatch algorithm to fix the background). Each photo becomes a mini-scene that you can navigate around (kinda like the "frozen" 360 degree pans in the Matrix). Next step is time-dimension, in other words 3D immersive movies where the viewer can move around almost anywhere while the movie unfolds. You can guess the first industry to adopt this...
Both of these can currently be done with flat images, processing power, and some human guidance. With depth sensing it can be faster, automated, and more accurate.
I imagine depth sensing will be more useful for Glass-like devices that can really benefit from hands-free gesture recognition. It seems like a much better fit than cell phones.
I also like the Oculus Rift hack that Occipital showed briefly here -- it makes a lot of sense for augmented/virtual reality, though I guess it means you can only interact with things when looking at your hands, as opposed to Sixense's controller-based approach.
> there have been stereo cameras on various Android models over the years
Hm. I now envision a (physical) addon that has a two mirrors, like a periscope, and fits over the end of the phone, and the "face" camera -- allowing the two cameras to be used for stereo imaging... Not sure if you'd be able to do just as well with motion sensor data coupled with just the main camera -- and having the user move the phone from side to side (using parallax for depth sensing) ... would make for an interesting algorithmic problem at least.
How often do you need to scan a 3D model of anything?
I'm sure it will be popular with the creative people. It's possible it will get popular in some niche markets, like real estate agents taking 3D shots of the apartments they offer (though not much hope here, they still fail to take decent photos even now).
How often do you need one of those auto-ma-whatsits anyway? They're really only good for a few blocks of downtown where the streets are flat cobblestone. They're useless on most trails. Just get a higher resolution horse and be happy with it.
If you want to buy, say furniture, a big barrier of web shopping is that you cannot automatically reason about their dimensions, because they are not exact enough (and even more seriously, they are usually available only as free text).
The 3D scanning technology would enable products that are add-ons to objects where the original manufacturer did not see value in adding an interface. If I can scan my car interior, I can attach my GPS to nearly any surface "perfectly". This gives aftermarket products very nice finish with a much lower cost.
You can even imagine previewing the furniture "live" in your apartment. I've been thinking about this for a while and Ikea made the first step with their 2014 catalogue. Ikea's version is pretty limited though, but that's a first step.
How often do you need to take panoramic pictures? Not that often, but it's fun and gimmicky enough that it's on the iPhone.
If (when) we can make a cheap and small sensor that can provide depth data to the camera to make cool looking 3D pictures, it'll likely find its way on smartphone cameras.
In 2050, will our pictures still be 2D arrays of RGB pixels?
Perhaps more to the point, how often do you "need" to take pictures at all? Smartphone cameras have proliferated even though most people don't "need" to be able to take pictures of their food or whatever.
