Here’s Amazon Smith 3D smartphone UI, straight from the Lab 126. Kinect like gesture controls in the works too
According to anonymous post on Hacker News, confirmed by Techcrunch through its own sources, Amazon is working on a smartphone with a 3D-eye tracking user interface, codenamed “Smith”. Instead of traditional 3D display, Smith will have a normal 2D screen, but will track user head and eye movement, and will shift the UI accordingly to create the impression of 3D. An effect similar to Apple’s 3D parallax wallpapers.
We can now confirm that Amazon is indeed working on devices with an eye/head movement tracking 3D user interface, and has been at it for at least 5 years. We just found one place where there are plenty of clues of what Amazon Smith 3D smartphone UI is about.
U.S Patent and Trademark Office.
Where Amazon filed quite a few patent applications describing their 3D user interface ideas, and a lot of them talk about things we are now hearing about Smith: device with a camera on each corner, eye and head movement tracking, adjusting user interface elements based on viewing angle, etc.
Interestingly enough, the first patent application for Amazon Phone I could find, was filed in May 2008, and lists Jeff Bezos himself as the sole inventor. It already describes a 3D user interface, operated via movement/gestures and user eye/head tracking. That’s how long Amazon CEO has been thinking about Smith.
By the way, those four front-cameras that Kindle smartphone should come equipped with? It seems that they are not really necessary to make Amazon’s faux-3D smartphone UI work. One, or at most two cameras should be plenty enough to accomplish the movement tracking task. Instead they are needed for Amazon’s Kinect like touchless input gesture user interface.
If the ideas from this patent app make it into the commercial release, you should be able to operate your Kindle Phone by hovering your finger and making various gestures above the device, without touching it. But following your finger movement and hand gestures requires much more detailed resolution and precision, than just figuring out your viewing angle. That’s where 4 front-cameras come into play.
Why would you need such a thing for a smartphone or tablet? Touch works just fine? Yep, multi-touch works great on a 2D UI’s of today. But to effectively navigate through the 3D interfaces, you also need the depth dimension. And that’s what Amazon’s multi-dimensional input UI is all about.
Another smartphone feature that Amazon has been working on for a while now, is constant awareness and user tracking. A bit similar to Moto X constant voice command monitoring for Google Now integration. But also able to track your whereabouts in the room through device cameras, and always ready for your voice or gesture commands. It’s described in some detail in these patent apps: App 1, App 2.
And why would Amazon go to all this trouble with their own smartphone? Well, to better integrate its services and sell you more stuff, of course. They think they can do it much better with own 3D UI, than with currently available 2D stuff. In Amazon’s own words:
“… an electronic marketplace might not display information through a conventional Web page in a browser, but might provide information to a device that can be displayed along one or more dimensions of a multi-dimensional interface and can be sorted and modified just as any data stored on the device. For example, a shopping view might show dimensions such as past purchases, browse history, reviews, wish list items, etc. A user can select such a view and navigate to content of interest using approaches discussed and suggested herein. Further, since the data can be associated with other data dimensions, the user can obtain information that would not previously have been readily determinable. For example, if a product on the marketplace was reviewed by a reviewer that the user recognizes, but cannot remember where the user met that reviewer, the user can change the view to combine contacts, location, or other such dimensions with the shopping dimension to locate the information.”
Looking at all those patent apps, upcoming Amazon smartphone sounds like a really interesting device. Let’s see how much of the features above make it into the commercial device.