HEARTLESS

According to a patent application recently uncovered, Microsoft's Kinect will recognize American Sign Language well enough that those who "speak" sign language will be able to navigate menus, enter text, and even participate in chats in real time.

Even more amazing, the device will apparently be finely tuned enough to recognize lip motions and recognize what is being "said" by those who can't speak. Here's the example from the patent:

"For example, when the user kills another user's character, that victorious, though speechless, user would be able to tell the other user that he had been 'PWNED'. In another embodiment, a user may be able to speak or make the facial motions corresponding to speaking words. The system may then parse those facial motions to determine the user's intended words and process them according to the context under which they were inputted to the system."

Actual lip-reading has been over-promised by movies and TV for many years, so I doubt that the Kinect will "really" be able to tell what a person is saying just by "looking" at him and her -- your lips don't move distincitvely enough to really be readable by anyone, without the aid of the sounds you're making as well. But American Sign Language should be easier for Kinect to understand.. .in fact, I'll bet a computer understanding ASL is "easier" than a computer understanding and/or translating speech.

A side note: Kinect will be able to count your freakin' toes. From the patent application:

"[Within the skeletal mapping system] a variety of joints and bones are identified: each hand, each forearm, each elbow, each bicep, each shoulder, each hip, each thigh, each knee, each foreleg, each foot, the head, the torso, the top and bottom of the spine, and the waist. Where more points are tracked, additional features may be identified, such as the bones and joints of the fingers or toes, or individual features of the face, such as the nose and eyes."

This may not prove to be a boon to hardcore gamers, but as technology, Kinect is very impressive. The thing will apparently know where you're looking.