Microsoft Research is presently developing a mechanical keyboard which can also sense your hand movements. According to Microsoft, this new keyboard is a type of augmented mechanical keyboard. It senses rich and expressive motion gestures performed both on and directly above the keyboard surface.
A low-resolution matrix of infrared (IR) proximity sensors is interspersed with the keys of a regular mechanical keyboard. This results in coarse but high frame-rate motion data. Microsoft has extended a machine learning algorithm, traditionally used for static classification only, to robustly support dynamic, temporal gestures. They propose the use of motion signatures a technique that utilizes pairs of motion history images and a random forest classifier to robustly recognize a large set of motion gestures. Their technique achieves a mean per-frame classification accuracy of 75:6% in leave–one–subject–out and 89:9% in half-test/half-training cross-validation. They also detail hardware and gesture recognition algorithm, provide accuracy results, and demonstrate a large set of gestures designed to be performed with the device.
