Facebook has showed off a mindreading prototype device and an augmented reality (AR) keyboard to replace the mouse and keyboard we have known for decades in future products.
The Facebook Reality Labs team demonstrated the prototype on a virtual call with the media earlier this week. Team execs discussed the underlying technologies. And videos of the projects played for the audience during the discussion.
The wrist device reads neural signals sent from a user’s brain to the hands. By theory, it can read these signals to make sense of what action a user wants and reproduce that action in a virtual reality (VR) or AR environment.
“You actually have more of your brain dedicated to controlling your wrist than any other part of your body, probably twice as many neurons controlling your wrist and the movement of your hands than is actually dedicated to your mouth for feeding and for speech,” said Facebook Reality Labs research science director TR Reardon.
The Reality Tabs team showed off “force” actions where a user could pinch fingers in real life to hold and navigate virtual objects in AR. The name “force” action refers to the Star Wars franchise. In that sci-fi series, some characters use the Force to control and move objects far from them.
The company also showed off electromyography wristbands. Users could wear these devices to type on any surface, as if typing on a physical keyboard. Even without an actual keyboard, the EMG wristbands could register the goals of a user’s finger strokes and type letters and words in a VR screen.
Facebook also has plans to release its first smart glasses this year. The device has a Ray-Ban branding and a partnership with Luxottica.
These devices are unlike our regular mobile handhelds. They do not rely on touchscreens for input. They are unlike Facebook’s Oculus VR headsets with handheld controllers. Smart glasses have no physical input devices for control. Facebook must work on all these VR and AR technologies to make the glasses viable.
Asked when these mindreading technologies will hit the shelves or online stores, Facebook CTO Mike Schroepfer said all these are in the early development stages.
“It’s hard to predict their timeline. How these things sequence out in the market, when they show up — are things I don’t have crisp answers to. What we’re focused on is hardening these technologies,” he added.