According to a recent patent filing, Apple is developing a device that could let people control Mac computers with their fingers. The gadget looks like a piece of band that would slip over your fingers. It’s part of a growing movement in the tech world to look for better ways to control devices. “Gestures allow product developers to capitalize on the ability to sense the position, location, and movement of the body and its parts,” Carla Diana, author of My Robot Gets Me, said in an email interview. “The way we move is intrinsically linked to how we express ourselves, so having gesture control can give us a more joyful and delightful way of interacting with our everyday products that also feels natural.”
Stay Clean without Touching
Apple’s band would contain sensors to detect a wide range of different gestures—such as swiping, tapping, or rotating—and transmit them to another device, like a Macbook, the patent filing shows. Or, you could make gestures against a surface of an object or wave your fingers to control the Mac using an optical sensor. Gesture controls are not going to replace mice and keyboards, but rather complement them, Thomas Amilien, CEO of Clay AIR, a company specializing in hand tracking and gesture controls, said in an email interview. “For example, gestural interfaces are particularly practical in situations where touch, voice, and biometrics have limitations, or where dealing with a controller breaks the immersion.” The most common use for gesture controls right now is linked to people’s concern on hygiene in public spaces, Amilien pointed out. Touchless gestures can be used to replace touch-based interfaces in public areas. You can interact with a self-service or wayfinding kiosk with simple gestures to navigate the menu, order, and pay. Touchless gestural interfaces can also complement other modes of interactions where voice and touch have limits, Amilien said. “In noisy environments like a factory, where employees wear protective gear, gesture controls to control an interface without taking off the gloves or to command an autonomous device at a distance can be very practical,” he added. Gestural interfaces are increasingly being integrated into car navigation systems in personal cars and fleets for safety issues, Amilien pointed out. Studies show it takes 18-25 seconds for a driver to perform an action on a touchscreen. “However, a few seconds of eyes-of-the-road increases the risk of a crash,” Amilien said. “Touchless in-car gesture controls are a good alternative to touch.” Gesture-recognition and hand-tracking technology have a wide range of applications in augmented and virtual reality, too.
Gesture Make VR Better
Clay AIR’s technology lets people interact with virtual content and navigate a menu or a workflow without using controllers, using the monochrome cameras located on the headset. “When gesture recognition is complemented with hand tracking, applications can include physical rehabilitation, training, hands-free navigation for remote assistance, interaction. In virtual reality, this feature’s main benefit is to keep users immersed. In augmented reality, it’s more about interactivity and ease of use,” Amilien said. But gesture controls come with privacy concerns, Diana said. The technology uses special cameras that can see an image and understand the three-dimensional profile of the physical world in front of it. “We need to think deeply about the pros and cons of having cameras embedded in everyday objects because of the privacy risks,” she added. Diana said a promising alternative comes in a new range of electronics that use radar to detect movements without using a camera. Google’s Project Soli, for example, offers developers a platform to detect gestures while providing more privacy than camera vision, she added. “It also offers the ability to detect gestures through other materials,” Diana said. “So it can be embedded inside objects and has the benefit of drawing less power to function.”
