With the AirPods Pro in Transparency mode, it should feel like somebody is walking next to you, giving you interesting facts and local recommendations about whatever you pass. Foursquare classes Marsbot as a virtual assistant, but apart from restaurant recommendations, what could it be used for? “We built Marsbot for AirPods to feel like you are walking down the street with a friend who knows everything about the city and is constantly pointing out the most interesting things to you,” writes Foursquare’s Dennis Crowley in a blog post. “Many of the notifications are designed to help you notice places and things you may have never noticed, even though you’ve walked by them 100 times before.”
Going Out
This is an interesting time to be launching an app based on going out. And yet, it’s also the perfect app to have some company while you take a socially-distant walk in your city. Foursquare made its reputation with check-ins, where you’d collect visits to bars, restaurants, and cultural spots, then it morphed into a crowd-sourced city guide. Audio AR is a logical next step. The app is meant to be a kind of beta/prototype hybrid; a test to see what would be possible with audio AR. We usually think of AR as a video overlay on a video stream of the world in front of us—gadgets like Google Glass could overlay visual information by projecting it into a pair of glasses. But AR can also be audio. In fact, many of us are already using it. If you have Apple’s AirPods Pro, you may already be familiar with Siri reading out incoming messages, and allowing you to reply to them, all without touching or looking at anything. Marsbot for AirPods takes this a step further, by adding location-based AR via your ears. It will pause podcasts, and lower your music’s volume when whispering in your ear. However, audio and video calls will not be interrupted. The idea isn’t to have a constant barrage of information. “You may only hear one audio snippet a day, or you may go days without hearing any audio snippets,” says Crowley. The advantage of audio AR is that you don’t have to take your eyes off anything to read it or look at it. Audio is already much more ambient than visual information. It also doesn’t require any special gear; AirPods Pro in Transparency mode might be the ideal equipment, but really, any headphones will work fine. Many folks are plugged in the entire time they’re out and about, so this really is a painless route to AR.
What Else Can Audio AR Do?
Notably, the majority of examples provided on Foursquare’s blog post are recommendations for coffee shops and restaurants. But you can also record your own audio snippets, which hang in the virtual space waiting for somebody else to pass by. You could leave restaurant and shopping tips, but what else could AR be used for? You could, for example, create your own virtual city guides. These could be themed around a particular interest or subculture. If apps like Marsbot allowed you to subscribe to a certain person’s audio snippets, then you could dip in and out of usually-hidden city subcultures. Or how about a virtual art gallery featuring notable graffiti and street art? More importantly, what about accessibility? Blind people face many challenges that “most vision able people aren’t aware of,” audio user interface designer Arthur Carabott, who also interned on Microsoft’s Project Tokyo, told Lifewire via direct message. “[Like] knowing who all the people in a room are without having to interrupt the conversation, [or] knowing who quietly joined a meeting and sat down across from you.” One advantage of audio information is that you don’t need to read it, meaning people with dyslexia can use it, too. Imagine if shop signs or even menus could be spoken aloud to you when you got up close? Or how about a kind of ambient map for the blind, something tied into Google Maps or Apple Maps. Not turn-by-turn directions, but something that blends into the background. You may have a sound for roads, another for pedestrian crossings, the stairs, or the entrances to buildings. With the kind of clever surround-sound audio used in movies and games, a blind person could hear these bits of infrastructure fading in and out as they pass by. Or how about using Bluetooth, or those fancy U1 ultra-wideband chips that let iPhones precisely measure the position and direction of other iPhones? That would be great for an AR social distancing app. Until people stop walking around with earbuds in their ears, audio AR is going to be way easier to deploy than any other kind of augmented reality. And if Marsbot can get us used to the idea, that’s a win for everyone.
