"Mobilio turns any smartphone into a navigation aid for blind and low-vision users"

A team of Harvard engineers has built a navigation aid that requires no specialized hardware at all — just the smartphone most people already carry. Mobilio, developed at the John A. Paulson School of Engineering and Applied Sciences, combines machine learning, the phone's built-in sensors, and personalized audio beeps to give blind and low-vision users turn-by-turn directions, path guidance, and obstacle avoidance in real time. In early testing, participants using Mobilio alongside a white cane finished an outdoor route about 13% faster than with Google Maps and a cane, and cut contact with obstacles on an indoor course by roughly 41%.

The technical setup is elegant because it leans on hardware that already exists. Mobilio pulls from the phone's camera, GPS, the inertial measurement unit that tracks motion and orientation, and the LiDAR sensor where one is available. A custom computer-vision model watches the live camera feed from a pedestrian's point of view and identifies walkable surfaces — sidewalks, crosswalks, roads — in real time. Professor Patrick Slade, who led the work with Ph.D. student Raymond Liu, described the output as "essentially a small autonomous vehicle plan on how to navigate them from where they are to where they want to go." The research is published in Nature Biomedical Engineering.

One of the subtlest and most important decisions in the project is what the model was trained on. Liu points out that most publicly available street-scene datasets are captured from cars, which means the imagery is dominated by a driver's-eye view — roads and vehicles ahead, not the sidewalks, curbs, and crosswalk edges a person on foot actually needs to read. So he trained a semantic-segmentation model specifically on pedestrian-view images instead. This is a cleaner example of a broader problem in computer vision than most people realize: a model's usefulness is bounded not just by how much data it saw, but by whose perspective that data was recorded from. Train on dashcam footage and you get an AI that sees the world like a car. Accessibility, in this case, forced the team to retrain the system to see like a pedestrian.

There's a second design constraint that reads as a feature once you think about it: everything runs on the phone itself, in real time, with no cloud round-trip. Liu calls it a major engineering challenge, since a smartphone has far less compute than the hardware these models usually target. But keeping inference on-device does three quietly valuable things. The camera feed never leaves the user's device, which matters enormously for privacy when the sensor is continuously pointed at the world around you. It works offline, in places with poor connectivity where a cloud-dependent tool would simply fail. And it removes the recurring subscription cost and server infrastructure that might otherwise stand between a navigation aid and the people who need it most.

The audio interface is where the system departs from conventional turn-by-turn apps. Instead of spoken prompts, Mobilio emits continuous beeps that originate from a direction — left or right — to steer the user, and it personalizes those cues on the fly. As a person walks, the system measures how accurately they follow the guidance and adjusts the pitch and pattern of the beeps to suit that individual. This "human-in-the-loop optimization" isn't a gimmick; it's a recognition that "blindness and visual impairment" is not one condition but a wide and diverse spectrum. A single fixed interface cannot fit everyone, so the interface adapts to the body using it rather than demanding the reverse.

The autonomous-vehicle framing is worth pausing on, because it inverts the usual relationship. In a self-driving car, the machine executes the plan and the human is the fallback. In Mobilio, the machine builds the plan but the human executes it — the AI converts a navigation trajectory into a stream of audio cues that a person, with their own judgment and a cane, acts on. That division of labor turns out to be strikingly effective: in the study, Mobilio's reliability at guiding users to a destination without major errors was comparable to a human sighted guide. That is a high bar, and it suggests the sweet spot for assistive AI is often augmenting human capability rather than trying to replace it.

The human element extends to how the project was shaped. The team first surveyed more than 100 blind and low-vision people, who converged on three must-haves: reliable turn-by-turn directions, continuous guidance along paths, and obstacle detection. Existing tools — canes, guide dogs, electronic travel aids — each handle part of that list, but rarely all three, and specialized systems are often expensive or hard to obtain. Liu, whose older brother is blind, has been explicit that his sibling's experience with independent navigation shaped the work, and that testing with real users was treated as non-negotiable: "It's impossible to predict how somebody is going to react to your device," especially across a "huge, diverse set of impairments." The fourteen volunteers who tested Mobilio came from the Carroll Center for the Blind in Newton, Massachusetts.

Zooming out, Mobilio lands in the middle of a quiet but genuinely useful wave of AI accessibility tools. Smartphone apps already help visually impaired users identify nearby objects, and smart glasses can connect them to sighted volunteers who describe what's in front of them. What distinguishes this work is the move from describing the environment to navigating through it continuously — from telling you what's there to getting you safely where you're going. The scale of the audience makes it non-trivial: the World Health Organization estimates at least 2.2 billion people live with some form of vision impairment, and for many, independent mobility is the single biggest daily challenge.

It's worth keeping the findings in perspective. Fourteen participants and a controlled outdoor path are a proof of concept, not a product launch, and the team's next step — testing across a wider range of realistic settings around Boston — is exactly the right one to see how well the system generalizes and how it slots into people's real routines. The results are also relative to a Google Maps baseline that was never designed for safe pedestrian use by this population, so part of the improvement is simply that Mobilio was purpose-built for the task. None of that diminishes the achievement; it just frames it accurately as an early, well-measured result rather than a finished solution.

The broader lesson is about where AI is creating unambiguous value. Amid a lot of noise about chatbots and image generators, assistive technology is one of the few places where the benefit is direct, measurable, and hard to argue with — people moving through the world faster and with fewer collisions. Mobilio's combination of pedestrian-aware training data, fully on-device inference, and adaptive audio guidance is a template for how to build AI that meets people where they are, using the device they already own. That's a more modest ambition than building a robot that navigates for you, but it may turn out to be the more useful one.

Sources

• A Smartphone Navigation App for People with Blindness and Low Vision — Harvard SEAS (primary source)
• AI smartphone app helps people with blindness navigate — Nature Biomedical Engineering press release
• Blindness and vision impairment — World Health Organization fact sheet
• Carroll Center for the Blind — community partner in the study