Augmented Reality Aids Nighttime Swiftwater Rescue

If you were blindfolded and dropped onto the banks of the Chattahoochee River near Johns Creek, Georgia, you might swear you were in the middle of nowhere. The Metropolitan River Protection Act, enacted in the 1970s, created an undisturbed vegetative buffer along an 84-mile-long section of the river, including the 14-mile stretch that forms the southern and eastern borders of the city. It feels like wilderness despite being part of Metro Atlanta.

Water released upstream at the Buford Dam amplifies that wild river feeling. When cold water from the depths of Lake Lanier courses down the Chattahoochee River during daily releases, it causes rapids and a rise in surface levels by as much as 11 feet. This swift, cold water forms an ideal habitat for trout, but it can imperil people when they’re caught off-guard by quickly changing conditions.

“There’s a website with a release schedule but if you didn’t check that, it’s pretty easy to be caught unaware,” said Nick O’Day, chief data officer at the City of Johns Creek, Georgia. “Our fire department gets calls at all times of the day and night to make rescues.”

In summer months, when the water temperature warms, river activity increases.

“It’s tubing season, and there’s always a lot of activity from sports fishermen, so it’s not uncommon for us to be called to the river weekly this time of year,” said Lt. Aaron Roberts of the Johns Creek Fire Department, and a leader of the Swiftwater Rescue Team.

Complicating the situation is the fact that there are few recognizable landmarks to determine locations while on the water. Dense riverbank vegetation makes spots along the river undistinguishable.

“The problem that everybody has is that it’s so heavily wooded that it’s very easy to lose track of where you are,” O’Day said. “The fire department knows where they are at launch but as they move toward a landmark it’s very easy to get disoriented.”

Also, dam release times and volumes vary, which causes constantly changing conditions that make it tough for even the most experienced operator.

“We have a few guys that know the ins and outs of the river because they’re on it so often,” Roberts said. “However, knowing the river when it’s low doesn’t mean that they can operate with the same efficiency when it’s high.”

At low water levels, it’s easy to see snags, sand bars and shoals. The fire department’s river rescue boat is hardened to these conditions, with a rock-proof bottom and jet drive that can operate safely in just six inches of water. “While we can slide across rocks and go over sandbars, we still want to avoid those areas because they can fling people from the boat,” Roberts said.

In addition to the physical obstacles, cold water mixing with warmer water can produce thick fog, and the air often fills with bugs.

“The fog is thick enough that at times you cannot see the shore or make out stars in the sky,” Roberts said. “It’s hard to explain the thickness of the insects, we wear eye protection, and you definitely can’t smile.”

These compounding conditions cause frustration for first responders.

“If we have a known location of somebody needing help but we can’t see to get there, we’re no help to that person,” Roberts said.

Augmented River Piloting

The City of Johns Creek set out to map a solution using AuGeo, an augmented reality application developed by Esri Labs. This software makes use of the location data that O’Day stewards for the city within its geographic information system (GIS). He equipped an iPad with the software and overlaid the city’s data at key points of interest. As they travel, firefighters can view the river through an iPad mounted to the boat to see key points in their field of view which helps to orient them.

“They needed a heads-up display, because they’re cruising down the river at 40 or 50 miles an hour,” O’Day said. “They don’t have time to pan, zoom or scroll.”

The solution also incorporates a night vision sensor fitted over the camera lens to deal with the darkness.

“It’s not just a matter of knowing the landmarks around you,” O’Day added. “The night-vision attachment we engineered cuts through the fog and darkness to show obstructions—rocks, branches, logs—as they speed toward their goal.”

The rescue boat has powerful lights and a thermal imaging sensor, but the lights can’t always cut through the fog and fog distorts the thermal imaging sensor. The night vision sensor and the augmented reality view provide a solution that works for the firefighters.

“We took Nick [O’Day] out on the river and showed him what we do, and he did the rest,” Roberts said. “The operation is super simple, which is big for us. It doesn’t take a techno-whiz or GIS specialist to figure it out.”

For O’Day, the key was to get inside the operational workflows and find the most appropriate use of the technology. This way he was able to create an app designed specifically for what the rescue team is trying to do.

Faster Response Times

Everyone involved in the effort is focused on reducing danger on the river and providing help quickly. The app has been crucial to meeting these objectives.

“It adds more situational awareness for the boat crew,” O’Day said. “Hopefully it eliminates guessing where they are and saves them minutes in their response times.”

As important as it is to have an idea of where one is located on the river, it’s critically important for those using the river to know the water release schedule and how long it takes to see a rise in the river where they are.

“Often times, people will get out in the river not knowing it will rise and they get stuck on a rock with swift water flowing around them,” Roberts said. “The gratitude and relief they feel knowing they get to go home that night can be overwhelming for them and for us.”

There are other times when the outcome is more tragic.

“About a year ago,” Roberts said. “A teenager got swept up by a flood and his friend jumped in to save him. Both drowned, and it was days before we found them. The families were worried that they got swept all the way to the Gulf. It was… closure for the family, but also sad.”

River safety and swiftwater rescue is a concern that goes beyond the city limits. The solution Johns Creek pioneered is being shared with other cities and agencies along the river including nearby fire departments, the National Park Service, and the Georgia Department of Natural Resources.

“We’re hoping that this prototype solution might help others,” O’Day said. “The more success we have with it, and the more we learn, the more we can share to raise the bar for everyone.”

Learn how public safety professionals uses GIS to monitor situations and make adjustments to keep a community safe and see what others can’t.