When Champlain Towers South in Florida’s Surfside suburb partially collapsed in June, the search and response (SAR) effort employed modern innovations. Some of the first devices on the scene were robots from Teledyne FLIR, a company specializing in sensing technologies. Its FirstLook automated ground robot, a self-righting device that can propel itself into tight spaces and send back images, helped in the early stages of the rescue effort.
SAR teams are increasingly turning to new technologies that can map out chaotic areas in disaster situations, from collapsed buildings to floods and hurricanes. Unmanned ground robots like Teledyne FLIR’s are a first choice in many scenarios that are either too unsafe or too small for human operators or dogs to navigate.
“The most immediate risk to humanity in the wake of one of these disasters may not be something you can see,” explains David Proux, vice president of Teledyne FLIR’s unmanned aerial vehicle business. There could be dangers ranging from ruptured gas lines to fires or toxic leaks. “The robot gets the first look and sends it in real time back to the operator for the planning of safer, more effective human interventions,” he continues.
While operators control these robots to a certain extent, they also need the autonomy to make their own decisions in a tight spot. Proux explains that they combine computer vision—a form of artificial intelligence (AI)—to understand what’s around them along with logic based on the direction and distance traveled. The team programs the system with trained behaviors so that it can optimize its path during explorations.
Getting devices several feet into a collapsed structure is one challenge. Getting data back out is another. Teledyne FLIR uses a mesh network that enables subterranean vehicles to continually maintain wireless communications with each other, relaying signals to the surface. This can get data back to operators, but if that data has to reach other people, then cellular operators might step in.
AT&T’s FirstNet, a nationwide wireless system built for first responders, features portable local cell stations either on land or in the air. It has a blimp, FirstNet one, which it deployed for the first time in September 2020 to relay communications in the aftermath of Louisiana’s Hurricane Laura.
From the ground to the air
Talking of airborne help, drones have become valuable tools in SAR missions, explains David Schedl, professor of visual computing at the University of Applied Sciences Upper Austria. He specializes in computer vision, computational photography and machine learning (ML).
“In comparison to a crewed helicopter, the operational costs for drones are considerably lower,” Schedl says. “Another factor in favor of drones, although not as obvious as in indoor scenarios, is the reduced risks for pilots and the crew during night flights and bad weather conditions.”
It takes a military drone or aircraft to stay aloft for long periods of time with the complex, high-resolution sensor packages necessary for wide-area SAR operations, say experts. However, short-range consumer-grade zones are useful for mapping smaller areas such as building collapses. These drones can be used to map disaster zones into grids and then assign SAR teams to different areas.
Depending on the operation, drones and aircraft can have different sensor types, explains Brian Spillane, vice president of business development for airborne law enforcement within Teledyne FLIR’s sensor systems business. “Light is just one part of the electromagnetic spectrum,” he says. Beyond visible light, there’s the low-light vision used in night-vision cameras, alongside thermal and short-range infra-red, both of which are useful at night and in seeing through smoke.
Depending on the environment and time of day, we can pick the camera that’s best suited to find our lost hiker or Alzheimer’s patient who’s wandered off or autistic child who’s gone from the neighborhood.
—Brian Spillane, VP of business development, Teledyne FLIR
Another sensor type, light detection and ranging (LIDAR), uses pulsed lasers to measure distance. While it can’t see through rubble, it can map uneven territory with high accuracy to build up a detailed area map. It can also detect humans in open terrain and measure the depth of the seafloor from above the water, making it useful in water rescue.
“Depending on the environment and time of day, we can pick the camera that’s best suited to find our lost hiker or Alzheimer’s patient who’s wandered off or autistic child who’s gone from the neighborhood,” Spillane adds.
Making sense of the data
The next leaps forward are in making sense of the data from all these sensors. “Right now, a human being has to interpret what they’re seeing,” says Spillane. In time, AI algorithms will handle more of that for SAR operators to remove the cognitive burden.
Ease of use is a big factor in SAR technology, explains Mark Neveau, a former fire chief and Federal Emergency Management Agency coordinating officer who has assisted in disasters ranging from the 2008 California wildfires to Hurricanes Sandy and Ike. “It must be easy to operate,” he advises. Rescue workers operate in busy, stressful conditions, so devices must present data as easily as possible.
FLIR’s moving target indicator for airborne systems already analyzes movement and color, spotting colors used to signal distress in maritime situations such as bright hues on life jackets.
In some cases, devices are using AI to see the unseeable by combining sensor data. Schedl and his colleagues have created a prototype drone that looks underneath forest canopies from the air by using computers to defocus the tops of trees in camera images. It combines this with thermal imaging that finds heat sources and then uses ML to analyze those hot spots, distinguishing human forms from animals. The prototype drone can do all this without human intervention, enabling widespread forest searches while reporting potential human locations back to SAR teams.
SAR technology will continue to evolve, with robots getting smaller and more innovative. A snake-like Japanese robot that slithers up pipes and ladders could one day be used in SAR missions. Quadrupedal and bipedal robots also show promise. As disasters continue to strike, there’s no shortage of room for human innovation to save lives.
Lead photo courtesy of Teledyne FLIR