Drone shows are becoming increasingly popular as large-scale light shows. These shows can feature hundreds or even thousands of aerial robots, each programmed to fly along flight paths that form intricate shapes and patterns in the air. When done right, drone shows can be spectacular. But when one or more drones malfunction, as has happened recently in Florida, New York and other areas, they can pose a significant danger to audiences on the ground.
The drone crash highlights the difficulty of maintaining safety in what engineers call “multi-agent systems” — robots, drones, self-driving cars and other computers that are programmed to communicate and cooperate with each other.
Now, a team of engineers at MIT has developed a method for training multi-agent systems that can ensure they operate safely even in crowded environments. The researchers found that when the method is used to train a small number of agents, the safety margins and controls that those agents learn can be automatically scaled to many more agents in a way that ensures the safety of the entire system.
In a real-world demonstration, the team trained a small number of palm-sized drones to safely execute a variety of tasks, from simultaneously switching positions in flight to landing on designated ground vehicles. In simulations, the researchers replicated a similar program trained on a small number of drones and scaled it to thousands of drones to show that a large-scale agent system could safely execute similar tasks.
“This could become the standard for any application that requires a group of agents, such as warehouse robots, search-and-rescue drones, or self-driving cars,” said Chuchu Huang, an associate professor of aeronautics and astronautics at MIT. “This provides a safety shield or filter that allows each agent to continue their mission and we teach them how to stay safe.”
Fan and his colleagues report their new approach in a study published this month in IEEE Transactions on Robotics, co-authored by MIT graduate students Songyuan Zhang and Oswin Soh, and Kunal Garg, a former MIT postdoctoral scholar who is now an assistant professor at Arizona State University .
Shopping mall profit margins
When engineers design safety into a multi-agent system, they often must consider the potential paths of each agent relative to every other agent in the system. This pairwise path planning is a time-consuming and computationally expensive process. It still does not guarantee safety.
“In a drone show, each drone is given a specific trajectory — a set of stopping positions and times — and the drones basically follow the plan with their eyes closed,” said Zhang, the study’s lead author. “They only know where they need to be and when, so they have no idea how to adapt when something unexpected happens.”
Instead, the MIT team aimed to develop a way to train a small number of safety control agents in a way that could scale efficiently to any number of agents in a system. And instead of planning a specific path for each agent, this approach allows the agents to continuously map safety margins or boundaries of potential unsafety. The agents can choose any number of paths to complete their task, as long as they stay within the safety margin.
In a way, the researchers say, this approach is similar to how humans visually navigate their surroundings.
“Let’s say you’re in a crowded shopping mall,” Seo explains, “and you’re not concerned with moving safely and not bumping into anyone except the people within five meters of you. Our work takes a localized approach as well.”
Safety Barrier
In the new study, the team presented a method they call GCBF+, short for “Graph Control Barrier Function.” A barrier function is a mathematical term used in robotics to calculate the type of safety barrier or boundary beyond which an agent is likely to become unsafe. For any agent, this safe zone can change from moment to moment as the agent moves among other agents moving through the system.
When designers calculate barrier functions for any agent in a multi-agent system, they often need to consider the potential paths and interactions with all other agents in the system. Instead, the MIT team’s method calculates safe zones for a small number of agents accurately enough to represent the dynamics of many other agents in the system.
“You can then copy and paste this barrier function onto each agent, and you’ll instantly have a graph of safe zones that will work for any number of agents in your system,” Seo said.
To calculate an agent’s barrier function, methods in the first group take into account the agent’s “sensing radius”, i.e., how much of its surroundings the agent can observe depending on its sensing capabilities. Similar to the shopping mall analogy, researchers assume that the agent only cares about agents within its sensing radius, in terms of staying safe and avoiding collisions with those agents.
Next, using a computer model that captured the agent’s specific mechanical capabilities and limitations, the team simulated a “controller” — a set of instructions for how the agent and a small number of similar agents should move. They then ran simulations of multiple agents moving along specific trajectories, recording whether they collided or otherwise interacted.
“Once we have these trajectories, we can calculate some laws that we want to minimize, such as the number of safety violations in the current controller,” Zhang said. “We then updated the controller to make it safer.”
In this way, the controller can be programmed to be a real-world agent, and the controller can continually map a safe zone based on other agents it senses around it, and then move within that safe zone to complete a task.
“Our controllers are reactive,” Fan said. “We don’t plan paths in advance. Our controllers are constantly receiving information about where the agent is going, how fast it’s going, and how fast other drones are going. They use all this information to make plans on the fly and re-execute the plans every time, so they can constantly adapt and stay safe, even as the situation changes.”
The research team demonstrated GCBF+ with a system of eight lightweight, palm-sized quadcopter drones tasked with flying and repositioning in the air. If the drones were to do so by taking the most direct path possible, they would surely collide. But after being trained using the team’s method, the drones were able to switch positions in flight, making real-time adjustments to move around each other while staying within their respective safe zones.
Similarly, the team tasked drones with flying around and landing on specific TurtleBots (wheeled robots with clam-like tops), which always move in wide circles to allow the Crazy Flies to avoid colliding with each other when landing.
“With our framework, you only need to tell the drone the destination point instead of the entire collision-free trajectory, and the drone can figure out how to get to the destination without colliding,” said Fan, who envisions the approach being applied to any multi-agent system for ensuring safety, including collision avoidance systems for drone shows, warehouse robots, autonomous vehicles, drone delivery systems and more.
This research was supported in part by the U.S. National Science Foundation, MIT Lincoln Laboratory’s Safety in Aerobatic Flight Regime (SAFR) program, and the Defense Science and Technology Agency of Singapore.
