Autonomous Convoy Routing via Drone Swarms and Multi-Modal Threat Detection

The final presentation of the AutoGators team in the AWS & Vanderbilt Mission Autonomy Hackathon on October 10, 2025.

This project won the "Most Innovative" Award (10k USD).

Code: https://github.com/ishaan-sen/vb-autonomy-hacks

1. Autonomous convoy routing via drone swarms and multi‑modal threat detection Krish Kapadia 1 , Yilin Liu 2 , Zhenjiang Mao 1 , Ishaan Sen 1 , Zhongzheng Zhang 1 , Zhouyang Zhou 1 10/06-10/10, 2025 Sponsored by AWS and Maximus By the  AutoGators [1] University of Florida [2] Vanderbilt University

2. 2 Team Members Krish Kapadia Yilin Liu Zhenjiang Mao Ishaan Sen Zhongzheng Zhang Zhouyang Zhou Sponsored by AWS and Maximus

3. Why Autonomous Ro ute P lanning Matters 3 Motivation Sponsored by AWS and Maximus Supply-chain convoys operate in dynamic and uncertain environments Human operators have limited situational awareness and can’t react quickly to fast-changing threats. Traditional centralized navigation systems fail under communication loss or incomplete data .

4. Project Goals 4 Sponsored by AWS and Maximus Enable real-time adaptive navigation through a swarm of drones that detect, analyze, and communicate threats.  Augment human decision-making rather than replace it.

5. P roblem Definition Sponsored by AWS and Maximus 5 Given a road network and a reconnaissance drone swarm flying over it, plan safe and efficient convoy routes that avoid detected hazards while keeping humans in the loop.

6. Technical Challenges Sponsored by AWS and Maximus 6 Dynamically detect threats along the convoy's path Autonomously reroute the ground convoy to avoid danger Find optimal safe path by minimizing travel time and distance Leave final decision-making authority with human operators Integrate efficiently with existing infrastructure

7. Novelty of the Solution Contributions 1. Three-Modality Threat Detection: Vision, Thermal, and Sound 3. Human-Centered Emergency Decision Making: Humans m ed iate the decision-making instead of end-to-end AI 2. Two-Stage Swarm-Guided Navigation Framework: Static Environment Scanning, Dynamic Route Planning Sponsored by AWS and Maximus 7

8. Demo Case Study: Nashville Area 1. Real-world map of Nashville, Tennessee 3. A demo based in Central Nashville Path planning and real-time decision making 2. Real/S ynthetic Dataset & Environment Sponsored by AWS and Maximus 8 34363 Nodes, 113462 Edges For Vision: i5500 Images, 4 threat categories

9. System Overview Solution Threats Detector Path Planning Algorithms “Optimal” Path Nashville City Map Static Environment Scanning Threats Detector Human Decision Making Final Path Dynamic Route Planning Sponsored by AWS and Maximus 9 Path Planning Algorithms Nashville City Map

10. AWS integration Solution AWS integration: IoT Core S3 SageMaker Bedrock DynamoDB EC2 Sponsored by AWS and Maximus 10

11. Threat Detection Solution Sound Anomaly Detection  RCNN AWS  Rekognition AWS S3 CNN Vision Anomaly Detection  Thermal Anomaly Detection  Sponsored by AWS and Maximus 11

12. Vision Anomaly Detection Evaluation AIDER Image Dataset: 5138 images, 4 Anomaly Classes F1 Precision Recall Tested Image Fire 0.990 0.990 0.990 105 Flood 0.990 1.000 0.981 106 Traffic 0.942 0.957 0.928 97 Collapsed Building 0.970 1.000 0.942 103 Sponsored by AWS and Maximus 12

13. Sound Anomaly Detection Evaluation Sound Wave Dataset: 2208 segments, 2 Anomaly Classes F1 Precision Recall Tested Segments Fire 1.000 1.000 1.000 288 Normal 1.000 1.000 1.000 288 Sponsored by AWS and Maximus 13

14. Thermal Anomaly Detection Evaluation Image Dataset: 1000 images, 2 Detection Classes F1 Precision Recall Tested Image Fire 0.9899 1.000 0.9800 50 Normal 0.9901 0.9804 1.000 50 Sponsored by AWS and Maximus 14

15. Path Planning The output of our solution is in the form of a path selection, which accounts for:  Road blockages Hazard zones Single points of failure Solutions Sponsored by AWS and Maximus 15

16. Path Planning The path planning algorithm uses a modified extension of the A* graph pathfinding algorithm for routing decisions.  Solutions Sponsored by AWS and Maximus 16

17. Cost Analysis System Specification Swarm Size: 20–45 drones Sensors: RGB camera, thermal sensor, and sound module Onboard Compute: Optional GPU/CPU for edge AI processing Hardware Cost Market Price Range: $5K–$12K per drone, including sensors DJI Mavic 3T Enterprise DJI Matrice 30T Sponsored by AWS and Maximus 17

18. Limitations and Future Directions Limitations Sound sensing accuracy drops in noisy or windy conditions Thermal and audio modules increase payload → shorter flight time Cloud-based model calls may introduce latency Future Directions Strengthen human–AI collaboration for faster, safer responses Design lightweight multi-modal fusion (vision + thermal + audio) models Integrate onboard AI chips to enable full edge autonomy Sponsored by AWS and Maximus 18

19. Key Takeaways Primary Objective: Safe  and  Optimal   Route Planning   in an  Emergency  Situation AutoGators ' Solution: Three-Modality Threat Detection Two-Stage Swarm-Guided Navigation Framework Human-guided navigation decisions Our Example Demonstration: Based on a real-world road map of Nashville Presenting a demo in the area around Vanderbilt's campus Sponsored by AWS and Maximus 19

20. Contact Information Yilin Liu: [email protected] Ishaan Sen: [email protected] Krish Kapadia: [email protected] Zhenjiang Mao: [email protected] Zhongzheng Zhang: [email protected] Zhouyang Zhou: [email protected] Sponsored by AWS and Maximus 20

21. Thank You! AWS Provides Cloud Services Maximus provides citizen services Sponsored by AWS and Maximus 21