240513_Thanh_LabSeminar[Learning and Aggregating Lane Graphs for Urban Automated Driving].pptx

Learning and Aggregating Lane Graphs for Urban Automated Driving

1. Learning and Aggregating Lane Graphs for Urban Automated Driving Tien-Bach-Thanh Do Network Science Lab Dept. of Artificial Intelligence The Catholic University of Korea E-mail: os fa19730 @catholic.ac.kr 202 4/05/13 Martin Buchner et al. CVPR 2023

2. Introduction LiDAR view Bird’s-eye-view

3. Introduction Automated driving vehicles rely on the knowledge of their surroundings to safely navigate environments Onboard sensors including LiDARs and cameras provide perception inputs to aggregate representations of the environment However, create and maintain HD maps in the form of lane graphs is a time-consuming and arduous task due to the large amount of detail requirements Propose a novel two-stage GNN named LaneGNN operates on single aerial color images for lane graph prediction

4. Related Works Road Graph Learning Road graphs encode the topological connections between road segments Lane graphs describe the locations and connectivity between all lanes Prior works investigate estimating road graphs from both onboard sensors and aerial images, or extract pixel-level road segmentation from images and extract graphical road representations [1,19] use morphological image operators or GNN to extract the connectivity between different roads within the image [2,22] investigate iterative methods and interpret road graph prediction as a sequential prediction task

5. Related Works Lane Graph Learning from Vehicle Data Onboard vehicle sensors such as cameras and LiDAR [17] aggregate onboard LiDAR data on highways and leverage a RNN to generate highway lane graphs [35] use OpenStreetMap database to predict from a vehicle ego-view into a map representation [31] propose online road map extraction system for onboard sensor and construct GR of road network [5,6] proposed methods for lane connectivity learning in intersection scenarios from onboard camera

6. Related Works Lane Graph Learning from Birds-Eye-View Data Not much works on aerial data [36] propose a lane centerline regression model jointly with a Graph R-CNN backbone to predict nodes and edges of the lane graph from a local aggregated bird’s-eye-view image crop [15] propose a two-stage graph estimation, extract lanes at non-intersection areas and subsequently predict the connectivity of each pair of lanes, and extract the valid turning lanes to complete the map

7. Technical Approach - Lane Graph Learning Lane Regression and Graph Construction

8. Technical Approach - Lane Graph Learning Formulate task as supervised learning problem where successor lane graph is estimated based on an aerial image I bev A directed graph G I = (V,E) covering relevant regions is constructed by sampling from likely region of I bev The graph that constitute potentially valid lane graph edges The graph is attributed using both node, edge features, where D geo and D bev denote the dimensionality of the involved edge features

9. Technical Approach - Lane Graph Learning Lane Regression and Graph Construction Train 2 regression networks Centerline regression network predicting the likelihood map of lane centerline S lane Segment network predicting all reachable lane S ego lane starting from the initial virtual agent pose at the bottom center of I bev PSPNet architectures with a ResNet-152 feature extractor, sample equally-distributed node positions using Halton sequences, filtered based o n the obtained ego-lane segmentation mask, serves as a ROI for sampling Directed edges E among nodes are intialized for pairs of nodes with Euclidean distance d ij ∈ [d min ,d max ] Initial node features X are based on their 2D positions, geometric edge features are defined as Generate aerial edge features: per edge, a small oriented region of I bev and the lane segmentation S lane is obtained based on the direction of the edge

10. Technical Approach - Lane Graph Learning Feature Encoding and Message Passing Estimate edge probabilities, node probabilities and whether a node is terminal Utilize a causal variant of neural message passing as proposed by Braso

11. Technical Approach - Lane Graph Learning Feature Encoding and Message Passing Encode predecessor and successor features during message passing Initial node and geometric edge features are encoded using MLP while aerial edge feature is transformed using ResNet-18 Ground truth graph GGT as a learning target is generated based on the map annotations for the given cropped region and the corresponding closest nodes

12. Technical Approach - Iterative Temporal Graph Aggregation Pruning Aggregate local successor graphs into a globally consistent lane graph Formulate the graph pruning problem as a search problem from a starting node to possibly multiple predicted terminal nodes Use Dijkstra’s algorithm to iteratively find high-score paths between the initial pose and terminal nodes, ordered from high to low scores until all terminal nodes are reached

13. Technical Approach - Iterative Temporal Graph Aggregation Aggregation Aggregate predicted successor lane graphs G pred = (V pred ,E pred ) into a globally consistent and complete graph G agg = (V agg ,E agg )

14. Technical Approach - Iterative Temporal Graph Aggregation Aggregation Predicted successor graph G t pred is added to the current aggregated graph G t-1 agg at time step t: G t agg <- aggregate(G t pred , G t-1 agg ) Only updated in a lateral sense while the longitudinal misalignment of the two sets of nodes is neglected Apply multiple iterations of Laplacian smoothing, which modifies the original node positions in order to even out position irregularities caused by sampling while keeping the adjacency

15. Experimental Results Dataset Compiled the UrbanLaneGraph dataset - first-of-its-kind dataset for large-scale lane graph estimation from aerial images Contain aerial images from the cities of Austin, Miami, Pittsburgh, Palo Alto, Detroit, Washington DC Connect all local lane graphs of each city and aggregate them into one globally consistent graph per city

16. Experimental Results Proposed tasks Propose 3 distinct and complementary tasks Successor lane graph prediction, aim at predicting a feasible ego-reachable successor lane graphs from the current pose of the virtual agent. Purpose of the Successor-LGP task is to measure the prediction quality of potential future driving paths when no HD map available Full-LGP, evaluate the quality of regionally aggregated lane graphs in the context of HD map estimation. Aim to measure the predictive power of full two-stage model performing lane graph inference and graph aggregation in conjunction Full ground truth lane graph of a given map area is compared to the aggregated prediction High-level path planning task on the predicted lane graphs, intended to analyze the fidelity of routes planned on the predicted graphs

17. Experimental Results Evaluation Metrics Graph IoU APLS metric TOPO/GEO metrics Split Detection Accuracy (SDA R )

18. Experimental Results Successor Lane Graph Prediction

19. Experimental Results Successor Lane Graph Prediction

20. Experimental Results Full Lane Graph Prediction

21. Experimental Results Full Lane Graph Prediction

22. Experimental Results Path Planning

23. Experimental Results Limitations Perform well in urban and suburban but struggles with highly complex graph topologies such as multi-lane intersections or roundabouts The inference time increases with the number of nodes and edges

24. Conclusion Novel lane graph estimation framework complemented with a novel dataset comprising aerial images Lane graph estimation problem as bottom-up GNN approach leveraging agent-centric views yields Novel aggregation scheme to merge successive lane graphs to produce large-scale solutions A first-of-its-kind dataset and benchmark for lane graph estimation from aerial images