Investigating Intersection Safety with 3D Object Detection and Digital Twin Technology from Video Data

D. Patel, A. Nayeem, R. Alfaris, M. Jalayer
Rowan University,
United States

Keywords: intersection safety, artificial intelligence, YOLO, digital twin, SSM


The use of digital twin technology to improve the safety and efficiency of intersections is gaining popularity. Using data from sensors and cameras, the technique generates a virtual representation of a physical intersection that is updated in real-time. The Federal Highway Administration (FHWA) annually documents more than 2.5 million intersection-related crashes. 40 percent of all crashes occur at intersections, making them the second most common category for crashes. 20 percent of fatal fatalities and 50 percent of serious injury crashes occur at intersections. This study proposes an artificial intelligence-based video analytic approach for evaluating intersection safety based on surrogate safety measures and non-compliance behavior. The system employs Post-encroachment Time (PET) and Time-to-Collision (TTC) to identify potential risks, such as rear-end and left-turning crashes, caused by interactions between vehicles and road users. To extract the high-resolution trajectory data, this project integrates a real-time AI detection model - YOLO-v8 with a 3D segmentation and in concurrence with tracking and digital twin. 18 hours of high-resolution video data were collected from two New Jersey intersections. Non-compliance behaviors, such as vehicles red-light running or speeding, and pedestrians not using the crosswalk, are captured to understand the risky behaviors at intersections better. Furthermore, the homography transformation converted high-resolution data with point cloud data points from the 3D segmentation. This framework aims to provide practitioners with previously inaccessible data with a high level of accuracy to enable engineers and policymakers to make sustainable decisions that improve intersection safety without compromising efficiency.