Towards Trustworthy and Explainable AI for Perception Models: From Concept to Prototype Vehicle Deployment
Researchers have developed a new trustworthy AI perception module designed for autonomous driving systems. This module integrates explainability features derived from the attention mechanism of a transformer-based detector, validated for faithfulness through consistency tests. It also includes calibrated uncertainty estimation and robustness-enhancing training methods. The system has been successfully deployed in a prototype vehicle, demonstrating real-time monitoring capabilities with an interface visualizing documentation, uncertainty, and saliency maps. AI
IMPACT Enhances safety and transparency in autonomous driving systems by providing explainable AI and uncertainty estimates.