Neural networks (NN) are rapidly gaining prominence in automated decision making, in particular in safety-critical tasks such as navigating autonomous cars through dense traffic. It is imperative in this context that the NN decisions meet central societal desiderata regarding dependability, perspicuity, explainability, and robustness. Yet, how to ensure that this is the case remains a grand challenge — indeed, one of the great challenges in AI at this time. That challenge can only properly be addressed from the bottom up. Foundational research is chiefly required to establish concepts and methodologies able to tackle the combined complexity of NN decision making together with the dynamics of complex environments.
To enable this foundational research, in a joint effort across Projects C2, C3, C5, C6, and E4, we have identified and implemented a set of natural and easy-to-control abstractions that connect the autonomous driving challenge to the formal modelling world of Markov decision processes. They are based on Racetrack, a popular benchmark in the AI decision making literature. Our orchestrated efforts have extended this benchmark in many directions, including continuous dynamics, traffic, stochastic events, resource consumption, and different navigation and vehicle scenarios (planes, drones), making it a joint use case basis creating lab conditions for perspicuity research. This has led to systematic, structured, and extensible studies of NN behaviour, NN learning performance, NN verification, and analysis techniques (e.g. the picture on the right shows crash probabilities of an NN action policy when started from different positions on the map), interactive visual analysis (in our TraceVis tool), and evidence-based continuous certification.
This joint effort seized a new, originally unplanned, opportunity to align our research agendas and to create new synergy across projects, reinforcing a common theme across the entire scientific landscape of CPEC, from Bachelor’s theses to award-winning publications. This joint agenda and infrastructure will be continued and extended in the second funding period.