Making Neural Networks More Trustworthy and Sustainable - Professor Michael Unser

12th Harry Nicholson Lecture in Control Engineering - Professor Michael Unser, Professor of Image Processing, École polytechnique fédérale de Lausanne (EPFL), Switzerland

Thursday 6 June 2024

Abstract

The use of deep neural networks (DNNs) is currently transforming many areas of science and engineering.  Although DNN-based techniques outperform traditional algorithms in most signal processing tasks, they can exhibit weaknesses such as reduced robustness and a tendency to produce hallucinations. These issues are linked to the DNN's Lipschitz constant, which typically worsens exponentially with the addition of layers.

In this work, we present a framework for the design of stable networks with maximal expressivity, Our scheme involves a combination of learnable 1-Lip activations and a few energy-preserving convolution layers. We train these activations using a second-order total variation penalty, leading to adaptive linear spline solutions and a corresponding representer theorem for Lip-1 deep splines.  We illustrate our method with the task of image reconstruction, and demonstrate state-of-the-art results.