Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

The coupling effect of Lipschitz regularization in deep neural networks

Abstract : We investigate robustness of deep feed-forward neural networks when input data are subject to random uncertainties. More specifically, we consider regularization of the network by its Lipschitz constant and emphasize its role. We highlight the fact that this regularization is not only a way to control the magnitude of the weights but has also a coupling effect on the network weights accross the layers. We claim and show evidence on a dataset that this coupling effect brings a tradeoff between robustness and expressiveness of the network. This suggests that Lipschitz regularization should be carefully implemented so as to maintain coupling accross layers.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [19 references]  Display  Hide  Download

https://hal-enac.archives-ouvertes.fr/hal-02090498
Contributor : Nicolas Couellan <>
Submitted on : Thursday, April 4, 2019 - 5:56:34 PM
Last modification on : Thursday, March 5, 2020 - 5:58:34 PM

File

LipANN.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02090498, version 1
  • ARXIV : 1904.06253

Citation

Nicolas Couellan. The coupling effect of Lipschitz regularization in deep neural networks. 2019. ⟨hal-02090498⟩

Share

Metrics

Record views

89

Files downloads

439