KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support - INRIA - Institut National de Recherche en Informatique et en Automatique Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support

Pierre Glaser
  • Fonction : Auteur
  • PersonId : 1118675
Arthur Gretton
  • Fonction : Auteur
  • PersonId : 943040

Résumé

We study the gradient flow for a relaxed approximation to the Kullback-Leibler (KL) divergence between a moving source and a fixed target distribution. This approximation, termed the KALE (KL Approximate Lower bound Estimator), solves a regularized version of the Fenchel dual problem defining the KL over a restricted class of functions. When using a Reproducing Kernel Hilbert Space (RKHS) to define the function class, we show that the KALE continuously interpolates between the KL and the Maximum Mean Discrepancy (MMD). Like the MMD and other Integral Probability Metrics, the KALE remains well-defined for mutually singular distributions. Nonetheless, the KALE inherits from the limiting KL a greater sensitivity to mismatch in the support of the distributions, compared with the MMD. These two properties make the KALE gradient flow particularly well suited when the target distribution is supported on a low-dimensional manifold. Under an assumption of sufficient smoothness of the trajectories, we show the global convergence of the KALE flow. We propose a particle implementation of the flow given initial samples from the source and the target distribution, which we use to empirically confirm the KALE's properties.
Fichier principal
Vignette du fichier
2106.08929.pdf (1.28 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03455473 , version 1 (29-11-2021)

Identifiants

  • HAL Id : hal-03455473 , version 1

Citer

Pierre Glaser, Michael Arbel, Arthur Gretton. KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support. NeurIPS 2021 -Thirty-Fifth Annual Conference on Neural Information Processing Systems, Dec 2021, Online, France. pp.1-29. ⟨hal-03455473⟩
65 Consultations
82 Téléchargements

Partager

Gmail Facebook X LinkedIn More