Skip to main content
ENHU
Home

Main navigation

  • Discover
    • News
    • Events
    • Tenders
  • Research fields
  • Resources
    • Publications
    • Downloads
  • About us
  • Partners
  1. Home
  2. Publications
(2021) Proceedings of Machine Learning Research 2640-3498 139 10214-10224

Moreau-Yosida f-divergences

arxiv.org/abs/2102.13416v2
Széchenyi Plusz RRF
Abstract

Variational representations of f-divergences are central to many machine learning algorithms, with Lipschitz constrained variants recently gaining attention. Inspired by this, we define the Moreau-Yosida approximation of f-divergences with respect to the Wasserstein-1 metric. The corresponding variational formulas provide a generalization of a number of recent results, novel special cases of interest and a relaxation of the hard Lipschitz constraint. Additionally, we prove that the so-called tight variational representation of f-divergences can be to be taken over the quotient space of Lipschitz functions, and give a characterization of functions achieving the supremum in the variational representation. On the practical side, we propose an algorithm to calculate the tight convex conjugate of f-divergences compatible with automatic differentiation frameworks. As an application of our results, we propose the Moreau-Yosida f-GAN, providing an implementation of the variational formulas for the Kullback-Leibler, reverse Kullback-Leibler, χ2, reverse χ2, squared Hellinger, Jensen-Shannon, Jeffreys, triangular discrimination and total variation divergences as GANs trained on CIFAR-10, leading to competitive results and a simple solution to the problem of uniqueness of the optimal critic.

Authors
Dávid Terjék
Institutes
Read more
Home

LinkedIn

Become a partner

Subscribe to newsletter

Send partnership request

Explore

  • News
  • Events
  • Tenders
  • Publications
  • Downloads
  • Partners

Research fields

  • Foundations of AI
  • Human Language Processing
  • Machine perception
  • Medical, Health and Biology
  • Security and Privacy
  • Sensors, IoT and Telecommunications

Contact us

Hungary, H-1111 Budapest,
Kende u. 13-17.
+36 1 279 6000
@email

© 2020-2021 Artifical Intelligence National Laboratory, Budapest