Mathematics > Statistics Theory
[Submitted on 19 Dec 2016 (v1), revised 23 Jan 2017 (this version, v3), latest version 18 Jan 2019 (v4)]
Title:Revisiting maximum-a-posteriori estimation in log-concave models: from differential geometry to decision theory
View PDFAbstract:Maximum-a-posteriori (MAP) estimation is the main Bayesian estimation methodology in many areas of data science such as mathematical imaging and machine learning, where high dimensionality is addressed by using models that are log-concave and whose posterior mode can be computed efficiently by using convex optimisation algorithms. However, despite its success and rapid adoption, MAP estimation is not theoretically well understood yet, and the prevalent view is that it is generally not proper Bayesian estimation in a decision-theoretic sense. This paper presents a new decision-theoretic derivation of MAP estimation in Bayesian models that are log-concave. Our analysis is based on differential geometry and proceeds as follows. First, we exploit the log-concavity of the model to induce a Riemannian geometry on the parameter space. We then use differential geometry to identify the natural or canonical loss function to perform Bayesian point estimation in that Riemannian manifold. For log-concave models this canonical loss is the Bregman divergence of the negative log posterior density, a similarity measure rooted in convex analysis that in addition to the relative position of points also takes into account the geometry of the space, and which generalises the Euclidean squared distance to non-Euclidean settings. We then show that the MAP estimator is the Bayesian estimator that minimises the expected canonical loss, and that the posterior mean or MMSE estimator minimises the expected dual canonical loss. Finally, we establish universal performance and stability guarantees for MAP and MMSE estimation in high dimensional log-concave models. These results provide a new understanding of MAP and MMSE estimation under log-concavity, and reveal new insights about their good empirical performance and about the roles that log-concavity plays in high dimensional inference problems.
Submission history
From: Marcelo Pereyra [view email][v1] Mon, 19 Dec 2016 12:16:26 UTC (15 KB)
[v2] Wed, 21 Dec 2016 14:49:10 UTC (15 KB)
[v3] Mon, 23 Jan 2017 13:08:42 UTC (13 KB)
[v4] Fri, 18 Jan 2019 14:22:16 UTC (1,068 KB)
Current browse context:
math.ST
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.