publications

2023

1. 

   Sample Path Regularity of Gaussian Processes from the Covariance Kernel

   Nathaël Da Costa, Marvin Pförtner, Lancelot Da Costa, and Philipp Hennig

   2023

   Abs arXiv Bib PDF

   Gaussian processes (GPs) are the most common formalism for defining probability distributions over spaces of functions. While applications of GPs are myriad, a comprehensive understanding of GP sample paths, i.e. the function spaces over which they define a probability measure on, is lacking. In practice, GPs are not constructed through a probability measure, but instead through a mean function and a covariance kernel. In this paper we provide necessary and sufficient conditions on the covariance kernel for the sample paths of the corresponding GP to attain a given regularity. We use the framework of Hölder regularity as it grants us particularly straightforward conditions, which simplify further in the cases of stationary and isotropic GPs. We then demonstrate that our results allow for novel and unusually tight characterisations of the sample path regularities of the GPs commonly used in machine learning applications, such as the Matérn GPs.

   @misc{DaCosta2023PathRegularityKernel,
     author = {Da Costa, Natha\"el and Pf\"ortner, Marvin and Da Costa, Lancelot and Hennig, Philipp},
     title = {Sample Path Regularity of Gaussian Processes from the Covariance Kernel},
     year = {2023},
     publisher = {arXiv},
     doi = {10.48550/arxiv.2312.14886},
     url = {https://arxiv.org/abs/2312.14886},
   }

2022

1. 

   Physics-Informed Gaussian Process Regression Generalizes Linear PDE Solvers

   Marvin Pförtner, Ingo Steinwart, Philipp Hennig, and Jonathan Wenger

   2022

   Abs arXiv Bib PDF Code

   Linear partial differential equations (PDEs) are an important, widely applied class of mechanistic models, describing physical processes such as heat transfer, electromagnetism, and wave propagation. In practice, specialized numerical methods based on discretization are used to solve PDEs. They generally use an estimate of the unknown model parameters and, if available, physical measurements for initialization. Such solvers are often embedded into larger scientific models with a downstream application and thus error quantification plays a key role. However, by ignoring parameter and measurement uncertainty, classical PDE solvers may fail to produce consistent estimates of their inherent approximation error. In this work, we approach this problem in a principled fashion by interpreting solving linear PDEs as physics-informed Gaussian process (GP) regression. Our framework is based on a key generalization of the Gaussian process inference theorem to observations made via an arbitrary bounded linear operator. Crucially, this probabilistic viewpoint allows to (1) quantify the inherent discretization error; (2) propagate uncertainty about the model parameters to the solution; and (3) condition on noisy measurements. Demonstrating the strength of this formulation, we prove that it strictly generalizes methods of weighted residuals, a central class of PDE solvers including collocation, finite volume, pseudospectral, and (generalized) Galerkin methods such as finite element and spectral methods. This class can thus be directly equipped with a structured error estimate. In summary, our results enable the seamless integration of mechanistic models as modular building blocks into probabilistic models by blurring the boundaries between numerical analysis and Bayesian inference.

   @misc{Pfoertner2022LinPDEGP,
     author = {Pf\"ortner, Marvin and Steinwart, Ingo and Hennig, Philipp and Wenger, Jonathan},
     title = {Physics-Informed {G}aussian Process Regression Generalizes Linear {PDE} Solvers},
     year = {2022},
     publisher = {arXiv},
     doi = {10.48550/arxiv.2212.12474},
     url = {https://arxiv.org/abs/2212.12474},
   }

2. 

   Posterior and Computational Uncertainty in Gaussian Processes

   Jonathan Wenger, Geoff Pleiss, Marvin Pförtner, Philipp Hennig, and John P. Cunningham

   In Advances in Neural Information Processing Systems, 2022

   Abs arXiv Bib PDF Supp Code

   Gaussian processes scale prohibitively with the size of the dataset. In response, many approximation methods have been developed, which inevitably introduce approximation error. This additional source of uncertainty, due to limited computation, is entirely ignored when using the approximate posterior. Therefore in practice, GP models are often as much about the approximation method as they are about the data. Here, we develop a new class of methods that provides consistent estimation of the combined uncertainty arising from both the finite number of data observed and the finite amount of computation expended. The most common GP approximations map to an instance in this class, such as methods based on the Cholesky factorization, conjugate gradients, and inducing points. For any method in this class, we prove (i) convergence of its posterior mean in the associated RKHS, (ii) decomposability of its combined posterior covariance into mathematical and computational covariances, and (iii) that the combined variance is a tight worst-case bound for the squared error between the method’s posterior mean and the latent function. Finally, we empirically demonstrate the consequences of ignoring computational uncertainty and show how implicitly modeling it improves generalization performance on benchmark datasets.

   @inproceedings{Wenger2022IterGP,
     author = {Wenger, Jonathan and Pleiss, Geoff and Pf\"ortner, Marvin and Hennig, Philipp and Cunningham, John P.},
     title = {Posterior and Computational Uncertainty in {G}aussian Processes},
     year = {2022},
     booktitle = {Advances in Neural Information Processing Systems},
     volume = {35},
     editor = {Koyejo, S. and Mohamed, S. and Agarwal, A. and Belgrave, D. and Cho, K. and Oh, A.},
     publisher = {Curran Associates, Inc.},
     pages = {10876--10890},
     doi = {10.48550/arxiv.2205.15449},
     url = {https://proceedings.neurips.cc/paper_files/paper/2022/hash/4683beb6bab325650db13afd05d1a14a-Abstract-Conference.html},
   }

2021

1. 

   ProbNum: Probabilistic Numerics in Python

   Jonathan Wenger, Nicholas Krämer, Marvin Pförtner, Jonathan Schmidt, Nathanael Bosch, Nina Effenberger, Johannes Zenn, Alexandra Gessner, Toni Karvonen, François-Xavier Briol, Maren Mahsereci, and Philipp Hennig

   2021

   Abs arXiv Bib PDF Code

   Probabilistic numerical methods (PNMs) solve numerical problems via probabilistic inference. They have been developed for linear algebra, optimization, integration and differential equation simulation. PNMs naturally incorporate prior information about a problem and quantify uncertainty due to finite computational resources as well as stochastic input. In this paper, we present ProbNum: a Python library providing state-of-the-art probabilistic numerical solvers. ProbNum enables custom composition of PNMs for specific problem classes via a modular design as well as wrappers for off-the-shelf use. Tutorials, documentation, developer guides and benchmarks are available online at http://www.probnum.org/.

   @misc{Wenger2021ProbNum,
     author = {Wenger, Jonathan and Kr\"amer, Nicholas and Pf\"ortner, Marvin and Schmidt, Jonathan and Bosch, Nathanael and Effenberger, Nina and Zenn, Johannes and Gessner, Alexandra and Karvonen, Toni and Briol, Fran\c{c}ois-Xavier and Mahsereci, Maren and Hennig, Philipp},
     title = {{ProbNum}: Probabilistic Numerics in Python},
     year = {2021},
     publisher = {arXiv},
     doi = {10.48550/arxiv.2112.02100},
     url = {https://arxiv.org/abs/2112.02100},
   }