Banded Matrix Operators for Gauss-Markov Models in the Automatic Differentiation Era

back to our research

Banded Matrix Operators for Gauss-Markov Models in the Automatic Differentiation Era

April 16 - 18, 2019 AISTATS, Naha, Okinawa, Japan

Authors: Nicolas Durrande, Vincent Adam, Lucas Bordeaux, Stefanos Eleftheriadis and James Hensman

Abstract: Banded matrices can be used to express several models including linear state-space models, some Gaussian processes, and Gauss-Markov random fields. Whilst software libraries such as TensorFlow, PyTorch and Stan are changing the face of machine learning and statistics, banded matrices have been underutilized; the banded representation of models can avoid the inefficient use of loops in high level programming languages and allows easy construction of more complex models from constituent parts (e.g. additive or deep GPs). In this work we revisit the banded representation of several models and examine which banded matrix operations are required to implement inference using variational inference or gradient-based sampling. We collect the necessary operators and derive their reverse-mode derivatives, which can all be executed in linear time.

Probabilistic Modelling

Banded Matrices

Markov Structures

TensorFlow

Gaussian Processes

Variational Inference


See paper

Join us to make AI that will change the world

join our team