AutoBNN：構成可能なベイジアンニューラルネットワークを用いた確率的時系列予測

<span class="byline-author">Posted by Urs Köster, Software Engineer, Google Research</span>

<img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgd5Wc54p1HvgIokpazxDsMo1u6i9wg3ovpNOiFc4-wYwebETvjs9-hm2wxZ4osNbBAxhet8To3hwGg-whFScksHQB_BP1kS4Z8Cu7FQT2bjVtJl4trPid-OxCyYocwyRTN66tuvAedu9z0FepBg4zZvmLbLxY6uuib8p5jVH2kfb3RxT_HMABsKMXuSFXr/s320/AutoBNN.jpg" style="display: none;" />

<p>

<a href="https://en.wikipedia.org/wiki/Time_series">Time series</a> problems are ubiquitous, from forecasting weather and traffic patterns to understanding economic trends. <a href="https://en.wikipedia.org/wiki/Bayesian_inference">Bayesian</a> approaches start with an assumption about the data's patterns (prior probability), collecting evidence (e.g., new time series data), and continuously updating that assumption to form a posterior probability distribution. Traditional Bayesian approaches like <a href="https://gaussianprocess.org/gpml/">Gaussian processes</a> (GPs) and <a href="https://blog.tensorflow.org/2019/03/structural-time-series-modeling-in.html">Structural Time Series</a> are extensively used for modeling time series data, e.g., the commonly used <a href="https://gml.noaa.gov/ccgg/trends/">Mauna Loa CO2</a> dataset. However, they often rely on domain experts to painstakingly select appropriate model components and may be computationally expensive. Alternatives such as neural networks lack interpretability, making it difficult to understand how they generate forecasts, and don't produce reliable confidence intervals.

</p>

<a name='more'></a>

<p>

To that end, we introduce <a href="https://github.com/tensorflow/probability/tree/main/spinoffs/autobnn">AutoBNN</a>, a new open-source package written in <a href="https://github.com/google/jax">JAX</a>. AutoBNN automates the discovery of interpretable time series forecasting models, provides high-quality uncertainty estimates, and scales effectively for use on large datasets. We describe how AutoBNN combines the interpretability of traditional probabilistic approaches with the scalability and flexibility of neural networks.

</p>

<div style="line-height: 40%;">

<br />

</div>

<h2>AutoBNN</h2>

<p>

AutoBNN is based on a <a href="https://proceedings.mlr.press/v28/duvenaud13.html">line</a> <a href="https://royalsocietypublishing.org/doi/10.1098/rsta.2011.0550">of</a> <a href="https://proceedings.mlr.press/v202/saad23a.html">research</a> that over the past decade has yielded improved predictive accuracy by modeling time series using GPs with learned <a href="https://www.cs.toronto.edu/~duvenaud/cookbook/">kernel</a> structures. The kernel function of a GP encodes assumptions about the function being modeled, such as the presence of trends, periodicity or noise. With learned GP kernels, the kernel function is defined compositionally: it is either a base kernel (such as <code>Linear</code>, <code>Quadratic</code>, <code>Periodic</code>, <code><a href="https://en.wikipedia.org/wiki/Mat%C3%A9rn_covariance_function">Matérn</a></code> or <code>ExponentiatedQuadratic</code>) or a composite that combines two or more kernel functions using operators such as <code>Addition</code>, <code>Multiplication</code>, or <code><a href="https://icml.cc/Conferences/2010/papers/170.pdf">ChangePoint</a></code>. This compositional kernel structure serves two related purposes. First, it is simple enough that a user who is an expert about their data, but not necessarily about GPs, can construct a reasonable prior for their time series. Second, techniques like <a href="https://www.stats.ox.ac.uk/~doucet/doucet_defreitas_gordon_smcbookintro.pdf">Sequential Monte Carlo</a> can be used for discrete searches over small structures and can output interpretable results.</p>

<p>

AutoBNN improves upon these ideas, replacing the GP with <a href="https://www.cs.toronto.edu/~duvenaud/distill_bayes_net/public/">Bayesian neural networks</a> (BNNs) while retaining the compositional kernel structure. A BNN is a neural network with a probability distribution over weights rather than a fixed set of weights. This induces a distribution over outputs, capturing uncertainty in the predictions. BNNs bring the following advantages over GPs: First, training large GPs is computationally expensive, and traditional training algorithms scale as the cube of the number of data points in the time series. In contrast, for a fixed width, training a BNN will often be approximately linear in the number of data points. Second, BNNs lend themselves better to GPU and <a href="https://cloud.google.com/tpu?hl=en">TPU</a> hardware acceleratio