Boosting Black-Box Variational Inference by Incorporating the Natural Gradient

In this paper we present a modification of thepopular Black-Box Variational Inference (BBVI) approachwhich significantly improves the computational efficiency of theinference. We achieve this performance boost by replacing thestandard gradient in the stochastic gradient ascent framework ofBBVI with the natural gradient. Our experimental results (e.g.training of neutral networks) show that the proposed methodoutperforms the original BBVI algorithm on both synthetic andreal data.