In Quantum Mechanics, Heisenberg’s Uncertainty Principle states that there is a fundamental limit to how well one can measure a particle’s **position** and **momentum**. In the context of machine learning systems, a similar principle has emerged, but relating **interpretability** and **performance**. By using a manually wired or shallow machine learning model, you’ll have no problem understanding the moving pieces, but you will seldom be happy with the results. Or you can use a black-box deep neural network and enjoy the model’s exceptional performance. Today we’ll see one simple and effective trick to make our deep black boxes a bit more intelligible. The trick allows us to convert neural network outputs into probabilities, with no cost to performance, and minimal computational overhead.

The desire to understand deep neural networks has triggered a flurry of research into Neural Network Visualization, but in practice we are often forced to treat deep learning systems as black-boxes. (See my recent Deep Learning Trends @ ICLR 2016 post for an overview of recent neural network visualization techniques.) But just because we can’t grok the inner-workings of our favorite deep models, it doesn’t mean we can’t ask more out of our deep learning systems.

There exists a simple trick for upgrading black-box neural network outputs into probability distributions.

The probabilistic approach provides confidences, or ”uncertainty” measures, alongside predictions and can make almost any deep learning systems into a smarter one. For robotic applications or any kind of software that must make decisions based on the output of a deep learning system, being able to provide meaningful uncertainties is a true game-changer.

The key ingredient is **dropout**, an anti-overfitting deep learning trick handed down from Hinton himself (Krizhevsky’s pioneering 2012 paper). Dropout sets some of the weights to zero during training, reducing feature co-adaptation, thus improving generalization.

Without dropout, it is too easy to make a moderately deep network attain 100% accuracy on the training set.

The accepted knowledge is that an un-regularized network (one without dropout) is too good at memorizing the training set. For a great introductory machine learning video lecture on dropout, I highly recommend you watch Hugo Larochelle’s lecture on Dropout for Deep learning. Geoff Hinton’s dropout lecture, also a great introduction, focuses on interpreting dropout as an ensemble method. If you’re looking for new research ideas in the dropout space, a thorough understanding of Hinton’s interpretation is a must.

But while dropout is typically used at training-time, today we’ll highlight the keen observation that **dropout used at test-time is one of the simplest ways to turn raw neural network outputs into probability distributions**. Not only does this probabilistic ”free upgrade” often improve classification results, it provides a meaningful notion of uncertainty, something typically missing in Deep Learning systems.

The idea is quite simple: to estimate the predictive mean and predictive uncertainty, simply collect the results of stochastic forward passes through the model using dropout.

**How to use dropout: 2016 edition**

- Start with a moderately sized network
- Increase your network size with dropout turned off until you perfectly fit your data
- Then, train with dropout turned on
- At test-time, turn on dropout and run the network T times to get T samples
- The mean of the samples is your output and the variance is your measure of uncertainty

**Bayesian Convolutional Neural Networks**

To be truly Bayesian about a deep network’s parameters, we wouldn’t learn a single set of parameters **w**, we would infer a distribution over weights given the data, p(**w**|**X**,**Y**). Training is already quite expensive, requiring large datasets and expensive GPUs.

Bayesian learning algorithms can in theory provide much better parameter estimates for ConvNets and I’m sure some of our friends at Google are working on this already.

But today we aren’t going to talk about such full Bayesian Deep Learning systems, only systems that ”upgrade” the model prediction **y** to p(**y**|**x**,**w**). In other words, only the network outputs gain a probabilistic interpretation.

An excellent deep learning computer vision system which uses test-time dropout comes from a recent University of Cambridge technique called SegNet. The SegNet approach introduced an Encoder-Decoder framework for dense semantic segmentation. More recently, SegNet includes a Bayesian extension that uses dropout at test-time for providing uncertainty estimates. Because the system provides a dense per-pixel labeling, the confidences can be visualized as per-pixel heatmaps. Segmentation system is not performing well? Just look at the confidence heatmaps!

The Bayesian SegNet authors tested different strategies for dropout placement and determined that a handful of dropout layers near the encoder-decoder bottleneck is better than simply using dropout near the output layer. Interestingly, Bayesian SegNet improves the accuracy over vanilla SegNet. Their confidence maps shown high uncertainty near object boundaries, but different test-time dropout schemes could provide a more diverse set of uncertainty estimates.

Confidences are quite useful for evaluation purposes, because instead of providing a single average result across all pixels in all images, we can sort the pixels and/or images by the overall confidence in prediction. When evaluation the top 10% most confident pixels, we should expect significantly higher performance. For example, the Bayesian SegNet approach achieves 75.4% global accuracy on the SUN RGBD dataset, and an astonishing 97.6% on most confident 10% of the test-set [personal communication with Bayesian SegNet authors]. This kind of sort-by-confidence evaluation was popularized by the PASCAL VOC Object Detection Challenge, where precision/recall curves were the norm. Unfortunately, as the research community moved towards large-scale classification, the notion of confidence was pushed aside. Until now.

**Theoretical Bayesian Deep Learning**

Deep networks that model uncertainty are truly meaningful machine learning systems. It ends up that we don’t really have to understand how a deep network’s neurons process image features to trust the system to make decisions. As long as the model provides uncertainty estimates, we’ll know when the model is struggling. This is particularly important when your network is given inputs that are far from the training data.

In a recent ICML 2016 paper, Yarin Gal and Zoubin Ghahramani develop a new theoretical framework casting dropout training in deep neural networks as approximate Bayesian inference in deep Gaussian processes. Gal’s paper gives a complete theoretical treatment of the link between Gaussian processes and dropout, and develops the tools necessary to represent uncertainty in deep learning. They show that a neural network with arbitrary depth and non-linearities, with dropout applied before every weight layer, is mathematically equivalent to an approximation to the probabilistic deep Gaussian process. I have yet to see researchers use dropout between every layer, so the discrepancy between theory and practice suggests that more research is necessary.

In conclusion, maybe we can never get both interpretability and performance when it comes to deep learning systems. But, we can all agree that providing confidences, or uncertainty estimates, alongside predictions is *always* a good idea. Dropout, the very single regularization trick used to battle overfitting in deep models, shows up, yet again. Sometimes all you need is to add some random variations to your input, and average the results over many trials. Dropout lets you not only wiggle the network inputs but the entire architecture.