Machine Learning: Breaking up the Blackbox

Yanick Zeder  7 August 2018 13:09:09

One of the most obvious differences between a predicion from an expert and an ai system is that you can simply ask a person on what his predictions are based. He is able to explain his thought-process. This makes it much easier to weight the value of what he is saying. On the other hand, while we often work with neural networks and machine learning, we often struggle to explain what realy happens inside. We are often not able to explain why the system behaves how it does. This makes it very hard to convince a customers to trust in the predicitons of machine learning models. But there is light on the horizon!

The problem of overfitting

One large problem in machine learning is that the model is not learning what it should. One symptom of this is poor performace in real live conditions compared to the performace while training the model. In other words, the ai system generalizes badly. This issue has its own name and is called overfitting. It basically means, that the system is learning the training samples by hart and therefore performs very well on the training data.

The black-box problem

It is relativelly easy to detect that overfitting is happening. One basic strategy for example is to sepparate the data in to two groups; one group is only used for training and the other one to check the generalisation. While detection is basically a solved problem, finding the cause turns out to be a much harder problem. This is mainly due to the black-box-like nature of neural networks. In that regards it is indeed very similar to our brain. Your options are basically reduced to observing the inputs and outputs.
If you are like me and you try to build solutions for customers, you know that it is hard to sell blackboxes where not even you fully understand the behalvior. Especially in safety-relevant circumstances where you want to be sure that your system does what in needs to do this is not acceptable.
So the problem comes down to the quesiton: How can i breack up this black-box system and see behind its desicions?


The idea of explainers

As descibed above, it would be great to see the imprortance of each input to the outputs. Exactly this is what an explainer does. In short, an explainer explores a machine learning model by finding out which input value was most important for its desicion. For example, to decide if a photo contains a male or a female, the explainer highlights the parts of the image which were most important. Sort of like a heatmap of attention. In a very similar way it can highlight other input data, like for example bithdate, address, age and so on.

How does this help?

It does not fully remove the problem of overfitting. But it is one of the best tools out there to find clues on what data the model may overfitt.

• Comments [0]