Jo, Jason, and Yoshua Bengio. “Measuring the tendency of CNNs to Learn Surface Statistical Regularities.” arXiv preprint arXiv:1711.11561 (2017).
CNN learn differently then we as humans learn. What CNNs learn can be filtered out in the frequency domain.
Interesting paper that looks at the generalization abilities of deep CNN architectures. They begin by highlighting the generalization gap or the difference between the learning curves of the test and transit and how typically, in deep CNN architectures, the gap is relatively small. They then go on to hight how that this small generalisation gap is often attributed to the fact that that deep CNNs learn high-level semantic meaning. They then counter that common notion by highlighting the recent and popular research in Adversarial Examples, and note the high sensitivity to adverbial pertubtations. If a CNN is learning semantic meaning, then why does adding static to the image break it? Also, related is the recent research by Zhang, Chiyuan, et al. “Understanding deep learning requires rethinking generalization.” in which they show that CNN arch. can perfectly fit random labels which leads us to more down the path that generalization capabilities of CNNs are currently unknown to the community. They then go on to introduce their experiment to try and isolate what CNNs are doing.
Given the fact that the noise added to images by adversarial perturbations so drastically affects CNNs, the authors posit, that what the CNNs are doing is much more simple than semantic representation. They introduce and argument that CNNs are simply learning patterns in the statistical distribution of the images. To test this, they introduce a filtering method called Fourier Filtering in which the original image is modified in a way such that is still easily perceivable by a human, however, the underlying distribution of each image is no longer unique. So if the system is not learning semantic meaning, then it should fail the task and the generalization gap should increase.
In figure 2 above you can see that the filtered images are still recognisable, however, unbeknownst to the human eye, the images have lost their original distribution of pixel intensities.
The Figure above shows the learning curves for a Resnet-based model trained CIFAR 10. The blue model on the bottom is trained on the original images. So if the CNN is learning semantics, then the generalization gap between the blue and the test tests should all be small. It is for the the unfiltered test however, it is NOT for the filtered tests sets. So what is the CNN learning???
The authors then go on to discuss the physical properties of natural images or photographs. The cite literature from computer vision studies which shows that natural images have particual patterns that CNNs learn and that it could not learn when those patterns were destroyed by the Fourier Filtering:
Drawing upon computer vision literature on the statistical regularity of natural images, we believe that it is possible for natural image training and test datasets to share many superficial cues. By learning these superficial cues, a machine learning model would be able to sidestep the issue of high level concept learning and generalize well. To this end, we posed our main hypothesis: the current incarnation of deep neural networks have a tendency to learn surface statistical regularities as opposed to high level abstractions.
The authors also touch on adversarial training which does mitigate this problem in that the model is introduced to the strange distributions. This is in the vein of augmenting your dataset and has as regularisation effect. However, they are also keen to point out that it is only a temporary fix and that the flaw in CNNs is systematic and that a new type of adversarial perturbation will put us back in the same boat b/c from what we can tell so far, CNNs are not learning semantics.