Neural networks have made remarkable advances in image processing, classification and speech recognition. These very useful tools are based on well-known mathematical methods, yet we know very little about why some models work and others don’t. Here’s a technique that enable us to look inside a network, or even create images like an artist.
Neural networks can be grouped into different architectures (FC, RNN, CNN, etc.) according to the task to be performed or by input type. In the field of image processing, an outstanding structure, CNN, as a result of convolution with different filters, can extract information from images. Such networks typically consist of 30 to 60 paired neurons depending on the complexity of the task to be performed. The images are fed to the input layer one by one, followed by the different layers, until we finally reach the output layer, which represents the decision to perform on the given task.
Teaching the network, and thus adjusting the parameters of the filters, is done with millions of teaching examples, gradually adjusting the weights of the network until we reach the desired grading quality.
We know that our network accurately classifies an animal in the picture, but we don’t know, which features were the base of its decision (this is the speciality of artificial intelligence because it does the logical connection independently, we didn’t define it). We need this information to understand the operation of the network. One of the possible ways to extract this information is to spin the network upside down. Instead of making a decision with the network based on an input image, we will examine what image will maximize the filter of a chosen layer. This can be achieved by iterating the gradient ascent (towards the input layer).
Images created with the usage of this method, can be evaluated and ensure us what the network finds essential and fundamentally related to certain classes.
CONVOLUTIONAL NEURAL NETWORKS FEATURE VISUALIZATION WITH THE GRADIENT METHOD
- Place an arbitrary image to the input (it can even be full white).
- Optimize the picture: maximize the chosen neural network layer with the gradient ascent method.
- Calculate the activation of the chosen layer, which we would like to maximize.
- Determine the gradient of the given activation with respect to the maximal value.
- Propagate the gradient to the input layer (backpropagation), but this time not for the weights, but with respect to the pixels of the input (opposite to traditional teaching)
- With the usage of the gradients, modify the image. This image will be noisy, caused by the high frequency components and the lack of connection between pixels.
- Regularize the image, which will decrease the effect caused by the gradient update.
- Use this image as the start image, so we can jump to the first step. Keep this up until you reach the result what you need.
Evaluating the propagated images, we can conclude that the deeper we are in the layers of the net, the more complex features are evaluated. A layer close to the input detects edges or corners, intermediate layers interpreting basic features, general shapes, or components (wheel, windscreen). The last few layers compile them into a specific object, these neurons react to complex things such as a car.
THE ALGORITHM BRIEFLY:
DEEPDREAM is an algorithm developed by Google, with the usage of the visualization above, which creates series of artistic images. From this sequence of images, a video can be generated later, with hallucinogenic effects appearing.
The reason for supplementing the original algorithm is that the pictures produced by conventional techniques are noisy.
THE RECURSIVE PROCESS IN STEPS (OCTAVE)
- Place an image to the input.
- With Gauss filter dim the image, to avoid artifacts.
- Downscale the image with respect to the predetermined scalar. (EXIT POINT)
- Add the image and the previous image together pixel by pixel. (ENTER POINT)
- Apply the above described algorithm on the image.
- Upscale the image to the original size.
An octave’s recursive embedding
We are confident that inside your company there are a lot of tasks which can be automated with AI: In case you would like to enjoy the advantages of artificial intelligence, then apply to our free consultation on one of our contacts.
 Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich, ArXiv, 2014