使用类优先级优化输入图像 [英] Optimize input image with class prior

问题描述

我正在尝试实施google博客条目的第一部分 Inceptionism:深入研究神经网络 TensorFlow.到目前为止，我已经找到了几种资源，可以用自然语言进行解释，也可以专注于在其他部分或提供其他框架的代码段.我了解针对某类先验优化随机输入图像的想法，以及此

I'm trying to implement the first part of the google blog entry Inceptionism: Going Deeper into Neural Networks in TensorFlow. So far I have found several resources that either explain it in natural language or focus on other parts or give code snippets for other frameworks. I understand the idea of optimizing a random input image with respect to a class prior and also the maths behind it given in the this paper, section 2, but I'm not able to implement it myself using TensorFlow.

来自

From this SO question and the helpful comment by etarion, I now know that you can give a list of variables to the optimizer, while all other variables are untouched. However, when giving the optimizer a random image as a variable leads to

File "mnist_test.py", line 101, in main
    optimizer2 = tf.train.AdamOptimizer(learning_rate).minimize(-cost, var_list=[rnd_img])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 198, in minimize
  name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 309, in apply_gradients
  (converted_grads_and_vars,))
ValueError: No gradients provided for any variable: ((None,<tensorflow.python.ops.variables.Variable object at 0x7feac1870410>),)

出于测试目的，我使用了一个简化的MNIST示例.我尽力使它尽可能短，同时仍然可读和可执行:

For testing purpose I used a stripped down MNIST example. I tried to keep it as short as possible while still being readable and executable:

def main():
    # parameters
    learning_rate = 0.001
    train_batches = 1000
    batch_size = 128
    display_step = 50

    # net parameters
    n_input = 784 #28x28
    n_classes = 10
    keep_prob = 0.75

    weights = {
        'wc1': tf.Variable(tf.truncated_normal([5, 5, 1, 32])),
        'wc2': tf.Variable(tf.truncated_normal([5, 5, 32, 64])),
        'wd1': tf.Variable(tf.truncated_normal([7*7*64, 1024])),
        'out': tf.Variable(tf.truncated_normal([1024, n_classes]))
    }

    biases = {
        'bc1': tf.Variable(tf.constant(0.1, shape=[32])),
        'bc2': tf.Variable(tf.constant(0.1, shape=[64])),
        'bd1': tf.Variable(tf.constant(0.1, shape=[1024])),
        'out': tf.Variable(tf.constant(0.1, shape=[n_classes]))
    }

    # tf inputs
    x = tf.placeholder(tf.float32, [None, n_input])
    y = tf.placeholder(tf.float32, [None, n_classes])
    dropout = tf.placeholder(tf.float32)

    # create net
    net = create_net(x, weights, biases, keep_prob)

    # define loss
    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(net, y))

    # define optimizer
    optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)

    # evaluation
    pred_correct = tf.equal(tf.argmax(net, 1), tf.argmax(y, 1))
    accuracy = tf.reduce_mean(tf.cast(pred_correct, tf.float32))

    print "loading mnist data"
    from tensorflow.examples.tutorials.mnist import input_data
    mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)

    sess = tf.Session()
    sess.run(tf.initialize_all_variables())

    for i in xrange(train_batches):
        batch_x, batch_y = mnist.train.next_batch(batch_size)

        sess.run(optimizer, feed_dict={x: batch_x, y: batch_y, dropout: keep_prob})

        if i % display_step == 0:
            loss, acc = sess.run([cost, accuracy], feed_dict={x: batch_x, y: batch_y, dropout: 1.0})
            print "batch: %i, loss: %.5f, accuracy: %.5f" % (i, loss, acc)

    acc = sess.run(accuracy, feed_dict={x: mnist.test.images, y: mnist.test.labels, dropout: 1.0})
    print "test accuracy: %.5f" % (acc)

    # ====== this is where the reconstruction begins =====

    rnd_img = tf.Variable(tf.random_normal([1, n_input]))
    one_hot = np.zeros(10)
    one_hot[4] = 1;

    # the next line causes the error
    optimizer2 = tf.train.AdamOptimizer(learning_rate).minimize(-cost, var_list=[rnd_img])

    for i in xrange(1000):
        session.run(optimizer2, feed_dict={x: rnd_img, y: one_hot, dropout: 1.0})

    sess.close()

if __name__ == "__main__":
    main()

我使用的辅助函数:

def create_net(x, weights, biases, dropout):
    x = tf.reshape(x, shape=[-1, 28, 28, 1])

    conv1 = conv2d_relu(x, weights['wc1'], biases['bc1'])
    conv1 = maxpool2d(conv1, 2)

    conv2 = conv2d_relu(conv1, weights['wc2'], biases['bc2'])
    conv2 = maxpool2d(conv2, 2)

    fc1 = fullyconnected_relu(conv2, weights['wd1'], biases['bd1'])
    fc1 = tf.nn.dropout(fc1, dropout)

    out = tf.add(tf.matmul(fc1, weights['out']), biases['out'])

    return out

def conv2d_relu(x, W, b, stride=1):
    conv = tf.nn.conv2d(x, W, strides=[1, stride, stride, 1], padding='SAME')
    conv = tf.nn.bias_add(conv, b)
    return tf.nn.relu(conv)

def maxpool2d(x, k=2, stride=2, padding='VALID'):
    return tf.nn.max_pool(x, ksize=[1, k, k, 1], strides=[1, stride, stride, 1], padding=padding)

def fullyconnected_relu(x, W, b):
    fc = tf.reshape(x, [-1, W.get_shape().as_list()[0]])
    fc = tf.add(tf.matmul(fc, W), b)
    fc = tf.nn.relu(fc)

我发现一些消息来源说，当输出和要优化的变量之间的计算图中没有路径时，就会发生此错误，但是我不明白为什么会出现这种情况.

I've found some sources saying that this error occurs when there is no path within the computation graph between the output and the variables to be optimize, but I don't see why this should be the case here.

我的问题是:

1. 为什么优化器无法应用任何渐变?
2. 这是实现类的可视化的正确方法吗?

谢谢.

在合并了接受的答案之后(对于有兴趣的人)，这又是完整的代码.无论如何，结果仍然不如预期，因为该脚本在经过100000轮重建后基本上会生成随机图像.欢迎提出想法.

Here is the complete code again, after incorporation of the accepted answer (for anyone who is interested). Anyway, the results are still not as expected, as the script basically produces random images after 100000 rounds of reconstruction. Ideas are welcome.

import tensorflow as tf
import numpy as np
import skimage.io

def conv2d_relu(x, W, b, stride=1):
    conv = tf.nn.conv2d(x, W, strides=[1, stride, stride, 1], padding='SAME')
    conv = tf.nn.bias_add(conv, b)
    return tf.nn.relu(conv)

def maxpool2d(x, k=2, stride=2, padding='VALID'):
    return tf.nn.max_pool(x, ksize=[1, k, k, 1], strides=[1, stride, stride, 1], padding=padding)

def fullyconnected_relu(x, W, b):
    fc = tf.reshape(x, [-1, W.get_shape().as_list()[0]])
    fc = tf.add(tf.matmul(fc, W), b)
    fc = tf.nn.relu(fc)

    return fc;

def create_net(x, weights, biases, dropout):
    x = tf.reshape(x, shape=[-1, 28, 28, 1])

    conv1 = conv2d_relu(x, weights['wc1'], biases['bc1'])
    conv1 = maxpool2d(conv1, 2)

    conv2 = conv2d_relu(conv1, weights['wc2'], biases['bc2'])
    conv2 = maxpool2d(conv2, 2)

    fc1 = fullyconnected_relu(conv2, weights['wd1'], biases['bd1'])
    fc1 = tf.nn.dropout(fc1, dropout)

    out = tf.add(tf.matmul(fc1, weights['out']), biases['out'])

    return out

def save_image(img_data, name):
    img = img_data.reshape(28,28)

    mi = np.min(img)
    ma = np.max(img)

    img = (img-mi)/(ma-mi)
    skimage.io.imsave(name, img)

def main():
    # parameters
    learning_rate = 0.001
    train_batches = 1000
    batch_size = 100
    display_step = 50

    # net parameters
    n_input = 784 #28x28
    n_classes = 10
    keep_prob = 0.75

    weights = {
        'wc1': tf.Variable(tf.truncated_normal([5, 5, 1, 32])),
        'wc2': tf.Variable(tf.truncated_normal([5, 5, 32, 64])),
        'wd1': tf.Variable(tf.truncated_normal([7*7*64, 1024])),
        'out': tf.Variable(tf.truncated_normal([1024, n_classes]))
    }

    biases = {
        'bc1': tf.Variable(tf.constant(0.1, shape=[32])),
        'bc2': tf.Variable(tf.constant(0.1, shape=[64])),
        'bd1': tf.Variable(tf.constant(0.1, shape=[1024])),
        'out': tf.Variable(tf.constant(0.1, shape=[n_classes]))
    }

    # tf inputs
    x = tf.placeholder(tf.float32, [None, n_input])
    y = tf.placeholder(tf.float32, [None, n_classes])
    dropout = tf.placeholder(tf.float32)

    # create net
    net = create_net(x, weights, biases, dropout)

    # define loss
    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(net, y))

    # define optimizer
    optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)

    # evaluation
    pred_correct = tf.equal(tf.argmax(net, 1), tf.argmax(y, 1))
    accuracy = tf.reduce_mean(tf.cast(pred_correct, tf.float32))

    print "loading mnist data"
    from tensorflow.examples.tutorials.mnist import input_data
    mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)

    sess = tf.Session()

    sess.run(tf.initialize_all_variables())

    for i in xrange(train_batches):
        batch_x, batch_y = mnist.train.next_batch(batch_size)

        sess.run(optimizer, feed_dict={x: batch_x, y: batch_y, dropout: keep_prob})

        if i % display_step == 0:
            loss, acc = sess.run([cost, accuracy], feed_dict={x: batch_x, y: batch_y, dropout: 1.0})
            print "batch: %i, loss: %.5f, accuracy: %.5f" % (i, loss, acc)

    acc = sess.run(accuracy, feed_dict={x: mnist.test.images, y: mnist.test.labels, dropout: 1.0})
    print "test accuracy: %.5f" % (acc)

    # reconstruction part
    rnd_img = tf.Variable(tf.random_normal([1, n_input]))

    one_hot = np.zeros((1, 10))
    one_hot[0,1] = 1;

    net2 = create_net(rnd_img, weights, biases, dropout)
    cost2 = -tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(net2, y))
    optimizer2 = tf.train.AdamOptimizer(learning_rate).minimize(cost2, var_list=[rnd_img])

    init_var_list = []

    for var in tf.all_variables():
        if(not tf.is_variable_initialized(var).eval(session=sess)):
            init_var_list.append(var)

    sess.run(tf.initialize_variables(init_var_list))

    save_image(rnd_img.eval(sess), "bevor.tiff")

    for i in xrange(100000):
        _, loss = sess.run([optimizer2, cost2], feed_dict={y: one_hot, dropout: 1.0})
        if(i%10000 == 0):
            cur_img = rnd_img.eval(session=sess)
            print "loss:", loss, "mi:", np.min(cur_img), "ma:", np.max(cur_img)

    save_image(rnd_img.eval(sess), "after.tiff")

    sess.close()

if __name__ == "__main__":
    main()

一些解释:用新的输入变量和优化器重建图之后，我不得不初始化新变量，即rnd_img和Adam优化器使用的一些辅助变量，因此遍历all_variables()并检查初始化状态.如果有人知道更优雅的方式，请告诉我.还是这就是为什么我没有得到任何结果的原因?

Some explanation: After rebuilding the graph with the new input variable and optimizer, I had to initialize the new variables, i.e. the rnd_img and some helper variables used by the Adam optimizer, hence the loop over all_variables() and checking for initialization status. If somebody knows a more elegant way, let me know. Or maybe that's the reason why I don't get any results?

推荐答案

rnd_img需要包含您优化的图形的一部分.在您的情况下，您只需创建一个变量并告诉优化器对其进行优化，但该变量与图中的损失无关.例如，您可以做的是使用rnd_image而不是x(但使用相同的权重！)使用另一个对create_net的调用，为此创建成本，然后为该成本创建一个最小化操作.然后为了进行优化，您只输入y.

The rnd_img needs to part of the graph that you optimize. In your case, you just create a variable and tell the optimizer to optimize it, but the variable is not connected to the loss in the graph. What you can for example do is use another call to create_net with rnd_image instead of x (but using the same weights!), create the cost for that and then create a minimization op for that cost. Then for optimization you only feed in y.

这篇关于使用类优先级优化输入图像的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！