在Keras中的GPU上微调VGG-16:内存消耗 [英] Finetuning VGG-16 on GPU in Keras: memory consumption

问题描述

我正在为任务调整VGG-16.我的想法是，我加载预训练的权重，删除最后一层(具有1000个输出的softmax)，并用具有几个输出的softmax替换它.然后冻结除最后一层以外的所有层，并训练模型.

I'm fine-tuning VGG-16 for my task. The idea is that I load the pretrained weights, remove the last layer (which is softmax with 1000 outputs) and replace it with a softmax with a few outputs. Then I freeze all the layers but the last and train the model.

这是构建原始模型并加载权重的代码.

Here is the code that builds the original model and loads the weights.

def VGG_16(weights_path=None):
    model = Sequential()
    model.add(ZeroPadding2D((1,1),input_shape=(224,224,3)))
    model.add(Conv2D(64, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(64, (3, 3), activation='relu'))
    model.add(MaxPooling2D((2,2), strides=(2,2)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(128, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(128, (3, 3), activation='relu'))
    model.add(MaxPooling2D((2,2), strides=(2,2)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(256, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(256, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(256, (3, 3), activation='relu'))
    model.add(MaxPooling2D((2,2), strides=(2,2)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(MaxPooling2D((2,2), strides=(2,2)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(MaxPooling2D((2,2), strides=(2,2)))

    model.add(Flatten())
    model.add(Dense(4096, activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(4096, activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(1000, activation='softmax'))

    if weights_path:
        model.load_weights(weights_path)

    return model

在我的案例中，Keras使用Tensorflow作为后端.Tensorflow构建为使用GPU(使用CUDA).我目前有一张比较旧的卡:带有2Gb内存的GTX 760.

Keras uses Tensorflow as a backend in my case. Tensorflow is built to use GPU (using CUDA). I currently have a rather old card: GTX 760 with 2Gb of memory.

由于内存不足错误，我什至无法加载整个模型(上面的代码).

On my card I cannot even load the whole model (the code above) because of an out of memory error.

此处作者说4Gb是还不够.

Here the author says that 4Gb is not enough as well.

此处 GTX 1070可以甚至可以训练VGG-16(不仅将其加载到内存中)，而且只能以一些批处理大小和不同的框架(不在Keras中)进行训练.看来GTX 1070总是完全有8Gb的内存.

Here GTX 1070 is able to even train VGG-16 (not just load it into memory), but only with some batch sizes and in different frameworks (not in Keras). It seems that GTX 1070 always have exactly 8Gb of memory.

所以看来4Gb显然不足以微调VGG-16，而8Gb 可能就足够了.

So it seems that 4Gb is clearly not enough for fine-tuning VGG-16, and 8Gb may be enough.

问题是:多少内存足以用Keras + TF微调VGG-16?6Gb是否足够，或者最低8Gb还可以，还是需要更大的容量?

And the question is: what amount of memory is enough to finetune VGG-16 with Keras+TF? Will 6Gb be enough, or 8Gb is minimum and ok, or something bigger is needed?

推荐答案

我已经在Tensorflow中对VGG-16进行了微调，批处理大小为32(GPU:8GB).我认为这与Keras使用Tensorflow的情况相同.但是，如果您想以更大的批量进行训练，则可能需要12或16 GB的GPU.

I have finetuned VGG-16 in Tensorflow with a batch size of 32 (GPU: 8GB). I think this would be the same for your case as Keras uses Tensorflow. However, if you want to train with a larger batch size then you might need 12 or 16 GB GPU.

这篇关于在Keras中的GPU上微调VGG-16:内存消耗的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！