将模型从 keras h5 转换为 pytorch - 全连接层不匹配 [英] Converted model from keras h5 to pytorch - fully connected layer mismatch

问题描述

我已经使用 mmdnn 将两个模型(vgg16 和 resnet50)从 Keras 和 TensorFlow 后端(从作为 model.save 文件)转换为 PyTorch.这是通过以下方式完成的:

I have converted two models (vgg16 and resnet50) from Keras with TensorFlow backend (from as model.save file) into PyTorch using mmdnn. This was done with the following:

mmconvert -sf keras -iw vgg.h5 -df pytorch -om keras_to_torch.pt

A = imp.load_source('MainModel','/weights/keras_to_torch.py')
model = torch.load('/weights/keras_to_torch.pt')

对同一数据集进行预测给了我一组不同的结果，因此我进行了进一步调查.

Predicting on the same data set gave me a different set of results so I investigated further.

我可以看到所有卷积层的权重都相同(转置后)，但最后的全连接层的权重却不同.

I can see that the weights for all the convolutional layers are the same (after transposing), however the weights of the fully connected layers at the end are not.

这应该是有原因的吗?据我所知，它们应该是等效的

Is there a reason this should be? As i understand they should be equivalent

推荐答案

问题一定出在您定义 keras 模型的方式上，因为我无法使用 MMdnn 包提供的 h5 文件来复制该问题.如果你想使用 resnet50 和 VGG19 模型你可以得到正确的权重如下:

The problem must be in the way you defined your keras model, since I cannot replicate the issue using the h5 file that is provided using the MMdnn package. If you want to use the resnet50 and VGG19 model you can get the correct weights as follows:

• 按照文档中的规定启动 MMdnn 容器下载 resnet50 的 keras 模型

• start MMdnn container as specified in the documentation download keras model for resnet50

mmdownload -f keras -n resnet50 -o ./

• 转换为pytorch模型

mmconvert -sf keras -iw ./imagenet_resnet50.h5 -df pytorch -om keras_to_torch.pt

然后从docker容器中提取生成的numpy文件keras_to_torch.pt和keras_to_torch.py​​(和imagenet_resnet50.h5进行比较).

Then extract the produced numpy file, keras_to_torch.pt and keras_to_torch.py from the docker container (and imagenet_resnet50.h5 for comparison).

在 Python 中加载 keras 模型

In Python load the keras model with

import keras
model = load_model('imagenet_resnet50.h5')

和使用

import imp
import torch
torch_weights = # path_to_the_numpy_weights
A = imp.load_source('MainModel','keras_to_torch.py')
weights_torch = A.load_weights(torch_weights)
model_torch = A.KitModel(torch_weights)

我还必须在 keras_to_torch.py​​ 文件开头的 load_weights(weight_file) 函数中设置 allow_pickle = True.不幸的是，torch.load('/weights/keras_to_torch.pt') 变体向我抛出了一个错误.

I also had to set allow_pickle = True in the load_weights(weight_file) function at the beginning of the keras_to_torch.py file. The torch.load('/weights/keras_to_torch.pt') variant threw an error for me unfortunately.

打印最后一个密集连接层的权重

Print the weights of the last densely connected layer

# keras model
model.layers[-1].weights

# Output:
#tensor([[-0.0149,  0.0113, -0.0507,  ..., -0.0218, -0.0776,  0.0102],
#        [-0.0029,  0.0032,  0.0195,  ...,  0.0362,  0.0035, -0.0332],
#        [-0.0175,  0.0081,  0.0085,  ..., -0.0302,  0.0549, -0.0251],
#        ...,
#        [ 0.0253,  0.0630,  0.0204,  ..., -0.0051, -0.0354, -0.0131],
#        [-0.0062, -0.0162, -0.0122,  ...,  0.0138,  0.0409, -0.0186],
#        [-0.0267,  0.0131, -0.0185,  ...,  0.0630,  0.0256, -0.0069]])

# torch model (make sure to transpose)
model_torch.fc1000.weight.data.T

# Output:
#[<tf.Variable 'fc1000/kernel:0' shape=(2048, 1000) dtype=float32, numpy=
# array([[-0.01490746,  0.0113374 , -0.05073728, ..., -0.02179668,
#         -0.07764222,  0.01018347],
#        [-0.00294467,  0.00319835,  0.01953556, ...,  0.03623696,
#          0.00350259, -0.03321117],
#        [-0.01751374,  0.00807406,  0.00851311, ..., -0.03024036,
#          0.05494978, -0.02511911],
#        ...,
#        [ 0.025289  ,  0.0630148 ,  0.02041481, ..., -0.00508354,
#         -0.03542514, -0.01306196],
#        [-0.00623157, -0.01624131, -0.01221174, ...,  0.01376359,
#          0.04087579, -0.0185826 ],
#        [-0.02668471,  0.0130982 , -0.01847764, ...,  0.06304929
#...

keras 和火炬模型的权重根据需要一致(最多 4 位数左右).只要您不想在将 keras 中的 VGG 和 ResNet 权重转换为 Pytorch 之前更新它们，此解决方案就有效.

The weights of the keras and torch model coincide as desired (up to 4 digits or so). This solution works as long as you don't want to update the VGG and ResNet weights in keras before converting them to Pytorch.

如果您确实需要在转换前更新模型权重，您应该共享您的代码来创建 Keras 模型.您可以进一步检查使用 mmdownload 模型获得的 imagenet_resnet50.h5 与您在 keras 中使用 model.save 保存的模型有何不同，并纠正任何差异.

If you do need to update the model weights before converting you should share your code for creating the Keras model. You could further inspect how the imagenet_resnet50.h5 obtained with mmdownload model differs from the one you saved with model.save in keras and correct for any differences.

这篇关于将模型从 keras h5 转换为 pytorch - 全连接层不匹配的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！