如何取两个网络权重的平均值? [英] How to take the average of the weights of two networks?

问题描述

假设在PyTorch中，我有model1和model2，它们具有相同的体系结构.他们接受了相同数据的进一步培训，或者一个模型是其他模型的早期版本，但在技术上与该问题无关.现在，我想将model的权重设置为model1和model2的权重的平均值.我将如何在PyTorch中做到这一点?

Suppose in PyTorch I have model1 and model2 which have the same architecture. They were further trained on same data or one model is an earlier version of the othter, but it is not technically relevant for the question. Now I want to set the weights of model to be the average of the weights of model1 and model2. How would I do that in PyTorch?

推荐答案

beta = 0.5 #The interpolation parameter    
params1 = model1.named_parameters()
params2 = model2.named_parameters()

dict_params2 = dict(params2)

for name1, param1 in params1:
    if name1 in dict_params2:
        dict_params2[name1].data.copy_(beta*param1.data + (1-beta)*dict_params2[name1].data)

model.load_state_dict(dict_params2)

来自 pytorch论坛.您可以获取参数，进行转换并将其加载回去，但要确保尺寸匹配.

Taken from pytorch forums. You could grab the parameters, transform and load them back but make sure the dimensions match.

我也很想知道您对这些发现的了解.

Also I would be really interested in knowing about your findings with these..

这篇关于如何取两个网络权重的平均值?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！