如何取两个网络的权重的平均值? [英] How to take the average of the weights of two networks?

问题描述

假设在 PyTorch 中我有 model1 和 model2，它们具有相同的架构.他们接受了相同数据的进一步训练，或者一个模型是另一个模型的早期版本，但在技术上与问题无关.现在我想将 model 的权重设置为 model1 和 model2 的权重的平均值.我将如何在 PyTorch 中做到这一点?

Suppose in PyTorch I have model1 and model2 which have the same architecture. They were further trained on same data or one model is an earlier version of the othter, but it is not technically relevant for the question. Now I want to set the weights of model to be the average of the weights of model1 and model2. How would I do that in PyTorch?

推荐答案

beta = 0.5 #The interpolation parameter    
params1 = model1.named_parameters()
params2 = model2.named_parameters()

dict_params2 = dict(params2)

for name1, param1 in params1:
    if name1 in dict_params2:
        dict_params2[name1].data.copy_(beta*param1.data + (1-beta)*dict_params2[name1].data)

model.load_state_dict(dict_params2)

取自 pytorch 论坛.您可以获取参数，转换并加载它们，但要确保尺寸匹配.

Taken from pytorch forums. You could grab the parameters, transform and load them back but make sure the dimensions match.

此外，我真的很想知道您对这些的发现..

Also I would be really interested in knowing about your findings with these..

这篇关于如何取两个网络的权重的平均值?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！