喀拉斯的学生-老师模型 [英] student-teacher model in keras

问题描述

我正在将下面url中的学生-教师模型转换为keras一个.

I'm converting student-teacher model in below url to keras one.

https://github.com/chengshengchan/model_compression/blob/master/teacher-student.py

我怎样才能给两种模型(学生，老师)提供输入，而仅从喀拉拉邦的学生那里获得一种输出? 我将trainable = false设置为老师的所有张量，并将损失函数设置为学生和老师的输出之间的差异，如下所示:

How can I give input to two model(student, teacher) and get one output from only student in keras? I'll set teacher's all tensors with trainable=false, and loss function as difference between student and teacher's output like below :

tf_loss = tf.nn.l2_loss(teacher - student)/batch_size

据我所知，在定义model.fit时，可以仅向一个模型提供输入.但是在这种情况下，我应该同时采用教师和学生模式.

As I know, it is possible to give input to only one model when defining model.fit. But in this cases, I should it to both of teacher and student model.

提前谢谢！

推荐答案

我在Keras中看到的唯一实现包括构建2个单独的函数，这些函数可以扩大或加深教师模型的权重层，作为学生模型的初始权重. 我不确定是否正是Hinton等人. (2015)升格说老实，但它是师生. https://github.com/fchollet/keras/issues/3491

The only implementation I have seen in Keras involves building 2 separate functions which either widen or deepen weight layers from the teacher model as initial weights for the student model. I am not sure if it is precisely that Hinton et al. (2015) distillation to be honest, but it is teacher-student. https://github.com/fchollet/keras/issues/3491

这篇关于喀拉斯的学生-老师模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！