如何在TensorFlow中设置重量成本强度？ [英] How to set weight cost strength in TensorFlow?

问题描述

我正在尝试将TensorFlow用于我的深度学习项目。

I'm trying to use TensorFlow with my deep learning project.

当我使用动量梯度下降法时，重量成本强度设置如何？

（此公式。）

When I use Momentum Gradient Descent, how is the weight cost strength set?
(The λ in this formula.)

推荐答案

重量成本/衰减这一术语在TensorFlow中不是优化程序的一部分。

The term for the weight cost/decay is not part of the optimizers in TensorFlow.

很容易将其包括在内，但是可以通过在成本函数中添加额外的罚款权重的L2损失：

It is easy to include, however, by adding the extra penalty to the cost function with the L2 loss on the weights:

C = <your initial cost function>
l2_loss = tf.add_n([tf.nn.l2_loss(v) for v in tf.trainable_variables()])
C = C + lambda * l2_loss

tf.nn.l2_loss（v） 链接只是 0.5 * tf.reduce_sum（v * v），各个权重的梯度将等于 lambda * w ，应等于链接的等式。

tf.nn.l2_loss(v) link is simply 0.5 * tf.reduce_sum(v * v) and the gradients for individual weights will be equal to lambda * w, which should be equivalent to your linked equation.

这篇关于如何在TensorFlow中设置重量成本强度？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！