如何在python中实现Leaky Relu的导数? [英] How to implement the derivative of Leaky Relu in python?

问题描述

如何在不使用 Tensorflow 的情况下在 Python 中实现 Leaky ReLU 的导数?

How would I implement the derivative of Leaky ReLU in Python without using Tensorflow?

还有比这更好的方法吗?我希望函数返回一个 numpy 数组

Is there a better way than this? I want the function to return a numpy array

def dlrelu(x, alpha=.01):
     # return alpha if x < 0 else 1

     return np.array ([1 if i >= 0 else alpha for i in x])

预先感谢您的帮助

推荐答案

您使用的方法有效，但严格来说，您正在计算关于损失或下层的导数，因此也通过来自较低层的值以计算导数 (dl/dx).

The method you use works, but strictly speaking you are computing the derivative with respect to the loss, or lower layer, so it might be wise to also pass the value from lower layer to compute the derivative (dl/dx).

无论如何，您可以避免使用对于大型 x 更有效的循环.这是一种方法:

Anyway, you can avoid using the loop which is more efficient for large x. This is one way to do it:

def dlrelu(x, alpha=0.01):
  dx = np.ones_like(x)
  dx[x < 0] = alpha
  return dx

如果你从下层传递错误，它看起来像这样:

If you passed the error from lower layer, it looks like this:

def dlrelu(dl, x, alpha=0.01):
  """ dl and x have same shape. """
  dx = np.ones_like(x)
  dx[x < 0] = alpha
  return dx*dl

这篇关于如何在python中实现Leaky Relu的导数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！