具有NumPy的ReLU衍生物 [英] ReLU derivative with NumPy

问题描述

import numpy as np

def relu(z):
    return np.maximum(0,z)

def d_relu(z):
    z[z>0]=1
    z[z<=0]=0
    return z

x=np.array([5,1,-4,0])
y=relu(x)
z=d_relu(y)
print("y = {}".format(y))
print("z = {}".format(z))

上面的代码打印出来:

y = [1 1 0 0]
z = [1 1 0 0]

代替

y = [5 1 0 0]
z = [1 1 0 0]

据我了解，我使用过的函数调用只应该按值传递，传递变量的副本.

From what I understand the function calls I've used should only be doing passing by value,passing a copy of the variable.

为什么我的d_relu函数影响y变量?

Why is my d_relu function affecting the y variable?

推荐答案

您的第一个错误是假设python通过值传递对象...否-它是通过​​赋值传递(类似于通过引用传递，如果您这样做，重新熟悉这个概念).但是，仅顾名思义，可变对象可以就地修改.其中包括numpy数组.

Your first mistake is in assuming python passes objects by value... it doesn't - it's pass by assignment (similar to passing by reference, if you're familiar with this concept). However, only mutable objects, as the name suggests, can be modified in-place. This includes, among other things, numpy arrays.

您不应该就地d_relu修改z，因为这就是通过z[...] = ...语法进行的操作.尝试使用广播比较构建遮罩，然后返回该遮罩.

You shouldn't have d_relu modify z inplace, because that's what it's doing right now, through the z[...] = ... syntax. Try instead building a mask using broadcasted comparison and returning that instead.

def d_relu(z):
    return (z > 0).astype(int)

这将返回一个新数组，而不是就地修改z，并且您的代码会打印

This returns a fresh array instead of modifying z in-place, and your code prints

y = [5 1 0 0]
z = [1 1 0 0]

如果您要构建分层体系结构，则可以在前向通过阶段利用计算出的蒙版:

If you're building a layered architecture, you can leverage the use of a computed mask during the forward pass stage:

class relu:
    def __init__(self):
        self.mask = None

    def forward(self, x):
        self.mask = x > 0
        return x * self.mask

    def backward(self, x):
        return self.mask

如果前馈输入大于0，则导数为1，否则为0.

Where the derivative is simply 1 if the input during feedforward if > 0, else 0.

这篇关于具有NumPy的ReLU衍生物的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！