如何在keras中定义自定义激活函数的导数 [英] how to define the derivative of a custom activation function in keras

问题描述

我有一个自定义激活函数及其派生类，尽管我可以使用该自定义激活函数，但我不知道如何告诉keras它的派生类是什么.

I have a custom activation function and its derivative, although I can use the custom activation function I don't know how to tell keras what is its derivative.

似乎它找到了自己，但是我有一个必须在函数及其派生函数之间共享的参数，那么我该怎么做呢?

It seems like it finds one itself but I have a parameter that has to be shared between the function and its derivative so how can I do that?

我知道在tensorflow中有一个相对简便的方法可以做到这一点，但是我不知道如何在keras中实现它.

I know there is a relatively easy way to do this in tensorflow but I have no idea how to implement it in keras here is how you do it in tensorflow

基于我得到的答案，也许我不够清楚.我想要的是为我的激活函数实现自定义派生类，以便它在反向传播期间使用我的派生类.我知道如何实现自定义激活功能.

based on the answer I got maybe I wasn't clear enough. What I want is to implement a custom derivative for my activation function so that it use my derivative during the backpropagation. I know how to implement a custom activation function.

推荐答案

看看定义Keras激活功能的源代码:

Take a look at the source code where the activation functions of Keras are defined:

keras/activations.py

例如:

def relu(x, alpha=0., max_value=None):
    """Rectified Linear Unit.

    # Arguments
        x: Input tensor.
        alpha: Slope of the negative part. Defaults to zero.
        max_value: Maximum value for the output.

    # Returns
        The (leaky) rectified linear unit activation: `x` if `x > 0`,
        `alpha * x` if `x < 0`. If `max_value` is defined, the result
        is truncated to this value.
    """
    return K.relu(x, alpha=alpha, max_value=max_value)

以及Keras层如何调用激活函数:self.activation = activations.get(activation) activation可以是字符串或可调用的.

And also how does Keras layers call the activation functions: self.activation = activations.get(activation) the activation can be string or callable.

因此，类似地，您可以定义自己的激活函数，例如:

Thus, similarly, you can define your own activation function, for example:

def my_activ(x, p1, p2):
    ...
    return ...

假设您想在Dense层中使用此激活，只需将您的函数放置如下:

Suppose you want use this activation in Dense layer, you just put your function like this:

x = Dense(128, activation=my_activ(p1, p2))(input)

---

如果您要实现自己的派生:

---

If you mean you want to implement your own derivative:

如果您的激活函数是用Tensorflow/Keras函数编写的，这些函数的操作是可区分的(例如K.dot(), tf.matmul(), tf.concat() etc.)，则将通过自动区分

If your activation function is written in Tensorflow/Keras functions of which the operations are differentiable (e.g. K.dot(), tf.matmul(), tf.concat() etc.), then the derivatives will be obtained by automatic differentiation https://en.wikipedia.org/wiki/Automatic_differentiation. In that case you dont need to write your own derivative.

如果您仍要重写衍生产品，请查看此文档 https://www.tensorflow .org/extend/adding_an_op ，您需要在其中使用tf.RegisterGradient

If you still want to re-write the derivatives, check this document https://www.tensorflow.org/extend/adding_an_op where you need to register your gradients using tf.RegisterGradient

这篇关于如何在keras中定义自定义激活函数的导数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！