具有重量限制的R-自定义Keras层 [英] R- Custom Keras Layer With Weight Constraints

问题描述

我正在尝试在R中编写一个可训练的权重的自定义Keras层，

I'm trying to write a custom Keras layer with trainable weights in R which:

• 获取输入向量x并返回值exp(A * X*A)，其中$ A $是对角线且可训练..

• Takes an input vector x and returns the value exp(A * X*A) where $A$ is diagonal and trainable..

其中 exp 是矩阵指数图.

推荐答案

请注意，了解批次大小在哪里非常重要，并且图层不能具有基于批次大小的权重(除非您定义了用batch_shape或batch_input_shape而不是shape输入-这将迫使您在模型中使用固定的批次大小.由于批次大小通常适用于单个"和独立"样本，因此在操作和混合样本中使用批次大小是不健康的！

Notice that it's very important that you understand where your batch size is, and that a layer CANNOT have weights with sizes based on the batch size (unless you define your inputs with batch_shape or batch_input_shape instead of shape -- this will force you to use a fixed batch size in the model). Since the batch size is usually for "individual" and "independent" samples, it's not healthy to use the batch size in operations and mixing samples!

也就是说，我假设这里的X具有形状(batch, dim, dim)，因此A将具有形状(dim, dim).

That said, I am assuming that X here has shape (batch, dim, dim), and that A will have shape (dim, dim) consequently.

为此，您将构建一个自定义层，例如: https://tensorflow. rstudio.com/guide/keras/custom_layers/

For this, you build a custom layer such as here: https://tensorflow.rstudio.com/guide/keras/custom_layers/

build的形状为(1, dim, 1)-

    build = function(input_shape) {
      self$kernel <- self$add_weight(
        name = 'kernel', 
        shape = list(1,input_shape[[2]], 1),
        initializer = initializer_random_normal(), #you may choose different initializers
        trainable = TRUE
      )
    },

然后通话将使用数学技巧来模拟对角线.

And call will use a mathematical trick to simulate the diagonal.

请注意，如果A是对角线，则A x X x A的结果将是B*X(基本上)，其中B是:

Notice that if A is diagonal, the result of A x X x A will be B*X (elementwise), where B is:

#supposing A has the elements [a, b, c, ...] in the diagonals,

B is:
[ [aa, ab, ac, ...],
  [ab, bb, bc, ...],
  [ac, bc, cc, ...],
  ...
]

因此，我们将不使用对角线，而是使用具有逐元素乘法的广播技巧:

Because of this, we will not use diagonals, but a broadcasting trick with elementwise multiplication:

    call = function(x, mask = NULL) {
      kernelTransposed <- tf$reshape(self$kernel, shape(1L, 1L, -1L)) #(1, 1, dim)
      B <- self$kernel * kernelTransposed #(1, dim, dim)
      tf$math$exp(x * B)
    },

输出形状不变:

    compute_output_shape = function(input_shape) {
      input_shape
    }

这篇关于具有重量限制的R-自定义Keras层的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！