将自定义层创建为单个神经元的堆栈 TensorFlow [英] Creating custom layer as stack of individual neurons TensorFlow

问题描述

因此，我尝试在 TensorFlow 2.4.1 中创建一个自定义层，使用我定义的神经元函数.

So, I'm trying to create a custom layer in TensorFlow 2.4.1, using a function for a neuron I defined.

# NOTE: this is not the actual neuron I want to use,
# it's just a simple example.
def neuron(x, W, b):
    return W @ x + b

W 和 b 的形状是 (1, x.shape[0]) 和 (1, 1) 分别.这意味着这就像密集层中的单个神经元.因此，我想通过堆叠任意数量的这些单个神经元来创建一个密集层.

Where the W and b it gets would be of shape (1, x.shape[0]) and (1, 1) respectively. This means this is like a single neuron in a dense layer. So, I want to create a dense layer by stacking however many of these individual neurons I want.

class Layer(tf.keras.layers.Layer):
    def __init__(self, n_units=5):
        super(Layer, self).__init__() # handles standard arguments
        
        self.n_units = n_units # Number of neurons to be in the layer

    def build(self, input_shape):
        # Create weights and biases for all neurons individually
        for i in range(self.n_units):
            # Create weights and bias for ith neuron
            ...

    def call(self, inputs):
        # Compute outputs for all neurons
        ...
        # Concatenate outputs to create layer output
        ...
        return output

我如何将层创建为单个神经元的堆栈(也以它可以训练的方式)?我在上面的代码中粗略地概述了层的想法，但答案不需要将其作为蓝图来遵循.

How can I create a layer as a stack of individual neurons (also in a way it can train)? I have roughly outlined the idea for the layer in the above code, but the answer doesn't need to follow that as a blueprint.

最后；是的，我知道要创建一个密集层，您不需要以这种迂回的方式进行(您只需要 1 个权重和偏置矩阵)，但在我的实际用例中，这是必要的.谢谢！

推荐答案

所以，在这里问这个问题的人，我已经找到了一种方法，通过动态创建变量和操作.
首先，让我们重新定义神经元以使用张量流操作:

So, person who asked this question here, I have found a way to do it, by dynamically creating variables and operations.
First, let's re-define the neuron to use tensorflow operations:

def neuron(x, W, b):
    return tf.add(tf.matmul(W, x), b)

然后，让我们创建图层(这使用了问题中布置的蓝图):

Then, let's create the layer (this uses the blueprint layed out in the question):

class Layer(tf.keras.layers.Layer):
    def __init__(self, n_units=5):
        super(Layer, self).__init__()

        self.n_units = n_units

    def build(self, input_shape):
        for i in range(self.n_units):
            exec(f'self.kernel_{i} = self.add_weight("kernel_{i}", shape=[1, int(input_shape[0])])')
            exec(f'self.bias_{i} = self.add_weight("bias_{i}", shape=[1, 1])')

    def call(self, inputs):
        for i in range(self.n_units):
            exec(f'out_{i} = neuron(inputs, self.kernel_{i}, self.bias_{i})')
        return eval(f'tf.concat([{", ".join([ f"out_{i}" for i in range(self.n_units) ])}], axis=0)')

如您所见，我们使用 exec 和 eval 来动态创建变量并执行操作.
就是这样！我们可以执行一些检查，看看 TensorFlow 是否可以使用它:

As you can see, we're using exec and eval to dynamically create variables and perform operations.
That's it! We can perform a few checks to see if TensorFlow could use this:

# Check to see if it outputs the correct thing
layer = Layer(5) # With 5 neurons, it should return a (5, 6)
print(layer(tf.zeros([10, 6])))

# Check to see if it has the right trainable parameters
print(layer.trainable_variables)

# Check to see if TensorFlow can find the gradients
layer = Layer(5)
x = tf.ones([10, 6])
with tf.GradientTape() as tape:
    z = layer(x)
print(f"Parameter: {layer.trainable_variables[2]}")
print(f"Gradient:  {tape.gradient(z, layer.trainable_variables[2])}")

这个解决方案有效，但不是很优雅......我想知道有没有更好的方法可以做到，一些神奇的TF方法可以映射神经元创建层，我暂时不知道.所以，如果您有(更好的)答案，请回答，我很乐意接受:)

This solution works, but it's not very elegant... I wonder if there's a better way to do it, some magical TF method that can map the neuron to create a layer, I'm too inexperienced to know for the moment. So, please answer if you have a (better) answer, I'll be happy to accept it :)

这篇关于将自定义层创建为单个神经元的堆栈 TensorFlow的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！