如何识别任意神经网络中的循环连接 [英] How to Identify Recurrent Connections in an Arbitrary Neural Network

问题描述

我正在尝试在C#中实现增强拓扑的神经进化.我遇到了经常性连接问题.我知道，对于循环连接，输出基本上是暂时移位的.

I am trying to implement Neuro-Evolution of Augmenting Topologies in C#. I am running into a problem with recurrent connections. I understand that, for a recurrent connection, the output is basically temporally displaced.

http://i.imgur.com/FQYjCLZ.png

在链接的图像中，我展示了一个非常简单的神经网络，具有 2个输入，3个隐藏节点和一个输出.如果没有激活函数或传递函数，我认为它将被评估为:

In the linked image, I show a pretty simple neural network with 2 inputs, 3 hidden nodes, and one output. Without an activation function or transfer function, I think it would be evaluated as:

n3[t] = (i1[t]*a + n6[t-1]*e)*d + i2[t]*b*c) * f

但是，我很难弄清楚如何识别链接e是循环连接这一事实.我读到的有关NEAT的论文显示了XOR问题的最小解和双极无速度问题都是如何反复出现的.

However I am having a hard time figuring out how to identify the fact that the link e is a recurrent connection. The paper that I read about NEAT showed how the minimal solutions of the XOR problem and the dual pole no velocity problem both had recurrent connections.

如果您具有固定的拓扑，这似乎很简单，因为您可以自己分析拓扑，并确定需要延时的连接.

It seems rather straight forward if you have a fixed topology, because you can analyze the topology yourself, and identify which connections you need to time delay.

您如何准确识别这些连接?

How exactly would you identify these connections?

推荐答案

当我开始撰写本文时，我遇到了类似的问题.我不知道您的网络在动态环境中是什么样子，所以我将向您解释我的工作.

I had a similar problem when i started implememting this paper. I don't know what your network looks like in the momen, so i'll explain to you what i did.

我的网络开始作为输入&仅输出层.为了创建连接和神经元，我实现了某种DNA(在我的情况下，这是一系列指令，例如将神经元2与神经元5连接，并将权重设置为0.4").我的网络中的每个神经元都有一个"layerNumber"，它告诉我神经元在我的网络中的位置.为每个输入和输出神经元设置此layerNumber.对于输入神经元，我使用Double.minvalue；对于输出神经元，我使用Double.maxvalue.

My network starts out as input & output layers only. To create connections and neurons i implemented some kind of DNA (in my case this is an array of instructions like 'connect neuron nr. 2 with neuron nr. 5 and set the weight to 0.4'). Every neuron in my network has a "layerNumber" which tells me where a neuron is in my network. This layerNumber is set for every in and output neuron. for inputneurons i used Double.minvalue and for outputneurons i used Double.maxvalue.

这是基本设置.从现在开始，在修改网络时只需遵循以下规则:

This is the basic setup. From now on just follow these rules when modifying the network:

• 每当您要创建连接时，请确保来自"神经元的layerNumber为< Double.maxValue

• Whenever you want to create a connection, make sure the 'from' neuron has a layerNumber < Double.maxValue

无论何时要创建连接，请确保"to"神经元的层号大于"from"神经元的层数.

Whenever you want to create a connection, make sure that the 'to' neuron has a bigger layerNumber than the 'from' neuron.

每当一个连接被拆分为2个连接和它们之间的一个神经元时，请将神经元layerNumber设置为NeuronFrom.layerNumber * 0.5 + NeuronTo.layerNumber * 0.5 这很重要，您不能将它们相加并简单地除以2，因为这可能会导致Double.maxValue +某物，这将返回一些奇怪的数字(我猜会发生溢出，因此您会得到一个负数吗?).

Whenever a connection is split up into 2 connections and a neuron between them, set the neurons layerNumber to NeuronFrom.layerNumber*0.5 + NeuronTo.layerNumber*0.5 This is important, you can't add them and simply divide by 2, because this would likely result in Double.maxValue + something, which would return some weird number (i guess an overflow would happen, so you would get a negative number?).

如果遵循所有规则，则应该始终仅具有转发连接.没有复发的.如果您需要循环连接，则可以通过交换'from'&来创建它们.创建新连接时将其设置为至".

If you follow all the rules you should always have forwarding connections only. No recurrent ones. If you want recurrent connections you can create them by just swapping 'from' & 'to' while creating a new connection.

专业技巧: 仅使用一个神经元ArrayList. 使DNA使用神经元的ID来查找它们，但是创建一个连接"类，它将神经元对象作为属性. 过滤连接/神经元时，请使用ArrayList.stream().filter()

Pro tricks: Use only one ArrayList of Neurons. Make the DNA use ID's of neurons to find them, but create a 'Connection' class which will have the Neuron objects as attributes. When filtering your connections/neurons use ArrayList.stream().filter()

稍后在网络中传播时，您可以按照layerNumber对神经元进行排序，设置inputValues并使用for()循环遍历所有神经元.只需计算神经元的输出值，然后将其传输到每个有联系的神经元，其中"from" ==当前神经元即可.

When later propagating trough the network you can just sort your neurons by layerNumber, set the inputValues and go trough all neurons using a for() loop. Just calculate the neurons outputvalue and transfer it to every neuron which has a connection where 'from' is == the current neuron.

希望它并不太复杂...

Hope it's not too complicated...

这篇关于如何识别任意神经网络中的循环连接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！