如何将多线程应用于反向传播神经网络训练？ [英] How can I apply multithreading to the backpropagation neural network training?

问题描述

对于我的大学项目，我正在创建一个神经网络，可以对信用卡交易是否具有欺诈性的可能性进行分类。我正在接受反向传播训练。我用Java写这个。我想应用多线程，因为我的电脑是四核i7。这让我花了几个小时的训练，看到我的大部分内核闲置。

For my university project I am creating a neural network that can classify the likelihood that a credit card transaction is fraudulent or not. I am training with backpropagation. I am writing this in Java. I would like to apply multithreading, because my computer is a quad-core i7. It bugs me to spend hours training and see most of my cores idle.

但是我如何应用多线程进行反向传播？ Backprop通过网络向后调整错误来工作。必须先完成一层，然后才能继续。有什么方法可以修改我的程序来做多核背景吗？

But how would I apply multithreading to backpropagation? Backprop works by adjusting the errors backwards through the network. One layer must be done before the other can continue. Is there any way that I can modify my program to do multicore backdrop?

推荐答案

首先不要使用反向传播。还有很多其他选择。我建议尝试RPROP（弹性传播）。对你的反向传播算法来说，这不会有太大的修改。您无需指定学习率或动力。它几乎就像你对神经网络中的每个连接都有一个单独的，可变的学习率。

First of all don't use backpropagation. There are many other options out there. I would suggest trying RPROP (resilient propagation). It won't be that big of modification to your backpropagation algorithm. You do not need to specify learning rate or momentum. Its really almost as if you have an individual, variable, learning rate for every connection in the neural network.

关于将多线程应用于反向传播。我刚刚写了一篇关于这个主题的文章。

As to applying multithreading to backpropagation. I just wrote an article on this topic.

http://www.heatonresearch.com/encog/mprop/compare.html

基本上我创建了许多线程并将其划分训练数据，因此每个线程的数量几乎相等。我正在计算每个线程中的渐变，并在reduce步骤中求和。如何将梯度应用于权重取决于所使用的传播训练算法，但权重更新是在关键部分中完成的。

Basically I create a number of threads and divide up the training data so each thread has a near equal amount. I am calculating the gradients in each thread and they are summed in a reduce step. How the gradients are applied to the weights depends on the propagation training algorithm used, but the weight update is done in a critical section.

当您拥有的训练样本多于权重代码在多线程渐变计算中花费的时间比临界区权重更新要多得多。

When you have considerably more training samples than weights the code spends much more time in the multi-threaded gradient calculation than the critical section weight update.

我在上面的链接中提供了一些性能结果。它确实加快了速度！

I provide some of the performance results at the above link. It does really speed things up!

这篇关于如何将多线程应用于反向传播神经网络训练？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！