梯度检查到底是什么? [英] What exactly is gradient checking?

问题描述

我是深度学习的初学者.我遇到了梯度检查"的概念.

I am a beginner in Deep Learning. I came through the concept of 'Gradient Checking'.

我只想知道，这是什么以及它如何有助于改善培训过程?

I just want to know, what is it and how it could help to improve the training process?

推荐答案

为什么需要进行渐变检查?

作为一种算法，后置道具有很多细节，要实现起来可能有些棘手.一个不幸的特性是，有许多方法可以使后置支撑中存在一些细微的错误.因此，如果您使用梯度下降或其他优化算法来运行它，则实际上看起来像是在工作.而且您的成本函数theta的J可能会在每次梯度下降迭代时最终减少.但这可能是正确的，即使您在反向道具的实现中可能存在一些错误.这样看来theta的J在减小，但是您可能会发现神经网络的错误级别比无错误实现的错误级别更高.您可能只是不知道有一个细微的错误使您的性能变差.那么，我们该怎么办?有一种称为梯度检查的想法，可以消除几乎所有这些问题.

Why do we need Gradient Checking?

Back prop as an algorithm has a lot of details and can be a little bit tricky to implement. And one unfortunate property is that there are many ways to have subtle bugs in back prop. So that if you run it with gradient descent or some other optimizational algorithm, it could actually look like it's working. And your cost function, J of theta may end up decreasing on every iteration of gradient descent. But this could prove true even though there might be some bug in your implementation of back prop. So that it looks J of theta is decreasing, but you might just wind up with a neural network that has a higher level of error than you would with a bug free implementation. And you might just not know that there was this subtle bug that was giving you worse performance. So, what can we do about this? There's an idea called gradient checking that eliminates almost all of these problems.

我们描述了一种方法，该方法用于对由您的代码计算出的导数进行数字检查，以确保实现正确.进行派生检查程序将大大提高您对代码正确性的信心.

We describe a method for numerically checking the derivatives computed by your code to make sure that your implementation is correct. Carrying out the derivative checking procedure significantly increase your confidence in the correctness of your code.

如果我要简短地说，梯度检查是一种调试back prop算法的方法.梯度检查基本上执行导数检查程序.

If I have to say in short than Gradient Checking is kind of debugging your back prop algorithm. Gradient Checking basically carry out the derivative checking procedure.

您可以在此处找到该过程.

这篇关于梯度检查到底是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！