张量流是否使用自动或符号渐变? [英] Does tensorflow use automatic or symbolic gradients?
问题描述
我无法找到清晰的关于tensorflow使用自动还是符号微分的陈述.
I haven't been able to find a clear statement of whether tensorflow uses automatic or symbolic differentiation.
我浏览了tensorflow paper ，他们提到了自动渐变，但是目前尚不清楚它们只是表示符号渐变，因为它们还提到它具有这种功能.
I skimmed the tensorflow paper and they mention automatic gradients, but it is unclear if they just mean symbolic gradients, as they also mention that it has that capability.
推荐答案
TF使用自动微分，尤其是反向模式自动微分.
TF uses automatic differentiation and more specifically reverse-mode auto differentiation.
有3种流行的计算导数的方法:
There are 3 popular methods to calculate the derivative:
- 数值微分
- 符号分化
- 自动区分
，在其中放置一个非常小的h并在两个位置评估函数.这是最基本的公式，在实践中，人们使用其他公式得出的估计误差较小.如果您不知道函数并且只能对其进行采样，则这种计算导数的方法最适合.此外，要实现高调功能，还需要进行大量计算.
, where you put a very small h and evaluate function in two places. This is the most basic formula and on practice people use other formulas which give smaller estimation error. This way of calculating a derivative is suitable mostly if you do not know your function and can only sample it. Also it requires a lot of computation for a high-dim function.
符号微分 操纵数学表达式.如果您曾经使用过matlab或mathematica，那么您在这里，每个数学表达式他们都知道导数，并使用各种规则(乘积规则，链式规则)来计算结果导数.然后他们简化最终表达式以获得最终的表达式.
Here for every math expression they know the derivative and use various rules (product rule, chain rule) to calculate the resulting derivative. Then they simplify the end expression to obtain the resulting expression.
自动区分 操纵计算机程序块.区分符具有获取程序每个元素派生的规则(当您在核心TF中定义任何op时，您需要为此操作注册渐变.它还使用链规则将复杂的表达式分解为更简单的表达式.这是一个一个很好的示例，它带有一些解释，如何在实际TF程序中工作.
Automatic differentiation manipulates blocks of computer programs. A differentiator has the rules for taking the derivative of each element of a program (when you define any op in core TF, you need to register a gradient for this op). It also uses chain rule to break complex expressions into simpler ones. Here is a good example how it works in real TF programs with some explanation.
您可能会认为自动微分"与符号微分"相同(在一个地方，他们对数学表达式进行操作，在另一地方，他们对计算机程序进行操作).是的，它们有时非常相似.但是对于控制流语句(如果，则循环")，结果可能非常不同:
You might think that Automatic differentiation is the same as Symbolic differentiation (in one place they operate on math expression, in another on computer programs). And yes, they are sometimes very similar. But for control flow statements (`if, while, loops) the results can be very different:
符号差异导致代码效率低下(除非小心操作) 完成)，并且面临着将计算机程序转换为 单一表达
symbolic differentiation leads to inefficient code (unless carefully done) and faces the difficulty of converting a computer program into a single expression
这篇关于张量流是否使用自动或符号渐变?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
