tensorflow einsum vs. matmul vs. tensordot [英] tensorflow einsum vs. matmul vs. tensordot

问题描述

在 tensorflow 中，tf.einsum、tf.matmul 和 tf.tensordot 函数都可以用于相同的任务.(我意识到 tf.einsum 和 tf.tensordot 有更一般的定义；我也意识到 tf.matmul 具有批处理功能.)在在可以使用这三个中的任何一个的情况下，一个函数是否往往是最快的?还有其他推荐规则吗?

In tensorflow, the functions tf.einsum, tf.matmul, and tf.tensordot can all be used for the same tasks. (I realize that tf.einsum and tf.tensordot have more general definitions; I also realize that tf.matmul has batch functionality.) In a situation where any of the three could be used, does one function tend to be fastest? Are there other recommendation rules?

例如，假设 A 是一个 rank-2 张量，而 b 是一个 rank-1 张量，并且您想要计算乘积 c_j =A_ij b_j.三个选项中:

For example, suppose that A is a rank-2 tensor, and b is rank-1 tensor, and you want to compute the product c_j = A_ij b_j. Of the three options:

c = tf.einsum('ij,j->i', A, b)

c = tf.matmul(A, tf.expand_dims(b,1))

c = tf.tensordot(A, b, 1)

通常比其他任何一个更可取吗?

is any generally preferable to the others?

推荐答案

Both tf.tensordot() 和 tf.einsum() 是包装一个或多个 tf.matmul()(虽然在某些特殊情况下 tf.einsum() 可以简化为更简单的元素tf.multiply()).

Both tf.tensordot() and tf.einsum() are syntactic sugar that wrap one or more invocations of tf.matmul() (although in some special cases tf.einsum() can reduce to the simpler elementwise tf.multiply()).

在极限情况下，我希望所有三个函数对于相同的计算都具有相同的性能.然而，对于较小的矩阵，直接使用 tf.matmul() 可能更有效，因为它会产生一个更简单的 TensorFlow 图，操作更少，因此每个操作的调用成本会更低.

In the limit, I'd expect all three functions to have equivalent performance for the same computation. However, for smaller matrices it may be more efficient to use tf.matmul() directly, because it would yield a simpler TensorFlow graph with fewer operations, and hence the per-operation invocation costs will be lower.

这篇关于tensorflow einsum vs. matmul vs. tensordot的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！