TensorFlow 中的 tf.nn.embedding_lookup_sparse 是什么意思? [英] What's the meaning of tf.nn.embedding_lookup_sparse in TensorFlow?

问题描述

我们花了大量时间阅读的API文档tf.nn.embedding_lookup_sparse.embedding_lookup_sparse 的含义很混乱，看起来和 embedding_lookup 有很大的不同.

We spend a lot of time in reading the API document of tf.nn.embedding_lookup_sparse. The meaning of embedding_lookup_sparse is confusing and it seems quite different from embedding_lookup.

这是我的想法，如果我错了，请纠正我.广深模型示例使用contrib.layers API 并调用embedding_lookup_sparse 进行稀疏特征列.如果它获得 SparseTensor(例如，稀疏的国家/地区)，它会创建实际上用于单主机编码的嵌入.然后调用to_weights_sum将embedding_lookup_sparse的结果作为prediction返回，将embedding作为variable返回.

Here's what I think and please correct me if I'm wrong. The example of wide and deep model uses contrib.layers APIs and call embedding_lookup_sparse for sparse feature colume. If it gets the SparseTensor(for example, country, which is sparse), it creates the embedding which is actually for one-host encoding. Then call to_weights_sum to return the result of embedding_lookup_sparse as prediction and the embedding as variable.

embedding_lookup_sparse的结果加上bias，成为损失函数和训练操作的logits.这意味着 embedding_lookup_sparse 对密集张量执行类似 w * x(y = w * x + b 的一部分)之类的操作.

The the result of embedding_lookup_sparse add bias and become the logits for loss function and training operation. That means the embedding_lookup_sparse do something like w * x(part of y = w * x + b) for dense tensor.

也许对于 one-hot encoding 或 SparseTensor，来自 embedding_lookup_sparse 的 weight 实际上是 w * x 的值，因为查找数据始终为1，无需添加其他0.

Maybe for one-hot encoding or SparseTensor, the weight from embedding_lookup_sparse is actually the value of w * x because the look-up data is always 1 and no need to add other 0s.

我说的也令人困惑.谁能帮忙详细解释一下?

What I said is also confusing. Can anyone help to explain this in detail?

推荐答案

嵌入查找和嵌入查找稀疏的主要区别在于稀疏版本期望 id 和权重为 SparseTensor 类型.

The main difference between embedding lookup and embedding lookup sparse is that the sparse version expects the id's and weights to be of type SparseTensor.

嵌入查找的工作原理:

您传入某个大小的张量，embedding_lookup_sparse 会将张量的切片(由 sp_ids 参数引用的切片)乘以某个权重(也作为 sp_weight 传入；默认值为 1)，然后您将返回新切片.

You pass in a tensor of some size and the embedding_lookup_sparse will multiply the slices of the tensors (slices referenced by sp_ids parameter) by some weight (also passed in as sp_weight; defaults to value 1) then you are returned the new slices.

没有偏差项.您可以通过引用多个作为元素包含在输出中的张量切片来将张量切片添加在一起.

There is no bias term. You can add the slices of the tensor together by referencing more than one to be included as an element in your output.

这篇关于TensorFlow 中的 tf.nn.embedding_lookup_sparse 是什么意思?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！