Tensorflow一个热编码器? [英] Tensorflow One Hot Encoder?
问题描述
张量流是否具有类似于scikit Learn的一个热编码器来进行处理分类数据?使用tf.string的占位符会表现为分类数据吗?
Does tensorflow have something similar to scikit learn's one hot encoder for processing categorical data? Would using a placeholder of tf.string behave as categorical data?
我意识到我可以在将数据发送到tensorflow之前对其进行手动预处理,但是将其内置非常方便.
I realize I can manually pre-process the data before sending it to tensorflow, but having it built in is very convenient.
推荐答案
从TensorFlow 0.8开始,现在有一个原生一键运算tf.one_hot
,可以将一组稀疏标签转换为密集的一键运算.这是对 tf.nn.sparse_softmax_cross_entropy_with_logits
的补充.让您直接在稀疏标签上计算交叉熵,而不是将它们转换为一键热.
As of TensorFlow 0.8, there is now a native one-hot op, tf.one_hot
that can convert a set of sparse labels to a dense one-hot representation. This is in addition to tf.nn.sparse_softmax_cross_entropy_with_logits
, which can in some cases let you compute the cross entropy directly on the sparse labels instead of converting them to one-hot.
上一个答案,以防您想采用旧方法: @Salvador的答案是正确的-过去(过去)没有本机操作.不过,您可以使用稀疏到密集运算符在tensorflow中本地执行此操作,而不是在numpy中执行此操作:
Previous answer, in case you want to do it the old way: @Salvador's answer is correct - there (used to be) no native op to do it. Instead of doing it in numpy, though, you can do it natively in tensorflow using the sparse-to-dense operators:
num_labels = 10
# label_batch is a tensor of numeric labels to process
# 0 <= label < num_labels
sparse_labels = tf.reshape(label_batch, [-1, 1])
derived_size = tf.shape(label_batch)[0]
indices = tf.reshape(tf.range(0, derived_size, 1), [-1, 1])
concated = tf.concat(1, [indices, sparse_labels])
outshape = tf.pack([derived_size, num_labels])
labels = tf.sparse_to_dense(concated, outshape, 1.0, 0.0)
标签的输出是一批次矩阵x数量为num_labels的矩阵.
The output, labels, is a one-hot matrix of batch_size x num_labels.
还请注意,自2016年2月12日起(我认为最终将成为0.7发行版的一部分),TensorFlow还具有tf.nn.sparse_softmax_cross_entropy_with_logits
op,在某些情况下,它可以让您进行培训而无需转换为一键式编码.
Note also that as of 2016-02-12 (which I assume will eventually be part of a 0.7 release), TensorFlow also has the tf.nn.sparse_softmax_cross_entropy_with_logits
op, which in some cases can let you do training without needing to convert to a one-hot encoding.
编辑后添加:最后,您可能需要显式设置标签的形状.形状推断无法识别num_labels组件的大小.如果您不需要动态的批量大小,则可以简化操作.
Edited to add: At the end, you may need to explicitly set the shape of labels. The shape inference doesn't recognize the size of the num_labels component. If you don't need a dynamic batch size with derived_size, this can be simplified.
修改了2016年2月12日,以更改以下每个注释的形状分配.
Edited 2016-02-12 to change the assignment of outshape per comment below.
这篇关于Tensorflow一个热编码器?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!