在 tensorflow 中拆分字符串 [英] Split strings in tensorflow

问题描述

我有一个维度为 [None, None] 的二维字符串张量.我想使用 tf.string_split() 函数拆分每个字符串.我试过使用下面的代码

I have a 2-D tensor of strings having dimension [None, None]. I want to split each string using tf.string_split() function. I have tried using the below code

sentences = tf.placeholder(shape=[None, None], dtype=tf.string)
# remove punctuation
normalized_sentences = tf.regex_replace(input=sentences, pattern=r"\pP", rewrite="")        
tokens = tf.map_fn(lambda x: tf.string_split(x, delimiter=" "), normalized_sentences, dtype=tf.string)

但是这个错误给我带来了错误.

But the error throws me an error.

TypeError: Failed to convert object of type <class 'tensorflow.python.framework.sparse_tensor.SparseTensor'> to Tensor. Contents: SparseTensor(indices=Tensor("map/while/StringSplit:0", shape=(?, 2), dtype=int64), values=Tensor("map/while/StringSplit:1", shape=(?,), dtype=string), dense_shape=Tensor("map/while/StringSplit:2", shape=(2,), dtype=int64)). Consider casting elements to a supported type.

推荐答案

代替 tf.string_split(x) 使用 tf.string_split(x).values:

import tensorflow as tf

tf.enable_eager_execution()

print(tf.string_split(['this is example sentence']).values)
# tf.Tensor([b'this' b'is' b'example' b'sentence'], shape=(4,), dtype=string)

这篇关于在 tensorflow 中拆分字符串的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！