Tensorflow:如何在排除填充值的同时从张量中选择随机值? [英] Tensorflow: How to select random values from tensor while excluding padded values?

问题描述

我有一批由 dataset.padded_batch 批处理的 Tensorflow 张量，因为张量的长度各不相同.

I have a batch of Tensorflow tensors which are batched by dataset.padded_batch since the tensors vary in length.

从这些张量中，我想选择几个随机值，不包括填充值.

From these Tensors, I would like to select several random values, excluding the padded values.

这是一个最小的示例，其中填充的值为-1".

Here is a minimal example, where the padded values are '-1'.

import math
import numpy as np
import tensorflow as tf

#Set up data
cells = np.array([[0,1,2,3], [2,3,4], [3,6,5,4,3], [3,9]])
mells = np.array([[0], [2], [3], [9]])
print(cells)

#Write data to tfrecords
writer = tf.python_io.TFRecordWriter('test.tfrecords')
for index in range(mells.shape[0]):
    example = tf.train.Example(features=tf.train.Features(feature={
        'num_value':tf.train.Feature(int64_list=tf.train.Int64List(value=mells[index])),
        'list_value':tf.train.Feature(int64_list=tf.train.Int64List(value=cells[index]))
    }))
    writer.write(example.SerializeToString())
writer.close()

#Open tfrecords using dataset api and batch data
filenames = ["test.tfrecords"]
dataset = tf.data.TFRecordDataset(filenames)
def _parse_function(example_proto):
    keys_to_features = {'num_value':tf.VarLenFeature(tf.int64),
                        'list_value':tf.VarLenFeature(tf.int64)}
    parsed_features = tf.parse_single_example(example_proto, keys_to_features)
    return tf.sparse.to_dense(parsed_features['num_value']), \
           tf.sparse.to_dense(parsed_features['list_value'])
# Parse the record into tensors.
dataset = dataset.map(_parse_function)
# Shuffle the dataset
dataset = dataset.shuffle(buffer_size=1)
# Repeat the input indefinitly
dataset = dataset.repeat()  
# Generate batches
dataset = dataset.padded_batch(3, padded_shapes=([None],[None]), padding_values=(tf.constant(-1, dtype=tf.int64)
                                                 ,tf.constant(-1, dtype=tf.int64)))
# Create a one-shot iterator
iterator = dataset.make_one_shot_iterator()
i, data = iterator.get_next()

#Number or random samples we want to get
size = tf.placeholder(tf.int32)

#Retrieve random samples from the batch
y1 = tf.py_func(lambda x, s: np.random.choice(x.reshape(-1),s), [data[0], size], tf.int64)
y2 = tf.py_func(lambda x, s: np.random.choice(x.reshape(-1),s), [data[1], size], tf.int64)
y3 = tf.py_func(lambda x, s: np.random.choice(x.reshape(-1),s), [data[2], size], tf.int64)

with tf.Session() as sess:
    print(sess.run([y1, y2, y3 ], {size:5}))

如何从随机选择中排除-1"?

How can I exclude '-1' from the random selection ?

推荐答案

试试 x[x>-1]:

y1 = tf.py_func(lambda x, s: np.random.choice(x[x>-1].reshape(-1),s), [data[0], size], tf.int64)
y2 = tf.py_func(lambda x, s: np.random.choice(x[x>-1].reshape(-1),s), [data[1], size], tf.int64)
y3 = tf.py_func(lambda x, s: np.random.choice(x[x>-1].reshape(-1),s), [data[2], size], tf.int64)

这篇关于Tensorflow:如何在排除填充值的同时从张量中选择随机值?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！