将张量组织成一批动态形状的张量 [英] Organizing tensor into batches of dynamically shaped tensors

问题描述

我有以下情况:

• 我想使用 Tensorflow Serving 部署人脸检测器模型:

  解决方案

  我最终使用 TensorArray 和 tf.while_loop 找到了解决方案:

  def batch_reconstructor(tensor, partitions, batch_size):"""取一个形状为 (batch_size, 4) 的张量和一个一维分区张量以及标量 batch_size并重建一个保留原始批处理的 TensorArray从分区中，我们可以获得批次内的最大张量.这将通知我们需要使用的填充.参数:- 张量:要转换为批次的张量- 分区:批量索引列表.位置 i 处的张量对应于批次 # partitions[i]"""tfarr = tf.TensorArray(tf.int32, size=batch_size, infer_shape=False)_, _, count = tf.unique_with_counts(partitions)maximum_tensor_size = tf.cast(tf.reduce_max(count), tf.int32)padding_tensor_index = tf.cast(tf.gather(tf.shape(tensor), 0), tf.int32)padding_tensor = tf.expand_dims(tf.cast(tf.fill([4], -1), tf.float32), axis=0) # 用 [-1, -1, -1, -1] 填充张量 = tf.concat([张量，padding_tensor]，轴=0)def cond(i, acc):返回 tf.less(i, batch_size)def body(i, acc):partition_indices = tf.reshape(tf.cast(tf.where(tf.equal(partitions, i)), tf.int32), [-1])partition_size = tf.gather(tf.shape(partition_indices), 0)# 用 padding_size * padding_tensor_index 连接 partition_indicespadding_size = tf.subtract(maximum_tensor_size, partition_size)padding_indices = tf.reshape(tf.fill([padding_size], padding_tensor_index), [-1])partition_indices = tf.concat([partition_indices, padding_indices],axis=0)返回 (tf.add(i, 1), acc.write(i, tf.gather(tensor, partition_indices)))_, 重构 = tf.while_loop(条件，身体，(tf.constant(0), tfarr),名称='batch_reconstructor')重建 = 重建.stack()回归重建

  I have the following situation:

  • I want to deploy a face detector model using Tensorflow Serving: https://www.tensorflow.org/serving/.
  • In Tensorflow Serving, there is a command line option called --enable_batching. This causes the model server to automatically batch the requests to maximize throughput. I want this to be enabled.
  • My model takes in a set of images (called images), which is a tensor of shape (batch_size, 640, 480, 3).
  • The model has two outputs: (number_of_faces, 4) and (number_of_faces,). The first output will be called faces. The last output, which we can call partitions is the index in the original batch for the corresponding face. For example, if I pass in a batch of 4 images and get 7 faces, then I might have this tensor as [0, 0, 1, 2, 2, 2, 3]. The first two faces correspond to the first image, the third face for the second image, the 3rd image has 3 faces, etc.

  My issue is this:

  • In order for the --enable_batching flag to work, the output from my model needs to have the 0th dimension the same as the input. That is, I need a tensor with the following shape: (batch_size, ...). I suppose this is so that the model server can know which grpc connection to send each output in the batch towards.
  • What I want to do is to convert my output tensor from the face detector from this shape (number_of_faces, 4) to this shape (batch_size, None, 4). That is, an array of batches, where each batch can have a variable number of faces (e.g. one image in the batch may have no faces, and another might have 3).

  What I tried:

  • tf.dynamic_partition. On the surface, this function looks perfect. However, I ran into difficulties after realizing that the num_partitions parameter cannot be a tensor, only an integer:

    tensorflow_serving_output = tf.dynamic_partition(faces, partitions, batch_size)

  If the tf.dynamic_partition function were to accept tensor values for num_partition, then it seems that my problem would be solved. However, I am back to square one since this is not the case.

  Thank you all for your help! Let me know if anything is unclear

  P.S. Here is a visual representation of the intended process:

  解决方案

  I ended up finding a solution to this using TensorArray and tf.while_loop:

  def batch_reconstructor(tensor, partitions, batch_size):
      """
      Take a tensor of shape (batch_size, 4) and a 1-D partitions tensor as well as the scalar batch_size
      And reconstruct a TensorArray that preserves the original batching
  
      From the partitions, we can get the maximum amount of tensors within a batch. This will inform the padding we need to use.
      Params:
          - tensor: The tensor to convert to a batch
          - partitions: A list of batch indices. The tensor at position i corresponds to batch # partitions[i]
      """
      tfarr = tf.TensorArray(tf.int32, size=batch_size, infer_shape=False)
  
      _, _, count = tf.unique_with_counts(partitions)
      maximum_tensor_size = tf.cast(tf.reduce_max(count), tf.int32)
  
      padding_tensor_index = tf.cast(tf.gather(tf.shape(tensor), 0), tf.int32)
  
      padding_tensor = tf.expand_dims(tf.cast(tf.fill([4], -1), tf.float32), axis=0) # fill with [-1, -1, -1, -1]
      tensor = tf.concat([tensor, padding_tensor], axis=0)
  
      def cond(i, acc):
          return tf.less(i, batch_size)
  
      def body(i, acc):
          partition_indices = tf.reshape(tf.cast(tf.where(tf.equal(partitions, i)), tf.int32), [-1])
  
          partition_size = tf.gather(tf.shape(partition_indices), 0)
  
          # concat the partition_indices with padding_size * padding_tensor_index
          padding_size = tf.subtract(maximum_tensor_size, partition_size)
          padding_indices = tf.reshape(tf.fill([padding_size], padding_tensor_index), [-1])
  
          partition_indices = tf.concat([partition_indices, padding_indices], axis=0)
  
          return (tf.add(i, 1), acc.write(i, tf.gather(tensor, partition_indices)))
  
      _, reconstructed = tf.while_loop(
          cond,
          body,
          (tf.constant(0), tfarr),
          name='batch_reconstructor'
      )
  
      reconstructed = reconstructed.stack()
      return reconstructed
  

  这篇关于将张量组织成一批动态形状的张量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！