tensorflow 计算得到最大框后记住索引 [英] tensorflow remember the index after calculating getting the maximum box

问题描述

假设我有两个盒子数组，每个盒子的形状分别为 (?, b1, 4) 和 (?, b2, 4) (处理? 作为未知的批量大小):

Assume that I have two arrays of boxes, each of which has the shape (?, b1, 4) and (?, b2, 4) respectively (treat ? as a unknown batch size):

box1: [[[1,2,3,4], [2,3,4,5], [3,4,5,6]...]...]
box2: [[[4,3,2,1], [3,2,5,4], [4,3,5,6]...]...]

(以上数字任意设定)

我想:

1. 在每批中，对于box1中的每个框A，在box2中找到B框> 与 A(当然在同一批次)具有最大 IOU(交集)，然后将元组 (A, B) 附加到列表中list_max.

1. in each batch, for each box A in box1, find in box2 the box B which has the maximum IOU (intersection over union) with A (in the same batch, of course), and then append the tuple (A, B) to a list list_max.

将box2中所有与box1中任意一个框没有最大IOU的框追加到list_nonmax中(以batch分隔)，当然)

append to list_nonmax all the boxes in box2 that does not have maximum IOU with any box in box1 (separated by batch, of course)

您可以假设:

1. b1 和 b2 都是 python 变量，不是 tensorflow 张量.

1. b1 and b2 are both python variables, not tensorflow tensor.

计算单个盒子之间或一批盒子之间的IOU的方法已经存在并且可以直接使用:

methods for calculating IOU between single box or between batch of boxes already exists and can be used literally:

iou_single_box(box1, box2) :box1 和 box2 都是 (4,) 形状.

iou_single_box(box1, box2) : both box1 and box2 are of shape (4,).

iou_multiple_boxes(bbox1, bbox2) : bbox1 和 bbox2 的形状都是 (b1, 4)和 (b2, 4) 分别.

iou_multiple_boxes(bbox1, bbox2) : both bbox1 and bbox2 are of shape (b1, 4) and (b2, 4) respectively.

iou_batch_boxes(bbbox1, bbbox2) : bbbox1 和 bbbox2 的形状都是 (?, b1, 4) 和 (?, b2, 4) 分别(将 ? 视为未知的批大小).

iou_batch_boxes(bbbox1, bbbox2) : both bbbox1 and bbbox2 are of shape (?, b1, 4) and (?, b2, 4) respectively (treat ? as a unknown batch size).

我发现这些在 tensorflow 中特别困难，尤其是对于 list_nonmax 的情况，因为虽然它很容易使用填充然后 tf.reduce_max() 来获取框具有最大 iou 的元组，不可能记住它们的索引然后提取 list_nonmax 的框.

I found these particularly hard in tensorflow, especially for the list_nonmax case, because, whereas it is easy to use padding and then tf.reduce_max() to get box tuples with maximum iou, it is impossible to remember their index and then extract out boxes for list_nonmax.

推荐答案

您需要 tf.nn.top_k() 为此.它返回最大值和它在最后一维的索引.

You need tf.nn.top_k() for this. It returns both the maximum value and the index it's at at the last dimension.

val, idx = tf.nn.top_k( iou_batch_boxes( bbbox1, bbbox2 ), k = 1 )

将为您提供 box2 索引以及每个 box1 和批次的最大 iou.

will give you the box2 index with the max iou for each box1 and batch.

要获得list_max，您需要tf.stack() box1 与 box2 的条目由 idx 与tf.gather_nd() 沿轴1.这是一个带有虚拟 iou 函数的工作代码:

To get your list_max you need to tf.stack() box1 with box2's entries by idx with tf.gather_nd() along axis 1. Here's a working code with a dummy iou function:

import tensorflow as tf

box1 = tf.reshape( tf.constant( range( 16 ), dtype = tf.float32 ), ( 2, 2, 4 ) )
box2 = tf.reshape( tf.constant( range( 2, 26 ), dtype = tf.float32 ), ( 2, 3, 4 ) )
batch_size = box1.get_shape().as_list()[ 0 ]

def dummy_iou_batch_boxes( box1, box2 ):
    b1s, b2s = box1.get_shape().as_list(), box2.get_shape().as_list()
    return tf.constant( [ [ [9.0,8,7], [1,2,3],
                            [0  ,1,2], [0,5,0] ] ] )

iou = dummy_iou_batch_boxes( box1, box2 )
val, idx = tf.nn.top_k( iou, k = 1 )
idx = tf.reshape( idx, ( batch_size, box1.get_shape().as_list()[ 1 ] ) )
one_hot_idx = tf.one_hot( idx, depth = box2.get_shape().as_list()[ 1 ] )
full_idx = tf.where( tf.equal( 1.0, one_hot_idx ) )
box1_idx = full_idx[ :, 0 : 2 ]
box2_idx = full_idx[ :, 0 : 3 : 2 ]
box12 = tf.gather_nd( box1, box1_idx )
box22 = tf.gather_nd( box2, box2_idx )
list_max = tf.stack( [ box12, box22 ], axis = 1 )

with tf.Session() as sess:
    res = sess.run( [ list_max ] )
    for v in res:
        print( v )
        print( "-----------------------------")

将输出:

[[[0.1.2.3.]
[ 2. 3. 4. 5.]]

[[[ 0. 1. 2. 3.]
[ 2. 3. 4. 5.]]

[[ 4.5.6.7.]
[10.11. 12. 13.]]

[[ 4. 5. 6. 7.]
[10. 11. 12. 13.]]

[[ 8. 9. 10. 11.]
[22.23. 24. 25.]]

[[ 8. 9. 10. 11.]
[22. 23. 24. 25.]]

[[12.13. 14. 15.]
[18.19. 20. 21.]]]

[[12. 13. 14. 15.]
[18. 19. 20. 21.]]]

如果你想把它作为一个列表或元组，你可以使用 tf.unstack() 在上面的 list_max 上.

If you want this as a list or tuple, you can use tf.unstack() on the above list_max.

要获得list_nonmax，您需要将索引合并到一个掩码中，我相信我已经在 您的另一个问题，但重要的部分是:

To get list_nonmax the thing you need is merging the indices into a mask, which I believe I've already answered in another one of your questions, but the important part is:

mask = tf.reduce_max( tf.one_hot( idx, depth = num_bbbox2 ), axis = -2 )

这会给你一个形状为 ( batch, num_box2 ) 的掩码告诉你每个批次和每个 box2 如果那个 box2 是 any box1 的最大 iou .

This will give you a mask with shape ( batch, num_box2 ) telling you for each batch and each of box2 if that box2 is the max iou one for any box1.

从这里，您可以使用掩码或通过 <获取索引列表code>tf.where() 像这样:

From here, you can either use the mask or get the list of indices with tf.where() like this:

was_never_max_idx = tf.where( tf.equal( 0, mask ) )

这篇关于tensorflow 计算得到最大框后记住索引的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！