带有tensorflow后端的keras:使用batch_size时的层实现,但在图形构建过程中为None [英] keras with tensorflow backend: layer implementation when batch_size is used, but it is None during graph construction

查看:118
本文介绍了带有tensorflow后端的keras:使用batch_size时的层实现,但在图形构建过程中为None的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我正在为视网膜网在对象检测中实现一个keras层AnchorTargets,它输入anchorsannotations(即地面真相边界框).代码的想法是由github中的 keras-retinanet 激发的.

I am implementing a keras layer AnchorTargets for retinanet in object detection, it inputs anchors and annotations (i.e. ground-truth bounding-boxes). The ideas for the codes are motivated by keras-retinanet in github.

但是,在labels_shape = (input_shape[0]*input_shape[1], num_classes)中出现错误,因为input_shape [0]表示批处理大小,这在网络构建期间不可用.关于这个问题有什么建议吗?谢谢.

However, I got error in labels_shape = (input_shape[0]*input_shape[1], num_classes) since input_shape[0] denotes batch size, which is unavailable during network construction. Is there any suggestions for this issue? Thanks.

def batch_anchor_targets_bbox(
anchors,
annotations,
num_classes,
mask_shape=None,
negative_overlap=0.4,
positive_overlap=0.5,
**kwargs):
""" Generate anchor targets for bbox detection.

Args
    anchors: np.array of annotations of shape (B, N, 4) for (x1, y1, x2, y2).
    annotations: np.array of shape (B, M, 5) for (x1, y1, x2, y2, label).
    num_classes: Number of classes to predict.
    mask_shape: If the image is padded with zeros, mask_shape can be used to mark the relevant part of the image.
    negative_overlap: IoU overlap for negative anchors (all anchors with overlap < negative_overlap are negative).
    positive_overlap: IoU overlap or positive anchors (all anchors with overlap > positive_overlap are positive).

Returns
    labels: np.array of shape (B, N, num_classes) where a row consists of 0 for negative and 1 for positive for a certain class.
    annotations: np.array of shape (B, N, 5) for (x1, y1, x2, y2, label) containing the annotations corresponding to each anchor or 0 if there is no corresponding anchor.
    anchor_states: np.array of shape (B, N) containing the state of an anchor (-1 for ignore, 0 for bg, 1 for fg).
"""
# anchor states: 1 is positive, 0 is negative, -1 is dont care    
anchors     = keras.backend.cast(anchors, 'float32')
annotations = keras.backend.cast(annotations, 'float32')
input_shape = keras.backend.int_shape(anchors)
annos_shape = keras.backend.int_shape(annotations)    

overlaps = batch_compute_overlap(anchors, annotations)   
argmax_overlaps_inds = keras.backend.argmax(overlaps, axis=2)
max_overlaps         = keras.backend.max(overlaps, axis=2)

# assign "dont care" labels
flag_pos = keras.backend.greater_equal(max_overlaps, positive_overlap)
anchor_states = keras.backend.cast(flag_pos, 'float32')
flag = backend.logical_and(keras.backend.greater_equal(max_overlaps, negative_overlap), \
                               keras.backend.less(max_overlaps, positive_overlap))
anchor_states = anchor_states - keras.backend.cast(flag, 'float32')

# reshape the values and indices of the max output
argmax_overlaps_inds = keras.backend.reshape(argmax_overlaps_inds,(-1,))
flag_pos = keras.backend.reshape(flag_pos,(-1,))

# compute box regression targets
annotations_o = keras.backend.reshape(annotations, (-1, annos_shape[2]))
annotations_o = backend.gather(annotations_o,argmax_overlaps_inds)
annotations = keras.backend.reshape(annotations_o, (-1, input_shape[1],annos_shape[2]))

# compute target class labels
ind_x = backend.where(flag_pos)
ind_y = keras.backend.cast(backend.gather(annotations_o[:,4], ind_x), 'int64')
indices = keras.backend.concatenate([ind_x,ind_y], axis=1)
labels_shape = (input_shape[0]*input_shape[1], num_classes)
values = keras.backend.ones(keras.backend.shape(ind_x)[0])
delta = backend.SparseTensor(indices, values, labels_shape)
labels = backend.sparse_tensor_to_dense(delta)
labels = keras.backend.reshape(labels, (-1, input_shape[1], num_classes))
return indices, annotations, anchor_states


class AnchorTargets(keras.layers.Layer):

def __init__(self, num_classes, negative_overlap=0.4, positive_overlap=0.5, *args, **kwargs):
    self.num_classes = num_classes
    self.negative_overlap = negative_overlap
    self.positive_overlap = positive_overlap

    super(AnchorTargets, self).__init__(*args, **kwargs)

def call(self, inputs, **kwargs): 
    # anchors (batch_size x N x 4): pre-defined anchors 
    # annotations (batch_size x #gt x 4): gt_bboxes.

    anchors, annotations = inputs 

    # calculate regression and labels
    labels, annotations, anchor_states = batch_anchor_targets_bbox(anchors, annotations,
                            self.num_classes, self.negative_overlap, self.positive_overlap)
    anchor_states = keras.backend.expand_dims(anchor_states,axis=2)
    labels = keras.backend.concatenate([labels, anchor_states], axis=2)
    regressions = keras.backend.concatenate([annotations, anchor_states], axis=2)
    return [regressions, labels]

def compute_output_shape(self, input_shape):
    return [(input_shape[0][0],input_shape[0][1],input_shape[0][2]+1),\
            (input_shape[0][0],input_shape[0][1],self.num_classes+2)]

def get_config(self):
    config = super(AnchorTargets, self).get_config()
    config.update({
        'num_classes':      self.num_classes,
        'negative_overlap': self.negative_overlap,
        'positive_overlap': self.positive_overlap
    })

    return config

推荐答案

没关系.我已经知道了.基本思想是避免在keras.layers.Layercall函数中使用input_shape [0](即批处理大小).如果有人对此感兴趣,我的实现如下.

Never mind. I have figured it out. The basic idea is to avoid the use of input_shape[0] (i.e. batch size) in call function of keras.layers.Layer. My implementation is as follows if anyone is interested in it.

def batch_anchor_targets_bbox(
anchors,
annotations,
num_classes,
mask_shape=None,
negative_overlap=0.4,
positive_overlap=0.5,
**kwargs):
""" Generate anchor targets for bbox detection.

Args
    anchors: np.array of annotations of shape (B, N, 4) for (x1, y1, x2, y2).
    annotations: np.array of shape (B, M, 5) for (x1, y1, x2, y2, label).
    num_classes: Number of classes to predict.
    mask_shape: If the image is padded with zeros, mask_shape can be used to mark the relevant part of the image.
    negative_overlap: IoU overlap for negative anchors (all anchors with overlap < negative_overlap are negative).
    positive_overlap: IoU overlap or positive anchors (all anchors with overlap > positive_overlap are positive).

Returns
    labels: np.array of shape (B, N, num_classes) where a row consists of 0 for negative and 1 for positive for a certain class.
    annotations: np.array of shape (B, N, 5) for (x1, y1, x2, y2, label) containing the annotations corresponding to each anchor or 0 if there is no corresponding anchor.
    anchor_states: np.array of shape (B, N) containing the state of an anchor (-1 for ignore, 0 for bg, 1 for fg).
"""
# anchor states: 1 is positive, 0 is negative, -1 is dont care    
anchors     = keras.backend.cast(anchors, 'float32')
annotations = keras.backend.cast(annotations, 'float32')
input_shape = keras.backend.int_shape(anchors)
annos_shape = keras.backend.int_shape(annotations)    

overlaps = batch_compute_overlap(anchors, annotations)   
argmax_overlaps_inds = keras.backend.argmax(overlaps, axis=2)
max_overlaps         = keras.backend.max(overlaps, axis=2)

# assign "dont care" labels
flag_pos = keras.backend.greater_equal(max_overlaps, positive_overlap)
anchor_states = keras.backend.cast(flag_pos, 'float32')
flag = backend.logical_and(keras.backend.greater_equal(max_overlaps, negative_overlap), \
                               keras.backend.less(max_overlaps, positive_overlap))
anchor_states = anchor_states - keras.backend.cast(flag, 'float32')

# reshape the values and indices of the max output
argmax_overlaps_inds = keras.backend.reshape(argmax_overlaps_inds,(-1,))
flag_pos = keras.backend.reshape(flag_pos,(-1,))

# compute box regression targets
annotations_o = keras.backend.reshape(annotations, (-1, annos_shape[2]))
annotations_o = backend.gather(annotations_o,argmax_overlaps_inds)
annotations = keras.backend.reshape(annotations_o, (-1, input_shape[1],annos_shape[2]))

# compute target class labels
anchor_states_o = -1*keras.backend.cast(keras.backend.less(max_overlaps, positive_overlap), 'float32')
anchor_states_o = keras.backend.reshape(anchor_states_o, (-1,))
ind_y = backend.where(keras.backend.greater_equal(anchor_states_o,-0.5), annotations_o[:,4], anchor_states_o)
labels = keras.backend.one_hot(keras.backend.cast(ind_y,'int32'), num_classes)
labels = keras.backend.reshape(labels, (-1, input_shape[1], num_classes))
return labels, annotations, anchor_states

这篇关于带有tensorflow后端的keras:使用batch_size时的层实现,但在图形构建过程中为None的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆