如何在Keras中为张量的2D子集分配另一个2D张量? [英] How do I assign a 2D subset of my tensor with another 2D tensor in Keras?

问题描述

如果我有两个3-D张量img和gen.如何为img的2D子集和gen的2D子集分配?由于tensorflow不允许直接分配张量，因此以下内容不起作用.

If I have two 3-D tensors img and gen. How do I assign a 2D subset of img with a 2D subset of gen? The following doesn't work as tensorflow doesn't allow direct assignment of tensors.

img[96:160 , 144:240 , :] = gen[96:160 , 144:240 , :]

这是周围的代码.所以我正在使用自定义的keras层.该层必须接收输入图像img和生成的图像x.它必须用x替换img的一部分，并且必须返回修改后的img.

This is the surrounding code. So I'm using a custom keras layer. This layer must receive the input image img and the generated image x. It must replace a portion of img with x and must return the modified img.

def patcher(tensors):
    img = tensor[1]
    gen = tensor[0]
    #This is where the slicing must happen
    img[96:160 , 144:240 , :] = gen[96:160 , 144:240 , :]
    return [img]

img = Input( .. )
x = Conv( .. )(img)
out = Lambda(patcher,lambda a : [a[1]] )([x , img])
model = Model(img, out)

推荐答案

当前，您无法以简单的方式替换张量的切片.实际上，我提出了一个问题，因为这是人们一直在要求的东西.使用当前的API，您必须设法找出构建所需张量的最佳方法.在这种情况下，假设img和gen都具有相同的形状，这是您可以做到的一种方法:

Currently, you cannot replace slices of a tensor in a straightforward manner. I actually opened an issue about it because it is something that people keep asking for. With the current API, you have to sort of figure out the best way to build the tensor you want. In this case, assuming img and gen have both the same shape, this is one way you could do that:

import tensorflow as tf
import numpy as np

# Input
img = tf.placeholder(tf.float32, [None, None, None])
gen = tf.placeholder(tf.float32, [None, None, None])
row_start = tf.placeholder(tf.int32, [])
row_end = tf.placeholder(tf.int32, [])
col_start = tf.placeholder(tf.int32, [])
col_end = tf.placeholder(tf.int32, [])
# Masks rows and columns to be replaced
shape = tf.shape(img)
rows = shape[0]
cols = shape[1]
channels = shape[2]
i = tf.range(rows)
row_mask = (row_start <= i) & (i < row_end)
j = tf.range(cols)
col_mask = (col_start <= j) & (j < col_end)
# Full mask of replaced elements
mask = row_mask[:, tf.newaxis] & col_mask
# Select elements from flattened arrays
img_flat = tf.reshape(img, [-1, channels])
gen_flat = tf.reshape(gen, [-1, channels])
mask_flat = tf.reshape(mask, [-1])
result_flat = tf.where(mask_flat, gen_flat, img_flat)
# Reshape back
result = tf.reshape(result_flat, shape)

这是一个小测试:

with tf.Session() as sess:
    # img is positive and gen is negative
    img_val = np.arange(60).reshape((4, 5, 3))
    gen_val = -img_val
    # Do img[2:4, 0:3, :] = gen[2:4, 0:3, :]
    result_val = sess.run(result, feed_dict={
        img: img_val,
        gen: gen_val,
        row_start: 2,
        row_end: 4,
        col_start: 0,
        col_end: 3,
    })
    # Print one channel only for clarity
    print(result_val[:, :, 0])

输出:

[[  0.   3.   6.   9.  12.]
 [ 15.  18.  21.  24.  27.]
 [-30. -33. -36.  39.  42.]
 [-45. -48. -51.  54.  57.]]

---

---

这是您发布的代码的可能实现.我在这里使用的是基于乘法的稍微不同的方法，当您有很多图像时，我认为这种方法会更好.

Here is a possible implementation for the code that you have posted. I am using a slightly different method here based on multiplication, which I think is better when you have many images.

import tensorflow as tf

def replace_slices(img, gen, row_start, row_end, col_start, col_end):
    # Masks rows and columns to be replaced
    shape = tf.shape(img)
    rows = shape[1]
    cols = shape[2]
    i = tf.range(rows)
    row_mask = (row_start <= i) & (i < row_end)
    j = tf.range(cols)
    col_mask = (col_start <= j) & (j < col_end)
    # Full mask of replaced elements
    mask = row_mask[:, tf.newaxis] & col_mask
    # Add channel dimension to mask and cast
    mask = tf.cast(mask[:, :, tf.newaxis], img.dtype)
    # Compute result
    result = img * (1 - mask) + gen * mask
    return result

def patcher(tensors):
    img = tensor[1]
    gen = tensor[0]
    img = replace_slices(img, gen, 96, 160, 144, 240)
    return [img]

img = Input( .. )
x = Conv( .. )(img)
out = Lambda(patcher, ambda a: [a[1]])([x , img])
model = Model(img, out)

这篇关于如何在Keras中为张量的2D子集分配另一个2D张量?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！