如何在Keras中对(回归)模型输出强制执行单调性? [英] How to enforce monotonicity for (regression) model outputs in Keras?

问题描述

我目前正在处理一个问题，该问题是为神经网络提供一个输入变量 a 和另一个输入 x ，它是N个数字的单调递增序列.

I am currently working on a problem where I provide a neural network with an input variable a, and another input x which is a monotonically increasing sequence of N numbers.

所以我的网络基本上看起来像这样:

So my network would basically looks something like this:

a_input = Input(shape=[1], name='a')
x_input = Input(shape=[N], name='x')
nn = concatenate([a_input, x_input])
nn = Dense(100, activation='relu')(nn)
nn = Dense(N, activation='relu')(nn)
model = Model(inputs=[a_input, x_input], outputs=[nn])
model.compile(loss='mean_squared_error', optimizer="adam")

我在输入空间上执行回归(对于每个 a 来说，序列 x 是唯一的) ，我希望网络为每组输入 a 和 x .

I perform a regression over the input space (where for each a the sequence x is unique), and I want the network to output a monotonically increasing sequence of (non-negative) N numbers for each set of inputs a and x.

现在，我已经注意到，到目前为止，我的输出并不是严格意义上的单调的，而是看起来就像您缩小"时的输出一样.我的意思是，对于给定的 a 和 x ，如果我想让输出数组看起来像:

Now, I have noticed that so far my outputs are not strictly speaking monotonic, but sort of look like they are if you 'zoom out'. By this I mean, for a given choice of a and x, if I want my output array to look like:

[0, 0.5, 0.51, 0.7, 0.75, 0.9, 1.], 

我可能会得到:

[0.001, 0.5, 0.48, 0.7, 0.75, 0.9, 1.].

因此，我想知道Keras中是否有标准方法或特定工具可以将模型约束为仅输出单调递增的序列?

Hence, I would like to know if there are standard ways, or specific tools already available in Keras, to constrain models to only output monotonically increasing sequences?

推荐答案

要强制执行非负输出，请在输出层中使用非负激活，例如ReLU或Sigmoid.

To enforce non-negative outputs, use a non-negative activation such as ReLU or sigmoid in your output layer.

我不知道有什么 neural 方法可以在您的输出中强制单调性，但是在我看来，明智的方法是更改​​输出表示形式，以使网络预测两个连续元素之间的差异.例如，您可以转换输出数组:

I am not aware of any neural method to enforce monotonicity in your output, but in my opinion a sensible approach would be to change the output representation to make the network predict the difference between two consecutive elements. For example, you could transform your output array:

a=[0, 0.5, 0.51, 0.7, 0.75, 0.9, 1.]

收件人:

b=[0, 0.5, 0.01, 0.19, 0.05, 0.15, 0.1]

，其中b[0] = a[0]和b[i] = a[i]-a[i-1]用于i>0.在这种情况下，将循环层用作输出层是有意义的，因为每个输出单元现在都依赖于先前的单元.您的原始表示形式可以很容易地恢复为a[0] = b[0]的a[0] = b[0]和a[i] = b[i]+a[i-1]，并且由于每个输出b[i]为非负值，因此所得序列将单调递增.

with b[0] = a[0] and b[i] = a[i]-a[i-1] for i>0. Within this context, it would make sense to use a recurrent layer as the output layer, since each output unit now depends on the previous ones. Your original representation can be easily recovered as a[0] = b[0] and a[i] = b[i]+a[i-1] for i>0, and the resulting sequence will be monotonically increasing because each output b[i] is non-negative.

更新1 . LSTM应该返回完整序列.您可以尝试按以下方式构建模型:

UPDATE 1. The LSTM should return the full sequence. You could try building the model as follows:

a_input = Input(shape=[1], name='a')
x_input = Input(shape=[N], name='x')
nn = concatenate([a_input, x_input])
nn = Dense(100, activation='relu')(nn)
nn = Dense(N, activation='relu')(nn)
nn = Lambda(lambda x: x[..., None])(nn)  # Output shape=(batch_size, nb_timesteps=N, input_dim=1)
nn = LSTM(1, return_sequences=True, activation='relu')(nn)  # Output shape=(batch_size, nb_timesteps=N, output_dim=1)
nn = Lambda(lambda x: keras.backend.squeeze(x, axis=-1))(nn)  # Output shape=(batch_size, N)
model = Model(inputs=[a_input, x_input], outputs=[nn])
model.compile(loss='mean_squared_error', optimizer="adam")

---

更新2 .具有一个隐藏单元的LSTM可能不够强大.我不确定这是否有帮助，但是您可以尝试在最后一个之前添加另一个具有更多单位(即10)的LSTM层:

---

UPDATE 2. The LSTM with one hidden unit might not be powerful enough. I am not sure if this will help, but you could try adding another LSTM layer with more units (i.e. 10) before the last one:

...
nn = Lambda(lambda x: x[..., None])(nn)  # Output shape=(batch_size, nb_timesteps=N, input_dim=1)
nn = LSTM(10, return_sequences=True)(nn)  # Output shape=(batch_size, nb_timesteps=N, output_dim=10)
nn = LSTM(1, return_sequences=True, activation='relu')(nn)  # Output shape=(batch_size, nb_timesteps=N, output_dim=1)
nn = Lambda(lambda x: keras.backend.squeeze(x, axis=-1))(nn)  # Output shape=(batch_size, N)
...

这篇关于如何在Keras中对(回归)模型输出强制执行单调性?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！