二进制数字而不是一个热向量 [英] Binary numbers instead of one hot vectors

问题描述

在进行逻辑回归时，通常的做法是使用一个热向量作为所需结果.因此，no of classes = no of nodes in output layer.我们不使用词汇表中的单词索引(或一般说来是一个类号)，因为这可能会错误地指示两个类的紧密程度.但是为什么我们不能使用二进制数而不是一元向量呢?

While doing logistic regression, it is common practice to use one hot vectors as desired result. So, no of classes = no of nodes in output layer. We don't use index of word in vocabulary(or a class number in general) because that may falsely indicate closeness of two classes. But why can't we use binary numbers instead of one-hot vectors?

即，如果有4个类，我们可以将每个类表示为00,01,10,11，从而在输出层中生成log(no of classes)个节点.

i.e if there are 4 classes, we can represent each class as 00,01,10,11 resulting in log(no of classes) nodes in output layer.

推荐答案

如果您使用二进制编码，那就很好.但是您可能需要根据任务和模型添加另一层(或过滤器).因为由于二进制表示，您的编码现在隐含了无效的共享功能.

It is fine if you encode with binary. But you probably need to add another layer (or a filter) depending on your task and model. Because your encoding now implicates invalid shared features due to the binary representation.

例如，输入(x = [x1, x2])的二进制编码:

For example, a binary encoding for input (x = [x1, x2]):

'apple' = [0, 0]
'orange' = [0, 1]
'table' = [1, 0]
'chair' = [1, 1]

这意味着orange和chair共享相同的功能x2.现在有了对两个类y的预测:

It means that orange and chair share same feature x2. Now with predictions for two classes y:

'fruit' = 0
'furniture' = 1

标记数据样本的线性优化模型(W = [w1, w2]和偏差b):

And linear optimization model (W = [w1, w2] and bias b) for labeled data sample:

(argmin W) Loss = y - (w1 * x1 + w2 * x2 + b)

每当将chair的w2权重更新为furniture时，您也会为此类选择orange获得积极的改进.在这种情况下，如果添加另一个图层U = [u1, u2]，则可能可以解决该问题:

Whenever you update w2 weights for chair as furniture you get a positive improvement of choosing orange for this class as well. In this particular case, if you add another layer U = [u1, u2], you can probably solve it:

(argmin U,W) Loss = y - (u1 * (w1 * x1 + w2 * x2 + b) +
                         u2 * (w1 * x1 + w2 * x2 + b) +
                         b2)

好吧，为什么不使用单热编码来避免这种未命中表示. :)

Ok, why not avoid this miss representation, by using one-hot encoding. :)

这篇关于二进制数字而不是一个热向量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！