新形状和旧形状必须具有相同数量的元素 [英] new shape and old shape must have the same number of elements
问题描述
出于学习目的，我正在使用Tensorflow.js，尝试将 fit 方法与批处理数据集(10乘10)一起使用以了解批处理过程时遇到错误.
我有一些要分类的图像600x600x3(2个输出，1或0)
这是我的训练循环:
const batchs =等待loadDataset()for(让i = 0; i< batchs.length; i ++){const batch =批处理[i]const xs = batch.xs.reshape([batch.size，600，600，3])const ys = tf.oneHot(batch.ys，2)console.log({xs:xs.shape，ys:ys.shape，})//{xs:[10，600，600，3]，ys:[10，2]}const history =等待model.fit(xs，ys，{batchSize:batch.size，时期:1})//< -----代码在这里抛出const损失= history.history.loss [0]常量精度= history.history.acc [0]console.log({损失，准确性})} 这是我定义数据集的方式
const块=块(examples，BATCH_SIZE)const batchs = chunks.map(批次=>{const ys = tf.tensor1d(batch.map(e => e.y)，'int32')const xs =批处理.map(e => imageToInput(e.x，3)).reduce((p，c)=> p?p.concat(c):c)返回{size:batch.length，xs，ys}}) 这是模型:
const模型= tf.sequential()model.add(tf.layers.conv2d({inputShape:[600，600，3]，kernelSize:60，筛选器:50，大步:20，激活:"relu"，kernelInitializer:'VarianceScaling'}))model.add(tf.layers.maxPooling2d({poolSize:[20，20]，大步走:[20，20]}))model.add(tf.layers.conv2d({kernelSize:5过滤器:100，大步:20，激活:"relu"，kernelInitializer:'VarianceScaling'}))model.add(tf.layers.maxPooling2d({poolSize:[20，20]，大步走:[20，20]}))model.add(tf.layers.flatten())model.add(tf.layers.dense({单位:2kernelInitializer:"VarianceScaling"，激活:"softmax"})) 在for循环的第一次迭代过程中，我从 .fit 出现以下错误:
错误:新形状和旧形状必须具有相同数量的元素.在Object.assert(/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/util.js:36:15)在reshape_(/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/ops/array_ops.js:271:10)在Object.reshape(/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/ops/operation.js:23:29)在Tensor.reshape(/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/tensor.js:273:26)在Object.derB [作为$ b](/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/ops/binary_ops.js:32:24)在_loop_1(/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/tape.js:90:47)在Object.backpropagateGradients(/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/tape.js:108:9)在/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/engine.js:334:20在/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/engine.js:91:22在Engine.scopedRun(/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/engine.js:101:23) 我不知道从中学到什么，也找不到关于该特定错误的文档或帮助，知道吗?
模型的问题在于 convolution 与 maxPooling 一起应用的方式>
第一层正在使用步幅为[20，20]和50个过滤器的kernelSize 60进行卷积.该层的输出将具有近似形状 [600/20，600/20，50] = [30，30，50]
最大池应用的步幅为 [20，20] .该层的输出也将具有近似的形状 [30/20，30/20，50] = [1，1，50]
从这一步开始，模型无法再使用kernelSize 5进行卷积.因为内核形状 [5，5] 大于输入形状 [1，1]导致抛出错误.该模型只能执行的卷积是大小为1的内核的卷积.显然，该卷积将在不进行任何变换的情况下输出输入.
同一规则适用于最后一个 maxPooling ，其 poolingSize 不能与1不同，否则将引发错误.
这是一个代码段:
const模型= tf.sequential()model.add(tf.layers.conv2d({inputShape:[600，600，3]，kernelSize:60，筛选器:50，大步:20，激活:"relu"，kernelInitializer:'VarianceScaling'}))model.add(tf.layers.maxPooling2d({poolSize:[20，20]，大步走:[20，20]}))model.add(tf.layers.conv2d({kernelSize:1过滤器:100，大步:20，激活:"relu"，kernelInitializer:'VarianceScaling'}))model.add(tf.layers.maxPooling2d({poolSize:1大步走:[20，20]}))model.add(tf.layers.flatten())model.add(tf.layers.dense({单位:2kernelInitializer:"VarianceScaling"，激活:"softmax"}))model.compile({optimizer:'sgd'，loss:'meanSquaredError'});model.fit(tf.ones([10，600，600，3])，tf.ones([10，2])，{batchSize:4});model.predict(tf.ones([1，600，600，3])).print() < html>< head><！-加载TensorFlow.js->< script src ="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@0.13.0"></script></head><身体></body></html> For learning purpose, I am using Tensorflow.js, and I experience an error while trying to use the fit method with a batched dataset (10 by 10) to learn the process of batch training.
I have got a few images 600x600x3 that I want to classify (2 outputs, either 1 or 0)
Here is my training loop:
const batches = await loadDataset()
for (let i = 0; i < batches.length; i++) {
const batch = batches[i]
const xs = batch.xs.reshape([batch.size, 600, 600, 3])
const ys = tf.oneHot(batch.ys, 2)
console.log({
xs: xs.shape,
ys: ys.shape,
})
// { xs: [ 10, 600, 600, 3 ], ys: [ 10, 2 ] }
const history = await model.fit(
xs, ys,
{
batchSize: batch.size,
epochs: 1
}) // <----- The code throws here
const loss = history.history.loss[0]
const accuracy = history.history.acc[0]
console.log({ loss, accuracy })
}
Here is how I define the dataset
const chunks = chunk(examples, BATCH_SIZE)
const batches = chunks.map(
batch => {
const ys = tf.tensor1d(batch.map(e => e.y), 'int32')
const xs = batch
.map(e => imageToInput(e.x, 3))
.reduce((p, c) => p ? p.concat(c) : c)
return { size: batch.length, xs , ys }
}
)
Here is the model:
const model = tf.sequential()
model.add(tf.layers.conv2d({
inputShape: [600, 600, 3],
kernelSize: 60,
filters: 50,
strides: 20,
activation: 'relu',
kernelInitializer: 'VarianceScaling'
}))
model.add(tf.layers.maxPooling2d({
poolSize: [20, 20],
strides: [20, 20]
}))
model.add(tf.layers.conv2d({
kernelSize: 5,
filters: 100,
strides: 20,
activation: 'relu',
kernelInitializer: 'VarianceScaling'
}))
model.add(tf.layers.maxPooling2d({
poolSize: [20, 20],
strides: [20, 20]
}))
model.add(tf.layers.flatten())
model.add(tf.layers.dense({
units: 2,
kernelInitializer: 'VarianceScaling',
activation: 'softmax'
}))
I get an error during the first iteration in the for-loop, from the .fit which is the following:
Error: new shape and old shape must have the same number of elements.
at Object.assert (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/util.js:36:15)
at reshape_ (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/ops/array_ops.js:271:10)
at Object.reshape (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/ops/operation.js:23:29)
at Tensor.reshape (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/tensor.js:273:26)
at Object.derB [as $b] (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/ops/binary_ops.js:32:24)
at _loop_1 (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/tape.js:90:47)
at Object.backpropagateGradients (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/tape.js:108:9)
at /Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/engine.js:334:20
at /Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/engine.js:91:22
at Engine.scopedRun (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/engine.js:101:23)
I don't know what to understand from that and found no documentation or help on that specific error, any idea?
The issue of the model lies in the way the convolution is applied along with the maxPooling
The first layer is doing a convolution of kernelSize 60 with a strides of [20, 20] and 50 filters.
The output of this layer will have the approximate shape [600 / 20, 600 / 20, 50] = [30, 30, 50]
The max pooling is applied with a stride of [20, 20]. The output of this layer will also have the approximate shape [30 / 20, 30 / 20, 50] =[1, 1, 50 ]
From this step, the model can no longer perform a convolution with a kernelSize 5. For the kernel shape [5, 5] is bigger than the input shape [1, 1] resulting in the error that is thrown. The only convolution the model can perform is that of a kernel whose size is 1. Obviously, that convolution will output the input without any transformation.
The same rule applies to the last maxPooling whose poolingSize cannot be different from 1, otherwise an error will be thrown.
Here is a snippet:
const model = tf.sequential()
model.add(tf.layers.conv2d({
inputShape: [600, 600, 3],
kernelSize: 60,
filters: 50,
strides: 20,
activation: 'relu',
kernelInitializer: 'VarianceScaling'
}))
model.add(tf.layers.maxPooling2d({
poolSize: [20, 20],
strides: [20, 20]
}))
model.add(tf.layers.conv2d({
kernelSize: 1,
filters: 100,
strides: 20,
activation: 'relu',
kernelInitializer: 'VarianceScaling'
}))
model.add(tf.layers.maxPooling2d({
poolSize: 1,
strides: [20, 20]
}))
model.add(tf.layers.flatten())
model.add(tf.layers.dense({
units: 2,
kernelInitializer: 'VarianceScaling',
activation: 'softmax'
}))
model.compile({optimizer: 'sgd', loss: 'meanSquaredError'});
model.fit(tf.ones([10, 600, 600, 3]), tf.ones([10, 2]), {batchSize: 4});
model.predict(tf.ones([1, 600, 600, 3])).print()
<html>
<head>
<!-- Load TensorFlow.js -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@0.13.0"> </script>
</head>
<body>
</body>
</html>
这篇关于新形状和旧形状必须具有相同数量的元素的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
