Tensorflow.js神经网络中的反向传播 [英] Backpropagation in an Tensorflow.js Neural Network

问题描述

当我尝试实现此功能时，将 tf.train.stg(learningRate).minimize(loss)嵌入我的代码中以进行反向传播.我遇到了多个错误，例如在variableGrads(f)中传递的f必须是一个函数.我如何将上述功能成功实现到下面的代码中?为什么会出现此错误?

When I have been attempting to implement this function tf.train.stg(learningRate).minimize(loss)into my code in order to conduct back-propagation. I have been getting multiple errors such The f passed in variableGrads(f) must be a function. How would I implement the function above into the code bellow successfully? and Why does this error even occur?

神经网络:

    var X = tf.tensor([[1,2,3], [4,5,6], [7,8,9], [10,11,12]])
    var Y = tf.tensor([[0,0,0],[0,0,0], [1,1,1]])
    var m = X.shape[0]
    var a0 = tf.zeros([1,3])
    var y_hat = tf.zeros([1,3])
    var parameters = {
        "Wax": tf.randomUniform([1,3]),
        "Waa": tf.randomUniform([3,3]),
        "ba": tf.zeros([1,3]),
        "Wya": tf.randomUniform([3,3]),
        "by": tf.zeros([1,3])
    }

   

    function RNN_cell_Foward(xt, a_prev, parameters){
        var Wax = parameters["Wax"]
        var Waa = parameters["Waa"]
        var ba = parameters["ba"]

        var a_next = tf.sigmoid(tf.add(tf.add(tf.matMul(xt, Wax), tf.matMul(a_prev , Waa)),ba))

        return a_next
    }
    function RNN_FowardProp(X, a0, parameters){
        var T_x  = X.shape[0]
        var a_next = a0
        var i = 1
        var Wya = parameters["Wya"]
        var by = parameters["by"]
        var l = 1
        
        
        for(; i <= T_x; i++){
            var X_i = X.slice([i-1,0],[1,-1])
            for(; l <= X.shape[1]; l++){
                var xt = X_i.slice([0,l-1],[1,1])
                var a_next = RNN_cell_Foward(xt, a_next, parameters)
            }
            var y_pred = tf.sigmoid((tf.add(tf.matMul(a_next, Wya), by)))
            l = 1
            
            if (i == 1){
                var y_pred1 = y_pred
            } else if (i == 2) {
                var y_pred2 = y_pred
            } else if (i == 3) {
                var y_pred3 = y_pred
            }
            
            
        }
        var y_predx = tf.concat([y_pred1, y_pred2, y_pred3])
        return y_predx
    }
        const learningRate = 0.01;
        var optimizer = tf.train.sgd(learningRate);
        var model = RNN_FowardProp(X, a0, parameters)
        var loss = tf.losses.meanSquaredError(Y, model)
    
    
    
        for (let f = 0; f < 10; f++) {
            optimizer.minimize(loss)
        }

这是一种用于情感分类的神经网络，具有多对一的结构.

This is a neural network for sentiment classification which has a many to one structure.

推荐答案

错误说明了一切:

在variableGrads(f)中传递的f必须是一个函数

The f passed in variableGrads(f) must be a function

optimizer.minimize 需要一个函数作为参数，而不是张量.由于代码正在尝试最小化meanSquaredError，因此 minimize 的 argument 可以是计算预测值和期望值之间的meanSquaredError的函数.

optimizer.minimize is expecting a function as parameter and not a tensor. Since the code is trying to minimize the meanSquaredError, the argument of minimize can be a function that computes the meanSquaredError between the predicted value and the expected one.

const loss = (pred, label) => pred.sub(label).square().mean();

for (let f = 0; f < 10; f++) {
            optimizer.minimize(() => tf.losses.meanSquaredError(Y, model))
}

它可以解决问题吗，还没有完全解决?该错误将因以下原因而改变:

Does it solve the issue, not completely yet ? The error will change for something like:

variableGrads() expects at least one of the input variables to be trainable

是什么意思?使用优化程序时，它期望作为参数传递的函数包含变量，变量的值将更新为 minimize minimize 函数输出.

What does it mean ? When the optimizer is used, it expects the function passed as argument to contains variables whose values will be updated to minimize the function output.

这是要进行的更改:

var Y = tf.tensor([[0,0,0],[0,0,0], [1,1,1]]).variable() // a variable instead

// var loss = tf.losses.meanSquaredError(Y, model)
// computed below in the minimize function

const learningRate = 0.01;
var optimizer = tf.train.sgd(learningRate);
var model = RNN_FowardProp(X, a0, parameters);

const loss = (pred, label) => pred.sub(label).square().mean();
for (let f = 0; f < 10; f++) {
    optimizer.minimize(() => tf.losses.meanSquaredError(Y, model))
}

这篇关于Tensorflow.js神经网络中的反向传播的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！