反向传播算法-误差导数计算

Question

在计算误差导数时，我正在使用以下方法，但不确定确切原因。

double errorDerivative = (-output * (1-output) *(desiredOutput - output));

当我从第一个输出中删除减号时，它就会失败，并达到最大历元限制。我假设这就是这个例子的样子http://homepages.gold.ac.uk/nikolaev/311imlti.htm它不使用减号运算符。

double errorDerivative2 = (output * (1-output) *(desiredOutput - output));

我目前正在研究一个修改现有的使用随机梯度下降的反向传播实现，并希望它只使用标准的反向传播算法。目前，它看起来是这样的。

public void applyBackpropagation(double expectedOutput[]) {
        // error check, normalize value ]0;1[
        /*for (int i = 0; i < expectedOutput.length; i++) {
            double d = expectedOutput[i];
            if (d < 0 || d > 1) {
                if (d < 0)
                    expectedOutput[i] = 0 + epsilon;
                else
                    expectedOutput[i] = 1 - epsilon;
            }
        }*/
        int i = 0;
        for (Neuron n : outputLayer) {
            System.out.println("neuron");
            ArrayList<Connection> connections = n.getAllInConnections();
            for (Connection con : connections) {
                double output = n.getOutput();
                System.out.println("final output is "+output);
                double ai = con.leftNeuron.getOutput();
                System.out.println("ai output is "+ai);
                double desiredOutput = expectedOutput[i];
                double errorDerivative = (-output * (1-output) *(desiredOutput - output));
                double errorDerivative2 = (output * (1-output) *(desiredOutput - output));
                System.out.println("errorDerivative is "+errorDerivative);
                System.out.println("errorDerivative my one is "+(output * (1-output) *(desiredOutput - output)));
                double deltaWeight = -learningRate * errorDerivative2;
                double newWeight = con.getWeight() + deltaWeight;
                con.setDeltaWeight(deltaWeight);
                con.setWeight(newWeight + momentum * con.getPrevDeltaWeight());
            }
            i++;
        }
        // update weights for the hidden layer
        for (Neuron n : hiddenLayer) {
            ArrayList<Connection> connections = n.getAllInConnections();
            for (Connection con : connections) {
                double output = n.getOutput();
                double ai = con.leftNeuron.getOutput();
                double sumKoutputs = 0;
                int j = 0;
                for (Neuron out_neu : outputLayer) {
                    double wjk = out_neu.getConnection(n.id).getWeight();
                    double desiredOutput = (double) expectedOutput[j];
                    double ak = out_neu.getOutput();
                    j++;
                    sumKoutputs = sumKoutputs
                            + (-(desiredOutput - ak) * ak * (1 - ak) * wjk);
                }
                double partialDerivative = output * (1 - output) * ai * sumKoutputs;
                double deltaWeight = -learningRate * partialDerivative;
                double newWeight = con.getWeight() + deltaWeight;
                con.setDeltaWeight(deltaWeight);
                con.setWeight(newWeight + momentum * con.getPrevDeltaWeight());
            }
        }
    }

Answer 1

对不起，我不会审查你的代码-没有时间了，你必须带着更具体的问题回来，然后我可以帮助你。

errorDerivave2工作的原因可能是您正在使用权重更新规则，如
deltaW = learningRate*errorDerivative2*input

通常，您所称的"errorDerivave2"被称为delta，并被定义为
-output * (1-output) *(desiredOutput - output)
对于具有S形传递函数的神经元

使用权重更新规则
deltaW = -learningRate*delta*input

所以基本上，它在errorDerivative2上没有减号，因为你在另一个地方也省略了减号。。