Transcription of Notes on Backpropagation
{{id}} {{{paragraph}}}
Example: confidence
Notes on Backpropagation
Notes on BackpropagationPeter SadowskiDepartment of Computer ScienceUniversity of California IrvineIrvine, CA document derives Backpropagation for some common neural Cross Entropy Error with Logistic ActivationIn a classification task with two classes, it is standard to use a neural network architecture witha single logistic output unit and the cross-entropy loss function (as opposed to, for example, thesum-of-squared loss function). With this combination, the output prediction is always between zeroand one, and is interpreted as a probability. Training corresponds to maximizing the conditionallog-likelihood of the data, and as we will see, the gradient calculation simplifies nicely with can generalize this slightly to the case where we have multiple, independent, two-class classi-fication tasks. In order to demonstrate the calculations involved in Backpropagation , we considera network with a single hidden layer of logistic units, multiple logistic output units, and where thetotal error for a given example is simply the cross-entropy error summed over the output cross entropy error for a single example withnoutindependent targets is given by the sumE= nout i=1(tilog(yi) + (1 ti) log(1 yi))(1)wheretis the target vector,yis the output vector.
Notes on Backpropagation Peter Sadowski Department of Computer Science University of California Irvine Irvine, CA 92697 peter.j.sadowski@uci.edu Abstract
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: