Maximum Likelihood, Logistic Regression, and Stochastic ...

Maximum Likelihood, Logistic Regression, and Stochastic Gradient TrainingCharles 10, 20141 Principle of Maximum likelihoodConsider a family of probability distributions defined by a set of parameters .The distributions may be either probability mass functions (pmfs) or probabilitydensity functions (pdfs). Suppose that we have a random sample drawn froma fixed but unknown member of this family. The random sample is a trainingset ofnexamplesx1toxn. An example may also be called an observation, anoutcome, an instance, or a data point. In general eachxjis a vector of values, and is a vector of real-valued parameters. For example, for a Gaussian distribution = , 2 .We assume that the examples are independent, so the probability of the set isthe product of the probabilities of the individual examples:f(x1,..,xn; ) = jf (xj; ).The notation above makes us think of the distribution as fixed and the examplesxjas unknown, or varying.

The distributions may be either probability mass functions (pmfs) or probability density functions (pdfs). Suppose that we have a random sample drawn from a fixed but unknown member of this family. The random sample is a training set of nexamples x 1 to x n. An example may also be called an observation, an outcome, an instance, or a data point.

Loading..

Tags:

  Functions, Mass, Mass functions

Information

Related search queries