Transcription of An Introduction to Machine Learning - Applied mathematics
1 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographyAn Introduction to Machine LearningResearch Group Meeting 2017 Melissa R. McGuirlFebruary 22, 2017An Introduction to Machine LearningFebruary 22, 20171 / 45 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographyTable of Contents1 Introduction2 Supervised LearningGeneralized Linear ModelsSupport Vector MachinesDecision Trees3 Unsupervised LearningManifold learningClusteringNeural Networks4 Model Selection and Evaluation5 Applications6 Pitfalls to avoid7 Machine Learning that
2 Matters8 BibliographyAn Introduction to Machine LearningFebruary 22, 20172 / 45 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographyIntroductionGener al Set-upSet-up and GoalSuppose we haveX1,X2,..,Xndata samples. Can wepredict properites about any givenXn+1,Xn+2,..,XN? Machine Learning systems attempt to predict properties ofunknown data based on the attributes or features of Introduction to Machine LearningFebruary 22.
3 20173 / 45 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographyIntroductionSuper vised learning1 Data comes with attributes that we want our algorithmto predict2 Machine Learning algorithm is given data attributes anddesired outputs and the goal is to learn a way to mapinputs to outputs in a general way32 main types of Learning :1 Classification: Data belongs to differentclasses/groups and we want to be able to predict whichclass/group unlabeled data belongs to2 Regression.
4 Data labeled with one or more continuousvariables (parameters) and the task is to predict thevalue of these variables for unknown dataAn Introduction to Machine LearningFebruary 22, 20174 / 45 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographyIntroductionImage source: Introduction to Machine LearningFebruary 22, 20175 / 45 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographyIntroductionUnsup ervised Learning1No labels are given to the algorithm2 Training data consists of a set of input vectors{X1,X2.}
5 ,Xn}with no target outputs33 Types of goals in this setting:1 Clustering: discover groups with similar features withinthe data2 Density Estimation:determine distribution of datawithin the input space3 Dimensionality Reduction:project data into a lowerdimensional space than input spaceAn Introduction to Machine LearningFebruary 22, 20176 / 45 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographySupervised LearningGeneralized Linear ModelsGeneralized Linear ModelsSet-up: Data in the form ofXi= (xi1,xi2.)
6 ,xip)Goal: RegressionFind y y(w,Xi) =w0+w1xi1+w2xi2+..wpxip,wherew0is called theinterceptandw1,..wpare Introduction to Machine LearningFebruary 22, 20177 / 45 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographySupervised LearningGeneralized Linear ModelsGLM: Ordinary Least SquaresFits linear model y(w,Xi) =w0+w1xi1+w2xi2+..wpxipwherew= (w1.
7 ,wp)coefficients obtained by solvingminw||Xw y||22 Model relies on independence of model termsIf terms are correlated then the least-square estimate ishighly sensitive to random errors in the observedresponse (large variance)Complexity:O(np2)An Introduction to Machine LearningFebruary 22, 20178 / 45 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographySupervised LearningGeneralized Linear ModelsGLM: Ridge RegressionFits linear model y(w,Xi) =w0+w1xi1+w2xi2+.
8 Wpxipwherew= (w1,..,wp)coefficients obtained by solvingminw||Xw y||22+ ||w||22 >0 is a complexity parameter that controls howrobust the coefficients are to linearityPenalty on the size of coefficients addresses issueswith ordinary least squares methodComplexity:O(np2)An Introduction to Machine LearningFebruary 22, 20179 / 45 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographySupervised LearningGeneralized Linear ModelsGLM: LassoFits linear model y(w,Xi) =w0+w1xi1+w2xi2+.
9 Wpxipwherew= (w1,..,wp)coefficients obtained by solvingminw12n||Xw y||22+ ||w||11l1norm means we will this linear model estimatessparse coefficientsThis method reduces number of variables upon whichthe solution is dependentUseful in compressed sensing and can be used forfeature selectionAn Introduction to Machine LearningFebruary 22, 201710 / 45 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographySupervised LearningSupport Vector MachinesSVM.
10 Mathematical SetupA SVM constructs a hyperplane (or set of hyperplanes) in ahigh or infinite dimensional space, which can be used forclassification, regression or other Introduction to Machine LearningFebruary 22, 201711 / 45 AnIntroduction toMachineLearningIntroductionSupervisedL earningGeneralized LinearModelsSupport VectorMachinesDecision TreesUnsupervisedLearningManifold learningClusteringNeural NetworksModelSelection andEvaluationApplicationsPitfalls toavoidMachineLearning thatMattersBibliographySupervised LearningSupport Vector MachinesSVM: Linear, separable caseWe want to find the hyperplane that maximizes the marginas follows:Mathematical formulationGivenX1,X2.
