Package ‘kernlab’ - The Comprehensive R Archive Network

1 Package kernlab November 12, Machine Learning LabDescriptionKernel-based machine learning methods for classification,regression, clustering, novelty detection, quantile regressionand dimensionality reduction. Among other methods ' kernlab 'includes Support Vector Machines, Spectral Clustering, KernelPCA, Gaussian Processes and a QP (>= )Importsmethods, stats, grDevices, graphicsLazyLoadYesLicenseGPL-2 SystemRequirementsC++11 NeedsCompilationyesAuthorAlexandros Karatzoglou [aut, cre],Alex Smola [aut],Kurt Hornik [aut],National ICT Australia (NICTA) [cph],Michael A. Maniscalco [ctb, cph],Choon Hui Teo [ctb]MaintainerAlexandros 07:05:02 UTCR topics ..3couple ..4csi ..5csi-class ..7dots.

2 9gausspr .. 11gausspr-class .. 1412 Rtopics documented:inchol .. 16inchol-class .. 18income .. 19inlearn .. 20ipop .. 22ipop-class .. 24kcca .. 25kcca-class .. 27kernel-class .. 28kernelMatrix .. 29kfa .. 31kfa-class .. 34kha .. 35kha-class .. 37kkmeans .. 38kmmd .. 41kmmd-class .. 44kpca .. 45kpca-class .. 47kqr .. 49kqr-class .. 52ksvm .. 54ksvm-class .. 61lssvm .. 63lssvm-class .. 67musk .. 69onlearn .. 70onlearn-class .. 71plot .. 73prc-class .. 77promotergene .. 78ranking .. 79ranking-class .. 82reuters .. 83rvm .. 84rvm-class .. 87sigest .. 89spam.

3 90specc .. 91specc-class .. 94spirals .. 95stringdot .. 96ticdata .. 97vm-class .. kernelMatrix class to matrix packagekernlabcan be used to coerce the kernelMatrix class to matrix ob-jects representing a kernel matrix. These matrices can then be used with the kernelMatrix interfaceswhich most of the functions ## S4 method for signature'matrix' (x, center = FALSE)Argumentsxmatrix to be assigned thekernelMatrixclasscentercenter the kernel matrix in feature space (default: FALSE)Author(s)Alexandros AlsokernelMatrix,dotsExamples## Create toy datax <- rbind(matrix(rnorm(10),,2),matrix(rnorm( 10,mean=3),,2))y <- matrix(c(rep(1,5),rep(-1,5)))### Use to label the cov.

4 Matrix as a kernel matrix### which is eq. to using a linear kernelK <- (crossprod(t(x)))Ksvp2 <- ksvm(K, y, type="C-svc")svp24couplecoupleProbabilit ies Coupling functionDescriptioncoupleis used to link class-probability estimates produced by pairwise coupling in multi-classclassification (probin, coupler = "minpair")ArgumentsprobinThe pairwise coupled class-probability estimatescouplerThe type of coupler to use. Currentlyminparandpkpdandvoteare supported(see reference for more details). Ifvoteis selected the returned value is a prim-itive estimate passed on given binary classification problems are much easier to solve many techniques exist to decomposemulti-class classification problems into many binary classification problems (voting, error codes,etc.)

5 Pairwise coupling (one against one) constructs a rule for discriminating between every pairof classes and then selecting the class with the most winning two-class decisions. By using Platt sprobabilities output for SVM one can get a class probability for each of thek(k 1)/2modelscreated in the pairwise classification. The couple method implements various techniques to combinethese matrix with the resulting probability (s)Alexandros Wu, Chih-Jen Lin, ruby C. WengProbability Estimates for Multi-class Classification by Pairwise CouplingNeural Information Processing Symposium 2003 ,ksvmcsi5 Examples## create artificial pairwise probabilitiespairs <- matrix(c( , , , , , ),2)couple(pairs)couple(pairs, coupler="pkpd")couple(pairs, coupler ="vote")csiCholesky decomposition with Side InformationDescriptionThecsifunction inkernlabis an implementation of an incomplete Cholesky decomposition algo-rithm which exploits side information ( , classification labels, regression responses) to computea low rank decomposition of a kernel matrix from the ## S4 method for signature'matrix'csi(x, y, kernel="rbfdot", kpar=list(sigma= ))

6 , rank,centering = TRUE, kappa = ,delta = 40 ,tol = 1e-5)ArgumentsxThe data matrix indexed by rowythe classification labels or regression responses. In classification y is am nmatrix wheremthe number of data andnthe number of classesyandyiis 1 ifthe corresponding x belongs to class kernel function used in training and predicting. This parameter can be setto any function, of classkernel, which computes the inner product in featurespace between two vector arguments. kernlab provides the most popular kernelfunctions which can be used by setting the kernel parameter to the followingstrings: rbfdotRadial Basis kernel function "Gaussian" polydotPolynomial kernel function vanilladotLinear kernel function tanhdotHyperbolic tangent kernel function laplacedotLaplacian kernel function besseldotBessel kernel function anovadotANOVA RBF kernel function splinedotSpline kernel stringdotString kernel6csiThe kernel parameter can also be set to a user defined function of class kernelby passing the function name as an list of hyper-parameters (kernel parameters).

7 This is a list which containsthe parameters to be used with the kernel function. Valid parameters for existingkernels are : sigmainverse kernel width for the Radial Basis kernel function "rbfdot"and the Laplacian kernel "laplacedot". degree,scale,offsetfor the Polynomial kernel "polydot" scale,offsetfor the Hyperbolic tangent kernel function "tanhdot" sigma,order,degreefor the Bessel kernel "besseldot". sigma,degreefor the ANOVA kernel "anovadot".Hyper-parameters for user defined kernels can be passed through the kpar pa-rameter as rank of the computed kernel matrixcenteringifTRUE centering is performed (default: TRUE)kappatrade-off between approximation of K and prediction of Y (default: )deltanumber of columns of cholesky performed in advance (default: 40)tolminimum gain at each iteration (default: 1e-4)DetailsAn incomplete cholesky decomposition calculatesZwhereK=ZZ Kbeing the kernel the rank of a kernel matrix is usually low,Ztends to be smaller then the complete kernelmatrix.

8 The decomposed matrix can be used to create memory efficient kernel-based algorithmswithout the need to compute and store a complete kernel matrix in the class labels, or regression responses to compute a more appropriate approximation forthe problem at hand considering the additional information from the response S4 object of class "csi" which is an extension of the class "matrix". The object is the decomposedkernel matrix along with the slots :pivotsIndices on which pivots where donediagresiduesResiduals left on the diagonalmaxresidualsResiduals picked for pivotingpredgainpredicted gain before adding each columntruegainactual gain after adding each columnQQR decomposition of the kernel matrixRQR decomposition of the kernel matrixslots can be accessed either byobject@slotor by accessor functions with the same name ( ,pivots(object))csi-class7 Author(s)Alexandros Karatzoglou (based on Matlab code by Francis R.)

9 Bach, Michael I. JordanPredictive low-rank decomposition for kernel of the Twenty-second International Conference on Machine Learning (ICML) 2005 ~ Alsoinchol,chol,csi-classExamplesdata(ir is)## create multidimensional y matrixyind <- t(matrix(1:3,3,150))ymat <- matrix(0, 150, 3)ymat[yind== (iris[,5])] <- 1datamatrix <- (iris[,-5])# initialize kernel functionrbf <- rbfdot(sigma= )rbfZ <- csi(datamatrix,ymat, kernel=rbf, rank = 30)dim(Z)pivots(Z)# calculate kernel matrixK <- crossprod(t(Z))# difference between approximated and real kernel matrix(K - kernelMatrix(kernel=rbf, datamatrix))[6,]csi-classClass "csi"DescriptionThe reduced Cholesky decomposition objectObjects from the ClassObjects can be created by calls of the formnew("csi".)

10 Or by calling :Object of class"matrix"contains the decomposed matrixpivots:Object of class"vector"contains the pivots performeddiagresidues:Object of class"vector"contains the diagonial residuesmaxresiduals:Object of class"vector"contains the maximum residuespredgainObject of class"vector"contains the predicted gain before adding each columntruegainObject of class"vector"contains the actual gain after adding each columnQObject of class"matrix"contains Q from the QR decomposition of the kernel matrixRObject of class"matrix"contains R from the QR decomposition of the kernel matrixExtendsClass"matrix", (object = "csi"): returns the diagonial residuesmaxresidualssignature(object = "csi"): returns the maximum residuespivotssignature(object = "csi"): returns the pivots performedpredgainsignature(object = "csi"): returns the predicted gain before adding each columntruegainsignature(object = "csi"): returns the actual gain after adding each columnQsignature(object = "csi"): returns Q from the QR decomposition of the kernel matrixRsignature(object = "csi"): returns R from the QR decomposition of the kernel matrixAuthor(s)Alexandros Alsocsi,inchol-classExamplesdata(iris)## create multidimensional y matrixyind <- t(matrix(1.))