Example: dental hygienist

Hierarchical Clustering - Princeton University

ElementsofMachineLearningPrincetonUniver sityK-Meansclusteringisagoodgeneral-purp osewaytothinkaboutdiscoveringgroupsindat a, ,itrequirestheusertospecifythenumberofcl ustersinadvance, , ,K-Meansisnondeterministic;thesolutionit findswilldependontheinitializationandeve ngoodinitializationalgorithmssuchasK-Mea ns++ , ,wepartitionorganismsintodifferentspecie s,butsciencehasalsodevelopedarichtaxonom yoflivingthings:kingdom,phylum,class, (usuallybinary) , clustersofclusters (HAC)startsatthebottom,witheverydatumini tsownsingletoncluster, ,thisalgorithmmaintainsan activeset , , shapedpaths,wherethelegsshow1 Algorithm1 HierarchicalAgglomerativeClusteringNote: writtenforclarity, :Input:Datavectors{xn}Nn=1,group-wisedis tanceD (G,G )2:A :forn :A A {{xn}} :endfor6:T A , :while|A|>1do :G 1,G 2 argminG1,G2 A;G1,G2 AD (G1,G2) :A (A\{G 1})\{G 2} :A A {G 1 G 2} :T T {G 1 G 2} :endwhile13: andavaliddendrogram ,thedistancebetweentwomergedgroupsGandG mustalwaysbegreaterthanorequaltothedista ncebetweenanyofthepreviously-mergedsubgr oupsthatformedGandG.

Hierarchical Clustering Ryan P. Adams COS 324 – Elements of Machine Learning Princeton University K-Means clustering is a good general-purpose way to think about discovering groups in data, but there are several aspects of it that are unsatisfying. For one, it …

Fullscreen Download

Tags:

  Learning, Hierarchical, Clustering, Hierarchical clustering

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of Hierarchical Clustering - Princeton University

1 ElementsofMachineLearningPrincetonUniver sityK-Meansclusteringisagoodgeneral-purp osewaytothinkaboutdiscoveringgroupsindat a, ,itrequirestheusertospecifythenumberofcl ustersinadvance, , ,K-Meansisnondeterministic;thesolutionit findswilldependontheinitializationandeve ngoodinitializationalgorithmssuchasK-Mea ns++ , ,wepartitionorganismsintodifferentspecie s,butsciencehasalsodevelopedarichtaxonom yoflivingthings:kingdom,phylum,class, (usuallybinary) , clustersofclusters (HAC)startsatthebottom,witheverydatumini tsownsingletoncluster, ,thisalgorithmmaintainsan activeset , , shapedpaths,wherethelegsshow1 Algorithm1 HierarchicalAgglomerativeClusteringNote: writtenforclarity, :Input:Datavectors{xn}Nn=1,group-wisedis tanceD (G,G )2:A :forn :A A {{xn}} :endfor6:T A , :while|A|>1do :G 1,G 2 argminG1,G2 A;G1,G2 AD (G1,G2) :A (A\{G 1})\{G 2} :A A {G 1 G 2} :T T {G 1 G 2} :endwhile13: andavaliddendrogram ,thedistancebetweentwomergedgroupsGandG mustalwaysbegreaterthanorequaltothedista ncebetweenanyofthepreviously-mergedsubgr oupsthatformedGandG.

2 Figure1bshowsadendrogramforasetofprofess ionalbasketballplayers, ,wemightobservethatthisstructureseemstoc orrespondtoposition;alloftheplayersinthe bottomsubtreebetweenDwightHowardandPaulM illsaparecentersorpowerforwards(exceptfo rPaulPiercewhoisconsideredmoreofasmallfo rward) ( ,StephenCurryandTonyParker)andshootinggu ards( ,DwayneWadeandKobeBryant).AtthetopareKev inDurantandLeBronJames, ; theD (G,G ) ,welookedatdistancesbetweendataitems; , ,weconsiderthedistancesbetweentwogroupsG ={xn}Nn=1andG ={ym}Mm=1,whereNandMare1 Thesearenotnecessarily distances (a)PairwiseDistances(b)Single-LinkageDen drogramFigure1:Thesefiguresdemonstratehi erarchicalagglomerativeclusteringofhigh- scoringprofes-sionalbasketballplayersint heNBA,basedonasetofnormalizedfeaturessuc hasassistsandreboundspergame, (a) :powerforwardsandcentersappearinthebotto mright,pointguardsinthemiddleblock,withs omeunusualplayersinthetopright.(b) linkages . -S L ({xn}Nn=1,{ym}Mm=1)=minn,m||xn ym||,(1)where||x y|| , , ,whenthealgorithmterminates, (subjecttonotaddingaloop), , ,twomergetwoclusterswiththesingle-linkag ecriterion, chaining , , :Ratherthanchoosingtheshortestdistance,i ncomplete-linkageclusteringthedistancebe tweentwogroupsisdeterminedbythelargestdi stanceoverallpossiblepairs, ,D -C L ({xn}Nn=1,{ym}Mm=1)=maxn,m||xn ym||,(2)whereagain||x y|| , :Ratherthantheworstorbestdistances,whenu singtheaverage-linkagecriterionweaverage overallpossiblepairsbetweenthegroups:D -A ({xn}Nn=1,{ym}Mm=1)=1 NMN n=1M m=1||xn ym||.

3 (3) ,3b, :Anotheralternativeapproachtocomputingth edistancebetweenclus-tersistolookatthedi fferencebetweentheircentroids:D -C {xn}Nn=1,{ym}Mm=1)=||#1NN n=1xn$ #1MM m=1ym$||.(4)Notethatthisissomethingthato nlymakessenseifanaverageofdataitemsissen sible; , , , ,orinstance-based, ,itisusefultothinkaboutthespaceofpossibl ethingsthatcanbelearnedfromdata, , ; (a)Single-Linkage(b)Complete-Linkage(c)A verage-Linkage(d)CentroidFigure2:Fourdif ferenttypesoflinkagecriteriaforhierarchi calagglomerativeclustering(HAC).(a)Singl elinkagelooksatminimumdistancebetweenall inter-grouppairs.(b)Completelinkagelooks atthemaximumdistancebetweenallinter-grou ppairs.(c)Averagelinkageusestheaveragedi stancebetweenallinter-grouppairs.(d) (2008). , , ,10,100, , ,wheretherowsareanimalssuchas lion and germanshepherd whilethefeaturesarecolumnssuchas tall and jungle .Figure6bshowsamatrixofHammingdistancesi nthisfeaturespace, ~ckemp/ Time(a)Single-Linkage1234564050607080901 00 DurationWait Time(b)Complete-Linkage12345640506070809 0100 DurationWait Time(c)Average-Linkage123456405060708090 100 DurationWait Time(d)CentroidFigure3:Thesefiguresshowc lusteringsfromthefourdifferentgroupdista ncecriteria, ,HACwasrun,thetreewastruncatedatsixgroup s,andthesegroupsareshownasdifferentcolor s.

4 (a)Single-Linkage(b)Complete-Linkage(c)A verage-Linkage(d)CentroidFigure4:Thesefi guresshowclusteringsfromthefourdifferent groupdistancecriteria,appliedto1500synth etic pinwheel ,HACwasrun,thetreewastruncatedatthreegro ups,andthesegroupsareshownasdifferentcol ors.(a)Thesingle-linkagecriterioncangive stringyclusters,soitcancapturethepinwhee lshapes(b-d)Complete,average, :NationalGovernmentsandDemographicsThese dataarebinarypropertiesof14nations(colle ctedin1965), ,Burma,China,Cuba,Egypt,India,Indonesia, Israel,Jordan,theNetherlands,Poland,theU SSR,theUnitedKingdom, ,socialstructures, , , 104 Interpoint Distances(a) 104 Interpoint Distances(b) 104 Interpoint Distances(c) 104 Interpoint Distances(d)1000 DimensionsFigure5 (zero)andmaximum( D) , , ; , , ,itfirstdividesthedataintoKclustersusing , ,K-MeansorK-Medoids, ,itsuffersfromallofthedifficultiesandnon -determinismofflatclustering, Chapter17ofManningetal.(2008)isfreelyava ilableonlineandisanexcellentresource.

5 Dudaetal.(2001) :writtenforclarity, :Input:Datavectors{xn}Nn=1,Flatclusterin gprocedureF C (G,K)2:3:functionS D (G,K) :{Hk}Kk=1 F C (G,K) :S 6:fork :if|Hk|=1then8:S S {Hk} :else10:S S S D (Hk,K) :endif12:endfor13:Return:S :endfunction15:16:Return:S D ({xn}Nn=1,K) ,PrabhakarRaghavan,andHinrichSch , , , , TODO8(a)AnimalsandFeatures(b)HammingDist ances(c)Average-LinkageDendrogramFigure6 :ThesefiguresshowtheresultofrunningHACon adatasetof50animals,eachwith85binaryfeat ures.(a)Thefeaturematrix,wheretherowsare animalsandthecolumnsarebinaryfeatures.(b ) (c) (a)NationsandFeatures(b)EuclideanDistanc es(c)Complete-LinkageDendrogramFigure7:T hesefiguresshowtheresultofrunningHAConad atasetof14nations,withbinaryfeatures.(a) Thefeaturematrix, (b) (c) (a)SenatorsandVotes(b)EuclideanDistances (c)Average-LinkingDendogramFigure8:These figuresshowtheresultofrunningHAConadatas etof104senatorsinthe113thUScongress,with binaryfeaturescorrespondingtovoteson172b ills.

6 (a)Thefeaturematrix, (b) (c)AdendrogramarisingfromHACwithcomplete linkage.

Show more

Documents from same domain

1 What is Machine Learning?

1 What is Machine Learning?

www.cs.princeton.edu

classification predicted rule prediction algorithm machine learning example new examples training labeled Figure 1: Diagram of a typical learning problem.

  Machine, Learning, Machine learning

An Introduction to MCMC for Machine Learning

An Introduction to MCMC for Machine Learning

www.cs.princeton.edu

emphasis on probabilistic machine learning. Second, it reviews the main building blocks of modern Markov chain Monte Carlo simulation, thereby providing and introduction to …

  Introduction, Machine, Learning, Machine learning, Introduction to mcmc for machine learning, Mcmc

Chapter 9 Basic Signal Processing - cs.princeton.edu

Chapter 9 Basic Signal Processing - cs.princeton.edu

www.cs.princeton.edu

Chapter 9 Basic Signal Processing ... Digital Signal Processing ... The key to understanding signal processing is to learn to think in the frequency do-

  Basics, Chapter, Understanding, Processing, Signal, Digital, Digital signal processing, Chapter 9 basic signal processing, Understanding signal processing

C Examples - Princeton University Computer Science

C Examples - Princeton University Computer Science

www.cs.princeton.edu

C Examples! Jennifer Rexford! 2 Goals of this Lecture ! • Help you learn about:! • The fundamentals of C! • Deterministic finite state automata (DFA)!

  Example, C examples

Modules - Princeton University Computer Science

Modules - Princeton University Computer Science

www.cs.princeton.edu

1 Modules CS 217 The C Programming Language • Systems programming language originally used to write Unix and Unix tools data types and control structures close to most machines

  Programming, Language, Module, The c programming language, Programming language

C Examples - cs.princeton.edu

C Examples - cs.princeton.edu

www.cs.princeton.edu

3 Overview of this Lecture! • C programming examples! • Echo input to output! • Convert all lowercase letters to uppercase! • Convert first letter of each word to uppercase!

  Programming, C programming

Introduction to Stochastic Simulation with the Gillespie ...

Introduction to Stochastic Simulation with the Gillespie ...

www.cs.princeton.edu

Introduction to Stochastic Simulation with the Gillespie Method David Karig April 18, 2005. Stochastic Systems • Many systems driven by random, discrete interactions • Traditional deterministic models may not accurately describe such systems $ Example: The Lambda Switch

  Introduction, With, Simulation, Stochastic, Introduction to stochastic simulation with the gillespie, Gillespie

Go programming language - Princeton University

Go programming language - Princeton University

www.cs.princeton.edu

Go programming language • history • basic constructs • simple programs • arrays & slices • maps • methods, interfaces • concurrency, goroutines

  Programming, Language, Go programming language

A Beginner’s Guide to LATEX September 12, 2005

A Beginner’s Guide to LATEX September 12, 2005

www.cs.princeton.edu

A Beginner’s Guide to LATEX David Xiao dxiao@cs.princeton.edu September 12, 2005 1 Introduction LATEX is the standard mathematical typesetting program.This document is for people who have never used LATEX before and just want a quick crash course to get started.I encourage all students in mathematics and

  Guide, Beginner, Latex, A beginner s guide to latex

Insertion Sort - Princeton University Computer Science

Insertion Sort - Princeton University Computer Science

www.cs.princeton.edu

13 Data analysis. Plot time vs. input size on log-log scale. Regression. Fit line through data points ! a Nb. Hypothesis. Running time grows quadratically with input size.

  Analysis, Princeton, Insertion, Sort, Insertion sort

Related documents

PointNet++: Deep Hierarchical Feature Learning on Point ...

PointNet++: Deep Hierarchical Feature Learning on Point ...

proceedings.neurips.cc

We will introduce a hierarchical feature learning framework in the next section to resolve the limitation. 3.2 Hierarchical Point Set Feature Learning While PointNet uses a single max pooling operation to aggregate the whole point set, our new architecture builds a hierarchical grouping of points and progressively abstract larger and larger local

  Feature, Learning, Deep, Hierarchical, Pointnet, Deep hierarchical feature learning, Hierarchical feature learning, Feature learning

PointNet++: Deep Hierarchical Feature Learning on Point ...

PointNet++: Deep Hierarchical Feature Learning on Point ...

arxiv.org

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space Charles R. Qi Li Yi Hao Su Leonidas J. Guibas Stanford University Abstract Few prior works study deep learning on point sets. PointNet [20] is a pioneer in this direction. However, by design PointNet does not capture local structures induced by

  Feature, Learning, Hierarchical, Pointnet, Hierarchical feature learning

Deep Learning: Methods and Applications

Deep Learning: Methods and Applications

www.microsoft.com

learning or hierarchical learning, has emerged as a new area of machine learning research [20, 163]. During the past several years, the techniques ... learn distributed and hierarchical feature representations, and to make effective use of both labeled and unlabeled data. Active researchers in this area include those at University of

  Feature, Learning, Hierarchical, Hierarchical features, Hierarchical learning

Stacked Convolutional Auto-Encoders for Hierarchical ...

Stacked Convolutional Auto-Encoders for Hierarchical ...

people.idsia.ch

Hierarchical Feature Extraction Jonathan Masci, Ueli Meier, Dan Cire¸san, and J¨urgen Schmidhuber Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA) Lugano, Switzerland {jonathan,ueli,dan,juergen}@idsia.ch Abstract. We present a novel convolutional auto-encoder (CAE) for unsupervised feature learning.

  Feature, Learning, Hierarchical, Convolutional, Hierarchical features, Feature learning

Introduction to Machine Learning Final Exam

Introduction to Machine Learning Final Exam

people.eecs.berkeley.edu

C: The kernel trick, when it is applicable, speeds up a learning algorithm if the number of sample points is substantially less than the dimension of the (lifted) feature space. D: If the raw feature vectors x;y are of dimension 2, then k(x;y) = x2 1 y 2 1 + x 2 2 y 2 2 is a valid kernel. A is correct; consider the Gaussian kernel from lecture.

  Feature, Machine, Learning, Machine learning

Machine Learning with Python - Tutorialspoint

Machine Learning with Python - Tutorialspoint

www.tutorialspoint.com

Machine Learning with Python ii About the Tutorial Machine Learning (ML) is basically that field of computer science with the help of which computer systems can provide sense to data in much the same way as human beings do.

  Python, With, Machine, Learning, Tutorialspoint, Machine learning with python

Knowledge-Enhanced Hierarchical Graph Transformer …

Knowledge-Enhanced Hierarchical Graph Transformer …

www.aaai.org

Knowledge-Enhanced Hierarchical Graph Transformer Network for Multi-Behavior Recommendation Lianghao Xia 1, Chao Huang 2, Yong Xu;3 4, Peng Dai , Xiyue Zhang1 Hongsheng Yang 2, Jian Pei5, Liefeng Bo South China University of Technology1, China, JD Finance America Corporation2, USA Communication and Computer Network Laboratory of …

  Hierarchical

Schema Theory and College English Reading Teaching

Schema Theory and College English Reading Teaching

files.eric.ed.gov

Reading is one of the important skills in English learning. It is acknowledged that while in communication between input and output, language comprehension is the very important key link that we can’t feel directly but it does exist (An, 2011). However, the current situation of college English reading teaching is not promising.

  Learning, Schema

Hierarchical Bayesian Modeling

Hierarchical Bayesian Modeling

astrostatistics.psu.edu

Hierarchical Modeling is a statistically rigorous way to make scientific inferences about a population (or specific object) based on many individuals (or observations). Frequentist multi-level modeling techniques exist, but we will discuss the Bayesian approach today. Frequentist: variability of sample

  Hierarchical

Related search queries

PointNet++: Deep Hierarchical Feature Learning, Hierarchical Feature Learning, Hierarchical, Feature learning, PointNet++, Learning, Hierarchical learning, Hierarchical Feature, Convolutional, Machine Learning, Feature, Machine Learning with Python, Tutorialspoint, Schema