Data Mining and Materials Informatics: a primer

1 Krishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 Data Mining and Materials Informatics: a primerKrishna RajanDepartment of Materials Science and EngineeringNSF Intl. Materials Institute Combinatorial Sciences & Materials Informatics CollaboratoryIowa State UniversityKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 What is data? primary, secondary , derivative sources of dataWhat do we mean by Mining data? data correlations dimensional analysis approachdata correlations data miningWhat can we learn from data Mining ?Classification data Mining databaseshierarchy of datatrends in dataPredictionpredicting structure-property relationshipscomputational informatics vs. computational Materials sciencedata Mining for building databasesOUTLINEK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 INFORMATICS Establish new correlations Identify outliers Enlarge database / virtual libraries Evaluate databases Establish predictionsWhat is it ?

2 Why ? Searching for patterns of behavior among multivariate data sets Can pattern recognition lead to predictions?Krishna RajanKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 COMPLEXITY OF BIOSYSTEMSK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 Data + Correlations + Theory = Knowledge-base SimulationsSimulations Combinatorially derived datasetsCombinatorially derived datasets Digital librariesDigital libraries Spectral and imaging librariesSpectral and imaging libraries Machine learning Machine learning Data compressionData compression Pattern recognitionPattern recognition Atomistic based calculationsAtomistic based calculations Continuum based theoriesContinuum based theoriesULTRA LARGE SCALE INFORMATION SPACEI nformatics-BasedDesignINFORMATICS STRATEGYM achine learning toolkit: Classification Prediction Hybrid toolsKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 Ideker and Lauffenburger.

3 (2003)INFORMATICS-BASED DESIGN STRATEGIESBIOLOGICAL ANALOGUECRYSTAL STRUCTURE ANALOGUENye (1950)Krishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 Statistical ToolsCombinatorial & Spectral LibrariesLatent Variables /Partial Least Squares(PLS)Principal Component Analysis(PCA)Support Vector Machine(SVM) Materials Databases Experimental data Computational deriveddata sets SimulationsAssociation Mining (AM)ClassificationPredictionInput & Output Krishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 Requirements Global minima Categorical data Missing / variable data Skewed distributions Large data sets ScalablePitfalls Local minima Categorical data difficult Convergence difficult for large data sets Few outliers can lead to poor performanceMACHINE LEARNING AND Materials DISCOVERYK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 DATA CURATIONKNOWLEDGE DISCOVERYD atabase administration & management thermodynamic, crystallographic & propertydata bases combinatorial experimental dataOracle, Unix, Supercomputing, SQL taxonomy and ontology of materialsscience data data sharing , networking / cyber infrastructureJAVA, HTML.

4 Python object oriented programming language visualization of high dimensional dataData Mining algorithms Clustering analysis Quantitative Structure-Activity Relationships (QSAR) for Materials designDATA REPRESENTATIONDATA STORAGEK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 Accelerated insertion of Materials into engineering systems Rapid multiscale design and optimization of Materials properties Establishment of new structure property correlations among large, heterogeneous and distributed data sets Discovery of new chemistries and compounds Formulation and / or refinement of new theories for Materials behavior Rapid identification of critical data and theoretical needs for future problems WHY COUPLE COMPUTATIONAL Materials SCIENCE AND INFORMATICS ?Krishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 OriginalDataTransformedDataPatternsKnowl edgeData WarehousingFeature ExtractionData Mining & VisualizationInterpretationSOFT MODELING vs.

5 HARD MODELINGData basesDimensionalAnalysis and/or TheoryConstitutive EquationsHard modelingHard modelingSoft modelingSoft modelingKrishna RajanKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th200622x22 (484 cells 2500 data points):Silicon nitride descriptorsDATA MAPK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 DIMENSIONALITY REDUCTIONK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 EIGENVALUE DECOMPOSITION eigenvalueeigenvectorIf V is nonsingular, this become the eigenvalue decompositionEigenvalues on the diagonal of this diagonal matrixEigenvectors forming the columns of this matrixSpp =SPP= 1 SPP = Loading Matrix=Eigenvector MatrixEigenvalue MatrixKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 PREDICTIONSPLS Upgradeable linear predictive tool Quadratic and interaction terms :flexibilityCalculate latent factors from original predictor variablesT=XW where T- latent factors, X predictors, W- weightsUse latent factors to predict responseY=TQ+E where Y - responses, Q loadings, E- noise Wmaximizes covariance between Y and TFactorsUTResponsesKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 INFORMATICS AND THE PERIODIC TABLEK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 DATA WAREHOUSINGK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th200602040050001000002000400012305100 2401020246020400500010000020004000123051 0024010202461 2 3 4 5 6 7 88 7 6 5 4 3 2 11.

6 # of atoms/unit cell 2. Valence electron # 3. Electronegativity 4. 1st Ionization potential 5. Atomic radius 6. Melting point 7. Boiling point 8. DensityBIVARIATE MAPSK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006-2-1012-6-4-20246-6-4-20246-2-10 12-6-4-20246-6-4-20246PC1 PC2 PC3PC3 PC2 PC1 SEEKING PATTERNSK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006-4-3-2-1012-3-2-1012345 Scores on PC 2 ( )Scores on PC 3 ( ) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58Na, Mg, AlK, Ca, Sc, Ti, V, Cr, Mn, Fe, Co, Ni, Cu, Zn, and GaSr, Y, Zr, Nb, Mo, Tc, Ru, Rh, Pd, Ag, Cd, In, and SnCs, Ba, La, Ce, Pr, Nd, Sm, Eu, Gd, Tb, Dy, Ho, Er, Tm, Yb, Lu, Hf, Ta, W, Re, Os, Ir, Pt, Au, Hg, Tl, Pb, and BiEach cluster represent each row in the periodic tableMENDELEEV SEQUENCINGK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 Location of property in loading plot indicates influence of property on PC Atomic weight has no influence on PC1, high influence on PC2 and 3 Clustered properties indicate relationships Electrical and thermal conductivity Melting point and density Molar volume and atomic radius Melting and boiling points, heats of fusion and vaporization.

7 And valence number Pauling electronegativity and first ionization potentialAtomicWeightPseudopotentialRadi usDensityPauling electronegativityFirstIonizationPotentia lThermalConductivityElectricalConductivi tySpecific HeatMartynov-BatsanovelectronegativityMo larVolumeCovalentradiusMetallicradiusHea t ofFusionHeat ofVaporizationValenceElectronnumberMelti ngPointBoilingPointDEVELOPING PHYSICAL LAWSK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 RPI Electrical Conductivity ( *ohm*m) Conductivity (W/(m*K))11010032. Germanium (Ge)6. Carbon, diamond (C)52. Tellurium (Te)83. Bismuth (Bi)29. Copper (Cu)34. Selenium (Se)DEVELOPING PHYSICAL LAWS2030405060705001000150020002500 Activation Energy, Q [kcal/mol]Tm [K]PbAgAuCuNiCoFePt18 RgLinear regressionBokshtein & ZhukhovitskiiAlActivation energy for self-diffusion versus melting pointFrom GlicksmanKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th20069 descriptors from NIST data- Lattice parameters (a, b, c, , , )- c/a, b/c- V/abcHexagonalMonoclinicTetragonal(P42 /mmc)-100-50050-50050-40-30-20-10010 6 1 2 9 22 8 10 23 3 15 21 16 4 25 26 31 27 34 32 28 14 12 13PC 1 ( ) 33 30 5 7 24 20 11 Scores Plot 18 29 19 17PC 2 ( )PC 3 ( ) 1 2 9 22 8 10 23 3 15 21 16 4 25 26 31 34 27 32 28 14 12 13 Tetragonal(I-42d)OrthorhombicCubicPCA Score plot (34 binary, ternary, quaternary compounds)PCA of 34 binary, ternary, quaternary compounds: Score plotPCA of 34 binary, ternary, quaternary compounds.

8 Score plotKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th200612345678910111213141516171819202 122232425262728293031323334 Cluster analysis of 34 binary, ternary, quaternary compoundsCluster analysis of 34 binary, ternary, quaternary compoundsCubic(Fm-3m)Tetragonal(I-42d)Cu bic(P41 32,P-43m,Pm-3,Pm-3,Fd-3ms)OrthorhombicCu bic(Im-3,Pm-3)Tetragonal(I4/mcm)Hexagona l (P63 /mmc,P6/mmm, P-6m2)/ Trigonal (P-3m1)Tetragonal(P42/mmc)Hexagonal (P-62m,P-6)MonoclinicHexagonal(P63 /mmc) , , = 90 , , 90 Krishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 Atomic packing of single elementLiAlCuScMgBCCFCCHCPFCCHCPsp electrons/atom 132112Ex.) Engel s model using # of sp electrons/atom BCC < FCC EXTRACTIONK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 PATTERN DETECTIONK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006-6-4-20246-6-4-20246-6-4-20246PC 2( )PC3( )PC1( )classificationGraduciz, Sad, SerbiaCNRS, MarseillescovalentionicpredictionDATA Mining DATABASESK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 COMPUTATIONAL ISSUESData can come across length and time scalesFocus on properties of signal / macroscopic behavior rather than noise/ error.

9 Assume complexity !!!Utilize data dimensionality reduction techniquesAnalyze variation and correlation in dataCovarianceEstablish correlations acrossdiverse data sets ( ie. length & time scales)Identify outliers: explore causeDevelop predictive models- Target requirements of missing data- Quantitatively assess data diversityModel relationships in data to seek heuristic relationships:Advanced statistical learning tools can deal with:- skewed data- missing data- differentiate between local and global minima- ultra large scale datasets- variable uncertainty Singular value decomposition Support vector machines Association Mining Fuzzy clusteringEstablish multivariate database:Seek DIVERSITY in datasetsKrishna RajanKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 CRYSTAL CHEMISTRY DESIGN1. Assess influence of latentvariables ( electronic structure parameters) on properties of known data2. Establish heuristic relationships on database of all input variables instead of phenomenological relationships in bivariate manner3.

10 Use statistical learning to predict new Materials behavioron new multivariate input data4. Inverse problem approach to formulate quantitative structure- property relationshipsChing J. Amer. 85 75-80 (2002)Krishna RajanKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 Virtual library:via informaticsRefractory metalsSuh and Rajan (2005, 2006) Real library:via first principles calculationsKrishna RajanKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 LINKING DISPARATE LENGTH SCALESV isualizing AssociationsKrishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006hcpfccSYNTHESIZING REMOTE DATA SETSK rishna RajanTMS / ASM Materials Informatics WorkshopCincinatti, OH October 15th2006 Search and retrieve Refining descriptors Developing predictions Filling in missing data Structure databases Diffraction spectra databases Hyper spectral imaging databasesMapping / Visualization of databases.