Example: barber

Lecture 2: Descriptive Statistics and Exploratory Data ...

Lecture 2: DescriptiveStatistics and ExploratoryData AnalysisFurther Thoughts on Experimental DesignPop 1 Pop 2 Repeat 2 times processing 16 samples in totalRepeat entire process producing 2 technicalreplicates for all 16 samplesRandomly sample 4 individuals from each popTissue culture and RNA extractionLabeling and array hybridizationSlide scanning and data acquisition 16 Individuals (8 each from two populations) with replicatesOther Business Course web-site: Homework due on Thursday not Tuesday Make sure you look at HW1 soon and seeeither Shameek or myself with questionsToday What is Descriptive Statistics and exploratorydata analysis? Basic numerical summaries of data Basic graphical summaries of data How to use R for calculating Descriptive statisticsand making graphsPopulationSampleInferential StatisticsDescriptiveStatisticsProbabili ty Central Dogma of StatisticsEDAB efore making inferences from data it is essential toexamine all your listen to the data:- to catch mistakes- to see patterns in the data- to find violations of statistical assumptions- to generate because if you don t, you will have trouble

Labeling and array hybridization Slide scanning and data acquisition • 16 Individuals (8 each from two populations) with replicates. Other Business ... –Display as little information as possible –Obscure what you do show (with chart junk) –Use pseudo-3d and color gratuitously –Make a pie chart (preferably in color and 3d)

Tags:

  Possible, Hybridization

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of Lecture 2: Descriptive Statistics and Exploratory Data ...

1 Lecture 2: DescriptiveStatistics and ExploratoryData AnalysisFurther Thoughts on Experimental DesignPop 1 Pop 2 Repeat 2 times processing 16 samples in totalRepeat entire process producing 2 technicalreplicates for all 16 samplesRandomly sample 4 individuals from each popTissue culture and RNA extractionLabeling and array hybridizationSlide scanning and data acquisition 16 Individuals (8 each from two populations) with replicatesOther Business Course web-site: Homework due on Thursday not Tuesday Make sure you look at HW1 soon and seeeither Shameek or myself with questionsToday What is Descriptive Statistics and exploratorydata analysis? Basic numerical summaries of data Basic graphical summaries of data How to use R for calculating Descriptive statisticsand making graphsPopulationSampleInferential StatisticsDescriptiveStatisticsProbabili ty Central Dogma of StatisticsEDAB efore making inferences from data it is essential toexamine all your listen to the data:- to catch mistakes- to see patterns in the data- to find violations of statistical assumptions- to generate because if you don t, you will have trouble laterTypes of DataCategoricalQuantitativecontinuousdis creteordinalnominalbinary2 categories more categories order mattersnumerical uninterruptedDimensionality of Data Sets Univariate: Measurement made on one variable persubject Bivariate.

2 Measurement made on two variables persubject Multivariate: Measurement made on many variables per subjectNumerical Summaries of Data Central Tendency measures. They are computedto give a center around which the measurements inthe data are distributed. Variation or Variability measures. They describe data spread or how far away the measurements arefrom the center. Relative Standing measures. They describe therelative position of specific measurements in the : MeanTo calculate the average of a set of observations, add theirvalue and divide by the number of observations:x! x =x1+x2+x3+..+xnn=1nxii=1n"Other Types of MeansWeighted means:Trimmed:! x =wixii=1n"wii=1n"Harmonic:Geometric:! x ="! x =n1xii=1n"! x =xii=1n"# $ % & ' ( 1nLocation: Median Median the exact middle value Calculation:- If there are an odd number of observations, find the middle value- If there are an even number of observations, find the middle two values and average them ExampleSome data:Age of participants: 17 19 21 22 23 23 23 38 Median = (22+23)/2 = Location Measure Is Best?)

3 0 1 2 3 4 5 6 7 8 9 10 Mean = 3 0 1 2 3 4 5 6 7 8 9 10 Mean = 4 Median = 3 Median = 3 Mean is best for symmetric distributions without outliers Median is useful for skewed distributions or data with outliersScale: Variance Average of squared deviations of values fromthe mean! " 2=(xi#x )2in$n#1 Why Squared Deviations? Adding deviations will yield a sum of ? Absolute values do not have nice mathematicalproperties Squares eliminate the negatives Result: Increasing contribution to the variance as you gofarther from the : Standard Deviation Variance is somewhat arbitrary What does it mean to have a variance of Or Or Nothing. But if you could standardize that value,you could talk about any variance ( deviation) inequivalent terms Standard deviations are simply the square root of thevarianceScale: Standard Deviation!

4 " =(xi#x )2in$n#11. Score (in the units that are meaningful)1. Score (in the units that are meaningful)2. Mean2. Mean3. Each score3. Each score s deviation from the means deviation from the mean4. Square that deviation4. Square that deviation5. Sum all the squared deviations (Sum of Squares)5. Sum all the squared deviations (Sum of Squares)6. Divide by n-6. Divide by n-117. Square root 7. Square root now the value is in the units we started with!!! now the value is in the units we started with!!!Interesting Theoretical ResultwithinAt least(1 - 1/12) = 0% .. k=1 ( 1 )(1 - 1/22) = 75% .. k=2 ( 2 )(1 - 1/32) = 89% ..k=3 ( 3 )Note use of (sigma) torepresent standard deviation. Note use of (mu) torepresent mean . Regardless of how the data are distributed, a certainpercentage of values must fall within k standard deviationsfrom the mean:Often We Can Do BetterFor many lists of observations especially if their histogram is 68% of the observations in the list lie within 1 standarddeviation of the of the observations lie within 2 standard deviations of theaverage + +.

5 Quartiles and IQR25%25%25%25% The first quartile, Q1, is the value for which 25% of theobservations are smaller and 75% are larger Q2 is the same as the median (50% are smaller, 50% arelarger) Only 25% of the observations are greater than the thirdquartileQ1Q2Q3 IQRP ercentiles (aka Quantiles)In general the nth percentile is a value such that n% of theobservations fall at or below or itn%Median = 50th percentileQ1 = 25th percentileQ2 = 75th percentileGraphical Summaries of DataA (Good) Picture IsWorth A 1,000 WordsUnivariate Data: Histograms andBar Plots What s the difference between a histogram and bar plot? Used for categorical variables to show frequency or proportion ineach category. Translate the data from frequency tables into a plotHistogram Used to visualize distribution (shape, center, range, variation) ofcontinuous variables Bin size importantEffect of Bin Size on HistogramFrequencyFrequencyFrequency Simulated 1000 N(0,1) and 500 N(1,1)More on Histograms What s the difference between a frequency histogramand a density histogram?

6 More on Histograms What s the difference between a frequency histogramand a density histogram?Frequency HistogramDensity HistogramBox PlotsQ3maximum DataVariable 1 Variable 2 DisplayCategoricalCategoricalCrosstabsSt acked Box PlotCategoricalContinuousBoxplotContinuo usContinuousScatterplotStacked Box PlotMultivariate Data Organize units into clusters Descriptive , not inferential Many approaches Clusters always producedClusteringData Reduction Approaches (PCA) Reduce n-dimensional dataset into much smaller number Finds a new (smaller) set of variables that retains most ofthe information in the total sample Effective way to visualize multivariate dataHow to Make a Bad GraphThe aim of good data graphics:Display data accurately and clearlySome rules for displaying data badly.

7 Display as little information as possible Obscure what you do show (with chart junk) Use pseudo-3d and color gratuitously Make a pie chart (preferably in color and 3d) Use a poorly chosen scaleFrom Karl Broman: ~kbroman/Example 1 Example 2 Example 3 Example 4 Example 5R Tutorial Calculating Descriptive Statistics in R Creating graphs for different types of data(histograms, boxplots, scatterplots) Useful R commands for working with multivariatedata (apply and its derivatives) Basic clustering and PCA analysis


Related search queries