Example: tourism industry

Data Science Tutorial - Carnegie Mellon University

12017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited1 Software Engineering InstituteCarnegie Mellon UniversityPittsburgh, PA 152132017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is UnlimitedData Science TutorialEliezer Kanal Technical Manager, CERTD aniel DeCapria Data Scientist, ETC2 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited2 About usEliezer KanalTechnical Manager, CERTR ecent projects: ML-based Malware Classifier Network traffic analysis Cybersecurity questionnaireoptimizationDaniel DeCapriaData Scientist, ETCR ecent projects: Cyber risk situationaldashboard Big Learning benchmarks3 Data Science Tutori

Carnegie Mellon University Pittsburgh, PA 15213 2017 SEI Data Science in Cybersecurity Symposium Approved for Public Release; Distribution is Unlimited Data Science Tutorial ... •Support Vector Machines Clustering •K-Means Clustering. …

Fullscreen Download

Tags:

  University, Vector, Carnegie, Carnegie mellon university, Mellon

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Advertisement

Transcription of Data Science Tutorial - Carnegie Mellon University

1 12017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited1 Software Engineering InstituteCarnegie Mellon UniversityPittsburgh, PA 152132017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is UnlimitedData Science TutorialEliezer Kanal Technical Manager, CERTD aniel DeCapria Data Scientist, ETC2 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited2 About usEliezer KanalTechnical Manager, CERTR ecent projects: ML-based Malware Classifier Network traffic analysis Cybersecurity questionnaireoptimizationDaniel DeCapriaData Scientist, ETCR ecent projects: Cyber risk situationaldashboard Big Learning benchmarks3 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release.

2 Distribution is Unlimited3 Today s presentation a tale of two rolesThe call center managerIntroduction todata Science capabilitiesThe master carpenterOverview of thedata Science toolkit4 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited4 Call center managerFirst day on !Goal:Reduce costsTask:Keep calls short!Data:Average call minutes (5:08)..very long!Number of employees:300 Average calls per day:~28,0005 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited5 Call center manager Gather dataGet the data!

3 Where is it? What will you use to analyze it? How accurate it is? How complete is it? Is it too big to easily read?6 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited6 Data cleaning = 90% of the work2 weeks (10 days) = 9 cleaning, 1 analyzing7 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited7 More structureLess structureCleaning the Data Structuring the DataGoal: Organize data in a table, = descriptor (age, weight, height)Row = individual, complete recordsHow can you get data out of these documents?

4 8 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited8 Cleaning the DataEven when you think your data should be clean, it might not Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited9 Cleaning the Data Call Center ExampleNameMgrDirCall LengthPhone LineProblem solved?CommentBeth JonesDan ThomasAnne Kim1 ThomasAnne Kim1 , BethDan ThomasAnne KeaneMark RyanTim KeaneMark RyanTim WoodTim WoodTim WoodTim Nominal Unstructured 10 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited10 Call Center manager Exploring data11 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release.

5 Distribution is Unlimited11 Exploratory Data Analysis (EDA) Mean Median Standard deviation Histograms!12 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited12 Distributions The majority of data will follow SOME distributionoWeight of all Americans:Gaussianophone call length:Exponential Determining distribution is a common Data Science task Multidimensional outliers: Insider Threat exampleImage Copyright 2001-2016 The Apache Software Foundation.

6 See Copyright slide for more Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited13 EDA Smart visualizations14 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited1415 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited1516 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited1617 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release.

7 Distribution is Unlimited17 Brief interruptionSkeptics in the audience18 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited18 Brief interruptionData Science helps you use data to get results. This is Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited19 Call center manager call duration histogramAverage (5:08)20 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited20 Call Center manager Insights!

8 Strategy update: Goodbye reduce call time Hello reduce callbacks How to measure? callbacks isn t currently captured21 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited21 Feature EngineeringNeed more useful data?Create it yourself!22 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited22 Feature Engineering Feature Engineering: coming up with new, useful ( ,informative) dataomean, sums, medians, , xy , sqrt (xy), etc.

9 Our case:o# of callbacksoCall during peak time?oOverall agent performance? (combination of factors)23 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited23 The role of Listening in Data ScienceData Science finds hidden patterns in dataExperts know what data & patterns are importantTalk to subject matter experts24 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release.

10 Distribution is Unlimited24 Call Center manager Predictive analyticsCan we predict staffing ..one day ahead? ..one week ahead? ..one month ahead?Can we determine what types of calls to ..for a product we haven t had before? ..for a market we ve never seen before?25 Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited25 Example Predictive Analytics QuestionsPredicting Current UnknownsOnline: Which ads are malicious?Security: Is the bank transaction fraudulent?

Show more

Documents from same domain

Insider Threat Control: Using Universal Serial Bus …

Insider Threat Control: Using Universal Serial Bus …

resources.sei.cmu.edu

Insider Threat Control: Using Universal Serial Bus (USB) Device Auditing to Detect Possible Data Exfiltration by Malicious Insiders George J. Silowash

  Serial, Using, Threats, Control, Universal, Insider, Insider threat control, Using universal serial bus

Delivering a CMMI-Compliant Project Plan in 30 …

Delivering a CMMI-Compliant Project Plan in 30 …

resources.sei.cmu.edu

Earned Value Management, ... Delivering a CMMI-Compliant Project Plan in 30 Minutes Author: ... Subject: Presentation: ...

  Project, Management, Plan, Presentation, Delivering, Complaints, Cmmi, Delivering a cmmi compliant project plan in

Product Line Analysis: A Practical Introduction

Product Line Analysis: A Practical Introduction

resources.sei.cmu.edu

CMU/SEI-2001-TR-001 i Table of Contents Abstract vii 1 Introduction 1 1.1 The Context of Product Line Analysis 2 1.2 About This Report 3 2 Product Line Requirements 5

  Analysis, Introduction, Practical, Line, A practical introduction, Line analysis

Insider Fraud in Financial Services - SEI Digital Library

Insider Fraud in Financial Services - SEI Digital Library

resources.sei.cmu.edu

Insider fraud is perpetrated by a malicious insider, which is a current or former employee, contractor, or other business partner who has or had authorized access to an organization’s network, system, or data and intentionally exceeded or misused that access in

  Insider

Analytic Approaches to Detect Insider Threats

Analytic Approaches to Detect Insider Threats

resources.sei.cmu.edu

Insider threat is the potential for a current or former employee, contractor, or business partner to accidentally or maliciously misuse their trusted access to harm the organization’s employees , customers, assets, reputation,

  Insider

Implementing Best Practices in the Joint Battlespace ...

Implementing Best Practices in the Joint Battlespace ...

resources.sei.cmu.edu

Implementing Best Practices in the Joint Battlespace Infosphere (JBI) Program at AFRL Presented by: Ellen Walker, Analyst Data and Analysis Center for Software

  Programs, Joint, Infosphere, Battlespace, Joint battlespace infosphere, Program at

The Business Case for Systems Engineering Study: Results ...

The Business Case for Systems Engineering Study: Results ...

resources.sei.cmu.edu

The Business Case for Systems Engineering Study: Results of the Systems Engineering Effectiveness Survey ... 6.1.2 Analysis Differences 68 6.1.3 Sampling Differences 68 ... . The Business Case for Systems Engineering . for ) ) ) ) ) ...

  Business, Analysis, System, Engineering, Survey, Case, Effectiveness, Business case for systems engineering, The systems engineering effectiveness survey

Quantitative Methods for Software Selection and Evaluation

Quantitative Methods for Software Selection and Evaluation

resources.sei.cmu.edu

in the COTS-Based Systems for Program Managers and COTS Software Product Evaluation for Practitioners training courses. The approach and subsequent results were captured in the

  Evaluation, Approach, Software

Internal Audit's Role in Information Security: An Introduction

Internal Audit's Role in Information Security: An Introduction

resources.sei.cmu.edu

Audit Committee on occasion internal auditors would participate in a consulting role for those functions. It's very important up front that management and the Board agree with that role for

  Information, Security, Introduction, Internal, Roles, An introduction, Role in information security

ATAM for NoSQL Database Selection - SEI Digital Library

ATAM for NoSQL Database Selection - SEI Digital Library

resources.sei.cmu.edu

ATAM for NoSQL Database Selection. Using Architecture (Not Products) to ... Dan McCreary. Kelly-McCreary & Associates. New NoSQL databases offer more options to the database architect. Selecting the right NoSQL database for your project has become a nontrivial task. Yet selecting the right ... Distilled info. Impacts. Kelly-McCreary ...

  Database, Selection, Nosql, Distilled, Tama, Atam for nosql database selection

Related documents

Computer Architecture: Vector Processing: SIMD/Vector/GPU ...

Computer Architecture: Vector Processing: SIMD/Vector/GPU ...

course.ece.cmu.edu

SIMD/Vector/GPU Prof. Onur Mutlu (edited by seth) Carnegie Mellon University Vector Processing: Exploiting Regular (Data) Parallelism Data Parallelism Concurrency arises from performing the same operations on different pieces of data Single instruction multiple data (SIMD) E.g., dot product of two vectors Contrast with data flow

  University, Vector, Carnegie, Mellon, Carnegie mellon university vector

Kalman Filtering Tutorial - Carnegie Mellon University

Kalman Filtering Tutorial - Carnegie Mellon University

biorobotics.ri.cmu.edu

Monash University, Clayton. 2 Introduction Objectives: 1. Provide a basic understanding of Kalman Filtering and assumptions behind its implementation. 2. Limit (but cannot avoid) mathematical treatment to broaden appeal. 3. Provide some practicalities and examples of implementation. ... smallest vector that summarises the past of the system in ...

  University, Vector, Kalman, Carnegie, Carnegie mellon university, Mellon

The Extended Cohn-Kanade Dataset (CK+): A complete …

The Extended Cohn-Kanade Dataset (CK+): A complete …

sites.pitt.edu

The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression Patrick Lucey 1;2, Jeffrey F. Cohn , Takeo Kanade , Jason Saragih , Zara Ambadar2 Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, 152131 Department of Psychology, University of Pittsburgh, Pittsburgh, PA, 152602 patlucey@andrew.cmu.edu, …

  University, Carnegie, Carnegie mellon university, Mellon

On the Sentence Embeddings from Pre-trained Language …

On the Sentence Embeddings from Pre-trained Language …

aclanthology.org

zLanguage Technologies Institute, Carnegie Mellon University {zhouhao.nlp,wangmingxuan.89,lileilab}@bytedance.com {bohanl1,junxianh,yiming}@cs.cmu.edu Abstract Pre-trained contextual representations like BERT have achieved great success in natu-ral language processing. However, the sen-tence embeddings from the pre-trained lan-

  University, Carnegie, Carnegie mellon university, Mellon

Fast stochastic optimization on Riemannian manifolds

Fast stochastic optimization on Riemannian manifolds

arxiv.org

Carnegie Mellon University Suvrit Sra suvrit@mit.edu Massachusetts Institute of Technology Abstract We study optimization of finite sums of geodesically smooth functions on Rieman-nian manifolds. Although variance reduction techniques for optimizing finite-sum problems have witnessed a huge surge of interest in recent years, all existing work

  University, Carnegie, Carnegie mellon university, Mellon

Mixture Models - Carnegie Mellon University

Mixture Models - Carnegie Mellon University

www.stat.cmu.edu

Chapter 20 Mixture Models 20.1 Two Routes to Mixture Models 20.1.1 From Factor Analysis to Mixture Models In factor analysis, the origin myth is that we have a fairly small number, q of real

  University, Carnegie, Carnegie mellon university, Mellon

Modern regression 2: The lasso - Carnegie Mellon University

Modern regression 2: The lasso - Carnegie Mellon University

www.stat.cmu.edu

Modern regression 2: The lasso Ryan Tibshirani Data Mining: 36-462/36-662 March 21 2013 Optional reading: ISL 6.2.2, ESL 3.4.2, 3.4.3 1

  University, Carnegie, Carnegie mellon university, Mellon

Related search queries

Vector, Carnegie Mellon University Vector, Kalman, Carnegie Mellon University, University