Developer Guide Version Latest - AWS Documentation

1 Amazon Machine LearningDeveloper GuideVersion LatestAmazon Machine learning Developer GuideAmazon Machine learning : Developer GuideCopyright 2018 Amazon Web Services, Inc. and/or its affiliates. All rights 's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any mannerthat is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks notowned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored Machine learning Developer GuideTable of ContentsWhat is Amazon Machine learning ?

2 1 Amazon Machine learning Key Concepts .. 1 Datasources .. 1ML Models.. 2 Evaluations.. 3 Batch Predictions .. 4 Real-time Predictions .. 4 Accessing Amazon Machine learning .. 4 Regions and Endpoints .. 5 Pricing for Amazon ML .. 5 Estimating Batch Prediction Cost .. 5 Estimating Real-Time Prediction Cost .. 7 Machine learning Concepts .. 8 Solving Business Problems with Amazon Machine learning .. 8 When to Use Machine learning .. 9 Building a Machine learning Application.. 9 Formulating the Problem.. 9 Collecting Labeled Data .. 10 Analyzing Your Data .. 10 Feature Processing.

3 11 Splitting the Data into Training and Evaluation Data .. 12 Training the Model .. 12 Evaluating Model Accuracy .. 14 Improving Model Accuracy .. 17 Using the Model to Make Predictions .. 18 Retraining Models on New Data .. 19 The Amazon Machine learning Process .. 19 Setting Up Amazon Machine learning .. 21 Sign Up for AWS .. 21 Tutorial: Using Amazon ML to Predict Responses to a Marketing Offer .. 22 Prerequisite .. 22 Steps.. 22 Step 1: Prepare Your Data .. 22 Step 2: Create a Training Datasource .. 24 Step 3: Create an ML Model .. 28 Step 4: Review the ML Model's Predictive Performance and Set a Score Threshold.

4 29 Step 5: Use the ML Model to Generate Predictions .. 31 Step 6: Clean Up.. 36 Creating and Using Datasources .. 38 Understanding the Data Format for Amazon ML .. 38 Attributes .. 38 Input File Format Requirements .. 39 Using Multiple Files As Data Input to Amazon ML .. 39 End-of-Line Characters in CSV Format .. 40 Creating a Data Schema for Amazon ML .. 40 Example Schema.. 40 Using the targetAttributeName Field .. 42 Using the rowID Field .. 42 Using the AttributeType Field .. 43 Providing a Schema to Amazon ML .. 44 Splitting Your Data .. 44 Pre-splitting Your Data .. 45 Sequentially Splitting Your Data.

5 45 Randomly Splitting Your Data .. 45 Version LatestiiiAmazon Machine learning Developer GuideData Insights.. 46 Descriptive Statistics .. 46 Accessing Data Insights on the Amazon ML console .. 47 Using Amazon S3 with Amazon ML .. 54 Uploading Your Data to Amazon S3 .. 55 Permissions .. 55 Creating an Amazon ML Datasource from Data in Amazon Redshift .. 56 Creating an Amazon ML Amazon Redshift IAM Role .. 56 Required Parameters for the Create Datasource Wizard .. 58 Creating a Datasource with Amazon Redshift Data (Console) .. 61 Troubleshooting Amazon Redshift Issues .. 63 Using Data from an Amazon RDS Database to Create an Amazon ML Datasource.

6 67 RDS Database Instance Identifier .. 68 MySQL Database Name .. 68 IAM Role Used to Create, Activate, and Execute the Data Pipeline .. 68 Database User Credentials .. 70 AWS Data Pipeline Security Information .. 70 Amazon RDS Security Information .. 70 MySQL SQL Query .. 70S3 Output Location.. 70 Configuring an IAM User or Role to Grant Permission to Enable Role Passing .. 71 Training ML Models .. 72 Types of ML Models .. 72 Binary Classification Model .. 72 Multiclass Classification Model.. 72 Regression Model .. 73 Training Process .. 73 Training Parameters .. 73 Maximum Model Size.

7 74 Maximum Number of Passes over the Data .. 74 Shuffle Type for Training Data .. 74 Regularization Type and Amount .. 75 Training Parameters: Types and Default Values .. 75 Creating an ML Model .. 76 Prerequisites .. 77 Creating an ML Model with Default Options .. 77 Creating an ML Model with Custom Options .. 77 Data Transformations for Machine learning .. 79 Importance of Feature Transformation .. 79 Feature Transformations with Data Recipes .. 79 Recipe Format Reference .. 80 Groups .. 80 Assignments.. 81 Outputs.. 81 Complete Recipe Example .. 82 Suggested Recipes .. 83 Data Transformations Reference.

8 84N-gram Transformation .. 84 Orthogonal Sparse Bigram (OSB) Transformation .. 85 Lowercase Transformation .. 85 Remove Punctuation Transformation .. 86 Quantile Binning Transformation .. 86 Normalization Transformation .. 86 Cartesian Product Transformation .. 87 Data Rearrangement .. 88 DataRearrangement Parameters .. 88 Evaluating ML Models.. 91 Version LatestivAmazon Machine learning Developer GuideML Model Insights.. 91 Binary Model Insights .. 92 Interpreting the Predictions .. 92 Multiclass Model Insights.. 95 Interpreting the Predictions.. 95 Regression Model Insights .. 96 Interpreting the Predictions.

9 96 Preventing Overfitting .. 98 Cross-Validation .. 99 Adjusting Your Models .. 100 Evaluation Alerts .. 100 Generating and Interpreting Predictions .. 102 Creating a Batch Prediction .. 102 Creating a Batch Prediction (Console) .. 102 Creating a Batch Prediction (API) .. 103 Reviewing Batch Prediction Metrics .. 103 Reviewing Batch Prediction Metrics (Console) .. 104 Reviewing Batch Prediction Metrics and Details (API) .. 104 Reading the Batch Prediction Output Files .. 104 Locating the Batch Prediction Manifest File .. 104 Reading the Manifest File .. 104 Retrieving the Batch Prediction Output Files.

10 105 Interpreting the Contents of Batch Prediction Files for a Binary Classification ML model .. 105 Interpreting the Contents of Batch Prediction Files for a Multiclass Classification ML Model .. 106 Interpreting the Contents of Batch Prediction Files for a Regression ML Model .. 107 Requesting Real-time Predictions .. 107 Trying Real-Time Predictions .. 108 Creating a Real-Time Endpoint .. 109 Locating the Real-time Prediction Endpoint (Console) .. 110 Locating the Real-time Prediction Endpoint (API) .. 110 Creating a Real-time Prediction Request .. 110 Deleting a Real-Time Endpoint .. 112 Managing Amazon ML Objects.