Developer Guide Version Latest - AWS Documentation

1 Amazon machine LearningDeveloper GuideVersion LatestAmazon machine learning Developer GuideAmazon machine learning : Developer GuideCopyright 2018 Amazon Web Services, Inc. and/or its affiliates. All rights 's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any mannerthat is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks notowned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored machine learning Developer GuideTable of ContentsWhat is Amazon machine learning ? .. 1 Amazon machine learning Key Concepts .. 1 Datasources .. 1ML Models.. 2 Evaluations.. 3 Batch Predictions .. 4 Real-time Predictions .. 4 Accessing Amazon machine learning .. 4 Regions and Endpoints .. 5 Pricing for Amazon ML .. 5 Estimating Batch Prediction Cost.

2 5 Estimating Real-Time Prediction Cost .. 7 machine learning Concepts .. 8 Solving Business Problems with Amazon machine learning .. 8 When to Use machine learning .. 9 Building a machine learning Application.. 9 Formulating the Problem.. 9 Collecting Labeled Data .. 10 Analyzing Your Data .. 10 Feature Processing .. 11 Splitting the Data into Training and Evaluation Data .. 12 Training the Model .. 12 Evaluating Model Accuracy .. 14 Improving Model Accuracy .. 17 Using the Model to Make Predictions .. 18 Retraining Models on New Data .. 19 The Amazon machine learning Process .. 19 Setting Up Amazon machine learning .. 21 Sign Up for AWS .. 21 Tutorial: Using Amazon ML to Predict Responses to a Marketing Offer .. 22 Prerequisite .. 22 Steps.. 22 Step 1: Prepare Your Data .. 22 Step 2: Create a Training Datasource .. 24 Step 3: Create an ML Model .. 28 Step 4: Review the ML Model's Predictive Performance and Set a Score Threshold.

3 29 Step 5: Use the ML Model to Generate Predictions .. 31 Step 6: Clean Up.. 36 Creating and Using Datasources .. 38 Understanding the Data Format for Amazon ML .. 38 Attributes .. 38 Input File Format Requirements .. 39 Using Multiple Files As Data Input to Amazon ML .. 39 End-of-Line Characters in CSV Format .. 40 Creating a Data Schema for Amazon ML .. 40 Example Schema.. 40 Using the targetAttributeName Field .. 42 Using the rowID Field .. 42 Using the AttributeType Field .. 43 Providing a Schema to Amazon ML .. 44 Splitting Your Data .. 44 Pre-splitting Your Data .. 45 Sequentially Splitting Your Data .. 45 Randomly Splitting Your Data .. 45 Version LatestiiiAmazon machine learning Developer GuideData Insights.. 46 Descriptive Statistics .. 46 Accessing Data Insights on the Amazon ML console .. 47 Using Amazon S3 with Amazon ML .. 54 Uploading Your Data to Amazon S3.

4 55 Permissions .. 55 Creating an Amazon ML Datasource from Data in Amazon Redshift .. 56 Creating an Amazon ML Amazon Redshift IAM Role .. 56 Required Parameters for the Create Datasource Wizard .. 58 Creating a Datasource with Amazon Redshift Data (Console) .. 61 Troubleshooting Amazon Redshift Issues .. 63 Using Data from an Amazon RDS Database to Create an Amazon ML Datasource .. 67 RDS Database Instance Identifier .. 68 MySQL Database Name .. 68 IAM Role Used to Create, Activate, and Execute the Data Pipeline .. 68 Database User Credentials .. 70 AWS Data Pipeline Security Information .. 70 Amazon RDS Security Information .. 70 MySQL SQL Query .. 70S3 Output Location.. 70 Configuring an IAM User or Role to Grant Permission to Enable Role Passing .. 71 Training ML Models .. 72 Types of ML Models .. 72 Binary Classification Model .. 72 Multiclass Classification Model.. 72 Regression Model .. 73 Training Process.

5 73 Training Parameters .. 73 Maximum Model Size .. 74 Maximum Number of Passes over the Data .. 74 Shuffle Type for Training Data .. 74 Regularization Type and Amount .. 75 Training Parameters: Types and Default Values .. 75 Creating an ML Model .. 76 Prerequisites .. 77 Creating an ML Model with Default Options .. 77 Creating an ML Model with Custom Options .. 77 Data Transformations for machine learning .. 79 Importance of Feature Transformation .. 79 Feature Transformations with Data Recipes .. 79 Recipe Format Reference .. 80 Groups .. 80 Assignments.. 81 Outputs.. 81 Complete Recipe Example .. 82 Suggested Recipes .. 83 Data Transformations Reference .. 84N-gram Transformation .. 84 Orthogonal Sparse Bigram (OSB) Transformation .. 85 Lowercase Transformation .. 85 Remove Punctuation Transformation .. 86 Quantile Binning Transformation .. 86 Normalization Transformation .. 86 Cartesian Product Transformation.

6 87 Data Rearrangement .. 88 DataRearrangement Parameters .. 88 Evaluating ML Models.. 91 Version LatestivAmazon machine learning Developer GuideML Model Insights.. 91 Binary Model Insights .. 92 Interpreting the Predictions .. 92 Multiclass Model Insights.. 95 Interpreting the Predictions.. 95 Regression Model Insights .. 96 Interpreting the Predictions.. 96 Preventing Overfitting .. 98 Cross-Validation .. 99 Adjusting Your Models .. 100 Evaluation Alerts .. 100 Generating and Interpreting Predictions .. 102 Creating a Batch Prediction .. 102 Creating a Batch Prediction (Console) .. 102 Creating a Batch Prediction (API) .. 103 Reviewing Batch Prediction Metrics .. 103 Reviewing Batch Prediction Metrics (Console) .. 104 Reviewing Batch Prediction Metrics and Details (API) .. 104 Reading the Batch Prediction Output Files .. 104 Locating the Batch Prediction Manifest File .. 104 Reading the Manifest File.

7 104 Retrieving the Batch Prediction Output Files .. 105 Interpreting the Contents of Batch Prediction Files for a Binary Classification ML model .. 105 Interpreting the Contents of Batch Prediction Files for a Multiclass Classification ML Model .. 106 Interpreting the Contents of Batch Prediction Files for a Regression ML Model .. 107 Requesting Real-time Predictions .. 107 Trying Real-Time Predictions .. 108 Creating a Real-Time Endpoint .. 109 Locating the Real-time Prediction Endpoint (Console) .. 110 Locating the Real-time Prediction Endpoint (API) .. 110 Creating a Real-time Prediction Request .. 110 Deleting a Real-Time Endpoint .. 112 Managing Amazon ML Objects .. 113 Listing Objects .. 113 Listing Objects (Console) .. 113 Listing Objects (API) .. 114 Retrieving Object Descriptions .. 115 Detailed Descriptions in the Console.. 115 Detailed Descriptions from the API .. 115 Updating Objects .. 115 Deleting Objects.

8 115 Deleting Objects (Console) .. 116 Deleting Objects (API) .. 116 Monitoring Amazon ML with Amazon CloudWatch Metrics .. 118 Logging Amazon ML API Calls by Using AWS CloudTrail .. 119 Amazon ML Information in CloudTrail .. 119 Understanding Amazon ML Log File Entries .. 120 Tagging Your Objects .. 123 Tag Basics .. 123 Tag Restrictions .. 124 Tagging Amazon ML Objects (Console) .. 124 Tagging Amazon ML Objects (API) .. 125 Amazon machine learning Reference .. 126 Granting Amazon ML Permissions to Read Your Data from Amazon S3 .. 126 Granting Amazon ML Permissions to Output Predictions to Amazon S3 .. 127 Controlling Access to Amazon ML Resources -with IAM.. 129 IAM Policy Syntax .. 130 Version LatestvAmazon machine learning Developer GuideSpecifying IAM Policy Actions for Amazon MLAmazon ML .. 130 Specifying ARNs for Amazon ML Resources in IAM Policies .. 131 Example Policies for Amazon MLs .. 131 Dependency Management of Asynchronous Operations.

9 133 Checking Request Status .. 134 System Limits .. 135 Names and IDs for all Objects .. 136 Object Lifetimes .. 136 Resources .. 137 Document History .. 138 Version LatestviAmazon machine learning Developer GuideAmazon machine learning Key ConceptsWhat is Amazon machine learning ?Welcome to the Amazon machine learning Developer Guide . Amazon machine learning (Amazon ML) isa robust, cloud-based service that makes it easy for developers of all skill levels to use machine learningtechnology. Amazon ML provides visualization tools and wizards that Guide you through the process ofcreating machine learning (ML) models without having to learn complex ML algorithms and your models are ready, Amazon ML makes it easy to obtain predictions for your applicationusing simple APIs, without having to implement custom prediction generation code, or manage section introduces the key concepts and terms that will help you understand what you need to do tocreate powerful machine learning models with Amazon you are new to machine learning , we recommend that you read machine LearningConcepts (p.)

10 8) before you continue. If you are already familiar with machine learning ,continue reading this Amazon machine learning Key Concepts (p. 1) Accessing Amazon machine learning (p. 4) Regions and Endpoints (p. 5) Pricing for Amazon ML (p. 5)Amazon machine learning Key ConceptsThis section summarizes the following key concepts and describes in greater detail how they are usedwithin Amazon ML: Datasources (p. 1) contain metadata associated with data inputs to Amazon ML ML Models (p. 2) generate predictions using the patterns extracted from the input data Evaluations (p. 3) measure the quality of ML models Batch Predictions (p. 4) asynchronously generate predictions for multiple input data observations Real-time Predictions (p. 4) synchronously generate predictions for individual data observationsDatasourcesA datasource is an object that contains metadata about your input data. Amazon ML reads your inputdata, computes descriptive statistics on its attributes, and stores the statistics along with a schema andother information as part of the datasource object.