Amazon SageMaker - Developer Guide

1 Amazon SageMakerDeveloper GuideAmazon SageMaker Developer GuideAmazon SageMaker : Developer GuideCopyright 2018 Amazon Web Services, Inc. and/or its affiliates. All rights 's trademarks and trade dress may not be used in connection with any product or service that is not Amazon 's, in any mannerthat is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon . All other trademarks notowned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored SageMaker Developer GuideTable of ContentsWhat Is Amazon SageMaker ? .. 1 Are You a First-time User of Amazon SageMaker ? .. 1 How It Works.

2 2 Machine Learning with Amazon SageMaker .. 2 How It Works: Next Topic .. 4 Explore and Preprocess Data .. 4 How It Works: Next Topic .. 4 Model Training .. 4 How It Works: Next Topic .. 6 Model Deployment .. 6 Hosting Services .. 7 Batch Transform.. 9 Validating Models .. 11 How It Works: Next Topic .. 12 Programming Model .. 12 How It Works: Next Topic .. 13 Getting Started .. 14 Step 1: Setting Up .. 14 Step : Create an Account .. 14 Step : Create an S3 Bucket .. 15 Step 2: Create an Amazon SageMaker Notebook Instance .. 16 Step : (Optional) Customize a Notebook Instance .. 20 Next Step .. 20 Step 3: Train and Deploy a Model.

3 21 Step : Create a Jupyter Notebook .. 22 Step : Download, Explore, and Transform Data .. 23 Step : Train a Model .. 25 Step : Deploy the Model .. 28 Step : Validate the Model .. 35 Step 4: Clean up.. 39 Step 5: Additional Considerations .. 40 Automatic Model Tuning .. 41 How Hyperparameter Tuning Works .. 41 Defining Objective Metrics .. 42 Defining Hyperparameter Ranges.. 43 Example: Hyperparameter Tuning Job .. 44 Prerequisites .. 44 Create a Notebook .. 44 Get the SageMaker boto3 Client .. 45 Get the Amazon SageMaker Execution Role .. 45 Specify a Bucket and Data Output Location .. 46 Download, Prepare, and Upload Training Data.

4 46 Configure and Launch a Hyperparameter Tuning Job .. 47 Monitor the Progress of a Hyperparameter Tuning Job .. 50 Clean up.. 52 Design Considerations.. 52 Choosing the Number of Hyperparameters.. 53 Choosing Hyperparameter Ranges.. 53 Use Logarithmic Scales for Hyperparameters.. 53 Choosing the Best Degree of Parallelism.. 53 Running Training Jobs on Multiple Instances .. 53 Using Notebook Instances .. 54 Creating a Notebook Instance .. 54 Limit Access to a Notebook Instance by IP Address .. 54iiiAmazon SageMaker Developer GuideAccessing Notebook Instances .. 55 Limit Access to a Notebook Instance by IP Address .. 55 Using Example Notebooks.

5 56 Set the Notebook Kernel .. 57 Installing External Libraries and Kernels in Notebook Instances .. 57 Using Built-in Algorithms .. 59 Common Information.. 61 Common Parameters .. 61 Common Data Formats .. 65 Suggested Instance Types .. 73 Logs.. 73 BlazingText .. 74 Input/Output Interface .. 76EC2 Instance Recommendation .. 78 Sample Notebooks.. 78 Hyperparameters.. 78 Model Tuning .. 82 DeepAR Forecasting .. 84 Input/Output Interface .. 84 Recommended Best Practices .. 86EC2 Instance Recommendations .. 87 Sample Notebooks.. 87 How It Works .. 87 Hyperparameters.. 90 Model Tuning .. 93 Inference Formats .. 95 Factorization Machines.

6 97 Input/Output Interface .. 97EC2 Instance Recommendation .. 97 Sample Notebooks.. 98 How It Works .. 98 Hyperparameters.. 99 Model Tuning .. 103 Inference Formats .. 105 Image Classification Algorithm.. 106 Input/Output Interface .. 107EC2 Instance Recommendation .. 108 Sample Notebooks.. 108 How It Works .. 108 Hyperparameters.. 109 Model Tuning .. 114K-Means Algorithm.. 115 Input/Output Interface .. 116EC2 Instance Recommendation .. 116 Sample Notebooks.. 116 How It Works .. 116 Hyperparameters.. 119 Model Tuning .. 121 Inference Formats .. 122K-Nearest Neighbors .. 123 Input/Output Interface .. 123 Sample Notebooks.

7 124 How It Works .. 124EC2 Instance Recommendation .. 125 Hyperparameters.. 126 Model Tuning .. 127 Training Formats .. 128ivAmazon SageMaker Developer GuideInference Formats .. 129 Latent Dirichlet Allocation (LDA) .. 132 Input/Output Interface .. 132EC2 Instance Recommendation .. 133 Sample Notebooks.. 133 How It Works .. 133 Hyperparameters.. 135 Model Tuning .. 136 Linear Learner.. 137 Input/Output Interface .. 137EC2 Instance Recommendation .. 138 Sample Notebooks.. 138 How It Works .. 138 Hyperparameters.. 139 Model Tuning .. 147 Inference Formats .. 149 Neural Topic Model (NTM) .. 151 Input/Output Interface .. 151EC2 Instance Recommendation.

8 152 Sample Notebooks.. 152 Hyperparameters.. 152 Model Tuning .. 155 Inference Formats .. 156 Object Detection Algorithm.. 157 Input/Output Interface .. 157EC2 Instance Recommendation .. 159 Sample Notebooks.. 159 How It Works .. 159 Hyperparameters.. 159 Model Tuning .. 163 Inference Formats .. 164 Principal Component Analysis (PCA).. 166 Input/Output Interface .. 166EC2 Instance Recommendation .. 166 Sample Notebooks.. 166 How It Works .. 167 Hyperparameters.. 168 Inference Formats .. 169 Random Cut Forest .. 170 Input/Output Interface .. 170 Instance Recommendations .. 171 Sample Notebooks.. 171 How It Works .. 171 Hyperparameters.

9 174 Model Tuning .. 175 Inference Formats .. 176 Sequence to Sequence (seq2seq) .. 178 Input/Output Interface .. 178EC2 Instance Recommendation .. 179 Sample Notebooks.. 179 How It Works .. 180 Hyperparameters.. 180 Model Tuning .. 188 XGBoost Algorithm.. 190 Input/Output Interface .. 190EC2 Instance Recommendation .. 191 Sample Notebooks.. 192vAmazon SageMaker Developer GuideHow It Works .. 192 Hyperparameters.. 192 Model Tuning .. 198 Using Your Own Algorithms .. 201 Using Your Own Training Algorithms .. 202 How Amazon SageMaker Runs Your Training Image .. 202 How Amazon SageMaker Provides Training Information .. 203 Signalling Algorithm Success and Failure.

10 205 How Amazon SageMaker Processes Training Output .. 205 Next Step .. 206 Using Your Own Inference Code .. 206 Using Your Own Inference Code (Hosting Services) .. 206 Using Your Own Inference Code (Batch Transform) .. 209 Example: Using Your Own Algorithms .. 211 Automatically Scaling Amazon SageMaker Models .. 212 Automatic Scaling Components.. 212 Required Permissions for Automatic Scaling .. 213 Service-Linked Role .. 213 Target Metric .. 214 Minimum and Maximum Capacity .. 214 Cooldown Period .. 214 Before You Begin .. 215 Related Topics .. 215 Configure Automatic Scaling for a Variant .. 215 Configure Automatic Scaling for a Variant (Console).