AWS Database Migration Service - docs.aws.amazon.com

1 AWS Database Migration Service User Guide API Version API Version 2016-01-01. AWS Database Migration Service User Guide AWS Database Migration Service : User Guide Copyright 2019 Amazon Web services , Inc. and/or its a liates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or Service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be a liated with, connected to, or sponsored by Amazon. AWS Database Migration Service User Guide Table of Contents What Is AWS Database Migration Service ? .. 1. Migration Tasks That AWS DMS Performs .. 1. How AWS DMS Works at the Basic Level .. 2. How AWS DMS Works .. 4. High-Level View of AWS DMS .. 4. Components.

2 5. Sources .. 9. Targets .. 10. With Other AWS services .. 10. Support for AWS CloudFormation .. 11. Constructing an ARN .. 11. Setting Up .. 14. Sign Up for AWS .. 14. Create an IAM User .. 14. Migration Planning for AWS Database Migration Service .. 16. Getting Started .. 17. Start a Database Migration .. 17. Step 1: Welcome .. 17. Step 2: Create a Replication Instance .. 18. Step 3: Specify Source and Target Endpoints .. 22. Step 4: Create a Task .. 25. Monitor Your Task .. 29. Security .. 31. IAM Permissions Required .. 31. IAM Roles for the CLI and API .. 34. Fine-Grained Access Control .. 38. Using Resource Names to Control Access .. 38. Using Tags to Control Access .. 40. Setting an Encryption Key .. 44. Network Security .. 46. Using SSL .. 47. Limitations on Using SSL with AWS Database Migration Service .. 48. Managing Certi cates .. 48. Enabling SSL for a MySQL-compatible, PostgreSQL, or SQL Server Endpoint.

3 49. SSL Support for an Oracle Endpoint .. 50. Changing the Database Password .. 54. Limits .. 56. Limits for AWS Database Migration Service .. 56. Replication Instance .. 57. Replication Instances in Depth .. 58. Public and Private Replication Instances .. 60. AWS DMS Maintenance .. 60. AWS DMS Maintenance Window .. 60. Replication Engine Versions .. 63. Deprecating a Replication Instance Version .. 63. Upgrading the Engine Version of a Replication Instance .. 63. Setting Up a Network for a Replication Instance .. 65. Network Con gurations for Database Migration .. 65. Creating a Replication Subnet Group .. 70. Setting an Encryption Key .. 71. Creating a Replication Instance .. 72. Modifying a Replication Instance .. 76. Rebooting a Replication Instance .. 78. Deleting a Replication Instance .. 80. API Version API Version 2016-01-01. iii AWS Database Migration Service User Guide Supported DDL Statements.

4 81. Endpoints .. 83. Sources for Data Migration .. 83. Using Oracle as a Source .. 84. Using SQL Server as a Source .. 100. Using Azure SQL Database as a Source .. 109. Using PostgreSQL as a Source .. 110. Using MySQL as a Source .. 122. Using SAP ASE as a Source .. 129. Using MongoDB as a Source .. 132. Using Amazon Simple Storage Service as a Source .. 138. Using IBM Db2 as a Source .. 144. Targets for Data Migration .. 147. Using Oracle as a Target .. 148. Using SQL Server as a Target .. 152. Using PostgreSQL as a Target .. 156. Using MySQL as a Target .. 159. Using Amazon Redshift as a Target .. 163. Using SAP ASE as a Target .. 170. Using Amazon Simple Storage Service as a Target .. 171. Using Amazon DynamoDB as a Target .. 175. Using Amazon Kinesis Data Streams as a Target .. 189. Using Amazon Elasticsearch Service as a Target .. 195. Using Amazon DocumentDB as a Target .. 198. Creating Source and Target Endpoints.

5 210. Tasks .. 214. Creating a Task Assessment Report .. 215. Creating a Task .. 218. Task Settings .. 224. Setting LOB Support .. 238. Creating Multiple Tasks .. 239. Continuous Replication Tasks .. 239. Replication Starting from a CDC Start Point .. 240. Modifying a Task .. 242. Reloading Tables During a Task .. 242. AWS Management Console .. 242. Table Mapping .. 245. Specifying Table Selection and Transformations by Table Mapping from the Console .. 245. Specifying Table Selection and Transformations by Table Mapping Using JSON .. 250. Using Source Filters .. 257. Monitoring Tasks .. 261. Task Status .. 261. Table State During Tasks .. 262. Monitoring Replication Tasks Using Amazon CloudWatch .. 263. Data Migration Service Metrics .. 265. Replication Instance Metrics .. 265. Replication Task Metrics .. 266. Managing AWS DMS Logs .. 267. Logging AWS DMS API Calls with AWS CloudTrail.

6 268. AWS DMS Information in CloudTrail .. 269. Understanding AWS DMS Log File Entries .. 269. Validating Tasks .. 272. Replication Task Statistics .. 273. Revalidating Tables During a Task .. 274. AWS Management Console .. 275. Troubleshooting .. 275. Limitations .. 276. API Version API Version 2016-01-01. iv AWS Database Migration Service User Guide Tagging Resources .. 277. API .. 278. Working with Events and Noti cations .. 280. AWS DMS Event Categories and Event Messages .. 281. Subscribing to AWS DMS Event Noti cation .. 282. AWS Management Console .. 283. AWS DMS API .. 284. Migrating Large Data Stores Using AWS DMS and Snowball .. 285. Process Overview .. 285. Step-by-Step Procedures for Migrating Data using AWS DMS and AWS Snowball .. 287. Step 1: Create an AWS Snowball Job .. 287. Step 2: Install SCT .. 287. Step 3: Install and Con gure the SCT DMS Agent .. 287. Step 4: Unlock the AWS Snowball Edge Device.

7 288. Step 5: Create a New AWS SCT Project .. 288. Step 6: Con gure the AWS SCT Pro le .. 288. Step 7: Register the DMS Agent .. 290. Step 8: Install the Source Database Driver .. 291. Step 9: Con gure AWS SCT to Access the Amazon S3 Bucket .. 293. Step 10: Creating a Local & DMS Task .. 293. Step 11: Running and monitoring the Local & DMS Task .. 295. Step 11: Manage the AWS Snowball Appliance .. 295. Snowball to Amazon S3 .. 296. Troubleshooting .. 297. Slow Running Migration Tasks .. 297. Task Status Bar Not Moving .. 298. Missing Foreign Keys and Secondary Indexes .. 298. Amazon RDS Connection Issues .. 298. Error Message: Incorrect thread connection string: incorrect thread value 0 .. 298. Networking Issues .. 298. CDC Stuck After Full Load .. 299. Primary Key Violation Errors When Restarting a Task .. 299. Initial Load of Schema Fails .. 299. Tasks Failing With Unknown Error .. 299.

8 Task Restart Loads Tables From the Beginning .. 300. Number of Tables Per Task .. 300. Troubleshooting Oracle Speci c Issues .. 300. Pulling Data from Views .. 300. Migrating LOBs from Oracle 12c .. 300. Switching Between Oracle LogMiner and Binary Reader .. 301. Error: Oracle CDC stopped 122301 Oracle CDC maximum retry counter exceeded.. 301. Automatically Add Supplemental Logging to an Oracle Source Endpoint .. 301. LOB Changes not being Captured .. 302. Error: ORA-12899: value too large for column <column-name> .. 302. NUMBER data type being misinterpreted .. 302. Troubleshooting MySQL Speci c Issues .. 302. CDC Task Failing for Amazon RDS DB Instance Endpoint Because Binary Logging Disabled .. 303. Connections to a target MySQL instance are disconnected during a task .. 303. Adding Autocommit to a MySQL-compatible Endpoint .. 303. Disable Foreign Keys on a Target MySQL-compatible Endpoint.

9 304. Characters Replaced with Question Mark .. 304. "Bad event" Log Entries .. 304. Change Data Capture with MySQL .. 304. Increasing Binary Log Retention for Amazon RDS DB Instances .. 305. Log Message: Some changes from the source Database had no impact when applied to the target Database .. 305. Error: Identi er too long .. 305. API Version API Version 2016-01-01. v AWS Database Migration Service User Guide Error: Unsupported Character Set Causes Field Data Conversion to Fail .. 305. Error: Codepage 1252 to UTF8 [120112] A eld data conversion failed .. 306. Troubleshooting PostgreSQL Speci c Issues .. 306. JSON data types being truncated .. 306. Columns of a user de ned data type not being migrated correctly .. 307. Error: No schema has been selected to create in .. 307. Deletes and updates to a table are not being replicated using CDC .. 307. Truncate statements are not being propagated.

10 307. Preventing PostgreSQL from capturing DDL .. 307. Selecting the schema where Database objects for capturing DDL are created .. 308. Oracle tables missing after migrating to PostgreSQL .. 308. Task Using View as a Source Has No Rows Copied .. 308. Troubleshooting Microsoft SQL Server Speci c Issues .. 308. Special Permissions for AWS DMS user account to use CDC .. 308. Errors Capturing Changes for SQL Server Database .. 309. Missing Identity Columns .. 309. Error: SQL Server Does Not Support Publications .. 309. Changes Not Appearing in Target .. 309. Troubleshooting Amazon Redshift Speci c Issues .. 309. Loading into a Amazon Redshift Cluster in a Di erent Region Than the AWS DMS Replication Instance .. 310. Error: Relation "awsdms_apply_exceptions" already exists .. 310. Errors with Tables Whose Name Begins with "awsdms_changes" .. 310. Seeing Tables in Cluster with Names Like.