AWS Snowball - Developer Guide

1 AWS SnowballDeveloper GuideAWS Snowball Developer GuideAWS Snowball : Developer GuideCopyright 2018 Amazon Web Services, Inc. and/or its affiliates. All rights 's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any mannerthat is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks notowned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored Snowball Developer GuideTable of Contents.. viiWhat Is a Snowball Edge? .. 1 AWS Snowball Edge Features .. 1 Prerequisites for Using Snowball Edge .. 2 Related Services .. 2 Accessing the AWS Snowball Service .. 3 Accessing an AWS Snowball Edge Device .. 3 Pricing for the AWS Snowball Edge .. 3 Are You a First-Time AWS Snowball User?

2 3 Device Differences .. 3 Use Case Differences .. 3 Hardware Differences .. 4 Tool Differences .. 5 Other Differences .. 7 How It Works .. 7 How Import Works .. 8 How Export Works .. 8 How Local Compute and Storage Works .. 9 Jobs.. 9 Job Details.. 10 Job Statuses.. 11 Import Jobs into Amazon S3 .. 13 Export Jobs from Amazon S3 .. 14 Local Compute and Storage Only Jobs.. 16 Cloning a Job in the Console.. 17 Canceling Jobs in the Console .. 17 Setting Up .. 18 Sign Up for AWS .. 18 Create an IAM User .. 18 Next Step .. 20 Getting Started .. 21 Sign Up for AWS .. 21 Create an Administrator IAM User .. 21 Getting Started: Your First Job .. 22 Create Your First Job .. 22 Receive the Snowball Edge .. 24 Connect to Your Local Network .. 25 Get Your Credentials and Tools .. 25 Download and Install the Snowball Client.

3 26 Unlock the Snowball Edge .. 26 Use the Snowball Edge .. 27 Stop the Snowball Client, and Power Off the Snowball Edge .. 27 Disconnect the Snowball Edge .. 27 Return the Snowball Edge .. 28 Monitor the Import Status .. 28 Get Your Job Completion Report and Logs .. 28 Where Do I Go from Here? .. 29 Best Practices .. 30 Performance .. 30 Performance Recommendations .. 31 Speeding Up Data Transfer .. 31 How to Transfer Petabytes of Data Efficiently .. 32 Planning Your Large Transfer .. 32 Calibrating a Large Transfer .. 34iiiAWS Snowball Developer GuideUsing a Snowball Edge .. 35 Changing Your IP Address .. 37 Using the Snowball Client .. 37 Downloading and Installing the Snowball Client .. 37 Commands for the Snowball Client .. 38 Using the Adapter .. 48 Getting and Using Local Amazon S3 Credentials .. 49 Unsupported Amazon S3 Features for Snowball Edge.

4 50 Batching Small Files.. 50 Supported CLI Commands .. 51 Supported REST API Actions .. 53 Using the File Interface .. 55 Overview of the File Interface .. 55 Starting the File Interface .. 57 Mounting a Bucket with the File Interface .. 57 Monitoring the File Interface .. 60 Using AWS Lambda .. 61 Before You Start .. 62 Getting Started with Lambda .. 63 Using Amazon EC2 .. 67 Overview.. 67 Prerequisites .. 68 Limits for Compute Instances on a Snowball Edge .. 69 Creating a Job with Compute Instances .. 70 Network Configuration for Compute Instances .. 71 Connecting to Your Compute Instance on a Snowball Edge using SSH.. 72 Transferring Data from Amazon EC2 Compute Instances to Amazon S3 Buckets on the SameSnowball Edge .. 73 Snowball Client commands for Compute Instances .. 73 Using the Amazon EC2 Endpoint .. 76 Supported Instance Metadata and User Data.

5 79 Troubleshooting Amazon EC2 .. 81 Using an AWS Snowball Edge Cluster .. 82 Clustering Overview.. 82 Snowball Edge Cluster Quorums .. 82 Cluster Job Considerations.. 83 Related Topics .. 83 Administrating a Cluster .. 84 Reading and Writing Data to a Cluster .. 84 Reconnecting an Unavailable Cluster Node .. 84 Removing an Unhealthy Node from a Cluster .. 85 Adding or Replacing a Node in a Cluster .. 85 Shipping Considerations.. 87 Preparing an AWS Snowball Edge for Shipping .. 87 Region-Based Shipping Restrictions .. 88 Shipping an AWS Snowball Edge .. 88 Shipping Carriers.. 88 Security .. 91 Encryption for AWS Snowball Edge .. 91 Server-Side Encryption .. 91 AWS Key Management Service in AWS Snowball .. 92 Using the AWS-Managed Customer Master Key for Snowball .. 93 Creating a Custom KMS Envelope Encryption Key .. 93 Authorization with the Amazon S3 API Adapter for AWS Snowball .

6 93 Other Security Considerations for AWS Snowball .. 94 Authentication and Access Control .. 95 Authentication.. 95ivAWS Snowball Developer GuideAccess Control in the AWS Cloud .. 96 Overview of Managing Access .. 96 Resources and Operations .. 97 Understanding Resource Ownership .. 97 Managing Access to Resources in the AWS Cloud .. 97 Specifying Policy Elements: Actions, Effects, and Principals .. 98 Specifying Conditions in a Policy .. 99 Using Identity-Based Policies (IAM Policies) .. 99 Permissions Required to Use the AWS Snowball Console .. 100 AWS-Managed (Predefined) Policies for AWS Snowball Edge .. 103 Customer Managed Policy Examples .. 105 Job Management API Permissions Reference .. 108 Data Validation .. 109 Checksum Validation of Transferred Data .. 109 Common Validation Errors .. 109 Manual Data Validation for Snowball Edge During Transfer.

7 110 Manual Data Validation for Snowball Edge After Import into Amazon S3 .. 110 Notifications.. 111 Specifications.. 112 Supported Network Hardware .. 113 Network Converter for a Second Connection to the Snowball Edge .. 115 Limits.. 116 Regional Limitations for AWS Snowball .. 116 Limitations on AWS Snowball Edge Jobs .. 116 Limitations on Transferring On-Premises Data with an AWS Snowball Edge Device .. 117 Limitations for Lambda Powered by AWS Greengrass .. 117 Limitations on Shipping an AWS Snowball Edge .. 117 Limitations on Processing Your Returned AWS Snowball Edge for Import .. 118 Troubleshooting .. 119 How to Identify Your Device .. 120 Troubleshooting Connection Problems .. 120 Troubleshooting Manifest File Problems .. 120 Troubleshooting Credentials Problems .. 120 Unable to Locate AWS CLI Credentials .. 120 Error Message: Check Your Secret Access Key and Signing.

8 121 Troubleshooting Data Transfer Problems .. 121 Troubleshooting Problems with Transferring Data with the File Interface .. 121 Troubleshooting AWS CLI Problems .. 122 AWS CLI Error Message: "Profile Cannot Be Null" .. 122 Null Pointer Error When Transferring Data with the AWS CLI .. 122 Troubleshooting Import Job Problems .. 123 Troubleshooting Export Job Problems .. 123 Appendices .. 124 Using the Snowball Client .. 124 Downloading and Installing the Snowball Client .. 124 Commands for the Snowball Client .. 124 Unlocking the AWS Snowball Edge Device .. 128 Clustering Overview.. 129 Snowball Edge Cluster Quorums .. 129 Cluster Job Considerations.. 130 Related Topics .. 130 Administrating a Cluster .. 130 Using the File Interface .. 133 Overview of the File Interface .. 134 Mounting a Bucket with the File Interface .. 135 Monitoring the File Interface.

9 137 Using AWS Lambda .. 139vAWS Snowball Developer GuideBefore You Start .. 140 Getting Started with Lambda .. 141 Document History .. 144 AWS Glossary .. 148viAWS Snowball Developer GuideThis Guide is for the Snowball Edge. If you are looking for documentation for the Snowball , see the AWSS nowball User Snowball Developer GuideAWS Snowball Edge FeaturesWhat Is an AWS Snowball Edge?AWS Snowball is a service that accelerates transferring large amounts of data into and out of AWS byusing physical storage devices, bypassing the internet. AWS Snowball Edge is a 100-TB data transferdevice with on-board storage and compute power for select AWS capabilities. In addition to transferringdata to AWS, Snowball Edge can undertake local processing and edge-computing AWS Snowball device can transport data at speeds faster than the internet. This transport is doneby shipping the data in the appliances through a regional carrier.

10 The appliances are rugged shippingcontainers, complete with E Ink shipping labels. The AWS Snowball Edge device differs from the standardSnowball because it can bring the power of the AWS Cloud to your local environment, with local storageand compute storage and compute functionality is built-in. With an AWS Snowball Edge device, there's no needfor a powerful workstation to handle encryption, because all encryption happens on the device are many options for transferring your data into AWS. The AWS Snowball service isintended for transferring large amounts of data. If you want to transfer less than 30 terabytesof data, or if you don t have a local computing or clustering need, using an AWS Snowball Edgedevice might not be your most economical Snowball Edge FeaturesAWS Snowball Edge devices have the following features: Each AWS Snowball Edge device has 100 TB of storage capacity, twice the storage space of the originalSnowball device.