AWS Storage Gateway is a hybrid cloud storage solution that is deployed on premise, and allows your applications to utilize AWS cloud storage services like S3, Glacier, EBS, etc. It is deployed as a virtual machine or a hardware gateway appliance, and by using storage protocols like NFS, iSCSI, and SMB it provides a very optimized data transfer capabilities, along with various other features.
AWS Storage Gateway provides integration between the on-premises IT environment and the AWS storage infrastructure. The user can store data in the AWS cloud for scalable, data security features and cost-efficient storage.
Storage Gateway allows storage of data in the AWS cloud for scalable and cost-effective storage while maintaining data security.
Exposes compatible iSCSI interface on the front end to easily integrate with existing backup applications and represents another disk drive
AWS Storage Gateway backs up the data in Amazon Storage as incremental EBS snapshots
AWS Storage Gateway can run either on-premises, as a virtual machine (VM) appliance, or in AWS, as an EC2 instance. So if the on-premises data center goes offline and there is no available host, the gateway can be deployed on an EC2 instance.
Gateways hosted on EC2 instances can be used for disaster recovery, data mirroring, and providing storage for applications hosted on EC2.
AWS Storage Gateway, by default, uploads data using SSL and provides data encryption at rest when stored in S3 or Glacier using AES-256.
AWS Storage Gateway performs compression of data in-transit and at-rest.
Types of Storage Gateway
There are three types of AWS Storage Gateway: File Gateway, Volume Gateway and Tape Gateway.
1. File Gateway
This storage gateway type provides access to files that are stored as objects in an Amazon S3 bucket by using SMB (versions 2 and 3 of the CIFS protocol) and NFS shares (protocol version 3 and 4.1). An SMB (Server Message Block) or NFS (Network File System) mount point must be configured in your operating system to be used to access files/objects in an S3 bucket.
The File Gateway supports the following Amazon S3 storage classes: S3 Standard, S3 Standard-Infrequent Access (IA), and S3 One Zone-IA. Versioning is supported – you can edit, delete, and rename files by accessing them via the NFS or SMB protocols and each file modification is stored as a new version in an S3 bucket. The main advantage of using versioning for a file (object) share is extended recovery capabilities. In addition to versioning, you can enable lifecycle management and cross-region replication for objects stored in Amazon S3.
It is using the technique NFS.
It is used to store the flat files in S3 such as word files, pdf files, pictures, videos, etc.
It is used to store the files to S3 directly.
Files are stored as objects in S3 buckets, and they are accessed through a Network File System (NFS) mount point.
Ownership, permissions, and timestamps are durably stored in S3 in the user metadata of the object associated with the file.
Once the objects are transferred to the S3, they can be used as the native S3 objects, and bucket policies such as versioning, lifecycle management, and cross-region replication can be directly applied to the objects stored in your bucket.
2. Volume Gateway:
Volume Gateway allows your servers and applications running on-premises to connect to the AWS block storage (EBS volumes) in the cloud by using the iSCSI protocol (Internet Small Computer Systems Interface). While SMB and NFS used by a file gateway are file level sharing protocols, iSCSI works at the block level. There are two types of Volume Gateway – Stored Volumes and Cached Volumes.
Volume gateways provide cloud-backed storage volumes that you can mount as Internet Small Computer System Interface (iSCSI) devices from your on-premises application servers. For Volume gateways all data is securely stored in AWS, the approach differs with how much data is stored on-premises
Volume Gateway is an interface that presents your applications with disk volumes using the Iscsi block protocol. The iSCSI block protocol is block-based storage that can store an operating system, applications and also can run the SQL Server, database.
Data written to the hard disk can be asynchronously backed up as point-in-time snapshots in your hard disks and stored in the cloud as EBS snapshots where EBS (Elastic Block Store) is a virtual hard disk which is attached to the EC2 instance. In short, we can say that the volume gateway takes the virtual hard disks that you back them up to the aws.
Snapshots are incremental backups so that the changes made in the last snapshot are backed up. All snapshot storage is also compressed to minimize your storage charges.
Volume Gateway – provides cloud-backed storage volumes that you can mount as iSCSI devices from your on-premises application servers.
Volume Gateway is of two types:
Stored Volumes:
If you need low-latency access to your entire dataset, first configure your on-premises gateway to store all your data locally. Then asynchronously back up point-in-time snapshots of this data to S3. Stored volumes can range from 1 GiB to 16 TiB in size and must be rounded to the nearest GiB. Each gateway configured for stored volumes can support up to 32 volumes.
It is a way of storing the entire copy of the data locally and asynchronously backing up the data to aws.
Stored volumes provide low-latency access to the entire datasets of your on-premise applications and offsite backups.
You can create a stored volume that can be a virtual storage volume which is mounted as iSCSI devices to your on-premise application services such as data services, web services.
Data written to your stored volume is stored on your local storage hardware, and this data is asynchronously backed up to the Amazon Simple storage services in the form of Amazon Elastic Block store snapshots.
The size of the stored volume is 1GB – 16 TB.
Cached Gateway:
You store your data in S3 and retain a copy of frequently accessed data subsets locally. Cached volumes can range from 1 GiB to 32 TiB in size and must be rounded to the nearest GiB. Each gateway configured for cached volumes can support up to 32 volumes.
It is a way of storing the most recently accessed data on site, and the rest of the data is stored in aws.
Cached Volume allows using the Amazon Simple Storage service as your primary data storage while keeping the copy of the recently accessed data locally in your storage gateway.
Cached Volume minimizes the need to scale your on-premises storage infrastructure while still providing the low-latency access to their frequently accessed data.
Cached Gateway stores the data that you write to the volume and retains only recently read data in on-premises storage gateway.
The size of the cached volume is 1GB – 32 TB.
Tape Gateway:
Tape Gateway acts as an industry-standard iSCSI-based Virtual Tape Library (VTL). Deployed on-premise, it consist of virtual media changer and virtual tape drives, and allows you to continue to rely on your existing backup workflows. At the same time your data is written to virtual tapes which are stored in durable S3 buckets and is ready to be archived to Glacier (further reducing the storage cost) when frequent access is no longer needed.
Tape Gateway is compatible with many common backup applications, such as Dell EMC NetWorker, Microsoft System Center Data Protection Manager, and many others.
Tape Gateway is used to back up data for long term archival to Amazon Glacier and store that data on virtual tapes. In fact, data is stored in Amazon S3 Glacier or Amazon S3 Glacier Deep Archive. In this case, the physical interface used to write data on tapes by connecting tape drives and tape libraries is replaced with a compatible Tape Gateway Library interface that allows you to store data in the Amazon cloud. The iSCSI protocol is used to connect existing backup devices to the Tape Gateway. Existing backup configuration and workflow can be preserved. You can save data to the cloud directly via the Tape Gateway or by using specialized data backup applications.
Tape gateways can be used to back up data without making significant changes to an existing backup configuration or as an alternative to physical tape drives and libraries (which are not cost-effective)
Tape Gateway is mainly used for taking backups.
It uses a Tape Gateway Library interface.
Tape Gateway offers a durable, cost-effective solution to archive your data in AWS cloud.
The VTL interface provides a tape-based backup application infrastructure to store data on virtual tape cartridges that you create on your tape Gateway.
It is supported by NetBackup, Backup Exec, Veeam, etc. Instead of using physical tape, they are using virtual tape, and these virtual tapes are further stored in Amazon S3.
Tape Gateway – archive backup data in Amazon Glacier.
Has a virtual tape library (VTL) interface to store data on virtual tape cartridges that you create.
Deploy your gateway on an EC2 instance to provision iSCSI storage volumes in AWS.
The AWS Storage Gateway service integrates Tape Gateway with Amazon S3 Glacier Deep Archive storage class, allowing you to store virtual tapes in the lowest-cost Amazon S3 storage class.
Tape Gateway also has the capability to move your virtual tapes archived in Amazon S3 Glacier to Amazon S3 Glacier Deep Archive storage class, enabling you to further reduce the monthly cost to store long-term data in the cloud by up to 75%.
Supports Write-Once-Read-Many and Tape Retention Lock on virtual tapes.
Advantages of AWS Storage Gateway
The main advantages of using Amazon Storage Gateway are:
⦁ Integration of hardware and software configurations with no hardware changes.
⦁ The ability to use on-premises storage and cloud storage in Amazon (the hybrid storage concept).
⦁ Smooth migration from physical infrastructure to the AWS cloud.
Conclusion
Amazon Storage Gateway is a hybrid cloud solution that allows you to use your current physical and virtual infrastructure with Amazon cloud storage without significant changes to your current hardware and software configuration. Standard storage protocols are used – SMB and NFS are used to provide access to files stored as objects in Amazon S3 on a file level and access to block storage (Amazon EBS volumes) is provided by using iSCSI.
Amazon Storage Gateway can be used to copy your data backups to AWS manually or with special backup solutions that can work with NFS, SMB or iSCSI protocols.