PDF_Slide__Hadirkan_AWS_untuk_Hybrid_Cloud_Storage_Anda.pdf

menjelaskan tentang penggunaan aws pada data

1. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Builders Online Series
TedyTirtawidjaja
Senior Solutions Architect
Amazon Web Services
Bring AWS to you with
hybrid cloud storage

2. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda layout
•Connectivity to AWS
•AWS Hybrid Cloud Storage Services
•Common customer use cases and patterns
•Demo
•Resources

3. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How much data is out there?
175 ZB
Zettabytes of data created by
2025, globally
10
21
1 zettabyte is
1,000,000,000,000,000,000,000
Sources:
https://www.idc.com/getdoc.jsp?containerId=prUS48247121
https://www.idc.com/getdoc.jsp?containerId=US47397521

4. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Network connectivity
to AWS

5. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Connecting to AWS
AWS Cloud Data center
Devices
Clients
Apps
Connectivity Options:

6. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud Data center
Devices
Clients
Apps
Direct Connect
Connectivity Options:
Connecting to AWS

7. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud Data center
Devices
Clients
Apps
Direct Connect
Site to Site VPN
Connectivity Options:
Connecting to AWS

8. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud Data center
Devices
Clients
Apps
Direct Connect
Site to Site VPN
Public Internet
Connectivity Options:
Connecting to AWS

9. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud Data center
Devices
Clients
Apps
Direct Connect
Site to Site VPN
Public Internet
What about with NO connection?
Connectivity Options:
Connecting to AWS

10. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Backup
AWS storage portfolio
AWS Storage
Gateway
AWS DataSyncAWS Snow
Family
FILE
Amazon FSxAmazon Elastic
File System
(EFS)
BLOCK
Amazon Elastic Block
Store (Amazon EBS)
AWS Transfer
Family
AWS Resilience Hub
AWS Elastic
Disaster Recovery
OBJECT
Amazon Simple Storage
Service (Amazon S3) and
Amazon S3 Glacier
Data protection & resiliency

11. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Backup
AWS storage portfolio
AWS Storage
Gateway
AWS DataSyncAWS Snow
Family
FILE
Amazon FSx Amazon
EFS
BLOCK
Amazon
EBS
AWS Transfer
Family
AWS Resilience Hub
AWS Elastic
Disaster Recovery
OBJECT
Amazon S3 and
Amazon S3 Glacier
Data protection & resiliency

12. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS DataSync

13. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data transfers are challenging
•Building and deploying scripts
•Encrypting and verifying data
•Recovering from errors
•Handling network availability
•Assuring performance

14. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is AWS DataSync?
Fast data transfer Cost effectiveEasy to use Secure and reliable Fully managed
Online data migration service
that simplifies, automates, and accelerates
copying file and object data to and from AWS storage
•Highly optimized,
parallel network
transfer (up to
100 TB/day)
•Transfers only
incremental changes
•Integrates with
AWS management
and monitoring
services
•Direct transfer
into AWS storage
services
•End-to-end
encryption
•End-to-end data
verification
•VPC endpoints with
AWS PrivateLink
•Schedule
transfers
•Throttle
bandwidth
•Filter by file
name patterns
•$0.0125/GB
transferred
•No minimums

15. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS DataSync use cases
Migrate
application data
to AWS
Replicate
for data protection
and recovery
Archive
on-premises
storage to free
up capacity
Move
data for timely
in-cloud processing

16. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS DataSync architecture
S I M P L I F I E S , A U T O M A T E S , A N D A C C E L E R A T E S D A T A T R A N S F E R T O O R F R O M A W S
On-Premises
AWS Region
DataSync
Service
Shared
file system
AWS DataSyncagent
Storage Services
Management
& Monitoring
deploys on VMware,
Hyper-V or Linux KVM
NFS/SMB/Object
Amazon
S3
Amazon EFS
FSxfor
Windows
File
Server
TLS
AWS KMS
AWS IAM
AWS
CloudTrail
AWS
CloudWatch
AWS
Management
Console

17. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What can you do with AWS DataSync?
Q U I C K L Y , S I M P L Y , A N D S E C U R E L Y M I G R A T E Y O U R D A T A T O A N D F R O M A W S S T O R A G E S E R V I C E S
Amazon EFS
Amazon S3
Amazon FSx
(all file systems)
On premises
NFS, SMB, objectHadoop (HDFS)
AWS DataSync
Amazon S3
on AWS Outposts
Edge
AWS Snowcone
AWS DataSync
Cross-cloud migrations
Google
Cloud Storage
Azure Files
AWS DataSync Amazon EFS
Amazon S3
Amazon FSx
(all file systems)
AWS DataSync
Azure Blob

18. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What can you do with AWS DataSync?
Q U I C K L Y , S I M P L Y , A N D S E C U R E L Y M I G R A T E Y O U R D A T A T O A N D F R O M A W S S T O R A G E S E R V I C E S
Amazon EFS
Amazon S3
Amazon FSx
(all file systems)
On premises
NFS, SMB, objectHadoop (HDFS)
AWS DataSync
Amazon S3
on AWS Outposts
Edge
AWS Snowcone
AWS DataSync
Cross-cloud migrations
Google
Cloud Storage
Azure Files
AWS DataSync Amazon EFS
Amazon S3
Amazon FSx
(all file systems)
AWS DataSync
Azure Blob

19. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What can you do with AWS DataSync?
Q U I C K L Y , S I M P L Y , A N D S E C U R E L Y M I G R A T E Y O U R D A T A T O A N D F R O M A W S S T O R A G E S E R V I C E S
Amazon EFS
Amazon S3
Amazon FSx
(all file systems)
On premises
NFS, SMB, objectHadoop (HDFS)
AWS DataSync
Amazon S3
on AWS Outposts
Edge
AWS Snowcone
AWS DataSync
Cross-cloud migrations
Google
Cloud Storage
Azure Files
AWS DataSync Amazon EFS
Amazon S3
Amazon FSx
(all file systems)
AWS DataSync
Azure Blob

20. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What can you do with AWS DataSync?
Q U I C K L Y , S I M P L Y , A N D S E C U R E L Y M I G R A T E Y O U R D A T A T O A N D F R O M A W S S T O R A G E S E R V I C E S
Amazon EFS
Amazon S3
Amazon FSx
(all file systems)
On premises
NFS, SMB, objectHadoop (HDFS)
AWS DataSync
Amazon S3
on AWS Outposts
Edge
AWS Snowcone
AWS DataSync
Cross-cloud migrations
Google
Cloud Storage
Azure Files
AWS DataSync Amazon EFS
Amazon S3
Amazon FSx
(all file systems)
AWS DataSync
Azure Blob

21. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
On premises and edge AWS Region
AWS
DataSync
Shared file system,
object storage, or
Hadoop cluster
NFS, SMB,
HDFS, S3
API
AWS storage services
Amazon EFS
Amazon S3
DataSync
agent(s)
Amazon S3 on
AWS Outposts
S3 API
DataSync
agent(s)
AWS Snowcone
Amazon FSx
(all file systems)
Other public clouds
Google
Cloud Storage
Azure Files
SMB
S3 API
AWS DataSync: How it works

22. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
On premises and edge AWS Region
AWS
DataSync
Shared file system,
object storage, or
Hadoop cluster
NFS, SMB,
HDFS, S3
API
AWS storage services
Amazon EFS
Amazon S3
DataSync
agent(s)
Amazon S3 on
AWS Outposts
S3 API
DataSync
agent(s)
1
Agents are deployed
to connect to
on-premises
storage, edge
locations, or storage
in other clouds
AWS Snowcone
Amazon FSx
(all file systems)
Other public clouds
Google
Cloud Storage
Azure Files
SMB
S3 API
1
1
1
AWS DataSync: How it works

23. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
On premises and edge AWS Region
AWS
DataSync
Shared file system,
object storage, or
Hadoop cluster
NFS, SMB,
HDFS, S3
API
AWS storage services
Amazon EFS
Amazon S3
DataSync
agent(s)
Amazon S3 on
AWS Outposts
S3 API
DataSync
agent(s)
1
Agents are deployed
to connect to
on-premises
storage, edge
locations, or storage
in other clouds
2
Create location
resources for
source and
destination
storage
AWS Snowcone
Amazon FSx
(all file systems)
Other public clouds
Google
Cloud Storage
Azure Files
SMB
S3 API
1
1
1
2
2
2
AWS DataSync: How it works

24. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
On premises and edge AWS Region
AWS
DataSync
Shared file system,
object storage, or
Hadoop cluster
NFS, SMB,
HDFS, S3
API
AWS storage services
Amazon EFS
Amazon S3
DataSync
agent(s)
Amazon S3 on
AWS Outposts
S3 API
DataSync
agent(s)
1
Agents are deployed
to connect to
on-premises
storage, edge
locations, or storage
in other clouds
2
Create location
resources for
source and
destination
storage
3
AWS DataSync
managed service
connects to AWS
storage and
coordinates the
transfer
AWS Snowcone
Amazon FSx
(all file systems)
Other public clouds
Google
Cloud Storage
Azure Files
SMB
S3 API
1
1
1
2
2
2
3
AWS DataSync: How it works

25. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Storage Gateway

26. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The hybrid cloud model
You have on-premisesdata and applications . . .
. . . that want to use storage and
services in the cloud
AWS
Existing
applications
and data
archives
End users
New data
sources

27. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid cloud storage challenges
On premises In the cloud
I want to run my existing applications without
change (files, backups, archive) . . .
. . . but leverage the infinite scale of the cloud
I need low-latency, on-premises access to
data in the cloud . . .
Compliance requires that I integrate with
enterprise security and management tools . . .
. . . while accessing the power of cloud
compute and analytics
. . . but I want to manage and monitor from a
single pane of glass
I am running out of on-premises storage
capacity . . .
. . . and need to use scalable, durable, and
secure cloud storage

28. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Storage Gateway architecture
AWS Cloud
Storage Gateway
Storage
Gateway
Managed
Service
Application
Server
User
Workstation
Backup
Server
On-premises
Local cache for
low latency access
Optimized
uploads
Deployment options: VMware, Hyper-V,
KVM, Amazon EC2, Hardware Appliance
Storage
Services
Management
& Monitoring
Standard Protocols
Secure
connection
P R O V I D E S O N-P R E M I S E S A C C E S S T O V I R T U A L L Y U N L I M I T E D C L O U D S T O R A G E

29. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Storage Gateway
P R O V I D I N G C L O U D-B A C K E D F I L E S H A R E S , V I R T U A L T A P E S , A N D B L O C K V O LU M E S
Amazon S3
File Gateway
Volume
Gateway
Tape
Gateway
Native file access, SMB
or NFS, to Amazon S3
for backups, archives,
and ingest
for data lakes
Block storage volumes
with snapshots, AWS
Backup integration, and
cloud recovery
Replace physical tape
infrastructure,
leveraging Amazon S3
archive tiers for long-
term retention
Amazon FSx
File Gateway
Replace on-premises
Windows file shares with
efficient, low-latency
Amazon FSx for
Windows File Server
based group file shares
and home directories

30. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Storage Gateway
P R O V I D I N G C L O U D-B A C K E D F I L E S H A R E S , V I R T U A L T A P E S , A N D B L O C K V O LU M E S
Amazon S3
File Gateway
Volume
Gateway
Tape
Gateway
Native file access, SMB
or NFS, to Amazon S3
for backups, archives,
and ingest
for data lakes
Block storage volumes
with snapshots, AWS
Backup integration, and
cloud recovery
Replace physical tape
infrastructure,
leveraging Amazon S3
archive tiers for long-
term retention
Amazon FSx
File Gateway
Replace on-premises
Windows file shares with
efficient, low-latency
Amazon FSx for
Windows File Server
based group file shares
and home directories

31. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Storage Gateway
P R O V I D I N G C L O U D-B A C K E D F I L E S H A R E S , V I R T U A L T A P E S , A N D B L O C K V O LU M E S
Amazon S3
File Gateway
Volume
Gateway
Tape
Gateway
Native file access, SMB
or NFS, to Amazon S3
for backups, archives,
and ingest
for data lakes
Block storage volumes
with snapshots, AWS
Backup integration, and
cloud recovery
Replace physical tape
infrastructure,
leveraging Amazon S3
archive tiers for long-
term retention
Amazon FSx
File Gateway
Replace on-premises
Windows file shares with
efficient, low-latency
Amazon FSx for
Windows File Server
based group file shares
and home directories

32. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Storage Gateway
P R O V I D I N G C L O U D-B A C K E D F I L E S H A R E S , V I R T U A L T A P E S , A N D B L O C K V O LU M E S
Amazon S3
File Gateway
Volume
Gateway
Tape
Gateway
Native file access, SMB
or NFS, to Amazon S3
for backups, archives,
and ingest
for data lakes
Block storage volumes
with snapshots, AWS
Backup integration, and
cloud recovery
Replace physical tape
infrastructure,
leveraging Amazon S3
archive tiers for long-
term retention
Amazon FSx
File Gateway
Replace on-premises
Windows file shares with
efficient, low-latency
Amazon FSx for
Windows File Server
based group file shares
and home directories

33. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Storage Gateway
P R O V I D I N G C L O U D-B A C K E D F I L E S H A R E S , V I R T U A L T A P E S , A N D B L O C K V O LU M E S
Amazon S3
File Gateway
Volume
Gateway
Tape
Gateway
Native file access, SMB
or NFS, to Amazon S3
for backups, archives,
and ingest
for data lakes
Block storage volumes
with snapshots, AWS
Backup integration, and
cloud recovery
Replace physical tape
infrastructure,
leveraging Amazon S3
archive tiers for long-
term retention
Amazon FSx
File Gateway
Replace on-premises
Windows file shares with
efficient, low-latency
Amazon FSx for
Windows File Server
based group file shares
and home directories

34. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Provides security and
supports compliance
FIPS, HIPAA, PCI, SOC*,
ISO**, and encryption
Cost-effective
Pay-as-you-go pricing
AWS integrated
Management, monitoring,
and in-cloud workloads
Standard protocols
No changes to existing apps
Local caching
Low-latency access to
frequently used data
Optimized data transfers
Minimized network traffic
*SOC (1, 2, 3) **ISO (9001, 27001, 27017, 27018)
Storage Gateway benefits
P R O V I D I N G O N-P R E M I S E S A P P L I C A T I O N A C C E S S T O V I R T U A L L Y U N L I M I T E D C L O U D S T O R A G E

35. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo
Amazon S3 File Gateway

36. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo Scenarios –Sophia and Joe
Sophiahas large data repositories stored on NAS Storage presented via NFS
which need to be archived and migrated into AWS, and still available via
NFS on-premises.
Joemanages Backups and has run out of on-premises backup storage –and
has realized the SQL backups are taking up a large portion of their
expensive primary storage.

37. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Region On-premises
S3 File
Gateway
Data Repositories
(NFS)
NFS
Amazon S3
(all storage classes)
Data archive
bucket
Demo set up 1 –S3 File Gateway (NFS)

38. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Region
SQL Backups
On-premises
S3 File
Gateway
SMB
Amazon S3
(all storage classes)
SQL backup
bucket
Demo set up 2 –S3 File Gateway (SMB)

39. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo set up –S3 File Gateway (NFS and SMB)
AWS Region
SQL Backups
On-premises
S3 File
Gateway
SMB
Amazon S3
(all storage classes)
Data Repositories
(NFS)
NFS
Amazon S3
(all storage classes)
SQL backup
bucket
Data archive
bucket

40. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo recap
•Amazon S3 File Gateway was used to provide low latency access to an
applicationsfor on-premises access.
•S3 File Gateway was used to transfer / migrate data from different sources
on premises (NFS and SMB) to Amazon S3
•A seamless way to connect to the cloud (via SMB / NFS) in order to store
backup images as durable objects in Amazon S3 cloud storage.
•Objects written through the S3 File Gateway can be accessed directly in S3.
•A single gateway can connect to multiple buckets.

41. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Common hybrid pattern

42. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Archive data to the cloud
MOVE CLOUD DATA FROM EXPENSIVE ON -PREMISES STORAGE SYSTEMS TO
DURABLE, SECURE LONG -TERM STORAGE
AWS Cloud
NFS/SMB
On premises
Amazon S3
NAS File
Storage
DataSyncAgent
AWS DataSync
Service
AWS Storage Resources
Application
Server(s)
AWS Storage
Gateway
AWS Storage
Gateway Service
Amazon FSxfor
Windows File
Server

43. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
FORMULA 1 uses AWS DataSync to
replicate video archive to Amazon S3
•400 TB video archive that needs
to be backed up regularly while
maintaining high accessibility
•Using AWS DataSync, FORMULA 1
copies up to 4 TB per day to
Amazon S3.
•FORMULA 1 uses AWS Storage
Gateway to provide standard file
access to data in Amazon S3 with
a local cache for low latency
access.
•AWS DataSynctransfers data
quickly and securely to Amazon S3.
•Data is stored durably and reliably
in Amazon S3, and all data is
encrypted at rest.
•AWS Storage Gateway gives users
on-premises access to the video
archive in Amazon S3, providing fast
recovery from failure scenarios.
SolutionChallenge Results
https://aws.amazon.com/blogs/storage/formula-1-transfers-race-car-data-into-the-aws-cloud-using-aws-datasync/

44. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
MartynasJuras
Cloud Engineer, FORMULA 1
AWS DataSync synced data from
our on-premises file server to
Amazon S3 at an average of 4 TB
per day, and did so with encryption
during transfer and at rest.
”
“

45. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Snow Family

46. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is AWS Snow Family?
Move petabytes of data to and from AWS,
or process data at the edge.
AWS
Snowcone
AWS
Snowball
<1TB 10s PB
AWS
Snowcone SSD

47. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Snow Family data migration workflow
1-2
+
days1-2
+
days
1-N* days
1-N days
Job
created &
provisioned
Device
shipped
Device
arrives
1-3 days
Device sent
to AWS
Device arrives
at AWS
Data copied
* variable file transfer speeds | + business days

48. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Recap
•Hybrid Cloud Storage Services
•AWS DataSync
•AWS Storage Gateway
•Snow Family Devices
•Demo of using AWS Storage Gateway for NFS and SMB on-premises

49. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Additional Resources
•AWS DataSync : https://aws.amazon.com/datasync
•AWS Storage Gateway : https://aws.amazon.com/storagegateway
•AWS Snow Family : https://aws.amazon.com/snow
•AWS Storage : https://aws.amazon.com/products/storage/
Get hands-on experience with the AWS online data migration workshop
https://github.com/aws-samples/aws-online-data-migration-workshop

50. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Access 600+ free digital courses with AWS Skill Builder
Focus on the cloud skills and services that are most relevant to
you across 30+ AWS solutions, including digital self-paced
learning plans and ramp-up guides.
AWS TRAINING & CERTIFICATION
LEARN YOUR WAY EXPLORE.SKILLBUILDER.AWS »
Validate your cloud expertise with an AWS Certification
Take the step towards earning an industry-recognisedcredential.
Learn more about how to become an AWS Certified Cloud
Practitioner, and AWS resources that can help you prepare.
ACCESS RESOURCES TO PREPARE FOR YOUR EXAM »

51. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you for attending AWS Builders Online Series
We hope you found it interesting! A kind reminder to complete the survey.
Let us know what you thought of today’s event and how we can improve the event
experience for you in the future.
[email protected]
twitter.com/AWSCloud
facebook.com/AmazonWebServices
youtube.com/user/AmazonWebServices
linkedin.com/company/amazon-web-services
twitch.tv/aws

52. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!
TedyTirtawidjaja
Senior Solutions Architect
Amazon Web Services