Provide simple, scalable, fully managed NFS file system for use with AWS Services and on-premises resources.
Build to scale on-demand to petabytes without interrupting applications. Eliminate to manage provision the storage, it is automatically scale your storage as needed.
EFS has 2 types: EFS standard and EFS Infrequent Access (IA).
EFS has lifecycle management (like S3 lifecycle manage) to help move files into EFS IA automatically.
EFS IA is cheaper file system.
Shared access to thousands of Amazon EC2 instances, enabling high level of aggregate throughput and IOPS with consistent low latencies.
Common use cases: Big data analytics, web serving and content management, application development & testing, database backups, containers storage…
EFS is Regional service storing data within and cross Available Zone for high availability and durability.
Amazon EC2 instances can access cross Available Zone, On-premises resources can access EFS via AWS DX and AWS VPN.
EFS can support over 10GB/s, more than 500,000 IOPS.
Using EFS Lifecycle management can reduce cost up to 92%.
Amazon EFS is compatible with all Linux-based AMIs for Amazon EC2.
You do not need to manage storage procurement and provisioning. EFS will grow and shrink automatically as you add or remove files.
AWS DataSync provides fast and secure way to sync existing file system to Amazon EFS, even from on-premise over any network connection, including AWS Direct Connect or AWS VPN.
Moving files to EFS IA by enabling Lifecycle management and choose age-off policy.
Files smaller than 128KB will remain on EFS standard storage, will not move to EFS IA even it is enabled.
Speed: EFS Standard storage is single-digit latencies, EFS IA storage is double-digit latencies.
50MB/s baseline performance per TB of storage.
100MB/s burst for 1TB
More than 1TB stored, storage can burst 100MB/s per TB.
Can have Amazon EFS Provisioned Throughput to provide higher throughput.
EFS’s objects are redundantly across Available Zone.
You can use AWS Backup to incremental backup your EFS.
Access to EFS:
Amazon EC2 instances inside VPC: access directly
Amazon EC2 Classic instance: via ClassicLink
Amazon EC2 instances in other VPCs: using VPC Peering Connection or VPC Transit Gateway.
EFS can store petabytes of storage. With EFS, you don’t need to provision in advance, EFS will automatically grow and shrink as files added or removed from the storage.
Mount EFS via NFS v4
EFS Access points to simplify application access to shared datasets on EFS. EFS Access points can work with AWS IAM to enforce an user or group, and a directory for every file system request made through the access point.
You can create multiple access points and provide to some specific applications.
EFS support encryption in transit and at rest.
You can configure the encryption at rest when creating EFS via console, api or CLI.
Encrypting your data is minimal effect on I/O latency and throughput.
EFS and On-premise access
To access EFS from on-premise, you have to have AWS DX or AWS VPN.
Standard tools like GNU to allow you to copy data from on-premise parallel. It can help faster copy. https://www.gnu.org/software/parallel/
Amazon FSx Windows workload
Window file server for Windows based application such as: CRM, ERP, .NET…
Backed by Native Windows file system.
Build on SSD storage.
Can access by thousands of Amazon EC2 at the same time, also provide connectivity to on-premise data center via AWS VPN or AWS DX.
Support multiple access from VPC, Available Zone, regions using VPC Peering and AWS Transit gateway.
High level throughput & sub-millisecond latency.
Amazon FSx for Windows File Server support: SMB, Windows NFS, Active Directory (AD) Integration, Distributed File System (DFS)
Amazon FSx can also mount to Amazon EC2 Linux based instances.
Amazon FSx for Luster
Fully managed file system that is optimized for HPC (high performance computing), machine learning, and media processing workloads.
Hundreds of GB per second of throughput at sub-millisecond latencies.
Can be integrated with Amazon S3, so you can join long-term datasets with a high performance system. Data can be automatically copied to and from Amazon S3 to Amazon FSx for Luster.
Amazon FSx for Lustra is POSIX-compliant, you can use your current Linux-based applications without having to make any changes.
Support read-after-write consistency and support File locking.
Amazon Luster can also be mounted to an Amazon EC2 instance.
Connect to onpremise via AWS DX, or AWS VPN.
EFS Data transfers between Region using AWS DataSync
EFS Data transfer within Region using AWS Transfer Family endpoint