What is Amazon File Cache?
Amazon File Cache is a managed high-speed cache that consolidates data from various storage sources into a unified namespace. The cache is based on Lustre technology and provides sub-millisecond latencies and hundreds of GB/s throughput for workloads that need fast access to distributed data.
The service supports on-premises NFS file systems and Amazon S3 as data sources. Data is automatically loaded into the cache on first access and is subsequently available at full Lustre performance.
Core Features
- Multi-Source Cache: Unification of data from on-premises NFS and S3 into a single namespace
- Lustre-Based Performance: Sub-millisecond latencies and hundreds of GB/s throughput
- Automatic Caching: Data is transparently loaded from sources on first access
- POSIX-Compatible: Standard file system interface for existing Linux applications
- Temporary Operation: Optimized for time-limited workloads without long-term storage
Typical Use Cases
Cloud Bursting: Organizations process data residing in on-premises NFS systems using AWS compute resources. File Cache provides local performance for remote data and eliminates the need to copy data entirely to AWS beforehand.
HPC Data Processing: Scientific computations and simulations access data from various sources. File Cache unifies access and delivers the performance required for HPC.
Batch Processing: Time-limited processing jobs create a temporary File Cache, process data at high performance, and remove the cache after completion. This minimizes storage costs.
Benefits
- Unified access to distributed data sources
- Lustre performance without permanent storage overhead
- Transparent caching without application changes
- Cost-effective for temporary and recurring workloads
Integration with innFactory
As an AWS Reseller, innFactory supports you with Amazon File Cache: hybrid storage architecture, cache configuration for HPC workloads, integration with on-premises NFS systems, and performance optimization.
Typical Use Cases
Frequently Asked Questions
What is Amazon File Cache?
Amazon File Cache is a high-speed cache that consolidates data from various sources (on-premises NFS, S3) into a unified, high-performance file system cache in AWS. The cache is based on Lustre technology.
How does the cache work?
File Cache creates a temporary, high-performance Lustre-based cache in AWS. Data is automatically loaded from configured sources on first access and cached locally. The cache provides sub-millisecond latencies for subsequent accesses.
When should I use File Cache instead of FSx for Lustre?
File Cache is ideal for temporary workloads that need data from multiple sources simultaneously, especially when data resides in on-premises NFS systems. FSx for Lustre is better suited for persistent workloads with S3 integration.