What is Parallelstore?
Parallelstore is a managed, high-performance parallel file system from Google Cloud. It targets high performance computing as well as AI and ML workloads that require high IOPS and high throughput at low latency. The parallel file system enables many clients to access the same data concurrently.
Note on product status: Access to Parallelstore currently requires allowlisting after contacting Google Cloud sales. Google is also positioning Google Cloud Managed Lustre as its new first-party Lustre file system for HPC and AI workloads. Based on current information, a phase-out of Parallelstore in favor of Managed Lustre has been announced or is foreseeable. Anyone planning a new storage architecture for parallel workloads should evaluate Managed Lustre as an alternative and check the current status with innFactory or directly with Google Cloud.
Conventional storage options often become a bottleneck for data-intensive training and simulations because GPUs and CPUs wait for data. Parallelstore addresses this with high aggregate throughput and low latency. The system is built on local SSDs with erasure coding and is therefore designed as fast scratch storage for temporary workloads, not as durable primary storage.
Core Features
- High throughput and IOPS: According to the provider, performance scales per TiB, with several GiBps of throughput and tens of thousands of IOPS depending on provisioned capacity.
- Low latency and concurrency: Support for several thousand concurrent client processes with low latency for small accesses.
- POSIX compliance and integration: The file system is POSIX-compliant and mounts to Compute Engine VMs and Google Kubernetes Engine (GKE) through a CSI driver.
- Fast Cloud Storage transfer: High-throughput batch data transfer to and from Cloud Storage.
Typical Use Cases
AI and ML training: Training large models requires delivering extensive datasets quickly and in parallel to many accelerators. Parallelstore reduces GPU idle time and thereby shortens training duration.
High performance computing: Simulations and scientific computations need concurrent access from many nodes to shared data. The parallel file system provides the aggregate throughput required at low latency.
Scratch storage for batch jobs: Compute-intensive pipelines use Parallelstore as fast intermediate storage. Data can be loaded from Cloud Storage at high speed, processed, and results written back.
Benefits
- High throughput and IOPS for data-intensive HPC and AI/ML workloads
- Shorter training and computation times through reduced accelerator idle time
- Integration with Compute Engine and GKE via a CSI driver
- Availability in EU regions for data residency requirements
Integration with innFactory
As a certified Google Cloud partner, innFactory supports you in evaluating Parallelstore against alternatives such as Managed Lustre, and with the adoption and operation of parallel file systems for HPC and AI workloads.
Typical Use Cases
Frequently Asked Questions
What is Parallelstore?
Parallelstore is a managed, high-performance parallel file system from Google Cloud for high performance computing as well as AI and ML workloads that need high throughput and low latency. The system is POSIX-compliant and supports concurrent multi-client access. Access currently requires allowlisting after contacting Google Cloud sales.
Is Parallelstore being deprecated?
Based on current information, Google plans to phase out Parallelstore in favor of Google Cloud Managed Lustre, its new first-party Lustre file system. For new projects, evaluating Managed Lustre as the successor is recommended; existing users should check Google Cloud's official announcement and migration timeline.
When should I use Parallelstore?
Parallelstore fits AI/ML training and inference, HPC simulations, and compute-intensive batch jobs where many clients access the same data concurrently. Because the system runs on local SSDs with erasure coding, it is designed for temporary scratch data rather than durable primary storage.
How much does Parallelstore cost?
Parallelstore is billed on provisioned capacity rather than used storage. You can find the exact per-region rates on the official Google Cloud pricing page.
What throughput and capacity does Parallelstore offer?
According to the provider, usable capacity ranges from 12 TiB to 100 TiB, with performance scaling per TiB (around 1.15 GiBps read and 0.5 GiBps write throughput, roughly 30,000 read and 10,000 write IOPS). It supports up to 4,000 concurrent client processes. See official documentation for current figures.
Note: All product information on this page has been compiled with care, but is provided without guarantee and may be outdated or incomplete. Cloud services evolve rapidly — features, pricing, SLAs, and availability change frequently. Authoritative and up-to-date information can only be found on the official product page of Google Cloud (official documentation). This page does not represent an offer by Google Cloud.
