Here is a number that should bother anyone running AI workloads at scale: large-model training jobs burn 20% to 40% of their accelerator time doing nothing. Not training. Not inferencing. Waiting for model weights to load, checkpoints to write, and training data to arrive from storage.
At $3 to $10 per GPU hour, idle time adds up fast. It is a storage problem that shows up on the GPU bill.
What Google Built for This
Hyperdisk ML is a storage type on Google Cloud designed specifically for loading large model weights onto accelerators. The key feature is multi-attach: a single storage volume can connect simultaneously to up to 2,500 GPU or TPU instances in read-only mode. Instead of copying model weights to every inference replica separately, you provision one volume and mount it across your entire fleet. A 70-billion-parameter model deployment that previously required hundreds of separate copies now requires one. The storage cost reduction is direct.
Rapid Storage addresses the training side. It delivers sub-millisecond read latency and loads training data roughly 20x faster than standard cloud object storage. Co-located with accelerator pods, it keeps GPUs and TPUs fed with data instead of idle while they wait. InstaDeep used Google Cloud TPUs and high-performance storage to train genomics models with over 20 billion parameters, achieving near-linear scaling across multi-host configurations where traditional storage would have created a bottleneck. The pattern is consistent: storage is usually the constraint, and Rapid Storage removes it.
Neither AWS nor Azure offers a managed equivalent to Hyperdisk ML’s multi-attach read-only semantics for inference fleet weight sharing. AWS has high-throughput HPC storage options, but they require manually provisioning and managing distributed file systems that have nothing to do with your AI roadmap. The managed, purpose-built layer is the differentiator.
Why This Matters to ISVs
If you are building on Google Cloud, the impact lands in two places.
Internally, if your team is training or fine-tuning models, Rapid Storage is the difference between your accelerators waiting on data and actually training. The infrastructure cost does not change. The output per dollar spent does.
On the product side, Hyperdisk ML multi-attach enables something that was previously hard to deliver: fast model rollouts across large inference fleets. An ISV building a model-serving or MLOps platform can use this to offer customers a real SLA on model deployment speed, backed by storage architecture rather than aspirational engineering. That is a differentiator you can put in a sales deck.
The questions worth asking your team: What is your current accelerator utilization during training runs? How long does it take to cold-start your inference fleet after a model update? How many separate copies of your model weights are you maintaining today? If none of those answers are already optimal, this is worth a closer look.
Want to go deeper?
- Google Cloud AI Hypercomputer Blog, Hyperdisk ML specs, multi-attach architecture, and Rapid Storage announcement.
- Hyperdisk ML Official Documentation, Multi-attach limits and model weight loading performance details.
- Rapid Storage Overview, Sub-millisecond object storage for AI training workloads co-located with accelerator pods.