Google Launches Three New Cloud Storage Options: Cloud Storage FUSE, Parallelstore, NetApp Volumes

InfoQ Homepage News Google Launches Three New Cloud Storage Options: Cloud Storage FUSE, Parallelstore, NetApp Volumes

Aug 29, 2023 2 min read

by

Steef-Jan Wiggers

Google recently launched three new cloud storage options: Cloud Storage FUSE for Artificial Intelligence (AI) applications that require file system semantics, a parallel file system Parallelstore for demanding AI and HPC applications that use GPUs, and NetApp Volumes for enterprise applications running in the cloud.

Cloud Storage FUSE was already available as an open-source version, allowing objects in Cloud Storage buckets to be accessed as files mounted as a local file system. The company enhanced it with new levels of portability, reliability, performance, and integration targeting AI Workloads. In a Google blog post, the authors Marco Abela, product manager, and Akshay Ram, senior product manager at Google, explain:

The new Cloud Storage FUSE is particularly important for AI workloads. Because applications can access data directly (rather than downloading it locally), there’s no custom logic to implement and less idle time for valuable resources like TPUs and GPUs while the data is copied over. Further, a new Cloud Storage FUSE CSI driver for Google Kubernetes Engine (GKE) allows applications to mount Cloud Storage using familiar Kubernetes API, and it’s offered as a turn-key deployment managed by GKE.

Overview of Cloud Storage FUSE (Source: Google blog post)

J.W. Davis, a Cloud evangelist, commented on the availability of FUSE file systems in GKE in a tweet:

This seems destined for many disasters. Useful for a very narrow range of use cases; most will misuse it and suffer.

In addition to Cloud Storage FUSE for AI applications, the company announces a private preview of a parallel file system called Parallelstore that helps users stop wasting precious GPU resources while at the same time, they wait for storage I/O by providing a high-performing parallel file storage solution for AI/ML and HPC workloads. The solution is based on the next-generation Intel Distributed Asynchronous Object Storage (DAOS) architecture.

Sameet Agarwal, VP/GM, Storage, and Sean Derrington, group product manager, Storage, explain in a Google blog post on the new cloud storage options:

Based on the next-generation Intel DAOS architecture, all compute nodes in a Parallelstore environment have equal access to storage, so VMs can get immediate access to their data. With up to 6.3x read throughput performance compared to competitive Lustre Scratch offerings., Parallelstore is well suited for cloud-based applications that require extremely high performance (IOPS and throughput) and ultra-low latency.

Finally, the third new cloud storage option is NetApp Volumes, a fully Google-managed, high-performance file storage service. This storage option is designed for enterprises that want to migrate their applications on top of NetApp storage arrays on-premises to the cloud. The service provides the capability to increase volumes from 100GiB to 100TiB for maximum scalability, implement ONTAP data management for hybrid workloads, and run either Windows or Linux applications as virtual machines without refactoring.

When asked by InfoQ about what is driving this investment from Google, here is what Derrington had to say:

As AI has become instrumental in automating data management, organizations are turning to the cloud to deliver the right storage solution for the right application. With Google Cloud’s new AI-optimized storage offerings, Cloud Storage FUSE and Parallelstore, we’re helping customers adapt to complex AI workloads by offering tailored storage solutions that simplify operations, unlock innovation, reduce costs, and more.

Cloud Storage FUSE and NetApp Volumes are available through the Google Cloud Console, while the Parallel Store is through a Google account manager.

The true promise of the cloud with ease, not cost. DoiT provides technology and cloud expertise to reduce cloud costs and boost engineer productivity. All from an AWS, Microsoft and Google Cloud Partner. Learn More.

Writing for InfoQ has opened many doors and increased career opportunities for me. I was able to deeply engage with experts and thought leaders to learn more about the topics I covered. And I can also disseminate my learnings to the wider tech community and understand how the technologies are used in the real world.

I discovered InfoQ’s contributor program earlier this year and have enjoyed it since then! In addition to providing me with a platform to share learning with a global community of software developers, InfoQ’s peer-to-peer review system has significantly improved my writing. If you’re searching for a place to share your software expertise, start contributing to InfoQ.

I started writing news for the InfoQ .NET queue as a way of keeping up to date with technology, but I got so much more out of it. I met knowledgeable people, got global visibility, and improved my writing skills.

Becoming an editor for InfoQ was one of the best decisions of my career. It has challenged me and helped me grow in so many ways. We'd love to have more people join our team.

InfoQ seeks a full-time Editor-in-Chief to join C4Media's international, always remote team. Join us to cover the most innovative technologies of our time, collaborate with the world's brightest software practitioners, and help more than 1.6 million dev teams adopt new technologies and practices that push the boundaries of what software and teams can deliver!

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

We protect your privacy.

You need to Register an InfoQ account or Login or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p