XenonStack

Enterprise Digital Platform

XenonStack Insights Rook Storage: Ceph on Kubernetes Explained

| Published | Updated

Rook storage on Kubernetes: Rook operator, Ceph cluster, block, file and object storage, persistent volumes for pods

Key takeaway

  • Rook is an open-source storage orchestrator that runs Ceph on Kubernetes. See how it works, how to set it up, its best practices and when to use it.

Rook is an open-source storage orchestrator that runs Ceph inside Kubernetes, so block, file and object storage can be deployed and managed like any other cluster workload. It is a CNCF graduated project, and it is built to give stateful containers persistent storage without a separate storage platform.

This guide is for platform and infrastructure teams running stateful workloads on Kubernetes. It covers what Rook does, how it works with Ceph, the order of a typical setup, operating practices, and where it fits beside other cloud native storage solutions and software-defined data centre designs.

What is Rook storage?

The Rook documentation describes Rook as an open-source cloud-native storage orchestrator that provides the framework for Ceph storage to run in Kubernetes. The project is hosted by the Cloud Native Computing Foundation at the graduated level, and the docs record that the Ceph operator was declared stable in December 2018 with Rook v0.9.

Two-column diagram. Rook operator: watch resources, deploy daemons, provision storage, scale and upgrade. Ceph storage: block, file, object, monitoring.

Ceph, in the same documentation, is "a distributed storage system that provides file, block and object storage". Rook does not replace Ceph. It deploys, configures and watches Ceph so that Kubernetes does the day-to-day work. This is what people mean by software-defined storage: the storage software is separated from the hardware it runs on and managed through code and APIs.

How does Rook work with Kubernetes and Ceph?

Rook is built around the operator pattern. According to its documentation, the operator builds on Kubernetes resources to deploy, configure, provision, scale, upgrade and monitor Ceph, with the aim of storage that is self-managing, self-scaling and self-healing.

You describe the storage you want as Kubernetes resources. The operator then starts the Ceph daemons as containers: monitors, managers, OSDs that manage the disks, and, where you ask for them, metadata servers for file storage and object gateways for object storage. Applications request volumes through storage classes and persistent volume claims, the same way they would with any other provisioner.

Which storage types does Rook provide?

Storage typeCeph componentWhat applications use it for
BlockRBDVolumes mounted by a single pod, such as databases
Shared file systemCephFSA file system several pods can mount at once
ObjectRGW (object gateway)Object storage reached over an API from inside or outside the cluster

The Rook site lists these three as separate sections. Where an application needs fast storage, you can build pools on faster disks and expose them through their own storage class; slower disks can back bulk data and backups. Check the current docs for how to define pools and classes for your release.

How do you set up Rook on Kubernetes?

The order of work is stable even though commands change between releases. Always follow the quickstart for the Rook version you install.

  1. Build or choose a Kubernetes cluster with spare raw disks or partitions on the nodes that will host storage.
  2. Deploy the Rook operator.
  3. Create the Ceph cluster resource, and wait for the daemons to become healthy.
  4. Create a storage class for block storage, and test it with a small persistent volume claim.
  5. Create a shared file system or object store if you need them, and test each one.
  6. Point real workloads at the storage classes once the tests pass.

Many walkthroughs, including older XenonStack guidance, were written against Ubuntu 22.04, Docker 20.10, RKE 1.3.14, Kubernetes 1.24 and Helm 3.11. Those versions are dated, and early minimum requirements such as Kubernetes 1.8 are no longer a reliable guide. Check the Rook documentation for the Kubernetes versions supported by the release you choose.

What are the best practices for running Rook?

  • Monitor cluster health: use the Ceph dashboard and the metrics Rook exposes to Prometheus.
  • Keep a debugging toolbox: Rook provides a toolbox container with Ceph client tools for troubleshooting.
  • Separate pools by performance need: do not put latency-sensitive databases and backups on the same disks.
  • Plan capacity and failure domains: adding disks or nodes grows the cluster, but replication needs enough independent hosts.
  • Practise upgrades and recovery: rehearse them in a test cluster before production.
  • Choose the deployment style deliberately: running storage on the same nodes as applications uses resources well, while dedicated storage nodes isolate them.

How does Rook compare with other storage approaches?

ApproachWhere storage runsFit
Rook with CephInside the Kubernetes cluster, managed by an operatorTeams that want one cluster-native platform for block, file and object storage
Cloud provider volumesIn the provider's managed storage serviceTeams that prefer to hand storage operations to the cloud provider
Standalone object store such as MinIOSeparate service for object data onlyTeams that mainly need object storage, see our block and object storage guide

If you are weighing hyper-converged designs, read our guide to hyper-converged infrastructure.

How do AI agents change storage operations?

Agents bring data needs to storage: model checkpoints, vector indexes, logs and working files all need durable volumes. They can also watch the storage layer, flagging a degraded pool or a full disk and drafting the fix. Both uses make access and retention rules important: someone should decide which agent may provision, resize or delete volumes. For the data and governance side of agent workloads, see ElixirData, XenonStack's Context OS.

What should you do first?

  1. List the stateful workloads that need persistent storage and which type each needs.
  2. Check the Rook release notes against your Kubernetes version.
  3. Stand up a small test cluster and run the quickstart end to end.
  4. Define storage classes by performance need, then test failure and recovery.
  5. Wire up monitoring before real data lands on it.

Related reading: why cloud native applications, cloud-native network traffic management, cloud native architecture design and patterns, cloud native DevOps and our cloud native development services.

To map one decision your agents make, and the data and policy it depends on, book a Context Workshop. See what happens in the workshop.

Frequently asked questions

What is Rook in Kubernetes?

Rook is an open-source, CNCF graduated storage orchestrator that runs Ceph in Kubernetes, automating deployment, configuration, scaling, upgrades and monitoring of the storage cluster.

What is the difference between Rook and Ceph?

Ceph is the distributed storage system that provides file, block and object storage. Rook is the orchestrator that deploys and manages Ceph on Kubernetes.

Does Rook support block, file and object storage?

Yes, through Ceph: block storage with RBD, shared file systems with CephFS and object storage with the RGW object gateway.

Which Kubernetes version does Rook need?

It depends on the Rook release. Check the documentation for the release you plan to install, because older minimum versions are out of date.

Is Rook only for Ceph?

The current getting-started documentation covers Ceph as the storage provider, so plan around Ceph when you adopt Rook.

Ready to put AI agents to work? See an ElixirClaw demo: the agentic OS to build, orchestrate and govern AI agents across your enterprise systems.

Table of Contents

navdeep-singh-gill

Navdeep Singh Gill

Global CEO and Founder of XenonStack

LinkedIn profile

Navdeep Singh Gill is serving as Chief Executive Officer and Product Architect at XenonStack. He holds expertise in building SaaS Platform for Decentralised Big Data management and Governance, AI Marketplace for Operationalising and Scaling. His incredible experience in AI Technologies and Big Data Engineering thrills him to write about different use cases and its approach to solutions.

Get the latest articles in your inbox

Subscribe Now

Interested in Solving your Challenges with XenonStack Team
Get Started

Get Started with your requirements and primary focus, that will help us to make your solution

Please Select your Industry
Banking
Fintech
Payment Providers
Wealth Management
Discrete Manufacturing
Semiconductor
Machinery Manufacturing / Automation
Appliances / Electrical / Electronics
Elevator Manufacturing
Defense & Space Manufacturing
Computers & Electronics / Industrial Machinery
Motor Vehicle Manufacturing
Food and Beverages
Distillery & Wines
Beverages
Shipping
Logistics
Mobility (EV / Public Transport)
Energy & Utilities
Hospitality
Digital Gaming Platforms
SportsTech with AI
Public Safety - Explosives
Public Safety - Firefighting
Public Safety - Surveillance
Public Safety - Others
Media Platforms
City Operations
Airlines & Aviation
Defense Warfare & Drones
Robotics Engineering
Drones Manufacturing
AI Labs for Colleges
AI MSP / Quantum / AGI Institutes
Retail Apparel and Fashion

Proceed Next