Mitch Murphy

Mitch Murphy

Improving humanity by teaching machines how to learn.

Adding Persistent Storage to RKE2 with Longhorn

Title: Adding Persistent Storage to RKE2 with Longhorn
Author: Mitch Murphy
Date: 2023-12-27

Introduction

What is Longhorn? Longhorn refers to a project that provides persistent storage for Kubernetes. Longhorn aims to simplify and enhance the management of persistent storage in Kubernetes clusters. It is an open-source distributed block storage system that can be deployed as a containerized application.

Some key features of Longhorn include:

  1. Distributed Storage: Longhorn distributes block storage across multiple nodes in a Kubernetes cluster, providing redundancy and fault tolerance.
  2. Snapshot and Backup: It supports snapshots, allowing you to capture the state of your persistent volumes at a specific point in time. Additionally, Longhorn includes features for backup and restore operations.
  3. Replication and Synchronization: Longhorn replicates data across nodes to ensure data integrity and provides mechanisms for data synchronization.
  4. User Interface: Longhorn includes a user-friendly web-based UI that allows users to manage and monitor their storage resources easily.
  5. Kubernetes Native: Longhorn is designed to work seamlessly with Kubernetes, integrating into the Kubernetes control plane and leveraging custom resources to define and manage persistent volumes.

It’s worth noting that the landscape of Kubernetes storage solutions may evolve, and new developments may have occurred since my last update. Therefore, I recommend checking the official Longhorn project documentation or other reliable sources for the latest information on Longhorn and its features.

Prerequisites

Before you begin this guide, you’ll need the following:

  1. A Kubernetes cluster running RKE2
  2. Flux CD installed on your cluster
  3. A GitHub repository to store your cluster configuration
  4. Storage provisioned, mounted and formatted on each node in your cluster
    • For this guide, I will be attaching a Virtual Hard Disk to each node/virtual machine in Hyper-V, mounted at /mnt/storage
    • After attaching the VHD to each node, you will need to format the disk with a filesystem. For this guide, I will be using ext4
    • Stay tuned as I will be writing a guide on how to automate this process using Ansible

Install Longhorn

We will be using Flux CD to install Longhorn on our cluster. If you haven’t already, you will need to install Flux CD on your cluster. You can find a guide on how to do this here. Once you have Flux CD installed, you will need to create a HelmRepository resource to point to the Longhorn Helm chart as well as a HelmRelease resource to install Longhorn on your cluster. Please see the structure in this repo

Configure Longhorn

Virtual Hard Disk

First, we need to attach a Virtual Hard Disk to each node/virtual machine in Hyper-V. This will be used as the storage for Longhorn. To do this, you will need to:

  1. Create a Virtual Hard Disk in Hyper-V (follow this guide)
  2. Attach it to each node/virtual machine in your cluster
  3. Format/partition the disk with a filesystem. For this guide, I will be using ext4
  4. Mount the disk to /mnt/storage on each node/virtual machine in your cluster. Make sure to add an entry to /etc/fstab so that the disk is mounted on boot
    • echo "/dev/sdb1 /mnt/storage ext4 defaults 0 0" >> /etc/fstab
  5. You can find a guide on how to do this here.

As we will be mounting our storage to /mnt/storage you must set the defaultSettings.defaultDataPath to /mnt/storage in the HelmRelease resource. This will ensure that Longhorn will use the storage we have provisioned on each node in our cluster.

Note: that this will create a default StorageClass on our Kubernetes cluster. This means that you do not need to specify the StorageClass when you create PersistentVolumeClaims, as the default StorageClass will be used to provision the PersistentVolume.

Conclusion

In this guide, we have installed Longhorn on our RKE2 cluster using Flux CD. We have also configured Longhorn to use the storage we have provisioned on each node in our cluster. This means that we can now create PersistentVolumeClaims without specifying the StorageClass as the default StorageClass will be used to provision the PersistentVolume.

You may also enjoy