Charmed Ceph manual install | Ubuntu

Your submission was sent successfully! Close
Thank you for contacting us. A member of our team will be in touch shortly. Close
You have successfully unsubscribed! Close
Thank you for signing up for our newsletter! In these regular emails you will find the latest updates about Ubuntu and upcoming events where you can meet our team.Close
Your preferences have been successfully updated. Close notification
Please try again or file a bug report. Close
The guide shows how to perform a general install of Charmed Ceph. It will provide detail on the fundamental concepts and show how a more customised Ceph cluster can be achieved. Ensure that the base requirements have been met.
The Ceph cluster will have three Monitors and six OSDs. The OSDs will be provided by three storage nodes, with two OSDs hosted per node (backed by devices /dev/sdb and /dev/sdc).
A Monitor will be containerised on each of the storage nodes. This means that you will require three machines for the Ceph cluster. One additional machine will be needed for the Juju controller. The MAAS cluster must therefore consist of at least four machines.
The MAAS nodes will be running Ubuntu 22.04 LTS (Jammy) as will any LXD containers created during the deployment. The use of Jammy will give us the opportunity to include the UCA in our configuration. Ceph Reef will be deployed.
Ensure that you have a host with Juju installed and that it has network connectivity to the MAAS cluster. You must also have a user created on the MAAS cluster.
Inform Juju about the remote MAAS cluster and provide credentials for accessing it. Here the MAAS cluster will be added to Juju as a cloud called ‘my-maas’.
We’ll then create a Juju controller management node as well as a Juju model to house the Ceph cluster. The controller will be called ‘my-controller’ and the model will be called ‘ceph’:
Please see the relevant Juju documentation if you need help with carrying out the above commands.
In this deployment charm configuration options will be placed in a YAML file as opposed to stating them directly on the command line. File ceph.yaml will contain the options for all the charms we will use:
The meaning of the above options are explained below, categorised by charm.
customize-failure-domain This option determines how a Ceph CRUSH map is configured. A value of ‘false’ (the default) will lead to a map that will replicate data across hosts (implemented as Ceph bucket type ‘host’). With a value of ‘true’ all MAAS-defined zones will be used to generate a map that will replicate data across Ceph availability zones (implemented as bucket type ‘rack’). This option is also supported by the ceph-osd charm, and its value must be the same for both charms.
expected-osd-count This option states the number of OSDs expected to be deployed in the cluster. This value can influence the number of placement groups (PGs) to use per pool. The PG calculation is based either on the actual number of OSDs or this option’s value, whichever is greater. The default value is ‘0’, which tells the charm to only consider the actual number of OSDs.
monitor-count This option gives the number of ceph-mon units in the monitor cluster (where one ceph-mon unit represents one MON). The default value is ‘3’ and is generally a good choice. For other monitor counts, it is good practice to set this explicitly to avoid a possible race condition during the formation of the cluster. The capacity for fault tolerance is based upon the preservation of quorum (majority of the original MON cluster). For example, a count of five can tolerate two failures (quorum: ⅗), a count of four can tolerate one failure (quorum: ¾), and a count of three can also tolerate one failure (quorum: ⅔).
source This option states the software sources. A common value is an OpenStack UCA release (e.g. ‘cloud:jammy-bobcat’). See Ceph and the UCA. The underlying host’s existing apt sources will be used if this option is not specified (this behaviour can be explicitly chosen by using the value of ‘distro’). This option is also supported by the ceph-osd charm, and its value must be the same for both charms.
customize-failure-domain This option’s description is the same as that of the identically-named ceph-mon option. Each charm’s option must have the same value.
osd-devices This option lists what block devices can be used for OSDs across the cluster. This list may affect newly added ceph-osd units as well as existing units (the option may be modified after units have been added). The charm will attempt to activate as Ceph storage any listed device that is visible by the unit’s underlying machine.
source This option’s description is the same as that of the identically-named ceph-mon option. Each charm’s option must have the same value.
In particular, we see that the OpenStack 2023.1 (code named Antelope) on Jammy has been chosen as the software source for Ceph. From Ceph and the UCA it can be deduced that the resulting Ceph version will be Reef.
Ceph storage is natively highly available. This means that, by design, data objects are stored in such a way that data resiliency is ensured in the advent of an OSD failure. By default the ceph-osd charm will maintain three replicas of each storage object across the cluster.
The storage of these objects is organised on a per pool basis, where each pool is of one of two types:
For a detailed explanation of these two pool types see section Pool types in this guide.
The actual deployment of Ceph is straightforward:
As planned, a containerised Monitor is placed on each storage node. We’ve assumed that the machines spawned in the first command are assigned the IDs of 0, 1, and 2. The latter is the default behaviour (i.e. a model’s first machine will have an ID of 0).
The output to the juju status command should look similar to this:
Above we see the three Monitors and the six OSDs (two per storage node). It should also be clear that each Monitor is containerised (e.g. 1/lxd/0 indicates “LXD machine 0 on m