Encrypt OS disks with a customer-managed key on Azure Red Hat OpenShift

By default, Microsoft Azure encrypts the OS disks of the virtual machines (VMs) in an Azure Red Hat OpenShift (ARO) cluster by using autogenerated keys. For extra security, you can encrypt the OS disks by using self-managed keys when you deploy an Azure Red Hat OpenShift cluster. This feature gives you more control by encrypting confidential data by using customer-managed keys (CMK).

Clusters created with customer-managed keys have a default storage class enabled with their keys. Therefore, both OS disks and data disks are encrypted by these keys. The customer-managed keys are stored in Azure Key Vault.

For more information about using Azure Key Vault to create and maintain keys, see Azure Key Vault basic concepts in the Microsoft Azure documentation.

By using host-based encryption, the data stored on the virtual machine host of your Azure Red Hat OpenShift node VMs is encrypted at rest and flows encrypted to the Storage service. Host-based encryption means the temp disks are encrypted at rest by using platform-managed keys.

The cache of OS and data disks is encrypted at rest with either platform-managed keys or customer-managed keys, depending on the encryption type set on those disks. By default, when using Azure Red Hat OpenShift, OS and data disks are encrypted at rest with platform-managed keys, meaning that the caches for these disks are also by default encrypted at rest with platform-managed keys.

You can specify your own managed keys by following the encryption steps in the next section. The cache for these disks is also encrypted by using the key that you specify in this step.

Limitation

It's the responsibility of customers to maintain the Key Vault and Disk Encryption Set in Azure. Failure to maintain the keys will result in broken Azure Red Hat OpenShift clusters. The VMs will stop working and, as a result, the entire Azure Red Hat OpenShift cluster will stop functioning.

The Azure Red Hat OpenShift engineering team can't access the keys. Therefore, they can't back up, replicate, or retrieve the keys.

For details about using Disk Encryption Sets to manage your encryption keys, see Server-side encryption of Azure Disk Storage in the Microsoft Azure documentation.

Prerequisites

• Verify your permissions. You must have permissions to create an Azure Red Hat OpenShift cluster, an Azure Key Vault, and an Azure Disk Encryption Set.

• You must have the Key Vault Crypto Officer role for permissions to create a key.

• If you have multiple Azure subscriptions, register the resource providers. For registration details, see Register the resource providers.

• You will need to have the EncryptionAtHost feature enabled on your subscription. You can enable it by running:

  az feature register --namespace Microsoft.Compute --name EncryptionAtHost
  

• You can check the current status of the feature by running:

  az feature show --namespace Microsoft.Compute --name EncryptionAtHost
  

Create an Azure Key Vault instance

You must use an Azure Key Vault instance to store your keys. Create a new Key Vault with purge protection enabled. Then, create a new key within the Key Vault to store your own custom key.

1. Set environment variables:

   export KEYVAULT_NAME=aro-enckv
   export KEYVAULT_KEY_NAME=aro-key
   export DISK_ENCRYPTION_SET_NAME=aro-des
   export LOCATION=<location-name>
   export RESOURCEGROUP=<resource-group-name>
   

2. Create a Key Vault:

   az keyvault create \
       --name $KEYVAULT_NAME \
       --resource-group $RESOURCEGROUP \
       --location $LOCATION \
       --enable-purge-protection true
   

   Note

   Purge protection must be enabled. See Set up your Azure Key Vault.

   Optional: Because purge protection is enabled, the Key Vault name can't be reused immediately after deletion. By adding --retention-days <NUMBER_OF_DAYS>, you can control how long the vault is held in a "soft-delete" state. The default is 90 days, but setting it to a lower value (minimum 7 days) allows for quicker name reclamation.

3. Create a key in the Key Vault:

   az keyvault key create \
       --vault-name $KEYVAULT_NAME \
       --name $KEYVAULT_KEY_NAME \
       --protection software
   

4. Set Key Vault resource ID and key URL variables:

   KEYVAULT_ID=$(az keyvault show --name $KEYVAULT_NAME --query "[id]" -o tsv)
   
   KEYVAULT_KEY_URL=$(az keyvault key show --vault-name $KEYVAULT_NAME --name $KEYVAULT_KEY_NAME --query "[key.kid]" -o tsv)
   

Create an Azure Disk Encryption Set

The Azure Disk Encryption Set is used as the reference point for disks in Azure Red Hat OpenShift clusters. It's connected to the Azure Key Vault that you created in the previous step, and pulls the customer-managed keys from that location.

1. Create the disk encryption set:

   az disk-encryption-set create \
       --name $DISK_ENCRYPTION_SET_NAME \
       --location $LOCATION \
       --resource-group $RESOURCEGROUP \
       --source-vault $KEYVAULT_ID \
       --key-url $KEYVAULT_KEY_URL
   

2. Set disk encryption set resource ID and identity variables:

   DES_ID=$(az disk-encryption-set show --name $DISK_ENCRYPTION_SET_NAME --resource-group $RESOURCEGROUP --query 'id' -o tsv)
   
   DES_IDENTITY=$(az disk-encryption-set show --name $DISK_ENCRYPTION_SET_NAME --resource-group $RESOURCEGROUP --query "[identity.principalId]" -o tsv)
   

Grant permissions for the Disk Encryption Set to access the Key Vault

Grant the Disk Encryption Set’s managed identity access to the Key Vault so it can use the encryption key.

You can assign a built-in role that provides the required key permissions, such as Key Vault Crypto Service Encryption User or Key Vault Crypto User. Alternatively, you can use a custom role, as long as it includes the necessary permissions (wrapkey, unwrapkey, and get) to perform key operations required by Disk Encryption Sets.

az role assignment create \
    --assignee $DES_IDENTITY \
    --role "Key Vault Crypto Service Encryption User" \
    --scope $KEYVAULT_ID

Create an Azure Red Hat OpenShift cluster

You can enable customer-managed keys during cluster creation by specifying the Disk Encryption Set using the --disk-encryption-set parameter. For example, --disk-encryption-set $DES_ID.

To create a cluster with a service principal, follow the instructions in Create an Azure Red Hat OpenShift cluster.

Example cluster creation with Disk Encryption Set parameter:

az aro create \
    --resource-group $RESOURCEGROUP \
    --name $CLUSTER \
    --vnet aro-vnet \
    --master-subnet master-subnet \
    --worker-subnet worker-subnet \
    --disk-encryption-set $DES_ID

To Create an Azure Red Hat OpenShift cluster with managed identities, you must grant the machine-api identity, the disk-csi-driver identity, and the ARO first party service principal Microsoft.Compute/diskEncryptionSets/read permissions for the Disk Encryption Set prior to creating the cluster.

Example role assignments for the machine-api identity, the disk-csi-driver identity, and the ARO first party service principal:

az role assignment create \
  --assignee-object-id "$(az identity show --resource-group $RESOURCEGROUP --name disk-csi-driver --query principalId -o tsv)" \
  --assignee-principal-type ServicePrincipal \
  --role <ROLE> \
  --scope $DES_ID"

az role assignment create \
  --assignee-object-id "$(az identity show --resource-group $RESOURCEGROUP --name machine-api --query principalId -o tsv)" \
  --assignee-principal-type ServicePrincipal \
  --role <ROLE> \
  --scope $DES_ID

az role assignment create \
  --assignee-object-id "$(az ad sp list --display-name "Azure Red Hat OpenShift RP" --query '[0].id' -o tsv)" \
  --assignee-principal-type ServicePrincipal \
  --role <ROLE> \
  --scope $DES_ID

After you create the Azure Red Hat OpenShift cluster, all VMs are encrypted with the customer-managed encryption keys.

To verify that you configured the keys correctly, run the following commands:

1. Get the name of the cluster Resource Group where the cluster's VMs, disks, and other infrastructure are located:

   CLUSTERRESOURCEGROUP=$(az aro show --resource-group $RESOURCEGROUP --name $CLUSTER --query 'clusterProfile.resourceGroupId' -o tsv | cut -d '/' -f 5)
   

2. Check that the disks have the correct Disk Encryption Set attached:

   az disk list --resource-group $CLUSTERRESOURCEGROUP --query '[].encryption'
   

   The field diskEncryptionSetId in the output must point to the Disk Encryption Set that you specified while creating the Azure Red Hat OpenShift cluster.