In Part 1 of this series, we looked at ways to leverage Azure Backup Soft Delete, Backup Alert Notifications and Azure Backup Policies to help protect our valuable assets in Azure from malicious deletion.
Azure Backup is our first line of defence, however we are not there yet. Someone could still remove Soft Delete from Azure Backup, delete our retained backups and then delete our virtual machines or Azure storage. While farfetched, it is feasible and mitigation measures should be explored.
This multi-part blog series aims to explore some alternative and relatively inexpensive ways to mitigate this risk. In this post, we’ll look at a way to protect Azure Virtual Machines from malicious deletion.
AZURE VIRTUAL MACHINES
Immutable Blob Storage
As mentioned in the first article in this series, Immutable Blob Storage can be used with time-based retention policies, which would prevent data from protected containers being deleted until the defined retention period has expired. Let’s explore what Immutable Blob Storage is.
Immutable Blob storage allows organizations to store critical data in a WORM state. WORM stands for (Write Once, Read Many). Once data is stored in WORM state, it cannot be deleted or modified by any user regardless of what level of access or Role-based Access Control (RBAC) they have. Even a subscription owner cannot delete WORM data until its retention period has expired or its hold is lifted.
There are two types of policies in Immutable Storage:
- Legal hold – Users can set policies to retain data for as long as the policies are in place. When this policy is set, data can be created and read but not modified or deleted. Once the policy is removed, data can be deleted
- Time-based retention – Users can set policies to store data for a specified interval. When this policy is set, data can be created and read, not modified or deleted. Once the retention period has expired, data can be deleted
For our purposes, we are interested in Time-based retention since the Legal hold can be administratively removed. It is worth noting you cannot store active VMs in Immutable Storage since the data cannot be modified. We will need to find a way to copy data into the Immutable container daily for safekeeping.
When using Time-based policies, we set the policy at the container level and any data written to the container will automatically inherit the policy.
In this scenario, not only is the data within the retention policy protected from deletion, the container itself and the Storage Account are also protected from deletion, so this is a perfect set of controls for our use case. Best of all, Immutable Blob Storage adds no additional cost on top of regular blob storage costs! Immutable Blob Storage is supported across all access tiers, Hot, Cool, Archive as well as all redundancy configurations such as Locally Redundant Storage (LRS) and Geo Redundant Storage (GRS). It is also supported in Premium Storage Accounts however this would lead to excessive costs, so the recommendation would be to stick with Standard Storage Accounts using LRS.
Setting the Retention Policy
As mentioned earlier, once this policy is configured, you will not be able to delete the data stored in the container, so while testing make sure to use very short retention times such as three days.
The first step is to create a container for the retention data.
Once you have your container, click it and you will select Access Policies from the settings and then select Add policy under Immutable Blob Storage.
Select Time-based retention, input the appropriate number of days and click OK.
Once the policy is created, the state is set to Unlocked and if you click the ellipsis on the right, you can edit it.
In this unlocked state, the retention policies are going to be honored though an authorized user can change the retention interval. For testing this is fine, however, once you put this into production you will want to click ‘Lock policy’ which will lock it in and no longer allow you to edit it. Notice the warning message you see when clicking Lock policy.
Once confirmed, the status will indicate Locked.
Creating a Process
At this point, all the parts are in place to create a script to copy the Managed Disk VHDs of your protected VMs to the immutable storage container. You can also set this script to run using Azure Automation on a daily schedule. The script can run under the context of a Run As account, with appropriate permissions. You can also add additional functionality to the script to purge data older than the retention range, which would be recommended to control costs. The sample script does not include this functionality at the time of this writing.
One last note, copying the VHD directly from the Managed Disk would require shutting off the VM, which obviously isn’t a good option. Instead, the script will take Managed Snapshots of each Managed Disk from the list of protected VMs, copy the VHDs from the Managed Snapshots into the Immutable Storage container and then delete the Managed Snapshots before exiting.
The script sample can be found at https://github.com/n-hoffman/Azure-Immutable-Backups
I hope you found this helpful and in the next part of the series, we will look at protecting non-VM resources such as Azure SQL.
For more of our Azure blogs, I recommend:
