When architecting solutions for Azure Infrastructure as a Service (IaaS) or Platform as a Service (PaaS); Business Continuity and Disaster Recovery (BCDR) is often factored into the design to allow an organisation to withstand an outage at any single point of failure or even an Azure regional outage event. To mitigate the risk of service disruption, we use redundant and resilient services such as:
- Multiple regions
- Availability Sets/Zones
- Global Redundant Storage (GRS)
- Azure Site Recovery (ASR)
- Redundant database copies
- Replication
Leveraging these tools can certainly help when there is an outage or even corruption of some kind within the configured workloads, however there is another more insidious risk which should be considered.
Specifically, this risk is malicious actors who may, with the appropriate privileged access, be able to permanently delete production data. Azure provides some protections across the platform to protect from accidental deletion, however, clients ultimately have complete control of their data and have the prerogative and the capacity to delete their data and services if desired.
One exception to this is Azure Immutable Blob Storage, which allows clients to set compliance-driven legal hold and time-based retention policies on blob storage data. Unfortunately, this wouldn’t natively support things like Virtual Machines or Azure SQL, since they live outside of blob storage.
Azure Resource Locks can also add some level of protection, however if a malicious actor gets hold of a privileged access account, then the locks can be removed.
Protecting administrative credentials with MFA, Conditional Access, Identity Protection, and Azure Privileged Identity Management (PIM) should absolutely be included in the strategy to minimize both accidental and malicious deletion events, however, some organisations may also want to go one step further and plan for the worst-case scenario.
This multi-part blog series explores some alternative and relatively inexpensive ways to recover from this risk. Identity and Security was also covered by Robert Buktenica and Edmund Davis on The Late Night Brew, which I recommend watching over your next coffee break.
AZURE BACKUP
Azure Backup is the first line of defense for many companies. Azure Backup is used to protect several types of resources which, at the time of this writing, include virtual machines, on-premises servers, SQL databases running on virtual machines, Azure Files storage and SAP HANA. Azure Backup is mainly a backup platform offering both short and long-term retention of data and also has some capabilities to protect against accidental or intentional deletion.
Soft Delete
While individual backups can be deleted from Recovery Service Vaults, Soft Delete is enabled by default on each vault. This allows any deleted backup to be recovered for 14 days. This would offer a level of protection if the Soft Delete was permanently enabled, however like many things in Azure it can be turned off by going to the vault properties, clicking Security Settings and disabling it.
Alerts
If Soft Delete was removed from a vault, a critical alert notification would be sent (assuming alerts are enabled) to the email recipients configured to receive alerts.
It is recommended to ensure alerts are configured for all vaults.
While this is a great start and we definitely want an alert triggered when Soft Delete is removed from a Recovery Services Vault, this alone will not save an organisation from this worst-case scenario. The alert will not prevent somebody from manually removing Soft Delete, deleting historical backups and finally deleting the original resources such as VMs or Azure Storage.
Azure Backup Center
It is also recommended to leverage Azure Backup Center, which provides a single pane of glass to govern, monitor, operate and analyse backups at scale. To support the full reporting capabilities in Azure Backup Centre, diagnostics should be enabled from the active Recovery Service Vaults to a Log Analytics Workspace.
Another good practice is to ensure all VMs matching specific criteria are enrolled in Backup. From within Azure Backup Centre, it is possible to create an Azure Policy to automatically enroll VMs in backup if they’re not currently enrolled. Prior to Azure Backup Centre, the existing Azure Policy capabilities were limited to only reporting on VMs not being backed up, however these newer policies can remediate by enrolling the non-compliant VM in a Recovery Services Vault.
I hope exploring the different backup tools, features and settings has been helpful. In the next part of the series, we will look at a clever way to leverage Immutable Storage to protect Azure Virtual Machines from malicious deletion. For further reading, try this Azure Information Protection Deployment series.
