Ransomware Resilience: Immutable Backups and Tested Recovery Engineering

A green backup job is evidence that a task completed. It is not evidence that the business can restart after a compromised administrator deletes infrastructure, encrypts application data, or revokes access to the keys used by the backups.

Resilience requires a chain: an intact recovery point, a separate authority that can retrieve it, a clean restore environment, working dependencies, and application acceptance. This guide provides a reference design and a repeatable exercise. Any recovery times shown are planning examples, not measured StackLocked customer results.

1. Define recovery at the application boundary

For each critical service, document the maximum acceptable data loss (RPO) and elapsed time to usable service (RTO). Name the business owner who accepts the recovered application. Capture identity, DNS, certificates, secrets, network routes, operating-system images, database dependencies, and license requirements alongside the data backup.

Use a simple timing model: recovery time equals incident decision and authorization time, environment preparation, data retrieval, restore or replay, and validation. Some tasks can overlap, but do not assume overlap without an exercise. A storage restore timer excludes many of the steps that determine customer-visible downtime.

As an illustrative lower bound, moving 2 TB at a sustained 200 MB/s takes about 2.8 hours before overhead, decompression, replay, and application checks. A four-hour RTO would leave little room for the other steps. Measure sustained performance under realistic constraints rather than quoting peak interface speed.

2. Separate the backup trust boundary

Keep backup administration distinct from production administration. Production agents should have only the access their supported backup workflow requires. Separate retention management, restore authorization, and key administration where practical. Audit who can assume each role, reset its credentials, change its federation trust, or deploy code into the backup service.

An online immutable repository is not a physical air gap. An offline medium disconnected from production provides a different boundary from a cloud repository with logical isolation. Choose the design explicitly. For an offline copy, document export, custody, rotation, offsite storage, encryption, and periodic retrieval tests. Include the risk of a malicious or corrupted backup being written before the medium is disconnected.

For a cloud copy, use a separate administrative account or subscription with narrowly controlled cross-boundary access. Do not let the same everyday administrator remove the repository protection and approve the destructive operation. Preserve a recovery path when the normal identity provider or corporate password manager is unavailable.

3. Engineer S3 retention around object versions

S3 Object Lock protects object versions and depends on versioning. Governance mode permits bypass by specially authorized callers; compliance mode imposes stronger retention that cannot simply be shortened by an administrator. Test retention requirements before an irreversible commitment. A default retention setting does not retroactively protect every old object version. See S3 Object Lock behavior and configuration requirements.

Use the following read-only inspection commands with an authorized audit profile. Replace the bucket, key, and version placeholders with a sacrificial test object’s actual values.

aws s3api get-object-lock-configuration --bucket BACKUP_BUCKET
aws s3api get-bucket-versioning --bucket BACKUP_BUCKET
aws s3api get-object-retention \
  --bucket BACKUP_BUCKET \
  --key TEST_OBJECT_KEY \
  --version-id TEST_VERSION_ID

Check the returned version’s mode and retain-until date. In a dedicated test repository, verify that the modeled compromised operator cannot delete that retained version or reduce its retention. Deleting a current key can create a delete marker while protected versions remain; test recoverability of the actual version rather than judging the bucket listing alone.

Object immutability does not preserve a deleted encryption key. Protect KMS administration, monitor key-disable and deletion scheduling, and test decryption using the recovery identity. AWS explicitly documents this distinction in Object Lock considerations.

4. Make Azure vault protection a deliberate commitment

For supported Azure Backup workloads, enable vault immutability, validate the policy and operational impact, and then lock it when the retention design is accepted. Locking is irreversible; verify vault type, region, workload support, and current WORM availability before relying on a particular implementation. See Azure immutable vault concepts and management steps.

Use Resource Guard and multi-user authorization where supported to place approval for protected operations under separate control. That separation is ineffective if one compromised operator controls both the vault and its approving authority. Soft delete, immutability, and separate authorization solve different problems; record which attack each control is expected to resist.

5. Build a clean recovery path

1. Select a recovery point that predates the suspected compromise, accounting for attacker dwell time rather than choosing the newest successful job automatically.
2. Prepare an isolated recovery network and known-good operating environment. Restore identity dependencies using a workload-specific recovery plan.
3. Recover secrets and certificates through the independent recovery authority; rotate compromised credentials before reconnecting services.
4. Restore databases with the required full, differential, and transaction-log chain where applicable. Record the last recovered transaction time.
5. Run database consistency checks and business-level validation: login, representative reads, writes, integrations, and scheduled work.
6. Reconnect only after containment and application acceptance. Preserve incident evidence and record any temporary restrictions.

Do not boot restored domain-connected systems into production casually. Identity recovery, duplicate machine identities, malware persistence, and stale trust relationships need their own handling. Instant recovery can accelerate availability but may run on constrained backup storage; measure the later migration and sustained application performance as well.

6. Test the failure you actually fear

Exercise recovery with production credentials assumed unavailable. Have a designated recovery operator obtain the runbook and recover a test service using only the approved emergency route. Measure time from the recovery decision to business acceptance, and compare recovered data age with the RPO.

The evidence pack should contain recovery-point identifiers, retention checks, authorization results, key-access verification, restore timestamps, consistency results, application test results, and sign-off. If the runbook depends on a single person’s undocumented knowledge, the exercise has identified a resilience defect even if the restore eventually succeeds.

Track failures by owner and retest after correction. Useful measures include services with a recent successful restore, objectives actually met, recovery dependencies still unresolved, and repositories whose retention has been verified. Backup success percentage alone cannot answer these questions.

For an environment-specific plan, see Backup & Disaster Recovery engineering. Protect the administrative entry point using our Entra ID hardening guide.