Backups and Disaster Recovery for Startups

Two numbers define what "recovered" means:

• Recovery point objective (RPO) is the amount of data loss you can tolerate, measured in time. Commvault defines it as "the maximum acceptable amount of data loss measured in time, such as hours or days."
• Recovery time objective (RTO) is how long you can take to resume normal operations, the "target window for restoring systems and services" (Commvault).

Commvault's example shows they are independent: a healthcare organization might tolerate up to 12 hours of data loss but need services back within 2 hours. Its steps for setting them are practical for a startup: identify critical systems and data, assess the impact of losing each, choose the acceptable data loss and downtime, check the targets are achievable with your resources, and review them regularly.

For a typical startup, the targets differ sharply by system:

The numbers above are illustrations, not standards. Commvault notes there are no industry-wide RTO and RPO standards, although some regulations impose requirements (Commvault). Set yours with the founders, because they are business decisions with a cost attached.

The 3-2-1 rule is the standard starting point for backup design. Backblaze, which has written about the rule for years, defines it as keeping "three copies of your data on two different media with one copy off-site" (Backblaze):

• Three copies: the original production data plus two more.
• Two different media: for example, a local or primary store plus a cloud storage service.
• One copy off-site: in a separate location, so a single physical or regional event cannot destroy everything.

Backblaze notes that the rule has held up since the late 2000s even as storage changed. For a cloud-native startup, "media" and "site" translate into different services, accounts and regions rather than tapes and buildings. A snapshot in the same account and region as the database is one copy, not two.

Veeam argues that the original rule "wasn't designed with ransomware, immutable storage, or public cloud in mind," and proposes an extended version, 3-2-1-1-0, which adds one immutable copy and zero recovery errors verified by testing (Veeam on 3-2-1). Both additions address the two most common ways backups fail: attackers delete them, or they turn out not to restore.

Ransomware operators go after backups on purpose. CISA warns that "many ransomware variants attempt to find and subsequently delete or encrypt accessible backups to make restoration impossible unless the ransom is paid," which is why it insists backups be maintained offline (CISA).

A Security Stack Exchange answer states the requirement plainly: if an attacker with admin rights on your systems can destroy all your backups, "then they don't really protect you against a ransomware attack." You need a backup that cannot be destroyed "even with full administrative rights." The same answer explains the history: offline backups on tapes or disconnected disks achieved this, and cloud providers introduced immutable backups to achieve the same effect without physical media (Security Stack Exchange).

Veeam defines an immutable backup as one that "cannot be modified, deleted, or encrypted for a defined period of time," stored in a write-once, read-many state. It notes that the immutability period needs care: too short and you lose the ability to roll back far enough, too long and storage costs grow (Veeam on immutable backup). Object storage with object lock is a common way to get it in the cloud.

The Stack Exchange discussion adds two cautions worth keeping. Some providers use "immutable" for what is really just an air gap, so read what the feature actually prevents. And immutability means trusting the provider's claim, which is why the useful question is "if we get compromised, how can I stop the ransomware group destroying all my backups as well?"

Practical options for a startup:

• A separate backup account. Copy backups into a different cloud account with its own credentials and MFA, which production admins cannot reach.
• Object lock or equivalent retention. Set a retention period long enough to cover how long an attacker might sit undetected before acting.
• A second provider. CISA suggests considering a multi-cloud approach for cloud-to-cloud backups, in case every account at one vendor is affected (CISA).
• Encryption with keys stored separately. CISA's instruction is offline and encrypted.

Startups often back up the database and forget everything around it. Walk through what you would need if your main cloud account vanished tomorrow:

CISA's guide recommends storing IT asset documentation securely with offline backups and physical hard copies, and making sure that any third party or managed provider responsible for your backups follows the same practices, written into the contract (CISA).

A green backup job proves that data was copied. It does not prove that you can restore it, that the restore is complete, or that it fits inside your RTO. CISA's guidance is to "regularly test the availability and integrity of backups in a disaster recovery scenario" (CISA).

Datto describes six testing methods that range from cheap to thorough (Datto):

1. Plan review. Stakeholders read the written plan to find missing steps and outdated contacts.
2. Tabletop exercise. The team talks through a disaster scenario to clarify roles and decisions.
3. Simulation testing. A controlled scenario, such as a network failure, with real recovery actions and measured response times.
4. Partial testing. Restoring one component, such as a key database, without a full exercise.
5. Full-scale testing. Running the whole plan, possibly including failover to backup infrastructure.
6. Parallel testing. Restoring critical systems in a separate environment while production keeps running, then comparing the result with live data.

For most startups, partial and parallel tests give the most value for the effort: restore last night's database into an isolated environment, run a few checks against it, and time the whole process. Cloud providers increasingly automate this. AWS Backup, for example, offers restore testing that runs on a schedule, restores specific or random recovery points and reports the results (AWS).

On frequency, Datto's rule of thumb is to test the plan at least once a year, after any significant infrastructure change, and more often, such as quarterly or twice a year, for critical or regulated systems (Datto). Automated restore checks for the production database can run far more often than that, which is how you reach the "zero recovery errors" in Veeam's extended rule.

Record each test: what was restored, from which point in time, how long it took, what failed and what you changed. Compare the measured times with your RTO and RPO. If a restore takes six hours and your target is two, you have found the most important item on your roadmap.

A runbook turns backups into recovery. Keep it short, keep a copy outside the systems it describes, and include:

DISASTER RECOVERY RUNBOOK                     Last tested: ________
1. Targets: RPO and RTO per system (table above)
2. Who decides to fail over or restore, and their backup
3. Where backups live: accounts, regions, retention, who has access
4. How to get credentials if the main identity provider is down
5. Restore steps per system, in dependency order:
   network and IAM -> secrets -> database -> storage -> app -> DNS
6. How to verify each restored system before reopening to users
7. Customer communication template and status page access
8. Results of the last restore test and open issues

Two details catch teams out during real recoveries. First, access: if your backup account is protected by the same single sign-on that just failed, you cannot reach it, so keep break-glass credentials for the backup account stored offline. Second, order: an application restored before its secrets or database will fail in confusing ways, so the runbook should list dependencies explicitly.

Review the runbook after every restore test and every significant infrastructure change. Datto lists keeping pace with changing infrastructure as one of the main reasons to test (Datto), and the runbook should change with it.