Protect Your Data Now So You Can Have It Later
I’m currently working on a big project with a lot of audio, roughly 1 TB of audio recordings. (I’m making a sample library, 8 mics all recording at 96KHz.) I want to protect that data so I have a few routines set up and thought I’d share what they are so perhaps you can copy it, learn from it, or spark a better way.
Where the data lives:
- I recorded the original session onto External Drive 1. This is the external drive I’m working off of.
- Carbon Copy Cloner clones that drive to another drive, External Drive 2, with their Safety Net feature on. It does this every three hours. (If a backup fails CCC will automatically email me so I can deal with it immediately.)
- External Drives 1 and 2 are constantly backing up to Backblaze’s online backup service. This online backup service is a temporary solution for short-term data loss, not permanent. If something happened such as I am in a car accident and my computers shut off for 31 days then this data would be lost. This is good for the “now” safety but not the “always” safety. (That comes later with the B2 service and Dropbox Business.)
- This project is for a sample library and I will be doing a lot of manipulation so the first thing I do when working is duplicate all raw audio files (the original recordings, untouched by processing) from each recording session and set those aside, they will remain forever untouched. (This is in case anything goes wrong or if in the future better tools come along and I want to process this audio again.)
- The original raw audio folders are then also uploaded to B2. I use Transmit 5 to access B2 as if it were a regular FTP server. (It is not.) B2 is the long-term “cold storage” from Backblaze. This is the one that will remain saved online for as long as I keep paying them or they go out of business. I have these on their system in two ways:
- As a single large zip file for downloading the entire thing
- As individual files in case I mess up an audio file while working, accidentally use one of the original raw files, or a single file corrupts and I need to go grab the original file to start over.
- The new audio folders—which I’m using to work from—are constantly being copied to Dropbox also using Carbon Copy Cloner. The Dropbox folder they’re being copied to is shared with the client and uses the Smart Sync setting of “cloud only” so it doesn’t eat up any of my hard drive space. The Carbon Copy Cloner is a series of routines which are grouped into a parent routine and that parent will only run when I manually do so. This routine only updates any changed files which keeps the process extremely quick after the initial sync. Additionally, I will manually schedule this process to run a few days before a meeting so when that meeting comes the client will have the most recent files, those will be available a few days prior to the meeting in case they want to download anything or everything for the meeting.
So to recap, my assets live on:
- External Drive 1
- original recorded files
- new working files
- External Drive 2
- clone of External Drive 1
- Backblaze Online
- clone of External Drive 1
- B2 Cloud Service
- all original recorded audio files in a zipped snapshot
- all original recorded audio files in a navigable folder system
- new working files updated manually