Paid to be Paranoid: Backup
05 March 2010
About 10 years ago I learned a lesson which changed the way I think about my data. A family member loaned me their photo collection and I spent the best part of a month scanning and archiving pictures of the last four generations of my family.
I thought I'd taken every precaution necessary, I was running RAID and having never suffered data loss on a drive before the thought of two mirrored discs failing was inconceivable.
Of course the inevitable happened - the drive I had opted to use, the then top of the line IBM DeskStar (aka DeathStar) had a design defect and I was one of many people to lose every byte of data committed to them.
Since that "little" incident (I haven't been able to get access to the photos, having lost contact with that family member) my attitude to backup is drastically different to say the least, I've used DLT, LTO, products such as Windows Home Server and as my home network has expanded begun to experiment with sync to make my data highly available and accessible, along with an extensive backup archive.
A year or three ago I switched over to using macs at home and so my attitude to backup had to change somewhat. Today we
ve quite a lot of macs at home: a mac mini entertainment box, three imacs, three macbooks, a mac pro and an nc10 'hackbook'. At any time my partner (currently a student) could need to work from practically any of them, making sync as vital as protecting our data reliably and these two approaches compliment one another well.
Along the way I've tried an absolute ton of stuff and always had issues, either suffering a loss of data or finding a small inflexibility that turned out to be a deal-breaker. Products such as SpiderOak, SugarSync, Live Mesh all came and went, each had their foibles which made them too unreliable/much hassle/restricting.
What we've gone for in the end is a mixture of Dropbox and Time Machine. I've 5tb of firewire storage hooked off the mac mini and each of the other macs use it as a backup target. Each machine creates a sparse bundle on the backup drive allowing them to use the same storage. Application support is backed up for each machine plus any 'key' apps but the big stuff (Adobe, Microsoft, Apple, games) is excluded. We backup documents from certain systems, partly because they're all also in dropbox and I'm keen not to waste bandwidth. Dropbox provides 1/100th the capacity of the time machine but it's usually a no-brainer to work out what gets retained where - the big stuff we need once in a blue moon goes to local disc while the small stuff which can be a bit world-ending if it gets lost is pushed to the cloud and between each machine.
Even handling iTunes content between the machines has been pretty straightforward, the 'automatically add to itunes' folder is shared from the Mac Mini and any new music gets dropped there, allowing the clients to grab stuff for remote playback using the Home Sharing. For most apps there's either been a feature or some kind of automator frippery to handle it.
Dropbox is syncing all of the user-generated content that we need between machines plus critical stuff like the 1Password keychain which in the new format syncs flawlessly (and doesn't have to live in app support nor use the old keychain format any more). One of my fave geeky features to dropbox is that if we change something on the LAN from one system it syncs locally between the dropbox peers rather than each machine grabbing their own copies from the cloud, as we're on 50mbit bandwidth isn't a huge concern but it's nice to see growl fire off on all the systems at once indicating they're in sync. The pack-rat addon gives us the same kind of undo history that Time Machine provides albeit a bit slower and has been a lifesaver on several occasions.
Email, bookmarks and calendars are synced using google bookmark sync for chrome and mobileme for safari.
My latest little trick has been to use soft links to re-point the user settings and addon folders in WoW to dropbox, saving me a curse.com subscription and making maintaining the other machines very straightforward.
What this means is that at any time all of our macs have the really vital stuff and you'd have to drop the bomb in several locations to obliterate all our data. I think it's pretty neat.
I thought I'd taken every precaution necessary, I was running RAID and having never suffered data loss on a drive before the thought of two mirrored discs failing was inconceivable.
Of course the inevitable happened - the drive I had opted to use, the then top of the line IBM DeskStar (aka DeathStar) had a design defect and I was one of many people to lose every byte of data committed to them.
Since that "little" incident (I haven't been able to get access to the photos, having lost contact with that family member) my attitude to backup is drastically different to say the least, I've used DLT, LTO, products such as Windows Home Server and as my home network has expanded begun to experiment with sync to make my data highly available and accessible, along with an extensive backup archive.
A year or three ago I switched over to using macs at home and so my attitude to backup had to change somewhat. Today we
ve quite a lot of macs at home: a mac mini entertainment box, three imacs, three macbooks, a mac pro and an nc10 'hackbook'. At any time my partner (currently a student) could need to work from practically any of them, making sync as vital as protecting our data reliably and these two approaches compliment one another well.
Along the way I've tried an absolute ton of stuff and always had issues, either suffering a loss of data or finding a small inflexibility that turned out to be a deal-breaker. Products such as SpiderOak, SugarSync, Live Mesh all came and went, each had their foibles which made them too unreliable/much hassle/restricting.
What we've gone for in the end is a mixture of Dropbox and Time Machine. I've 5tb of firewire storage hooked off the mac mini and each of the other macs use it as a backup target. Each machine creates a sparse bundle on the backup drive allowing them to use the same storage. Application support is backed up for each machine plus any 'key' apps but the big stuff (Adobe, Microsoft, Apple, games) is excluded. We backup documents from certain systems, partly because they're all also in dropbox and I'm keen not to waste bandwidth. Dropbox provides 1/100th the capacity of the time machine but it's usually a no-brainer to work out what gets retained where - the big stuff we need once in a blue moon goes to local disc while the small stuff which can be a bit world-ending if it gets lost is pushed to the cloud and between each machine.
Even handling iTunes content between the machines has been pretty straightforward, the 'automatically add to itunes' folder is shared from the Mac Mini and any new music gets dropped there, allowing the clients to grab stuff for remote playback using the Home Sharing. For most apps there's either been a feature or some kind of automator frippery to handle it.
Dropbox is syncing all of the user-generated content that we need between machines plus critical stuff like the 1Password keychain which in the new format syncs flawlessly (and doesn't have to live in app support nor use the old keychain format any more). One of my fave geeky features to dropbox is that if we change something on the LAN from one system it syncs locally between the dropbox peers rather than each machine grabbing their own copies from the cloud, as we're on 50mbit bandwidth isn't a huge concern but it's nice to see growl fire off on all the systems at once indicating they're in sync. The pack-rat addon gives us the same kind of undo history that Time Machine provides albeit a bit slower and has been a lifesaver on several occasions.
Email, bookmarks and calendars are synced using google bookmark sync for chrome and mobileme for safari.
My latest little trick has been to use soft links to re-point the user settings and addon folders in WoW to dropbox, saving me a curse.com subscription and making maintaining the other machines very straightforward.
What this means is that at any time all of our macs have the really vital stuff and you'd have to drop the bomb in several locations to obliterate all our data. I think it's pretty neat.
0
