I’ve just about decided that Time Machine is about as close to a perfect piece of software as I’ve ever seen.
Backing up your data is hard. I don’t mean hard in the sense of laborious; it’d be silly to describe any operation that involves sitting in a comfortable chair sipping a beverage and clicking a button a few times as laborious. I mean it’s hard in the sense that it’s very difficult to get it just right, and unless you get it just right, all you’ve accomplished is to waste a lot of time and give yourself a false sense of security.
But Time Machine applies a startling level of cleverness to the problem, making something that’s really very tricky into something very easy. More important than that, it takes something very tricky and makes it automatic; no attention, or at worst very little attention, is required to take advantage of Time Machine.
Before Time Machine, there were a few ways to create attentionless backups. I used to have a Power Mac G5 under my desk with two identical 250 GB hard drives in it. Every day at noon and again at 6:30 p.m., an excellent little program called SuperDuper would run, synchronizing the two disks. If at any time my primary disk had failed, I would have been able to reboot off my secondary disk and get right back to work on a backup that was, at most, just a few hours out of date.
But there were a couple of problems with my scheme. The first was that SuperDuper, for all its virtues, was a pretty obtrusive program. Every time it ran, it had to do a lot of disk operations to compare the two drives, and that resulted in a lot of churning and grinding, making my computer pretty annoying to use for the duration of the backup. That’s why I had it set to run only twice a day: when I was out to lunch, and just after I’d left for the evening. At noon, it would catch all the work I’d done during the morning, and at night it would catch all the work I’d done during the afternoon. Running it any more frequently would have made my day pretty unpleasant.
The other problem with SuperDuper was that it only protected me from a hardware failure. If I changed a document in a way I turned out not to be happy with and wanted to go back to a previous version, I had a very narrow window of opportunity to do so. If I made the change at 8:00 a.m. and wanted to revert at 11:00 a.m., I was okay; I could go back to the backup that had been made the previous evening. But if I made the change at 8:00, reflected on my changes over lunch and then decided to revert at 1:00 in the afternoon, I was screwed. SuperDuper had, dutifully, already backed up my new version, overwriting the work of the previous day.
So my system worked, fundamentally, but it only protected me in limited ways. I was neurotically aware of them all the time, so it was all right, but it was far from perfect.
Of course, the system I used on that computer wouldn’t really work at all on my laptop. I like to roam; I don’t enjoy sitting in the same chair all the time. So while I have an external FireWire drive for backups, I don’t keep it plugged in all the time. If I wanted to reinstate my SuperDuper system of backups now, I’d have to go out of my way to plug in my FireWire drive twice a day, leaving my laptop alone for an hour or so to give the program the time to do its work. That’d really cramp my style. And, as I said before, it would only protect me in limited ways.
Time Machine is different, though. The first major difference is that backups are unobtrusive. You’ll notice them, of course; you’ll hear your external drive spin up and crunch-crunch-crunch for a few seconds before going to sleep again. But it’s nothing like the hour-long grind-grind-grind of a SuperDuper backup. That’s not an indictment of SuperDuper; Apple had to make some pretty large-scale changes to the operating system just for Time Machine. I wouldn’t be surprised if David Nanian incorporates the under-the-hood enhancements that power Time Machine into a new version of SuperDuper sooner or later.
But the other thing about Time Machine that makes it remarkable is the fact that it doesn’t really matter very much just when you plug your computer in. With my home-grown SuperDuper scheme, my computer had to be on and idle every day at noon and again at 6:30, otherwise my backups just wouldn’t happen. But Time Machine doesn’t operate on a schedule, making my life a heck of a lot more flexible.
Okay, it’s technically untrue to say that Time Machine doesn’t operate on a schedule. It does. It’s just a very loose one. By default, Time Machine backs your computer up every hour. I’m not entirely sure where it starts counting, because I didn’t pay attention when I first turned Time Machine on. But I think it just does your first full backup, then says “Okay, we’ll run again in an hour.” If that happens to be at 7:37 in the morning, then your next backup will run at 8:37. No big deal.
But here’s the thing: If you’re not plugged in at 8:37 — say you’re sitting on your porch enjoying your morning coffee — Time Machine doesn’t mind. It doesn’t throw any big, annoying error message at you. It just says — silently, in the background, recorded only in a log file you really have to look for to find — “Okay, we can’t back up now; we’ll just back up again whenever this guy plugs in his external disk.”
And then … it does.
The next time you plug in, Time Machine runs automatically, backing up whatever files have changed or been created since the last time it backed you up.
Let’s run through this one more time, because it’s a little complicated. You run your very first Time Machine backup at 7:37 in the morning. Time Machine says “Okay, the next backup is scheduled for 8:37,” and then goes to sleep for an hour. At 8:25, you decide you want to enjoy the morning on the porch, so you unplug your laptop and go outside. At 8:37, Time Machine wakes up, looks around and says “Hey, there’s no backup disk here. We’ll just run another backup the next time the disk is available. Oh, by the way, the next scheduled backup is now at 9:37.”
At 8:52, you go back inside and plug your external disk in. Time Machine wakes up, sees the disk and says “Okay, better run that backup we missed at 8:37.” It backs up all your stuff, then goes to sleep.
At 9:37, Time Machine wakes up again and runs another regularly scheduled backup.
At 9:45, because you’re a nerd, you decide to look at the contents of your backup disk to see what the heck is going on. You find three folders there, each with a seemingly cryptic name: 2007-11-02-073715, 2007-11-02-085259 and 2007-11-02-093701.
It doesn’t take you long to figure out that these gibberishy names are actually dates and times. The first one refers to November 2, 2007, at 7:37:15 in the morning. That’s what time the backup was taken. (I think it’s technically what time the backup completed.) As we’ve already observed, each of these folders contains a complete copy of your computer’s files at the time the backup was taken. Time Machine uses some cleverness to keep from having to actually copy files that haven’t changed since the last backup, but for all intents and purposes, each of these folders is a complete backup of everything on your computer.
Thanks to its adaptive scheduler, Time Machine will run every hour, or every time you plug in your backup disk, whichever is more frequent. It will keep your hourly backups for a day, then “thin” them, discarding all but one backup for each 24-hour period. It’ll keep those daily backups for a month, then start “thinning” them in favor of weekly backups, which will be retained as long as space is available.
What this all boils down to is this: I haven’t had to pay any attention to making sure my computer gets backed up. The only change I’ve had to make in my routine is to make sure my backup disk is plugged in long enough to get a backup at least once a day. Whenever I’m working on something important, I just sit at my desk with my backup disk plugged in, and Time Machine will wake up every hour and back up everything I’m doing, just to be safe. If I screw something up — and I did, once, over the weekend — I can just restore to the previous backup, and I never lose more than an hour’s worth of work.
Here’s an interesting idea, though. Hard drives are getting very small; the hard drive in the iPod Classic is tiny, but it holds 160 GB. That’s as big as the hard drive on my laptop. I wonder what it would take for Apple to engineer some kind of widget that can hold two miniature hard drives in the same space as one normal-sized laptop drive, so folks would have the option of having two identical, medium-sized hard drives inside their laptops? There’s a good chance you’d want to keep an external drive around for backups anyway, to guard against your laptop being lost, stolen or physically damaged. But having Time Machine keep an up-to-date set of backups inside your computer at all times, whether you’re at home or on an airplane or what, is a pretty seductive idea to me. Because that would take a virtually zero-attention operation and turn it into an actual zero-attention operation. You’d never even need to think about your backups; they’d just happen.
Comments
All comments are the property of their owners and do not reflect the opinions of this Web site or, well, basically anybody at all. The author of this Web site reserves the right to edit the hell out of any and all comments. Participate at your own risk.
What a waste of time and effort. All of the features of time machine - ALL of them - are native in ZFS. Apple should have just built ZFS into OSX 10.5 instead of wasting their time creating yet another closed, proprietary solution.
Closed = bad, open = good, apple. Get with the program.
Calli
Monday, November 5th, 2007, 11:35 am
I’ve done a lot of research on ZFS over the past year or so. It’s a neat idea and it has a lot of potential, but there are some excellent reasons why Apple didn’t use it for Time Machine, or indeed built it into Mac OS X in any significant sense at all.
First of all, ZFS is ridiculously complicated. There’s been a lot of criticism of the “big floppy” approach to storage management, but the one undeniable virtue it has going for it is that it’s incredibly easy to understand. One device maps conceptually to one volume. If a device fails or is removed, that volume disappears. Easy to understand.
ZFS changes all that. ZFS virtualizes devices into structures called “pools,” where pools are composed of “virtual devices” and virtual devices are made up of actual devices. This layer-cake of abstraction opens up a lot of potential functionality, but at significant cost on a single-user, single-device system like a laptop. Remember that Apple sells more laptops than desktops now, and even the popular iMac model is built like a laptop. Only the Mac Pro has room inside for more than one block device.
Of course it’s possible to abstract a lot of ZFS’s complexity away behind a user interface — after all, Apple has successfully abstracted away a lot of the complexity of partitioning, raw versus block devices and filesystems away behind Disk Utility. But ZFS is considerably more complex than a traditional filesystem, so a lot of work would be required to make it simple for the user.
Those are just the logistical problems of rolling out ZFS. They’re solvable with nothing more than time and effort — although of course time and effort are limited resources. But in addition to the logistical problems of hiding ZFS’s complexity, Apple would also have to look long and hard at some of ZFS’s inherent logical shortcomings. For example, reads and writes that are smaller than the filesystem block size on ZFS impart a significant penalty in terms of CPU utilization, and the default filesystem block size on ZFS is a whopping 128 kilobytes. So reading or writing any chunk of data smaller than 128 KB is considerably slower on ZFS than it is on a traditional filesystem.
ZFS also has significant fragmentation problems. The copy-on-write model of filesystem I/O has a lot going for it, but it quickly leads to a severely fragmented filesystem. That’s a problem Apple solved with automatic HFS defragmentation and the “hotfiles” system on Mac OS X, but those solutions would have to be totally reengineered for ZFS.
Logical volume management isn’t a new idea. In a lot of ways, ZFS provides no more core functionality than something like XLV or XVM did on IRIX ten years ago. And like XLV and XVM, ZFS is handy for large-scale server or supercomputing environments where a full-time staff of engineers is in place to keep the computers and storage working optimally, but it’s overkill for single-user workstations and a downright liability for laptops.
Will the day come when ZFS is the default filesystem for Mac OS X? I don’t know. Maybe. It’s certainly got potential that I think is worth exploring. But it’s not necessary today — as demonstrated by the fact that Apple implemented Time Machine only by making two low-level changes to Mac OS X without having to adopt a new filesystem at all — and the difficulty of implementing it in a user-friendly way far outweighs the potential benefits.
Jeff Harrell
Monday, November 5th, 2007, 11:48 am
Wow. All I was going to say was that I do find that my year-old MBP chokes if the Time Machine backup runs while I’m doing something system intensive (coughcoughWoWcough), so I unplug the drive while I’m raiding. Which is when I’m not generating anything new to back up anyway.
Tiff
Monday, November 5th, 2007, 12:05 pm
Time Machine is enough to appease the nerd and non-nerd Mac users out there.
Derek Giromini
Monday, November 5th, 2007, 12:21 pm
[…] is too complex’ Argument Posted November 5, 2007 Jeff Harell has a nice blog, The Shape of Days, and he has a nice love entry about Time Machine. One of the comments explains that Time Machine is […]
The ‘ZFS is too complex’ Argument « The Lame Leopard
Monday, November 5th, 2007, 2:48 pm
Geoff, I don’t agree with your statement about ZFS at all. I did a larger piece about that on my own blog … http://thelameleopard.wordpress.com/2007/11/05/the-zfs-is-too-complex-argument/
Leo
Monday, November 5th, 2007, 2:49 pm
@Calli: What absolute bullshit. First, Apple’s implementing ZFS now, it’s just taking a while to do a reliable file system implementation. Second, the major backend changes that make Time Machine possible is all open source as part of Darwin (like hard links to folders), and a new frontend on this to actually make the backups isn’t a lot of coding.
Jesper
Monday, November 5th, 2007, 2:55 pm
Here’s the comment I just posted to that anonymous “Leopard sux” blog.
As a side note, if you’re going to start an axe-grinding blog to spread misinformation about a single product, at least have the balls to sign your name. Otherwise you just come across as a dickweed with too much free time and an inflated sense of your own self-importance.
Your apparent inability to spell my own name right, even though it’s sprayed about four times across this very page, doesn’t help your case any.
Jeff Harrell
Monday, November 5th, 2007, 3:05 pm
Two great articles about Time Machine, thanks. And your blgo layout is beautiful.
Olivier
Monday, November 5th, 2007, 4:49 pm
I’ve read a couple of posts about Time Machine and have a question. Can it be used to backup an external drive as well? Or does it only backup the boot drive? It makes sense that it would but…I haven’t seen that mentioned anywhere. I’ve got all of my photos and music on an external FireWire drive. I’ve been using SuperDuper to back the boot drive and the external “media” drive up to a third drive. Can I still do that with Time Machine?
Joanna
Monday, November 5th, 2007, 6:40 pm
While I think Time Machine is great and all, I’m in many ways more interested in how 3rd party devs (such as David Nanian as you mentioned) utilise FSEvents and directory hardlinks to make new or improved software.
For example: Time Machine-like sync over ssh to replace rsync as my offsite backup tool of choice.
With the FSEvents log it should be blazingly fast and maintaining several copies of one hard disk in multiple locations would be easy and possible wherever an internet connection could be found, you wouldn’t even have to plug that FireWire drive into your MacBook.
Also, using FSEvents to make quick monolithic, bootable backups (similar to how you used SuperDuper!, but much faster).
Anonymous Coward
Monday, November 5th, 2007, 6:52 pm
There’s no need for shoehorning multiple separate hard drives into a small case. Just split one drive into two partitions (assuming that Time Machine won’t balk at such a configuration). These need not even be equal sized. Depending on personal usage patterns, it may make more sense to give the Time Machine partition more space than your main partition, so that it can keep more historical data.
Of course, this also means that you don’t have Time Machine available for true external backup usage, as it doesn’t initially support multiple backup drives (e.g. for onsite and offsite backups).
I’m also psyched to see SuperDuper incorporate the new FSEvents support into its SmartUpdate functionality. Then SuperDuper users will get a bootable external backup that syncs as fast as TimeMachine itself. Rock!
John Whitley
Monday, November 5th, 2007, 8:11 pm
Backing up your data from one half of your hard drive to the other is kind of like trying to save the passengers on the Titanic by herding them to the stern of the ship.
Using one partition to back up the data from your other partition does nothing more than open up the opportunity for a hard-drive failure to destroy both your system disk and all your backups all at once.
Jeff Harrell
Monday, November 5th, 2007, 10:21 pm
The thing is, SuperDuper! did provide a bootable backup disk. Time Machine doesn’t. They don’t even pretend to do the same job. I don’t think they are direct competitors, so to speak.
Time Machine for me is really unintuitive. It did not run automatically when I plugged my laptop back into the backup drive. I just got two hypens (—) beside “Next Backup”. There was nowhere I could see to configure it. I had to hunt through the help files to figure out how to run it manually. This isn’t going to work for me.
If I did, though, it would complement SuperDuper! beautifully. As you say, SuperDuper! does take ages to back up, even with Smart Update. Combined with Time Machine, it could be ok to run it less frequently. I could never do away with it, though! If my HDD dies, ok I have my files backed up with Time Machine but I still can’t work! SuperDuper takes care of that.
Vicki
Tuesday, November 6th, 2007, 2:00 am
Time Machine only runs when you’re plugged into a power source, presumably to prevent a backup from being left in an inconsistent state due to battery exhaustion. I believe this is documented, but I can’t confirm it right this second. And yes, this could be made clearer in the Time Machine prefs pane.
Also, keep in mind that a bootable backup that also stores all the old backup sets is impossible, so a compromise had to be struck. You can boot the Leopard install DVD to restore your system from the Time Machine backup disk in the case of a catastrophic failure, or if you want to replace your system disk.
Jeff Harrell
Tuesday, November 6th, 2007, 9:40 am
Time Machine also works over a network. We have two laptops here that automatically back up to a large drive attached to a PowerMac G5, whenever they’re in the house and plugged in to power. This is even more convenient than attaching an external drive, and really approaches effortless backup. There are some glitches and caveats, and it doesn’t yet work with Airport Disk or third-party file servers, but these seem like issues that can be fixed with point-version upgrades.
Apple’s approach of using FSEvents and rolling their own backup system is, in the end, much more flexible and can work in a wider variety of scenarios than just throwing ZFS at it and requiring Mac users to be Unix sysadmins. (I say this despite being a Unix sysadmin since FreeBSD 2 and very much looking forward to ZFS for server environments)
Christopher Masto
Tuesday, November 6th, 2007, 10:56 am
Great article on Time Machine, thanks Jeff. Please could you clarify the procedure to restore the system if your internal hard disk fails beyond repair and you had to buy a new Mac?
Thanks.
Ken Laji
Wednesday, November 21st, 2007, 3:11 pm
I was wondering why you used Super Duper instead of mirroring your drives (as a raid). That way you wouldn’t have the slowdown of a backup application churning away… and you would have a completely backed up bootable drive at the ready anytime.
ford
Thursday, November 29th, 2007, 7:24 pm
Because mirroring doesn’t protect you from human error. Delete a file, or change it in a way you don’t like, and you’re stuffed.
Jeff Harrell
Thursday, November 29th, 2007, 7:41 pm