offline_blown.jpg

Late last week, upon concluding some final tweaks to the visuals and soundtrack of episode one, I set up After Effects to render a final uncompressed export. To date, all we’d exported was the compressed Flash video file currently posted on the homepage; because we’d worked so hard to post episode one on our targeted date (Independence Day), we still had some tweaks we wanted to make before outputting the final files. This was to be our full-quality export used to generate other file formats (the iTunes podcast we’re working on, for example), and was to be the file we’d always have in the archives in case of future hard drive failure.

I hit “Render,” ready to finally be done with episode one. Then, five seconds later, in the middle of the screen, a little dialogue box with a red exclamation mark popped up. Sometime between tweaking the last few keyframes and starting the export, our external hard drive—containing all 280GB of The West Side to date—turned into a doorstop. And poof. Just like that, it was gone.

Filmmaking is often a case study in Murphy’s Law, but timing-wise, this was excruciating to the point of seeming like a surreal joke. If there was any one point at which our hard drive was going to fail, wouldn’t the worst possible time be exactly when we went to output the final export for our archives? If the drive failed one hour later, we’d be fine—at least we’d have the backup file. But now, instead, we have nothing.

This post will get a bit technical as we explain how, exactly, we just lost all of our data. As someone who considers himself well-versed in the technological side of filmmaking, I feel the need to explain how we were in such a precarious position, to avoid anyone posting a comment of the “always back up your data” nature.

Fundamentally, what it comes down to is this: as a no-budget filmmaker, what you don’t have is money. What you do have, is time. So while we’ve spent countless hours over the last year writing, casting, scheduling, and location scouting for The West Side, we haven’t spent a lot money.

Some things time can’t buy, however: a redundant storage system for all our data, for example. That takes money, and if we had it, we would’ve spent it long ago. But the fact is, we’re both in our mid-twenties—actually, Zack won’t even technically be in his mid-twenties until next month—and we live in New York City, which you might have heard is kind of expensive to live in. So it’s a lack of funds, not a lack of willingness to spend funds, that left us at the onset of production with a single LaCie 500GB d2 drive (Firewire 800 interface, 2X250GB SATA drives, striped as RAID 0). Months earlier, as I formatted the array for the Mac OS, I considered mirroring the drives rather than striping them, as the unit had failed in the past (after which I replaced the dead drive). For those of you who haven’t had the pleasure of dealing with these issues, “mirroring” a drive means you have two copies of every file across two drives, so if one drive fails, you still have it on the other. “Striping” is when half of each file is written onto each drive—riskier, but it gives you twice the speed and twice the space. It was the latter consideration that prompted me to stripe the drive—and, in retrospect, we would have already run out of space if I’d mirrored them. So I’m not second-guessing that decision… although that depends on how this all plays out.

Hard drive failures go with the territory of digital filmmaking—video editing demands more of a hard drive than any other task I can think of. While I rebuilt the array less than six months ago, by no means did I think we’d get through the whole season of The West Side using this sole drive. However, until the day came when we spent the money for a new, redundant storage solution, our plan was to simply use the LaCie for editing, and upon completion of each episode, copy the critical data over to a cheap external USB hard drive as a backup. If my After Effects render had completed, we would’ve had that file to copy over. Instead, here I am writing this post, about to take the drive into a data recovery service and hope they don’t charge an outrageous amount to get our files off the drive. Unfortunately, they could easily charge more than our entire budget for episode one.

To avoid this fee, we could re-capture all the footage from the DV tapes onto a new drive (our project files from Premiere Pro and After Effects are stored on the internal hard drive, and I also have FTP backups of those). But much of the post work done on The West Side—chiefly our Foley (sound-effects) work—doesn’t exist on those DV tapes. We’d have to spend a week or two just to re-finish episode one, when instead we want to be working on getting episode two out there. Also, we edited and posted episode one in Adobe’s beta/prerelease software—more on that decision in a later blog post—and so I’m not 100{b16cf85ee9cb4f6912492a96cf327bf5dfd43ef1609cd6fa23d14576a0d3e208} sure that everything would recapture correctly now that we have the final release of CS3.

Ultimately, I’m glad we were able to finish episode one for less than a thousand dollars. As a result, we’re not in debt and we don’t have to bother you with advertising on the site. But The West Side is no longer speculative—meaning, we have assets to protect and standards to live up to now that episode one’s live—so we’re going to have to eat a little less and spend a little more money on storage. Still, it’s a lot more tempting to spend that money on lighting equipment, lenses, or props—things that actually affect the on-screen production—instead of on a safety measure. If the choice of what to spend our money on is a matter of risk and reward, though, I suppose not having to write this post would itself be the reward.

Stay tuned.

The failure of the (hard drive) system | posted on July 22, 2007 10 comments
  1. Cecil 23 Jul 2007

    That sucks but a lesson learned. I hope this doesn’t hold up ep 2 for a long time. I’m really looking forward to it.

  2. David 23 Jul 2007

    I can turn you on to a company that does data recovery in California – they were the cheapest I found.
    Also, you can try to switch the controller board on the hard drive for another one – they say 90{b16cf85ee9cb4f6912492a96cf327bf5dfd43ef1609cd6fa23d14576a0d3e208} of hard drive failure is fixable without even looking at the platters.

  3. steve p 24 Jul 2007

    external hard drives are cheap. Buy two 500gb firewire drives. Total cost: $300 if you buy the enclosures and drives from tigerdirect and put them together yourself.

    Do daily backups as the project progresses, or at least do backups each time any significant amount of work gets completed. Just kick off the backup right before you stop for the day.

    Alternate which of the two backup drives you use each day. So you always have day-1 and day-2 backups. For the truly paranoid, take one backup drive offsite each day in case the building