The Data Storage Conundrum

We all need to store data: our documents, photos, music files, video files, and more. As time goes on, we have more and more data to store. In addition, we need to backup all that data. I have often said that is not a question of whether a hard drive will die, but when it will.

As such, developing a strategy for storing data can be complicated. You have data on your computer, and if you have a large music and/or video library, you most likely ha additional data on an external hard drive. In addition, you need backups for all that data. The best backup strategy includes multiple backups: one or more Time Machine backups, clones of your startup drive, and redundant backups of your media. Because never forget that one back up isn’t enough: you should always have at least two, in case you lose your original data and you find that your backup is corrupted.

I have a 27″ iMac with a 256 GB internal SSD, and a 4 TB external drive for my media. I also have an additional 2 TB drive for other data: software installers, archives, and other miscellaneous files.

I use two Time Machine drives to back up my startup drive and my music library. I have two redundant backups for my media drive; this means that my music files are backed up both by Time Machine and these redundant backups. My video files, mostly rips of DVDs and Blu-rays that I own, are only backed up twice. As for that extra 2 TB drive, it, too, has double backups.

All this comes at a price. I have lots of hard drives. I have a total of five units, four of which each hold two hard drives. Two of these units are connected to my Mac by a Thunderbolt, and the other three are USB-3 drives.

I would love to simplify this. I would love to have, say, one unit to store all my data, and another unit to back it up. But it’s not that simple. I’m not comfortable with a RAID unit, because the data is not recoverable unless the hard drives are in the exact same RAID unit. In addition, RAID units are noisy. Since they have so many drives, and processors, they need fans. All of the hard drive units I have are fanless, and the only noise they make is that the hard drives spinning. My drives in the shelf unit with boxes in front of them to dampen the noise.

You can buy enclosures that hold multiple drives and don’t use RAID, or configure a RAID unit as JBOD, or “just a bunch of drives.” In that case, each drive appears as a single drive on your computer, whereas a RAID unit shows all of the storage as if it were one drive. But these devices have the same problem: they have fans, and they are noisy.

Another option is using network drives. They would allow me to use either a RAID unit or a multiple-drive enclosure in a location other than my office. However, the limitation of network speed would be problematic at times. Gigabit ethernet may sound fast, but when you’re copying a lot of files, it’s not. Both Thunderbolt and USB-3 are much faster. As such, any device that is connected to a computer will copy files more quickly. This isn’t a big problem for, say, incremental backups, where only new or changed files get copied. If these happen over the network in the background, it doesn’t slow much down, and since these generally run at night (with the exception of Time Machine backups), I wouldn’t notice them anyway. But when you do need access to large files, it is slow. In addition, I would have to run an ethernet cable into another room, because Wi-Fi isn’t fast enough.

So what’s the solution? For now, I haven’t found an ideal solution. Perhaps larger hard drives will make all of this easier: instead of meeting, saying, two 4 TB drives, one 8 TB drive would be enough. So I could cut the number of drives I use in half. But I still need at least two separate drives for Time Machine backups, and at least two separate drives to backup my media files. So I’m not even sure that larger drives will make that much of a difference. Because of the fragility of hard drives, storing data really is a conundrum.