Professional and hobbyist photographers alike face a common problem – storage space. This article explains which brand is the best hard drive for photographers to use for long-term storage of their irreplaceable photo library.
For long-term storage of a photo library, photographers should consider the following:
- Use 2TB or 4TB SATA III (6GB/s) 7,200 RPM 3.5” (not the 2.5” drives for laptops) drive(s) with 64MB cache (avoid the 3TB size).
- Avoid “green” drives as they are designed to be more power efficient at the cost of performance, which might be OK for long-term storage except that they are also designed to start and stop a lot more which tends to cause early failures (sort of like how a light bulb blows when you turn on the switch).
- New drives tend to fail early (within the first 12-18 months) or late (after 3+ years), so after you get a new drive it is good to put it through a “stress test” before you trust it for your photo library. One good way to do this would be a “secure erase” utility. There are several suggestions on tools to do that here.
- Plan to replace your long term storage hard drive(s) about every 3 years. Some last beyond that, but the odds increase significantly after year 3 of using a magnetic drive that it will fail.
- As of the creation of this article, here are the top 3 consumer brands:
- HGST Deskstar is the top brand to trust. While all sizes of these drives seem to do very well, the 2TB and 4TB size had smaller failure rates that the 3TB size.
- Seagate Barracuda drives narrowly have the best price to performance (as far as failure rate) of any brand. You will want to avoid the 3TB size, specifically the Seagate Barracuda 7200.14 3 TB that had an unusually high failure rate.
- Western Digital Red drives actually have a smaller failure rate than Seagate drives, but they tend to be more expensive than HGST or Seagate without more reliability (there are other benefits). Again, stay away from the 3TB size (what is up with 3TB?).
You are free to disagree with the recommendation, but please read on to get some of the gory details behind it before you do.
If you haven’t had it happen yet, just give it some time and it will – the click of death. Magnetic hard drives, the big ones you are using for at least your long term storage, are ticking time-bombs. These hard drives are a necessary evil for photographers. Necessary because digital photography means we have a lot of pretty large digital files that have to be stored on drives. Evil because these drives with large amounts of storage (1TB up to 6TB) are mechanical and they will fail. There is nothing you can do to prevent it. Although there are some things you can do to prolong their life – more on that later.
No matter how long you have been using hard drives, you do not have enough experience to know which brand is most reliable. If you are like me you may have sworn off some brand or other because at one point you were bit when one went bad and you didn’t have a good backup.
However, not only is there no way you have had enough experience with hard drives to draw that type of conclusion, this isn’t something that stays constant over time. Just like everything, manufacturers leap frog each other over time. Don’t fall into the trap of thinking every hard drive from some manufacturer is terrible because you had one fail on you after a short 6 months and their customer service was awful.
While cloud storage may not be the best way to backup your photo library, there is a huge side benefit to the cloud storage frenzy going on right now with providers like Google, Microsoft, Apple, Facebook (kind of), and others. They are getting experience with hard drives on a scale that makes failure rate statistics actually meaningful vs. anecdotal. I just wish all of them would publish that information.
In January 2015, Backblaze – a decent sized cloud backup provider – published some data about their experience with the hard drives that make-up their cloud offering in a report called What is the Best Hard Drive? As an IT professional by day, I find the report to be incredibly interesting, especially because many in the tech industry don’t agree with the conclusions. Experts refute the report saying it is invalid because Backblaze is using drives designed (and priced) for consumers in a way they were not intended – in a data center. However, I think this actually works to a photographer’s advantage. We buy consumer hard drives and then put them through the paces more than an average consumer.
In 2014 Backblaze increased the number of hard drives running in the data center from 27,134 to 41,213. That is about a 65% increase, which means their business is doing well, but it also means they have a lot of experience with hard drives. Far more experience than any photographer can possibly have, so we have to rely on some kind of 3rd party data to help us here.
I do agree with the argument that Backblaze wasn’t doing a scientific study. They were watching their bottom line, which means there weren’t controls like an equal number of drives from each vendor (far fewer from WD than the others due to their cost). Still, I don’t know of a better source for the data, and anecdotal evidence doesn’t seem to contradict the findings much, so I am going with it.
What Does It Mean?
To me, whether you agree with the method of the report or not, there is some really good news in there. The 4TB sized drives Backblaze added to their data center in 2014 had very low failure rates. The brand with the highest failure rate (Seagate) was a very acceptable 2.6%, and some of the models had failure rates below 1%! That is really encouraging. Sure, you could still end up inside that failure rate, which would not be fun, but it should mean that you have a really good chance that a 4TB drive from any of the manufacturers is going to live a good life.
The data suggests a photographer should have the same confidence about 2TB drives that have been in Backblaze’s data center for even longer and more than half of them have gone better than 3 years without failure. Still, as I mentioned at the start of the article, it would be best to plan to replace any magnetic hard drive after 3 years of service. You could move them to be scratch disks or backup drives perhaps, but don’t store your main photo library on them after 3 years.
The bad news from the report is that the 3TB sized hard drives in 2013 were quite a bit worse. Not sure what happened there, Backblaze has said they have a theory they are going to write about later, but all of the manufacturers seemed to struggle with the 3TB size (HGST was actually just fine and Seagate significantly worse than the other two). Thus, the recommendation to stay away from that size of drive if you are buying today.
When Will Your Drive(s) Fail?
I imagine all of this discussion about drive failures has made you wonder what you are using, and how much time you have. Did you stop reading and go check to see if you have any Seagate 3TB drives? Sure glad I didn’t pick up any of those 3TB drives. Oh, wait, my 44,498 long-term photo library storage is currently on a single Western Digital Red 3TB drive. It has been in service since March 2014 for me. Besides the silliness that it is on a single drive, which of course I wouldn’t recommend, am I living on borrowed time? According to a report from Google, it is nearly impossible to tell.
For a long time now, modern hard drives have had a feature in them called “Self-Monitoring, Analysis and Reporting Technology” (SMART). The idea is that as the drive is used over time the little computer inside the drive can keep track of the errors (which happen way more than you want to know) and warn you the drive is about to give up. Cool idea, but in a much more scientific report, Google says that SMART data is not a good indicator of pending failure. In fact, Google says “Given the lack of occurrence of predictive SMART signals on a large fraction of failed drives, it is unlikely that an accurate predictive failure model can be built based on these signals alone.” Which I take to mean it isn’t possible for such a thing to work.
If you are hoping somehow your computer or NAS device is going to tell you that a drive is on the way out and give you some time to order a new one before it actually gives up, don’t count on it. The most likely way you will know your drive is in trouble is when your computer (or NAS) can no longer read it.
So, back to my 3TB WD drive. Google, Backblaze, and other studies agree that the fact it has nearly made it 12 months means it is more likely to go at least 3 years before failing. No guarantees, it could die as I am writing this article, but the odds are in my favor that my library is safe for another 2 years. At least it isn’t a 3TB Seagate Barracuda 7200.14, which Backblaze reported as having a failure of more than 40%.
Burn-In Test a New Drive
You may have heard of “burn-in” type testing before. It is very common among tech enthusiasts who overclock their computers (run them faster than they were intended) to get more performance. Doing so can cause instability, so they run tests that are designed specifically to put a lot of load on the computer and validate that the settings they are using won’t cause the computer to fail when they are doing something real.
You should really consider doing the same thing with a new hard drive before you trust it to store your photo library. I already talked about how the statistics have shown drives tend to fail either in their first year, or after year 3. One way to test if your brand new drive may fail in the first year is to do a really good “burn-in” test. Meaning you need to find a way to read and write to every spot (called a sector) of the drive multiple times. Formatting your hard drive, even a full format, doesn’t do this (unless you do something specific on Mac).
A really good option for this would be tools designed to securely erase a hard drive. These are tools that wipe the hard drive clean so that if you sell it or give it away your data is no longer on it. In order to make data recovery from the hard drive impossible, every sector on the drive has to be written over multiple times. For this reason secure erasing a drive is often called “zeroing” the drive because zeros are written to every sector. Again, formatting the drive does not do this.
This is exactly what you want to do with a new hard drive, because if it can make it through writing zeros to every sector over and over (like say for hours), it should be able to handle the far less frequent writes you will be making to the drive with your photos.
Gizmodo has a pretty good article on a number of secure erase tools that are available for this purpose here. As a PC user, my personal favorite is Darik’s Boot and Nuke (DNAB). It is a free and open source tool that can thrash a hard drive for as long as you want it to. I also like Steve Gibson’s commercial Spinrite tool for not only testing a hard drive, but recovering them. Both tools require you to create a bootable CD or USB dongle and booting a PC from them, so they won’t help any Mac users (although Steve Gibson is working on a new version that will work with Mac) and can actually be a bit of a pain for PC users.
For Mac users, you can use the native format capabilities in OSX for this so long as you follow these steps, paying special attention to step 8:
- Connect the drive to be wiped to your Computer.
- Press Command-Space Bar to bring up Spotlight Search.
- Type Disk Utility and press Return to launch.
- When Disk Utility launches, select the drive to be wiped from the left pane. Be sure to select the drive partition and not the drive itself.
- Click on the Erase tab in the right pane.
- Select the Format for the new partition.
- Name the drive something like WIPED or TESTED.
- Click Security Options and select the option that makes you most comfortable. I recommend the DOE Compliant (3 Pass) option to handle most situations.
- Close the Security Options window.
- Click Erase
Depending on the size of the drive, a DOE-Compliant wipe will take 2-6 hours to finish. Start the wipe at the beginning or end of your workday and let it run in the background. You may want to run it 2 or 3 times too, depending on how comfortable you want to be with the drive testing good before using it.
Should You Buy “Enterprise” Drives?
My recommendation is no, don't spend the extra money it takes to get an “enterprise” or “NAS” class drive – although I couldn't find anything else on Amazon for the HGST brand. Anyway, even though a photographer is going to use a hard drive a lot more harshly than many other consumers, you still aren't going to challenge it as hard as a data center would. Again I want to point to a post from Backblaze here where they talk about their experience with both types of drives.
Backblaze has far more experience with consumer rated drives than those rated as enterprise class, but in their limited 2 year experience with enterprise drives the failure rates so far have been very similar. This is not to say there aren't other reasons to buy enterprise class drives – the warranties are much better for example. But as far as failure rates Backblaze says “Are Enterprise Drives Worth The Cost? From a pure reliability perspective, the data we have says the answer is clear: No.”
I am going to continue forward with consumer grade hard drives for the foreseeable future. I may change my opinion here as I see more data in the future, like if data comes out several years from now saying that enterprise class drives have a longer lifetime than the 3 years you really have with consumer drives. Even then it may not be worth it since the drive technology is also changing and there are likely benefits to be gained from getting newer drives every 3 years even if the drives can last longer than that. For now I don't think there is a big enough advantage to my use of a hard drive as a photographer to merit spending extra money on an enterprise class drive.
Whether you are adding in internal drive, putting a drive in a USB or Thunderbolt enclosure, or filling up a RAID appliance (like a Drobo, G-Tech, Synology, Buffalo, etc), I would use either 2TB or 4TB drives made by HGST. I would avoid 3TB drives because for whatever reason they seem to have a higher failure rate across all the manufacturers.
When you get a new drive, put it through some work like use a “secure erase” tool before entrusting it with your precious photos (formatting the drive is not a good test). This should help validate it isn’t going to fail early on.
As soon as you buy a hard drive, plan to replace it within 3 years after starting to use it.
Save a little cash on the drives and stick with the consumer rated versions, the “enterprise” or “NAS” rated drives don't seem to change the failure rates.
May the odds every be in your favor J