New rules on data retention, such as the Markets in Financial Instruments Directive (Mifid), are prompting many storage managers to take a closer look at how they archive information over long periods of time. But choosing the right technology for the job is not an easy decision.
A range of media, including spinning disk, tape and removable disk formats, are available, and each one has its pros and cons.
With the cost of high-capacity hard disks plummeting over the past few years, many vendors have been pushing Raid arrays as the ideal storage solution for data that is rarely accessed but needs to be kept safe for a set period nevertheless.
Mike Walters, consultant systems engineer at NetApp, which supplies disk arrays based on low-cost Serial ATA (Sata) drives backed up by software enhancements such as de-duplication, said his firm’s solutions are designed to enable companies to store more data in less physical space.
Spinning Disk not cheap
Walters admitted that spinning disk archiving solutions are not the cheapest option, but added that they have advantages over other technologies that more than made up for the price premium. “It is the ongoing management costs that make spinning disk more attractive. Re-tensioning tapes and moving data from one tape technology to another every few years is labour intensive and prone to data loss, whereas migrating Raid storage is easier, proven and more reliable,” he said.
Both NetApp and EMC are recommending the use of spinning disk for long-term archiving, and data de-duplication can make that more economical from a cost perspective. Steve Mackey, UK area sales director at storage specialist Quantum, agreed that disk is the right medium for short-term archiving and data reduction. But he believes that tape is rapidly becoming the dominant medium for long-term archiving because it is both cheaper to buy and maintain over long periods.
“Disk or de-duplicated disk is ideal for short-term retention, but anything that is going to be retained for years is recorded onto tape,” Mackey said. “The customer has to ask how long it wants to store the data for, whether 12 months or 12 years, for example, and look at how much it costs not only in acquisition costs but to power, maintain and cool those spinning disks as well.”
Mackey added that Linear Tape-Open (LTO) is guaranteed for about 30 years, “though the challenge is more about having the drive technology available to read it”. He also pointed out that keeping tape media on a shelf consumes no power. “You have to have reasonable environmental conditions if you want to keep the data intact long-term, but it does not have to be air conditioned,” Mackey explained.
Though many consider the use of optical write once, read many (Worm) media as a niche technology, Dave Lipsey, information systems infrastructure manager at mapping agency Ordnance Survey, believes Plasmon’s UDO (Ultra Density Optical) media is ideal for his storage requirements.
UDO disks are more expensive to buy than tape cartridges, and some complain that UDO arrays do not scale well. The disks are guaranteed for 50 years, however, and do not have to be stored in special environmental conditions to ensure data integrity, pointed out Steve Tongish, Plasmon’s European marketing manager.
“Not many people use tape for long-term archiving, it is used more for backup,” Tongish said.
Lifespan and migration issues will never go away
Despite UDO’s longevity, Lipsey conceded that the technology is no different from tape in that there will always be lifespan and migration issues to be considered, particularly when new read hardware is released, and when applications and operating systems become obsolete. “At a certain stage, it will not be surprising if there is an obsolescence issue, which might lead to migration problems. When there is a big jump in technology, we need to be aware of the implications for existing data. In some cases it might be worth migrating it, in others not,” he said.
Quantum’s Mackey agreed those migration issues actually vary very little across different storage mediums. Any company progressing from a simple backup strategy towards implementing a long-term archiving solution will have to address them, irrespective of the hardware and software they use.
“Technology lifecycles are not vastly dissimilar for tape, spinning disk or optical disk. Whatever media you choose, if you have to archive it for longer than seven years, you have to plan to migrate it from one technology to another,” Mackey said.
NetApp’s Walters said the first step should be for companies to identify the specific applications they want to archive before they choose the technology, with email archiving having different requirements to other forms of data. “Cost is always going to be important, but it is not the overriding factor,” he added. “Putting data onto the platform that best suits how the company wants to recover it is more vital.”
‹ www.netapp.com ‹ www.quantum.com
Ordnance Survey does not trust in tape
The Ordnance Survey (OS) is Britain’s national mapping agency, employing over
1,400 people worldwide. Each aerial photograph it takes is around 700MB in size
once 3D and spatial information has been added, all of which needs to be stored
safely for many decades.
Last year, the agency deployed a combination of Bridgehead Software’s HT FileStore software and Plasmon’s G638 UDO write once read many (Worm) removable disk library in order to accommodate the 40TB of pictures and related information it estimates it will need to archive annually.
The UDO cartridges are guaranteed to keep data safe for 50 years, while the HT FileStore software handles automatic migration of data to new media according to the customer’s requirements.
By choosing UDO, which it will replace with UDO2 this year, Ordnance Survey believes it can operate on an eight- to 12-year data migration cycle, which is better than either spinning disk or tape archiving solutions that operate on equivalent cycles of three to six years, according to information systems infrastructure manager, Dave Lipsey.
“I do not trust tape. If you keep it in a safe environment for 10 years it might be OK, but as soon as you take it out and chuck it in a van, it might fail,” Lipsey said. “CDs and DVDs need careful handling and storage, and there are often problems with migration. Transfers do not happen over a weekend; they can take many painful months.”
Lipsey estimated that the solution will save the OS almost £500,000 in
storage costs compared to using online disk storage, and will break even within
four years.
The previous hard disk storage system was costing around £2,500 per TB, whereas
using UDO and HT FileStore costs around £1,200 per TB in the short-term, falling
to £600 per TB in the long-term, Lipsey added.
‹ www.bridgeheadsoftware.com
‹ www.plasmon.com
