Agilent Technologies is the world's premier measurement company and a technology leader in communications, electronics, life sciences, and chemical analysis. A spin-off of Hewlett Packard in 1999, Agilent has 19,000 employees, serves customers in more than 110 countries, and had revenue of USD$5.4B in fiscal year 2007.

Backup Challenges

One of our key research and development divisions is based in America in Southern California. This division has about 30TB of storage and about 200 computer-savvy engineers.

In 2005, the backup environment for this Southern California site consisted of predominantly network-based backup to tape. We had about 300 systems on the network being backed up to centralized tape libraries. To meet our SLAs, we were keeping copies of the backups locally so we could do quick file restores, and we were also duplicating tapes for off-site storage and disaster recovery. The whole environment was getting to be very challenging to support. Our key challenges included:

• Long backup windows–approaching or exceeding 24 hours
• Backups were dragging down system performance
• Complicated management
• Low reliability
• No remote office support
• Labor-intensive tape duplication process
• Increasing costs

Tape just wasn't scaling for us. We were seeing huge data growth year over year, and continuing to address that problem with backup to tape was no longer getting the job done.

Storage Infrastructure Overhaul

We were already pursuing the idea of centralizing the business data that was spread across our distributed systems down to a few storage devices in order to make it easier for the R&D staff to manage their data and to improve our storage utilization and data availability. However, the real clincher on this strategy came when we spoke to NetApp and learned about SnapMirror® and SnapVault technology available with NetApp storage systems. We knew we had found a backup solution that offered the sea change we needed to overcome our existing backup limitations.

We wound up deploying a number of frontline NetApp storage systems–FAS900 and FAS3000 series systems–and centralizing all of our data on them for primary storage. We also deployed a NetApp R200 as a second tier. That allows us to use our FAS storage for what it is intended–high-performance storage–and use less expensive storage for backups.

Client systems primarily access the centralized storage using NFS and CIFS, which provides our engineers the most flexibility in reaching their data from anywhere on the network. We had also been running Fibre Channel SAN in the past for a few servers that required greater I/O performance, but found SAN to be terribly complex to support and expensive to maintain. For the situations where we still needed SAN levels of performance, we started relying on iSCSI, which we´ve found to be very fast, reliable, and much more cost effective.

We adopted SnapVault for backing up our primary storage systems to our secondary storage; we back up that secondary storage to tape using NDMP for disaster recovery purposes. Because we only do tape backups once a week now, as opposed to backing up to tape every day, we have the luxury of taking as much time as we need to complete those tape backups and have dropped from a couple dozen tape drives down to four.

Figure 1: Agilent Network Architecture

For those few systems where for whatever reason we couldn't use centralized storage, we adopted Open Systems SnapVault (OSSV) to allow those legacy systems to utilize the new backup paradigm. We also use SnapVault to back up data from remote office storage systems in Santa Clara and Atlanta to the R200 in our main location. We typically run those backups on the weekend, and just keep enough local Snapshot™ copies on the remote storage systems to allow end users to do any necessary file recoveries during the week.

SnapVault Impact

The biggest benefit for Agilent is that SnapVault has significantly cut our backup times. In some cases, we went from 24-hour time spans down to minutes. The whole process of replicating data from the primary storage systems is pretty darn fast. Plus, it's nowhere near as system intensive as the old backup to tape, where you always had to walk through the whole file system, creating a big performance drag. SnapVault knows what blocks have to move in advance and just gets it done. By eliminating the backup burden from our servers and primary storage, we're able to truly use those systems to meet the business needs that we bought them for in the first place. 

Here´s a list of what I see as the key benefits:

• Backup times cut from many hours to minutes
• Eliminated burden on primary storage and servers
• End users can do their own file restores without assistance from IT, which improves their productivity and ours
• Much greater reliability
• Cost savings as a result of ease of management

If we had tried to scale our old backup architecture to meet the needs of all three sites, we would have had to invest well over $400k in tape hardware and software in order to meet our backup SLAs, and we still would have been struggling with the same old labor-intensive support issues that had always caused us so much pain. By contrast, the SnapVault-based backup solution cost us less than $300k to deploy, and reduced our support burden by about 500 person hours a year over the old solution.

Overall, it's just been a much happier experience dealing with this new backup environment. We have a lot more confidence, and the user population is a lot happier. The IT team is a lot happier. We're saving money on the deal. So I think it's just been a real win all the way around.

---