The IT challenge is how to provision functional services such as e-mail with the right data service attached at the right cost. Or to put it differently, e-mail has to have the right number of backups, the CEO's e-mail must not be deleted, and the e-mail service has to tolerate the right number of failures without breaking the budget. Failure to do so results in down time that immediately affects business, especially for mission-critical applications like Microsoft® Exchange Server.

Data ONTAP® provides many of the basic building blocks that can be used to deliver the appropriate data service at a fraction of the cost. For example, consider an application that requires hourly backups. Using Snapshot™, a storage administrator can perform a near instantaneous backup with minimal downtime.

The problem is that for most applications, any data management operation such as backup must be coordinated with the application and in many cases with the underlying operating system. Without such coordination, the backup is only crash consistent.

To address this challenge, NetApp has created a suite of intelligent data management offerings that allow applications and storage to coordinate activities to make backup fast and space efficient, speed the restore process, and simplify common data management tasks. NetApp users today can choose from a suite of five SnapManager offerings that coordinate data management activities with many of the most popular enterprise software programs.

A Short History: Why Integrate Storage with Microsoft Exchange?

Our application integration strategy and SnapManager for Exchange product began life when NetApp proactively partnered with Microsoft on new ways that we could add value to Microsoft applications. The Microsoft Exchange team, in particular, learned about our Snapshot technology and asked whether we could develop a script that would allow Exchange to be backed up very frequently using Snapshot copies. Dennis Chapman, a senior technical director at NetApp, was asked to look into the problem. Dennis quickly determined that a scripted solution was possible, but he observed that a scripted solution would force Exchange to shut down while the Snapshot copy was in progress. In his exploration he discovered that it was possible to make Exchange think that taking a Snapshot copy was just a really, really fast tape backup. The advantage of this approach was that Exchange could stay online during the entire backup process.

Based on this insight, NetApp built the original SnapManager for Exchange as a tool targeted to Exchange administrators that enabled very fast backups of Exchange using Snapshot copies. Over the next several years, as Exchange and Data ONTAP evolved, so did SnapManager for Exchange.

The Creation of SnapManager and SnapDrive

The big change that occurred for SnapManager for Exchange was the introduction of FC and iSCSI support in Data ONTAP 6.3. Until that point, SnapManager for Exchange was tightly coupled with Microsoft Windows®, Exchange, and Data ONTAP. One engineering challenge was that every release of Windows, Exchange, or Data ONTAP required a new release of SnapManager for Exchange. The other challenge was that the team wanted to add more instances of SnapManager, in particular SnapManager for SQL Server™. The team decided at that point to split SnapManager for Exchange into two parts:

Figure 1) Roles of SnapDrive and SnapManager.

• SnapDrive understands and coordinates activities between the host operating system and file system and NetApp storage.
• SnapManager understands and coordinates with the application (and with SnapDrive).

Splitting the functionality makes it possible to leverage the underlying capabilities of SnapDrive in the construction of new SnapManager offerings tailored to specific applications, thereby simplifying the development process. SnapDrive and SnapManager are able to reconcile the host's notion of where blocks live with the storage system's notion so the host can leverage all the underlying capabilities of NetApp storage such as Snapshot, FlexVol®, FlexClone®, SnapMirror®, etc.

Host/Storage Integration – Automatic, Near-Instant Operations
NetApp Tool: SnapDrive

Applications running on a host with SnapDrive installed (even without an appropriate SnapManager product) are able to use LUNs on NetApp storage systems as if they were local resources, while benefiting from underlying NetApp technologies such as Snapshot and cloning. Most tasks occur in seconds, so operations are not disrupted, and the operator doesn't have to know anything about NetApp storage or the specific networking technology (such as iSCSI or Fibre Channel).

The current generation of SnapDrive products includes SnapDrive for Windows and SnapDrive for UNIX® (with support for Solaris™, HP-UX, AIX, Red Hat, and SUSE Linux®). Both solutions support iSCSI and Fibre Channel transports with multipathing for performance and reliability. SnapDrive for UNIX also works over NFS.

SnapDrive helps host administrators provision storage and manage it directly from the host, enabling them to flexibly define backup policies and resize storage on the fly without any disruption of application service. SnapDrive understands the operating system, volume manager, and file system details necessary to coordinate activities with the host and to appropriately map host file systems to NetApp storage and WAFL®. Both versions simplify storage and data management by hiding the complexity of steps that must be executed on both the storage system and the host system, and by removing the dependency on the storage administrator.

Key SnapDrive functionality includes storage provisioning on the host, consistent Snapshot copies, and rapid data recovery from Snapshot copies. SnapDrive complements the native file system and volume manager technology and integrates seamlessly with clustering technology for high availability. Role-based access control (RBAC) prevents administrators from performing actions on LUNs for which they don't have permission.

Making these operations transparent to the user is not as simple as you might think. For instance, to create a consistent Snapshot copy, you must ensure that all file system data and metadata is flushed to disk and the file system is quiescent. That requires the following general steps:

1. Create a list of LUNs to be included in the Snapshot copy. A Snapshot copy can span volumes or even multiple storage systems.
2. Stop activity on all affected volumes.
3. Flush data from the host operating system for all volumes in the Snapshot copy. (In Windows 2003, this requires integration with Microsoft Volume Shadow Copy Service, VSS.)
4. Create the necessary Snapshot copies on the storage systems.
5. Resume operations.

The NetApp and SnapDrive advantage – aside from the fact that all this happens more or less automatically without the operator having to know anything about the underlying storage – is that the entire Snapshot copy creation process can be done in a few seconds, so operations aren't disrupted.

Once you have a consistent Snapshot copy on the storage system, you can keep it on primary storage in case you need to restore a file or even roll back to that point in time. You can also back the Snapshot copy up to tape if desired, use NetApp SnapVault® to back up the Snapshot copy to secondary disk, or use NetApp SnapMirror technology to make a copy for disaster recovery. SnapDrive integrates with SnapMirror and SnapVault to do this automatically, when configured to do so.

NetApp continues to enhance SnapDrive functionality. To learn more about recent SnapDrive features, NetApp CSE Nick Triantos describes thin provisioning and space reclamation in SnapDrive for Windows 5.0 and self-service storage management with SnapDrive for UNIX in two excellent posts on his blog.

Application Support Streamlines Backup, Recovery, & More
NetApp Tool: SnapManager

The SnapManager set of offerings currently supports five popular enterprise applications (Microsoft Exchange, SQL Server, and SharePoint®, plus Oracle and SAP), which coordinate activities at the application level while leveraging the underlying capabilities of SnapDrive to coordinate with the host OS and storage system. (Another feature in this issue highlights the unique capabilities of SnapManager for SharePoint.)

For example, consider the backup process with SnapManager for SQL Server (SMSQL), which carries out the necessary steps on the application side while SnapDrive for Windows performs essentially the same steps as described in the previous section:

1. A DBA requests a backup of a database: MySqlDb.
2. SnapManager for SQL freezes MySqlDb.
3. SnapManager for SQL generates a list of the drives used by MySqlDb.
4. SnapManager for SQL passes the list to SnapDrive.
5. SnapDrive determines which LUNs the drives are attached to and the corresponding FlexVol volumes.
6. SnapDrive stops activity on those LUNs and flushes data cached in the host operating system.
7. SnapDrive performs the Snapshot copy.
8. SnapDrive resumes host operations and signals success to SnapManager for SQL.
9. SnapManager for SQL unfreezes the database.

Figure 2) Database Backup Process.

Again, despite the seemingly complicated process, the whole thing happens in a matter of seconds and the DBA doesn't need to know anything about how data is organized on the storage system or how it is sent over the network. SnapManager and SnapDrive work together to enable the application administrator to get the job done without having to constantly involve the storage administrator.

Typical capabilities that the SnapManager tools provide for applications include backup, restore, cloning, and disaster recovery, as well as scheduling and retention. Each tool typically has features that are specific to that application. For instance, Single Mailbox Restore is an option with SnapManager for Exchange, and SnapManager for Oracle has the ability to integrate and coordinate with RMAN, ASM, and other critical features of Oracle. A command line interface (CLI) makes it possible to write scripts to integrate SnapManager actions such as Snapshot copies as part of the normal operation of an application.

Customized Application Management beyond Popular Platforms
NetApp Tool: SnapDrive (Command Line Interface)

Even though NetApp is committed to continue to add integration for popular software, we obviously will never be able to provide specific versions of SnapManager to cover all the myriad commercial applications in existence (or in-house applications that aren't built on SQL Server, Oracle, or some other database).

However, SnapDrive does include a command line interface sdcli, so that any application that can invoke a script can access SnapDrive functionality for backup, restore, etc. The following simple batch file is an example that one NetApp customer used to back up a database using SnapDrive for Windows. The database was put in hot backup mode and the script was called. When the script returns, hot backup mode is turned off.

Get More from Your Application

If you need host-level storage integration or support for a homegrown app, SnapDrive by itself can provide an excellent solution that can simplify and accelerate many of the most common data management tasks. If you need to integrate your NetApp storage with Exchange, SQL Server, SAP, Oracle, or SharePoint, the SnapManager suite of tools can help you get the most value from your storage and your application. Using these tools, server and application administrators can perform important storage tasks without storage administrator assistance.