| Home > News > Tech OnTap    

 
Printable Page
Getting More from NetApp Storage
Workload Prioritization for Optimized Performance

Akshay Bhargava
Product Engineer

Akshay Bhargava Akshay Bhargava is a product engineer on the NetApp core system team with responsibility for Data ONTAP® and the WAFL® file system. Prior to joining NetApp, Akshay served in a variety of engineering management and software development roles working on distributed computing, clustering, storage virtualization, and network security. He holds a degree in computer science from Carnegie Mellon University.


The below article was originally published in the July 2006 edition of the Tech OnTap newsletter. To receive the newsletter monthly and enjoy other great benefits, sign up today.

For years the most common approach to storage deployment has been to dedicate one or more storage systems for each major application or workload. That way, if one application overloads a storage system, no other application or group of users suffers the consequences.

Today—with storage system capacity and performance continuing to grow—this traditional approach has become less and less cost-effective. The one-storage-system-per-application approach can lead to a huge number of storage systems, each of which may be underutilized, tying up budget dollars, consuming valuable floor space, wasting energy, and complicating management.

A Simple Solution: Workload Prioritization

To minimize the need for dedicated storage, NetApp has introduced a new priority of service feature—FlexShare™—that enables you to dynamically prioritize storage traffic at the volume level and control service levels. This means you can give higher priority to the volumes used by more critical workloads and safely run multiple applications on a single, consolidated storage system.

The best news is that FlexShare is a standard feature of the NetApp operating system starting with Data ONTAP 7.2. There is no additional cost to take advantage of this advanced feature. No other mainstream storage vendor offers non-disruptive dynamic tuning of storage service levels.

Three Independent, Tunable Parameters

Each workload in your storage environment has a different level of importance to your business, so a "one-size-fits-all" approach to storage resource allocation doesn't make sense. FlexShare offers three parameters that can be configured independently on each volume to tune your storage to the needs of every application:
  • Relative priority. Assign a priority from "VeryLow" to "VeryHigh" to each volume. A higher priority gives a volume a greater percentage of available resources when a system is fully loaded. If higher priority applications aren't busy, lower priority applications can use available resources without limitation.
  • User versus system priority. Prioritize user workloads (application and end-user traffic) versus system work (backup, replication, and so on) or vice versa. For instance, give your OLTP application priority over SnapMirror®.
  • Cache utilization. Configure the cache to retain data in cache or reuse the cache depending on workload characteristics. Optimizing cache usage can significantly increase performance for data that is frequently read and/or written.
These parameters can be configured dynamically, so you can change all or some of the settings on-the-fly as your needs change, even if they change daily or hourly. Once FlexShare is configured, it directs the way storage system resources are used to provide an appropriate level of service to each application. See the sidebar for more details.

Four Ways You Can Get Started with FlexShare

By assigning an appropriate priority to the volumes used by each application, you can mix workloads on the same storage system without having lower priority work impact critical workloads. Some of the ways FlexShare might be used to increase storage efficiency and utilization in your environment include...

Consolidating Different Workloads

Most organizations have a variety of workloads—such as databases, business applications, e-mail, and home directories—that could be consolidated to simplify storage administration, increase storage ROI, and reduce the consumption of valuable resources and physical space.

With and Without FlexShare With FlexShare, you can consolidate two or more such workloads together on the same system. For instance, you might consolidate your Oracle® Database with user home directories, setting the priority of the database higher than the home directories to ensure that home directory traffic doesn't interfere with important database operations.

A database is a prime example of an application that may also benefit from the ability of FlexShare to tune cache utilization. Volumes that store log files, which are typically written but not reread, can be set to reuse cache space immediately, while volumes containing data files storing the most active indices and table spaces that are frequently read and updated can be set to keep data in cache. (For optimal results, the keep setting must be used with care. The active data set configured with this policy should be smaller than the cache.)

Mixing Fibre Channel and SATA Storage

Many NetApp storage systems support mixed storage—combining Fibre Channel disks and SATA disks on the same system. The new NetApp FAS6000 series, for example, can provide hundreds of terabytes of either or both types of storage. (See last month's discussion on FAS6070 design.) Customers typically put higher priority data on Fibre Channel disks and lower priority data on SATA disks. Using FlexShare you can ensure that access to the data on Fibre Channel volumes has priority over access to SATA volumes.

Incidentally, if you're considering mixed Fibre Channel and SATA storage, I highly recommend checking out the best practices outlined on page six of the Storage Best Practices and Resiliency Guide (PDF)

Setting the Priority of Backup and DR Operations

One of the unique features of FlexShare is that it gives storage administrators the ability to set the priority of system operations (backup, replication, maintenance tasks, and so on) relative to end-user and application traffic.

Many NetApp sites have active user volumes that are backed up or replicated at frequent intervals using NDMP, NetApp SnapVault®, or SnapMirror. It may be advantageous to give higher priority to user access during the day (peak hours). This way, backup and replication occurring during that time won't impact users. During off-peak hours, you can grant higher priority to these critical tasks to ensure that they complete in as short a time as possible.

For organizations that have a primary site and a backup or disaster recovery site, you may choose to grant higher priority to user access at the primary site, while setting higher priority for system access at the secondary site.

Multiple Instances of the Same Application

A single storage system might also be used to host multiple instances of the same application, such as several database instances. For example, Database 1 might be executing an OLTP workload while Database 2 is doing data mining. Using FlexShare to give higher priority to volumes used by the OLTP database ensures that a storage system can support both activities without the data mining work affecting critical OLTP transactions.

Get the Most from NetApp Storage with FlexShare

FlexShare gives storage administrators a new level of control over storage system resource allocation and performance. By bundling advanced features such as FlexShare into its base operating environment at no additional cost, NetApp is helping you get more from your NetApp storage than ever before.

You can learn more about FlexShare from my recently published FlexShare Design and Implementation Guide. This guide provides detailed information on how FlexShare works, how it is administered, best practices for deploying FlexShare, and more.


The information in this article is useful to me.
1 2 3 4 5
Strongly Disagree Strongly Agree

What other types of information would you like to see included in this article?


Additional comments:

  Thanks! 



 
New NetApp FAS6000! Data Center Proven
Related Information
FlexShare Design and Implementation Guide (PDF)
FlexShare Demo
Architecting Storage for Resiliency
FlexShare in Action
The following graph shows how a high-priority application can benefit from FlexShare:


Click to launch the FlexShare demo.

To perform this test:

  • A single NetApp system was configured with two identical volumes.
  • SIO was used to simulate an OLTP workload against each volume (R/W 40:60, random: 70%, block size: 8KB).


    With FlexShare off, each volume delivered a workload of about 6,000 disk ops per second for a total of 12,000. Priority on one volume was then set to "VeryHigh," while the other was set to "VeryLow."

  • The high-priority volume climbed to an average of around 9,000 ops, 50% more than without FlexShare and 3X the low-priority volume.
  • The low-priority volume dropped to an average of about 3,000 ops.


    Total operations remained at 12,000, indicating that FlexShare has no significant system overhead.

    Watch the FlexShare demo.
    How FlexShare Controls System Resource Allocation
    FlexShare works by controlling the allocation of critical system resources based on the volume priority level and buffer cache policy set by the storage administrator.

  • CPU. Disk operations on higher priority volumes are preferentially scheduled for processing.
  • Disk I/O. Higher priority volumes are allowed more concurrent disk reads than lower priority volumes. FlexShare maintains a maximum number of concurrent disk reads allowed per volume.
  • NVRAM. A volume's priority dictates how much NVRAM it can consume relative to other volumes to ensure high-priority applications do not get write starved.
  • Memory. Memory consumption is dictated by the configured buffer cache policy, which provides hints to the buffer cache manager:


    – Keep. The cache manager preferentially keeps items in the cache.
    – Reuse. The cache manager preferentially flushes these items when clearing space in cache.
    – Default. The cache manager gives data items normal priority. Keep items have higher priority, and reuse items have lower priority.


    Want to learn more? Read the NetApp technical report.
    The Versatile Storage Platform
    To truly appreciate the versatility of the NetApp architecture, it is important to view how the storage is managed and how the storage is accessed as related functionality. Too often storage vendors separate these two concepts, which creates overly specialized storage systems that eventually become isolated islands.

    In his May Tech OnTap article, NetApp user Ben Rockwood provided an overview of how the NetApp Data ONTAP operating system manages data on disk and how that data is accessed from client systems.

    Highlighted technologies include:

  • RAID, RAID-DP™, and traditional volumes
  • Aggregates and FlexVol™ volumes
  • Snapshot™ functionality and FlexClone™ technology
  • LUN creation and masking


    Read a User Perspective on Data ONTAP 7G.