If your company is like most organizations, some unlucky IT staffer (probably you!) spends many painful hours managing and backing up servers with direct-attached or internal storage.

Every day, I work with our top enterprise customers to help them build consolidated networked storage environments. As soon as we start talking about storage, I find most people already have an opinion about which protocols are the "right" ones.

However, "one-size-fits-all" doesn't work for T-shirts, and it certainly doesn't fit your complicated IT environment. Protocol decisions should be based on data criticality, availability, performance, existing infrastructure, IT expertise, and budget. Frankly, one's philosophical bent also plays an important role in the decision-making process.

Here are key considerations to keep in mind as you evaluate your requirements and consider different storage protocol options.

Do your applications require block or file I/O?

Application requirements should always be the first thing to dictate your storage decisions. Many of the enterprises I work with are focused on consolidating direct-attached storage for applications that require block-based I/O, so that's what I'll focus on here.

What levels of redundancy do your applications require?

Mission-critical environments demand full redundancy with no single point of failure. Multipathing capabilities and clustered host configurations are common.

I used to automatically recommend Fibre Channel for full redundancy environments because all of the original iSCSI solutions offered NIC teaming rather than a true multipathing capability. Although NIC teaming is a proven and mature technique, it does not provide a fully redundant configuration because the aggregated interfaces must be connected to the same network, often to the same switch.

However, the technology has evolved.

Today you can achieve the same levels of redundancy for Windows® environments using the Microsoft® iSCSI software initiator 2.0, which provides a proven and stable approach to multipathing: MPIO (multipathed I/O). In fact, MPIO is not the only approach to achieving path redundancy in a Windows iSCSI environment. The iSCSI specification provides its own features for high availability and bandwidth aggregation, including multiple connections per session (MCS).

Both MCS and MPIO allow failover among multiple NICs, and both can support I/O load balancing as well. MCS offers multiple TCP/IP connections between the server and the storage device during the same iSCSI session. These can be on different physical links as well. MPIO lets the server log multiple sessions to the storage array.

Net:Net — Fibre Channel is currently the only option enabling full redundancy for block-based applications in UNIX® and Linux® environments but is no longer the default choice for Windows servers.

Do your applications use small or large block I/Os, how many I/Os per second (IOPS) are being pushed, and how much bandwidth do you need?

In my experience, more than half of our customers do not have a firm grasp of their performance requirements. For small block I/O applications, for example, the critical factor is IOPS-not bandwidth.

Depending on the application block size, the same number of IOPS may have significantly different bandwidth requirements. For example, an application requiring 5000 IOPS with a 4KB block size would result in a bandwidth requirement of about 20MB/sec. In contrast, an application that uses 16KB blocks with the same number of IOPS needs significantly higher bandwidth.

•	5000 IOPS x 4KB blocks = 20MB/sec
•	5000 IOPS x 16KB blocks = 80MB/sec

Fibre Channel provides much higher performance than iSCSI in terms of bandwidth, so it is the clear choice for streaming audio and video applications or any applications with sustained high bandwidth requirements.

The majority of Intel® servers and applications, however, cannot sustain 1GB/sec performance, so the bandwidth gains of Fibre Channel are usually irrelevant. IOPS capabilities are quite similar between the two protocols, for 70% to 80% of Windows and Linux servers, iSCSI performance is more than adequate.

The UNIX space is harder to generalize. For block-based UNIX applications with very high bandwidth requirements, Fibre Channel is your only option. Older midrange UNIX servers and low-end UNIX servers typically cannot sustain more than 1GB/sec performance, so bandwidth does not need to be a key consideration.

Don't forget to take seasonality into account. Retailer billing applications, for example, may normally do 20MB/sec but see dramatic spikes during the holiday shopping season.

Is your current infrastructure pure Ethernet, or do you have Fibre Channel fabric?

Companies relying exclusively on TCP/IP often lean toward Ethernet-based iSCSI. Fibre Channel is certainly an option, but if you decide to go that route you'll need to invest in additional hardware and budget for training or more staff with FC expertise.

In companies with established FC fabrics, storage and system administrators often lean toward Fibre Channel as a known and trusted solution. However, depending on your infrastructure and application requirements, iSCSI might be a good option if you already have a Gigabit Ethernet infrastructure in place.

Every enterprise is different. One of the top U.S. telecom carriers recently approached us about iSCSI but ultimately decided to stick with their existing Fibre Channel network. Why? They were using 100 megabit Ethernet and didn't want to invest in gigabit switches. Instead, they bought a couple of host bus adapters (HBAs), implemented a NetApp Fibre Channel SAN, and used available ports on their FC switch.

How do requirements differ among your main data center, regional data centers, disaster recovery (DR) facilities, and remote offices?

Doing a site-by-site review of application requirements as well as the existing network infrastructure is a key element in planning your consolidation strategy.

Application criticality, performance requirements, infrastructure, and on-site staff resources can vary significantly among corporate data centers, regional data centers, remote offices, and DR sites. Your decision at one site shouldn't lock you into using the same technologies at all locations. It helps, of course, to work with a storage vendor that supports both protocols.

For example, one major enterprise relied almost exclusively on Fibre Channel at its main data center, but its 15 remote data centers used only direct-attached storage (DAS). In addition to being painful to manage, there were no centralized backups to protect data in the event of a site failure because the cost of buying host-based software to enable centralized replication and backup was too steep. After determining that expanding its Fibre Channel infrastructure would be too expensive, the customer deployed iSCSI to locally consolidate storage and replicate data back to the main data center.

It's a mistake to overlook smaller data centers because they house less critical data. In my experience, about 75% of an enterprise's data resides in regional and small remote data centers. In many cases, Fibre Channel has not penetrated these areas simply because of cost. That means the majority of servers are connected to DAS or operate with internal storage. If you are developing a plan to consolidate local storage, consider a technology like iSCSI.

Is getting the "lowest cost" solution a top priority?

Cost should not be your primary concern, but is an area where iSCSI has a significant advantage. Even companies with existing Fibre Channel infrastructures may find it cost-effective to connect at least some servers with iSCSI.

	Fibre Channel (Low End)	iSCSI* (Low End)
HBA (each)	$800	Free Microsoft initiator available for download
Cables (each)	$150	~$15
Switch (cost per port)	$500**	$80
One extra GigE server card for full redundancy (iSCSI only)		$100 (GigE NIC)
Cost to connect each server with full redundancy	$2900	<$300

In a Nutshell

There is no one-size-fits-all solution that works for everyone. But answering the questions in this article will help you understand the necessary trade-offs and choose the best technology for your environment.

---