spc Archives

February 21, 2013December 24, 2020

NetApp EF540, something familiar, something new

StorageIO Industry trends and perspectives image

NetApp announced the other day a new all nand flash solid-state devices (SSD) storage system called the EF540 that is available now. The EF540 has something’s new and cool, along with some things familiar, tried, true and proven.

What is new is that the EF540 is an all nand flash multi-level cell (MLC) SSD storage system. What is old is that the EF540 is based on the NetApp E-Series (read more here and here) and SANtricity software with hundreds of thousands installed systems. As a refresher, the E-Series are the storage system technologies and solutions obtained via the Engenio acquisition from LSI in 2011.

Image of NetApp EF540 via ntapgeek.com
Image via www.ntapgeek.com

The EF540 expands the NetApp SSD flash portfolio which includes products such as FlashCache (read cache aka PAM) for controllers in ONTAP based storage systems. Other NetApp items in the NetApp flash portfolio include FlashPool SSD drives for persistent read and write storage in ONTAP based systems. Complimenting FlashCache and FlashPool is the server-side PCIe caching card and software FlashAccel. NetApp is claiming to have revenue shipped 36PB of flash complimenting over 3 Exabytes (EB) of storage while continuing to ship a large amount of SAS and SATA HDD’s.

NetApp also previewed its future FlashRay storage system that should appear in beta later in 2013 and general availability in 2014.

In addition to SSD and flash related announcements, NetApp also announced enhancements to its ONTAP FAS/V6200 series including the FAS/V6220, FAS/V6250 and FAS/V6290.

Some characteristics of the NetApp EF540 and SANtricity include:

Two models with 12 or 24 x 6Gbs SAS 800GB MLC SSD devices
Up to 9.6TB or 19.2TB physical storage in a 2U (3.5 inch) tall enclosure
Dual controllers for redundancy, load-balancing and availability
IOP performance of over 300,000 4Kbyte random 100% reads under 1ms
6GByte/sec performance of 512Kbyte sequential reads, 5.5Gbyte/sec random reads
Multiple RAID levels (0, 1, 10, 3, 5, 6) and flexible group sizes
12GB of DRAM cache memory in each controller (mirrored)
4 x 8GFC host server-side ports per controller
Optional expansion host ports (6Gb SAS, 8GFC, 10Gb iSCSI, 40Gb IBA/SRP)
Snapshots and replication (synchronous and asynchronous) including to HDD systems
Can be used for traditional IOP intensive little-data, or bandwidth for big-data
Proactive SSD wear monitoring and notification alerts
Utilizes SANtricity version 10.84

Poll, Are large storage arrays day’s numbered?

EMC and NetApp (along with other vendors) continue to sell large numbers of HDD’s as well as large amounts of SSD. Both EMC and NetApp are taking similar approaches of leveraging PCIe flash cards as cache adding software functionality to compliment underlying storage systems. The benefit is that the cache approach is less disruptive for many environments while allowing improved return on investment (ROI) of existing assets.

	EMC	NetApp
Storage systems with HDD and SSD	VMAX, VNX	FAS/V, E-Series
Storage systems with SSD cache	FastCache,	FlashCache
All SSD based storage	VMAX, VNX	EF540
All new SSD system in development	Project X	FlashRay
Server side PCIe SSD cache	VFCache	FlashAcell
Partner ecosystems	Yes	Yes

The best IO is the one that you do not have to do, however the next best are those that have the least cost or affect which is where SSD comes into play. SSD is like real estate in that location matters in terms of providing benefit, as well as how much space or capacity is needed.

What does this all mean?
The NetApp EF540 based on the E-Series storage system architecture is like one of its primary competitors (e.g. EMC VNX also available as an all-flash model). The similarity is that both have been competitors, as well as have been around for over a decade with hundreds of thousands of installed systems. The similarities are also that both continue to evolve their code base leveraging new hardware and software functionality. These improvements have resulted in improved performance, availability, capacity, energy effectiveness and cost reduction.

Whats your take on RAID still being relevant?

From a performance perspective, there are plenty of public workloads and benchmarks including Microsoft ESRP and SPC among others to confirm its performance. Watch for NetApp to release EF540 SPC results given their history of doing so with other E-Series based systems. With those or other results, compare and contrast to other solutions looking not just at IOPS or MB/sec (bandwidth), also latency, functionality and cost.

What does the EF540 compete with?
The EF540 competes with all flash-based SSD solutions (Violin, Solidfire, Purestorage, Whiptail, Kaminario, IBM/TMS, up-coming EMC Project “X” (aka XtremeIO)) among others. Some of those systems use general-purpose servers combined SSD drives, PCIe cards along with management software where others leverage customized platforms with software. To a lesser extent, competition will also be mixed mode SSD and HDD solutions along with some PCIe target SSD cards for some situations.

What to watch and look for:
It will be interesting to view and contrast public price performance results using SPC or Microsoft ESRP among others to see how the EF540 compares. In addition, it will be interesting to compare other storage based, as well as SSD systems beyond the number of IOPS. What will be interesting is to keep an eye on latency, as well as bandwidth, feature functionality and associated costs.

Given that the NetApp E-Series are OEM or sold by third parties, let’s see if something looking similar or identical to the EF540 appear at any of those or new partners. This includes traditional general purpose and little-data environments, along with cloud, managed service provider, high performance compute and high productivity compute (HPC), super computer (SC), big data and big bandwidth among others.

Poll, Have SSD been successful in traditional storage systems and arrays

The EF540 could also appear as a storage or IO accelerator for large-scale out, clustered, grid and object storage systems for meta data, indices, key value stores among other uses either direct attached to servers, or via shared iSCSI, SAS, FC and InfiniBand (IBA) SCSI Remote Protocol (SRP).

Keep an eye on how the startups that have been primarily Just a Bunch Of SSD (JBOS) in a box start talking about adding new features and functionality such as snapshots, replication or price reductions. Also, keep an eye and ear open to what EMC does with project “X” along with NetApp FlashRay among other improvements.

For NetApp customers, prospects, partners, E-Series OEMs and their customers with the need for IO consolidation, or performance optimization for big-data, little-data and related applications the EF540 opens up new opportunities and should be good news. For EMC competitors, they now have new competition which also signals an expanding market with new opportunities in adjacent areas for growth. This also further signals the need for diverse ssd portfolios and product options to meet different customer application needs, along with increased functionality vs. lowest cost for high capacity fast nand SSD storage.

Some related reading:

Disclosure: NetApp, Engenio (when LSI), EMC and TMS (now IBM) have been clients of StorageIO.

Ok, nuff said

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

July 12, 2012April 18, 2022

Give HP storage some love and short strokin

Following up from my last post over at InfoStor about metrics that matter, here is a link to a new piece that I did on storage vendors benchmarking and related topics. This new post looked at an storage performance council (SPC1) benchmark that HP did with their P10000 (e.g. 3PAR) storage system under assertions by some in the industry that they were short stroking to meet better performance.

Amazon Web Services (AWS)

I’m surprised some creative technical marketer, blogger or prankster has yet to rework Clarence Carters (e.g. Dr. CC) iconic song into something about storage performance and capacity short strokin.

Ok, nuff said before I get a visit from the HP truth squads, in the meantime, give HP a hug and some love if so inclined.

Cheers Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

October 25, 2011November 25, 2019

Congratulations to IBM for releasing XIV SPC results

Over the past several years I have done an annual post about IBM and their XIV storage system and this is the fourth in what has become a series. You can read the first one here, the second one here, and last years here and here after the announcement of the IBM V7000.

IBM XIV Gen3
IBM recently announced the generation 3 or Gen3 version of XIV along with releasing for the first time public performance comparison benchmarks using storage performance council (SPC) throughout SPC2 workload.

The XIV Gen3 is positioned by IBM as having up to four (4) times the performance of earlier generations of the storage system. In terms of speeds and feeds, the Gen3 XIV supports up to 180 2TB SAS hard disk drives (HDD) that provides up to 161TB of usable storage space capacity. For connectivity, the Gen3 XIV supports up to 24 8Gb Fibre Channel (8GFC) or for iSCSI 22 1Gb Ethernet (1 GbE) ports with a total of up to 360GBytes of system cache. In addition to the large cache to boost performance, other enhancements include leveraging multi core processors along with an internal InfiniBand network to connect nodes replacing the former 1 GbE interconnect. Note, InfiniBand is only used to interconnect the various nodes in the XIV cluster and is not used for attachment to applications servers which is handled via iSCSI and Fibre Channel.

IBM and SPC storage performance history
IBM has a strong history if not leading the industry with benchmarking and workload simulation of their storage systems including Storage Performance Council (SPC) among others. The exception for IBM over the past couple of years has been the lack of SPC benchmarks for XIV. Last year when IBM released their new V7000 storage system benchmarks include SPC were available close to if not at the product launch. I have in the past commented about IBMs lack of SPC benchmarks for XIV to confirm their marketing claims given their history of publishing results for all of their other storage systems. Now that IBM has recently released SPC2 results for the XIV it is only fitting then that I compliment them for doing so.

Benchmark brouhaha
Performance workload simulation results can often lead to applies and oranges comparisons or benchmark brouhaha battles or storage performance games. For example a few years back NetApp submitted a SPC performance result on behalf of their competitor EMC. Now to be clear on something, Im not saying that SPC is the best or definitive benchmark or comparison tool for storage or other purpose as it is not. However it is representative and most storage vendors have released some SPC results for their storage systems in addition to TPC and Microsoft ESRP among others. SPC2 is focused on streaming such as video, backup or other throughput centric applications where SPC1 is centered around IOPS or transactional activity. The metrics for SPC2 are Megabytes per second (MBps) for large file processing (LFP), large database query (LDQ) and video on demand delivery (VOD) for a given price and protection level.

What is the best benchmark?
Simple, your own application in as close to as actual workload activity as possible. If that is not possible, then some simulation or workload simulation that closets resembles your needs.

Does this mean that XIV is still relevant?
Yes

Does this mean that XIV G3 should be used for every environment?
Generally speaking no. However its performance enhancements should allow it to be considered for more applications than in the past. Plus with the public comparisons now available, that should help to silence questions (including those from me) about what the systems can really do vs. marketing claims.

How does XIV compare to some other IBM storage systems using SPC2 comparisons?

System	SPC2 MBps	Cost per SPC2	Storage GBytes	Price tested	Discount	Protection
DS5300	5,634.17	$74.13	16,383	417,648	0%	R5
V7000	3,132.87	$71.32	29,914	$223,422	38-39%	R5
XIV G3	7,467.99	$152.34	154,619	1,137,641	63-64%	Mirror
DS8800	9,705.74	$270.38	71,537	2,624,257	40-50%	R5

In the above comparisons, the DS5300 (NetApp/Engenio based) is a dual controller (4GB of cache per controller) with 128 x 146.8GB 15K HDDs configured as RAID 5 with no discount applied to the price submitted. The V7000 system which is based on the IBM SVC along with other enhancements consists of dual controllers each with 8GB of cache and 120 x 10K 300GB HDDs configured as RAID 5 with just under a 40% discount off list price for system tested. For the XIV Gen3 system tested, discount off list price for the submission is about 63% with 15 nodes and a total of 360GB of cache and 180 2TB 7.2K SAS HDDs configured as mirrors. The DS8800 system with dual controllers has a 256GB of cache, 768 x 146GB 15K HDDs configured in RAID5 with a discount between 40 to 50% off of list.

What the various metrics do not show is the benefit of various features and functionality which should be considered to your particular needs. Likewise, if your applications are not centered around bandwidth or throughput, then the above performance comparisons would not be relevant. Also note that the systems above have various discount prices as submitted which can be a hint to a smart shopper where to begin negotiations at. You can also do some analysis of the various systems based on their performance, configuration, physical footprint, functionality and cost plus the links below take you to the complete reports with more information.

DS8800 SPC2 executive summary and full disclosure report

XIV SPC2 executive summary and full disclosure report

DS5300 SPC2 executive summary and full disclosure report

V7000 SPC2 executive summary and full disclosure report

Bottom line, benchmarks and performance comparisons are just that, a comparison that may or may not be relevant to your particular needs. Consequently they should be used as a tool combined with other information to see how a particular solution might be a fit for your specific needs. The best benchmark however is your own application running as close to possible realistic workload to get a representative perspective of a systems capabilities.

Ok, nuff said
Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

October 23, 2009May 18, 2021

EPA Energy Star for Data Center Storage Update

Following up on a recent post about Green IT, energy efficiency and optimization for servers, storage and more, here are some additional thoughts, perspectives along with industry activity around the U.S. Environmental Protection Agency (EPA) Energy Star for Server, Data Center Storage and Data Centers.

First a quick update, Energy Star for Servers is in place with work now underway on expanding and extending beyond the first specification. Second is that Energy Star for Data Center storage definition is well underway including a recent workshop to refine the initial specification along with discussion for follow-on drafts.

Energy Star for Data Centers is also currently undergoing definition which is focused more on macro or facility energy (notice I did not say electricity) efficiency as opposed to productivity or effectiveness, items that the Server and Storage specifications are working towards.

Among all of the different industry trade or special interests groups, at least on the storage front the Storage Networking Industry Association (SNIA) Green Storage Initiative (GSI) and their Technical Work Groups (TWG) have been busily working for the past couple of years on taxonomies, metrics and other items in support of EPA Energy Star for Data Center Storage.

A challenge for SNIA along with others working on related material pertaining to storage and efficiency is the multi-role functionality of storage. That is, some storage simply stores data with little to no performance requirements while other storage is actively used for reading and writing. In addition, there are various categories, architectures not to mention hardware and software feature functionality or vendors with different product focus and interests.

Unlike servers that are either on and doing work, or, off or in low power mode, storage is either doing active work (e.g. moving data), storing in-active or idle data, or a combination of both. Hence for some, energy efficiency is about how much data can be stored in a given footprint with the least amount of power known as in-active or idle measurement.

On the other hand, storage efficiency is also about using the least amount of energy to produce the most amount of work or activity, for example IOPS or bandwidth per watt per footprint.

Thus the challenge and need for at least a two dimensional model looking at, and reflecting different types or categories of storage aligned for active or in-active (e.g. storing) data enabling apples to apples, vs. apples to oranges comparison.

This is not all that different from how EPA looks at motor vehicle categories of economy cars, sport utility, work or heavy utility among others when doing different types of work, or, in idle.

What does this have to do with servers and storage?

Simple, when a server powers down where does its data go? That’s right, to a storage system using disk, ssd (RAM or flash), tape or optical for persistency. Likewise, when there is work to be done, where does the data get read into computer memory from, or written to? That’s right, a storage system. Hence the need to look at storage in a multi-tenant manner.

The storage industry is diverse with some vendors or products focused on performance or activity, while others on long term, low cost persistent storage for archive, backup, not to mention some doing a bit of both. Hence the nomenclature of herding cats towards a common goal when different parties have various interests that may conflict yet support needs of various customer storage usage requirements.

Figure 1 shows a simplified, streamlined storage taxonomy that has been put together by SNIA representing various types, categories and functions of data center storage. The green shaded areas are a good step in the right direction to simplify yet move towards realistic and achievable befits for storage consumers.

Figure 1 Source: EPA Energy Star for Data Center Storage web site document

The importance of the streamlined SNIA taxonomy is to help differentiate or characterize various types and tiers of storage (Figure 2) products facilitating apples to apples comparison instead of apples or oranges. For example, on-line primary storage needs to be looked at in terms of how much work or activity per energy footprint determines efficiency.

Figure 2: Tiered Storage Example

On other hand, storage for retaining large amounts of data that is in-active or idle for long periods of time should be looked at on a capacity per energy footprint basis. While final metrics are still being flushed out, some examples could be active storage gauged by IOPS or work or bandwidth per watt of energy per footprint while other storage for idle or inactive data could be looked at on a capacity per energy footprint basis.

What benchmarks or workloads to be used for simulating or measuring work or activity are still being discussed with proposals coming from various sources. For example SNIA GSI TWG are developing measurements and discussing metrics, as have the storage performance council (SPC) and SPEC among others including use of simulation tools such as IOmeter, VMware VMmark, TPC, Bonnie, or perhaps even Microsoft ESRP.

Tenants of Energy Star for Data Center Storage overtime hopefully will include:

Reflective of different types, categories, price-bands and storage usage scenarios
Measure storage efficiency for active work along with in-active or idle usage
Provide insight for both storage performance efficiency and effective capacity
Baseline or raw storage capacity along with effective enhanced optimized capacity
Easy to use metrics with more in-depth back ground or disclosure information

Ultimately the specification should help IT storage buyers and decision makers to compare and contrast different storage systems that are best suited and applicable to their usage scenarios.

This means measuring work or activity per energy footprint at a given capacity and data protection level to meet service requirements along with during in-active or idle periods. This also means showing storage that is capacity focused in terms of how much data can be stored in a given energy footprint.

One thing that will be tricky however will be differentiating GBytes per watt in terms of capacity, or, in terms of performance and bandwidth.

Here are some links to learn more:

EPA Energy Star for Data Center Storage Presentation
SNIA GSI Resources
Energy-Efficient Servers & Storage: The EPA’s Energy Star Ratings Are Coming
Chapter 5 (Measurement, Metrics, and Management) and Chapter 8 (Data Storage) in the book The Green and Virtual Data Center (CRC).
Chapter 10 (Performance and Capacity Planning): Resilient Storage Networks (Elsevier)
Green IT Efficiency and optimization calculators/tools
Green Storage is Alive and Well: ENERGY STAR Data Center Storage
Update: Energy Star for Server Workshop and more here
EPA Energy Star for Data Center Storage site
Twitter tag #ESStorage

Stay tuned for more on Energy Star for Data Centers, Servers and Data Center Storage.

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

October 14, 2009March 11, 2022

PUE, Are you Managing Power, Energy or Productivity?

With a renewed focus on Green IT including energy Efficiency and Optimization of servers, storage, networks and facilities, is your focus on managing power, energy, or, productivity?

For example, do you use or are interested in metrics such as Greengrid PUE or 80 Plus efficient power supplies along with initiatives such as EPA Energy Star for servers and emerging Energy Star for Data Center for Storage in terms of energy usage?

Or are you interested in productivity such as amount of work or activity that can be done in a given amount of time,or how much information can be stored in a given footprint (power, cooling, floor space, budget, management)?

For many organizations, there tends to be a focus and in both managing power along with managing productivity. The two are or should interrelated, however there are some disconnects with some emphasis and metrics. For example, the Green grid PUE is a macro facilities centric metric that does not show the productivity, quality or measure of services being delivered by a data center or information factory. Instead, PUE provides a gauge of how the habitat, that is the building and power distribution along with cooling are efficient with respect to the total energy consumption of IT equipment.

As a refresher, PUE is a macro metric that is essentially a ratio of how much total power or energy goes into a facility vs. the amount of energy used by IT equipment. For example, if 12Kw (smaller room/site) or 12Mw (larger site) are required to power an IT data center or computer room for that matter, and of that energy load, 6kWh or 6Mw, the PUE would be 2. A PUE of 2 is an indicator that 50% of energy going to power a facility or computer room goes towards IT equipment (servers, storage, networks, telecom and related equipment) with the balance going towards running the facility or environment which typically has had the highest percentage being HVAC/cooling.

In the case of EPA Energy Star for Data Centers which initially is focused on the habitat or facility efficiency, the answer is measuring and managing energy use and facility efficiency as opposed to productivity or useful work. The metric for EPA Energy Star for Data Center initially will be Energy Usage Effectiveness (EUE) that will be used to calculate a ratting for a data center facility. Those data centers in the top25 percentile will qualify for Energy Star certification.

Note the word energy and not power which means that the data center macro metric based on Green grid PUE rating looks at all source of energy used by a data center and not just electrical power. What this means is that a macro and holistic facilities energy consumption could be a combination of electrical power, diesel, propane or natural gas or other fuel sources to generate or create power for IT equipment, HVAC/Cooling and other needs. By using a metric that factor in all energy sources, a facility that uses solar, radiant, heat pumps, economizers or other techniques to reduce demands on energy will make a better rating.

By using a macro metric such as EUE or PUE (ratio = Total_Power_Used / IT_Power_Needs), a starting point is available to decide and compare efficiency and cost to power or energize a facility or room also known as a habitat for technology.

Managing Productivity of Information Factories (E.g. Data Centers)
What EUE and PUE do not reflect or indicate is how much data is processed, moved and stored by servers, storage and networks within a facility. On the other hand or extreme from macro metrics are micro or component metrics that gauge energy usage on an individual device basis. Some of these micro metrics may have activity or productivity indicator measurements associated with them, some don’t. Where these leave a big gap and opportunity is to fill the span between the macro and micro.

This is where work is being done by various industry groups including SNIA GSI, SPC and SPEC among others along with EPA Energy Star among others to move beyond macro PUE indicators to more granular effectiveness and efficiency metrics that reflect productivity. Ultimately productivity is important to gauge, the return on investment and business value of how much data can be processed by servers, moved via networks or stored on storage devices in a given energy footprint or cost.

In Figure 1 are shown four basic approaches (in addition to doing nothing) to energy efficiency. One approach is to avoid energy usage, similar to following a rationing model, but this approach will affect the amount of work that can be accomplished. Another approach is to do more work using the same amount of energy, boosting energy efficiency, or do same amount of work (or storage data) however with less energy.

Tiered Storage
Figure 1 the Many Faces of Energy Efficiency Source: The Green and Virtual Data Center(CRC)

The energy efficiency gap is the difference between the amount of work accomplished or information stored in a given footprint and the energy consumed. In other words, the bigger the energy efficiency gap, the better, as seen in the fourth scenario, doing more work or storing more information in a smaller footprint using less energy. Clock here to read more about Shifting from energy avoidance to energy efficiency.

Watch for new metrics looking at productivity and activity for servers, storage and networks ranging from MHz or GHz per watt, transactions or IOPS per watt, bandwidth, frames or packets processed per watt or capacity stored per watt in a given footprint. One of the confusing metrics is Gbytes or Tbytes per watt in that it can mean storage capacity or bandwidth, thus, understand the context of the metric. Likewise watch for metrics that reflect energy usage for active along with in-active including idle or dormant storage common with archives, backup or fixed content data.

What this all means is that work continues on developing usable and relevant metrics and measurement not only for macro energy usage, also, to gauge the effectiveness of delivering IT services. The business value proposition of driving efficiency and optimization including increased productivity along with storing more information in a given footprint is to support density and business sustainability.

Additional resources and where to learn in addition to those mentioned above include:

EPA Energy Star for Data Center Storage

Storage Efficiency and Optimization – The Other Green

Performance = Availability StorageIOblog featured ITKE guest blog

SPC and Storage Benchmarking Games

Shifting from energy avoidance to energy efficiency

Green IT Confusion Continues, Opportunities Missed!

Green Power and Cooling Tools and Calculators

Determining Computer or Server Energy Use

Examples of Green Metrics

Green IT, Power, Energy and Related Tools or Calculators

Chapter 10 (Performance and Capacity Planning)
Resilient Storage Networks (Elsevier)

Chapter 5 (Measurement, Metrics and Management of IT Resources)
The Green and Virtual Data Center (CRC)

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

July 6, 2009December 24, 2020

SPC and Storage Benchmarking Games

There is a post over in one of the LinkedIn Discussion forums about storage performance council (SPC) benchmarks being miss-leading that I just did a short response post to. Here’s the full post as LinkedIn has a short post response limit.

While the SPC is far from perfect, it is at least for block, arguably better than doing nothing.

For the most part, SPC has become a de facto standard for at least block storage benchmarks independent of using IOmeter or other tools or vendor specific simulations, similar how MSFT ESRP is for exchange, TPC for database, SPEC for NFS and so forth. In fact, SPC even recently rather quietly rolled out a new set of what could be considered the basis for Green storage benchmarks. I would argue that SPC results in themselves are not misleading, particularly if you take the time to look at both the executive and full disclosures and look beyond the summary.

Some vendors have taken advantage of the SPC results playing games with discounting on prices (something that’s allowed under SPC rules) to show and make apples to oranges comparisons on cost per IOP or other ploys. This proactive is nothing new to the IT industry or other industries for that matter, hence benchmark games.

Where the misleading SPC issue can come into play is for those who simply look at what a vendor is claiming and not looking at the rest of the story, or taking the time to look at the results and making apples to apples, instead of believing the apples to oranges comparison. After all, the results are there for a reason. That reason is for those really interested to dig in and sift through the material, granted not everyone wants to do that.

For example, some vendors can show a highly discounted list price to get a better IOP per cost on an apple to oranges basis, however, when processes are normalized, the results can be quite different. However here’s the real gem for those who dig into the SPC results, including looking at the configurations and that is that latency under workload is also reported.

The reason that latency is a gem is that generally speaking, latency does not lie.

What this means is that if vendor A doubles the amount of cache, doubles the number of controllers, doubles the number of disk drives, plays games with actual storage utilization (ASU), utilizes fast interfaces from 10 GbE iSCSI to 8Gb FC or FCoE or SAS to get a better cost per IOP number with discounting, look at the latency numbers. There have been some recent examples of this where vendor A has a better cost per IOP while achieving a higher number of IOPS at a lower cost compared to vendor B, which is what is typically reported in a press release or news story. (See a blog entry that also points to a CMG presentation discussion around this topic here.

Then go and look at the two results, vendor B may be at list price while vendor A is severely discounted which is not a bad thing, as that is then the starting list price as to which customers should start negotiations. However to be fair, normalize the pricing for fun, look at how much more equipment vendor A may need while having to discount to get the price to offset the increased amount of hardware, then look at latency.

In some of the recent record reported results, the latency results are actually better for a vendor B than for a vendor A and why does latency matter? Beyond showing what a controller can actually do in terms of levering the number of disks, cache, interface ports and so forth, the big kicker is for those talking about SSD (RAM or FLASH) in that SSD generally is about latency. To fully effectively utilize SSD which is a low latency device, you would want a controller that can do a decent job at handling IOPS; however you also need a controller that can do a decent job of handling IOPS with low latency under heavy workload conditions.

Thus the SPC again while far from perfect, at least for a thumb nail sketch and comparison is not necessarily misleading, more often than not it’s how the results are utilized that is misleading. Now in the quest for the SPC administrators to try and gain more members and broader industry participation and thus secure their own future, is the SPC organization or administration opening itself up to being used more and more as a marketing tool in ways that potentially compromise all the credibility (I know, some will dispute the validity of SPC, however that’s reserved for a different discussion ;) )?

There is a bit of Déjà here for those involved with RAID and storage who recall how the RAID Advisory Board (RAB) in its quest to gain broader industry adoption and support succumbed to marketing pressures and use or what some would describe as miss-use and is now a member of the “Where are they now” club!

Don’t get me wrong here; I like the SPC tests/results/format, there is a lot of good information in the SPC. The various vendor folks who work very hard behind the scenes to make the SPC actually work and continue to evolve it also all deserve a great big kudos, an “atta boy” or “atta girl” for the fine work that have been doing, work that I hope does not become lost in the quest to gain market adoption for the SPC.

Ok, so then this should all then beg the question of what is the best benchmark. Simple, the one that most closely resembles your actual applications, workload, conditions, configuration and environment.

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

April 26, 2009January 23, 2019

U.S. EPA Looking for Industry Input on Energy Star for Storage

Following up on previous blog posts, here is some information that the U.S. EPA is looking for comments from industry on an Energy Start for enterprise storage program following on the heels of the Energy Star for Server program.

U.S. EPA Energy Star Wants and Needs You!

Here’s the message received from the EPA via their mailing list this past week (in italics below):

Dear Enterprise Storage Equipment Manufacturers and Other Interested Parties:

Please see the attached letter from the U.S. Environmental Protection Agency (EPA) announcing their intent to pursue development of an ENERGY STAR specification for Enterprise Storage equipment. If you have any questions or concerns, please contact Andrew Fanara, EPA, at fanara.andrew@epa.gov or Stephen Pantano, ICF International, at spantano@icfi.com.

Thank you for your support of ENERGY STAR.

Here’s the intro letter excerpted from the above email notification (in italics below):

April 23, 2009

Dear Enterprise Storage Equipment Manufacturers and Other Interested Parties:

This letter is intended to inform all stakeholders that the U.S. Environmental Protection Agency (EPA) intends to continue its efforts towards the development of an ENERGY STAR® specification for enterprise data storage equipment. Following is an outline of EPA’s general goals and next steps.

ENERGY STAR is a voluntary partnership between government, businesses, and purchasers designed to encourage the manufacture, purchase, and use of efficient products to help protect the environment. Products that earn the ENERGY STAR prevent greenhouse gas emissions by meeting strict energy efficiency guidelines. Manufacturers that qualify their products to meet ENERGY STAR requirements may use the label as a tool to educate their customers about the enhanced value of these products.

To date:
•More than 2,000 manufacturers are partnering with ENERGY STAR,
•More than 40,000 product models carry the ENERGY STAR label across more than 50 product categories,
•More than 70% of Americans recognize the ENERGY STAR label,
•Consumers have purchased more than 2.5 billion ENERGY STAR qualified products, and
•Americans, with the help of ENERGY STAR, saved enough energy in 2008 to avoid greenhouse gas emissions equivalent to those from 29 million cars — while saving $19 billion on utility bills.

In the last several years, the energy saving opportunities in data centers have been well documented. However, barriers to energy efficiency still persist and need to be addressed. EPA is pursuing a dual strategy to overcome these challenges by helping purchasers more easily identify energy efficient IT equipment with the use of the ENERGY STAR designation, and by encouraging organizations to benchmark the energy performance of their data centers.

In pursuit of this strategy, EPA will introduce an ENERGY STAR Computer Server specification in the coming weeks. In addition, EPA recently conducted a scoping effort to evaluate enterprise storage products for inclusion in the ENERGY STAR program. EPA reviewed available market research and facilitated discussions with product manufacturers, industry associations, and other interested parties. EPA concluded that IT purchasers would benefit from access to standardized information about the energy performance of storage equipment made available through the ENERGY STAR program. As a result, EPA intends to begin the specification development process. Details on this process will be forthcoming in the next several weeks.

To be added to the enterprise storage e-mail distribution list, please send your full contact information to Stephen Pantano at spantano@icfi.com. To stay informed about the ENERGY STAR specification development process for computer servers and other EPA data center initiatives please visit: www.energystar.gov/datacenters.

Thank you for your continued support of ENERGY STAR and please direct additional questions to Andrew Fanara at fanara.andrew@epa.gov or Stephen Pantano of ICF International, at spantano@icfi.com.

Sincerely,

Andrew Fanara
U.S. Environmental Protection Agency, Climate Protection Partnerships Division ENERGY STAR Program Manager

My take on the Energy Star programs is that as long as they add value including reflecting how energy is effectively used both when IT equipment such as servers and storage are in use, as well as in energy saving or avoidance modes are reflected, they can and should be a good thing.

However industry will need to work together across different trade and focus groups as well as factor in how supporting metrics will be applicable and reflective thus accepted by IT data center environments. This means metrics and measurements for both active or working while in use energy efficiency modes such as IOPS, bandwidth, messages or transactions, files or videos per watt of energy, as well as metrics for in-active or dormant data such as capacity per watt per usable footprint. Check out Chapter 5 (Measurements and Metrics) in "The Green and Virtual Data Center" (CRC) to learn more.

Various industry trade and focus groups including Storage Performance Council (SPC), SNIA GSI, Green Grid, SPEC and others are working on various metrics and aligning themselves to work with EPA. If you are in an IT data center involved with servers or storage, consider getting involved with one or more of these groups to help influence and shape what these programs will look like or affect your organization in the future.

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

April 27, 2008January 23, 2019

Power, Cooling, Floor-space, Environmental (PCFE) and Green Metrics

The Metrics and Measurement page on www.greendatastorage.com has been updated along with other pages covering IT data center PCFE and green topics for servers, storage, networks and facilities. Have a look.

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

April 19, 2008July 22, 2021

Do Disk based VTLs draw less power than Tape?

The tape is dead debates rage on as they have for a decades which make for good press and discussion or debate during slow times, similar to coverage of what Britney Spears or Paris Hilton are or are not wearing.

In the on-going debates and Greenwashing of what technology or vendor is greener to prevent global warming, some recent tape is dead flare-ups have occurred including one hinting that tape libraries can draw more power than a disk based VTL with de-dupe are discussed over on Tony Pearson of IBM fame blog site as well as Beth Pariseau of TechTarget StorageSoup site.

I posted some comments on those sites along along with a link to a StorageIO Industry Trends and Perspective report titled “Energy Savings without Performance Compromise” as an example (look for an updated version of the comparison charts in the report in the not so distant future). The report looks at how different storage tiers including on-line disk, MAID, MAID 2.0 and tape libraries vary to address different PCFE (power, cooling, floor-space, environment) issues while supporting various service levels including performance, availability, capacity and energy use.

Additional related material can be found at www.storageio.com and www.greendatastorage.com including the Industry Trends and Perspective Report Business “Benefits of Data Footprint Reduction in general covering archiving, compression (on-line and off-line) along with de-duplication

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

April 3, 2008December 4, 2019

Moving Beyond the Benchmark Brouhaha

Heres a Link to a new article/column/opinion piece over at Enterprise Storage Forum by me about “Moving Beyond the Benchmark Brouhaha” in the storage industry.

Have a read for yourself.

Cheers
GS

January 29, 2008December 4, 2019

Was today the proverbal day that he!! froze over?

Storage I/O trends

The upper Midwest, well, make that the Midwest in general was hit by high winds and a nasty cold front today, nothing all that unusual for January, especially in the Minneapolis area where the temperature yesterday was 40 to 45 with light rain and by about noon today about zero “F” without the wind chill. So in the face of old man winter and the cold, I had a chuckle today reading the announcement that an SPC (Storage Performance Council) SPC1 (IOPS) benchmark had finally been published for an EMC CLARiiON CX3-40.

Now for those of you who cover or track or cheerlead or dread EMC, you probably know there position on benchmarks, or at least of some of their bloggers and in particular, SPC and if not, read here for some perspective. My position has always been the best benchmark is your actual application in a real and applicable workload scenario, however also realizing that not everybody can simulate or test their applications, there is the need for points of reference comparison benchmarks such as SPC, Microsoft ESRP, TPC, SPEC among others.

Heres’ the caveat, take these benchmarks with a grain of salt; use them as a gauge along with other tools as they are an indicator of a particular workload. I am a fan of benchmarks that make sense, can be reproduced consistently and that are realistic representations, not a substitute for your actual applications. They are tools to help you make a better informed decision however that is all they are, a relative comparison.

Nuff rambling for now on that, why did I chuckle this morning and think that He!! had perhaps finally froze over, and don?t get me wrong, Minneapolis and the Midwest for that matter is far from being He!!!, granted its cold as crap during the winter months. The reason I chuckled is that EMC did not in fact submit the SPC1 benchmarks for the CLARiiON CX3-40, instead, one of their competitors namely Network Appliance (aka NetApp) did the honors for EMC along with a submission for their FAS3040.

So besides the fact that there is plenty of wiggle and debate room in the test for example NetApp using RAID6 (e.g. RAID-DP) and Mirroring on the EMC (I?m sure we will hear EMC cry foul), EMC can keep their hands clean on their party line about not submitting an SPC1 result, or at least that?s a card they could choose to play.I would like to see the DMX4 particularly with the new FLASH based SSDs in a future SPC test submission; however, I?m not going to hold my breath at least yet.

However it is ironic that EMC has in fact submitted other benchmark tests scenarios in the past including for Microsoft ESRP among others. Speaking of SPC submissions, TMS (Texas Memory Systems) also posted some new SPC results the other day.So maybe he!! did not really freeze over today with EMC finally submitting an SPC test, however it made for good warming chuckle on a cold morning.

Now, even though EMC has not officially submitted the SPC1 result, even though it is posted on the SPC website, that leaves only one major storage vendor yet to have their midrange open storage systems represented on the SPC results, and that would be HDS with their AMS series, maybe He!! will still freeze over…

Cheers

Greg Schulz – www.storageio.com

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this: