Server StorageIO Data Infrastructure Insights and Analysis
| ||||||||||||||||
Ok, nuff said (for now)
Cheers
Gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
With the largest installment yet of a VMworld in terms of attendance, there were also many announcements today (e.g. Monday) and many more slated for out the week. Here are a synopsis of some of those announcements.
VMware made a series of announcements today that set the stage for many others. Not surprisingly, these involved SDDC, SDN, SDS, vSphere 5.5 and other management tool enhancements, or the other SDM (Software Defined Management).
VMware NSX (SDN) combines Nicira NVPTM along with vCloud Network and Security
VMware Virtual SAN (VSAN) not to be confused with virtual storage appliances (VSAs)
VMware vCloud Suite 5.5
VMware vSphere 5.5 (includes support for new Intel Xeon and Atom processors)
VMware vSphere App HA
VMware vSphere Flash Read Cache software
VMware vSphere Big Data Extensions
VMware vCloud Automation Center
VMware vCloud
Note that while these were announced today, some will be in public beta soon and general availability over the next few months or quarters (learn more here including pricing and availability). More on these and other enhancements in future posts. However for now check out what Duncan Epping (@DuncanYB) of VMware has to say over at his Yellowbook site here, here and here.
Buzzword Bingo
Dell did some announcements as well for cloud and virtual environments in support of VMware from networking to servers, hardware and software. With all the recent acquisitions by Dell including Quest where they picked up Foglight management tools, along with vRanger, Bakbone and others, Dell has amassed an interesting portfolio. On the hardware front, check out the VRTX shared server infrastructure, I want one for my VMware environment, now I just need to justify one (to myself). Speaking of Dell, if you are at VMworld on Tuesday August 27 around 1:30PM stop by the Dell booth where I will be presenting including announcing some new things (stay tuned for more on that soon).
HP had some announcements today. HP jumped into the SDDC and SDN with some Software Defined Marketing (SDM) and Software Defined Announcements (SDA) in addition to using the Unified Data Center theme. Today’s announcements by HP were focused more around SDN and VMware NSX along with the HP Virtual Application Networks SDN Controller and VMware networking.
NetApp (Both #1417) announced more integration between their Data ONTAP based solutions and VMware vSphere, Horizon Suite, vCenter, vCloud Automation Center and vCenter Log Insight under the them theme of SDDC and SDS. As part of the enhancement, NetApp announced Virtual Storage Console (VSC 5.0) for end-to-end storage management and software in VMware environments. In addition, integration with VMware vCenter Server 5.5. Not to be left out of the SSD flash dash NetApp also released a new V1.2 of their FlashAccel software for vSphere 5.0 and 5.1.
Here is one that you probably have not seen or heard much about elsewhere, which is Nlyte announcement of their V1.5 Virtualization Connector for Data Center Infrastructure Management (DCIM). Keep in mind that DCIM is more than facilities, power, and cooling related themes, particular in virtual data centers. Thus, some of the DCIM vendors, as well as others are moving into the converged DCIM space that spans server, storage, networking, hardware, software and facilities topics.
Interested in or want to know more about DCIM, and then check out these items:
Data Center Infrastructure Management (DCIM) and Infrastructure Resource Management (IRM)
Data Center Tools Can Streamline Computing Resources
Considerations for Asset Tracking and DCIM
Quantum announced that Commvault has added support to use the Lattus object storage based solution as an archive target platform. You can learn more about object storage (access and architectures) here at www.objectstoragecenter.com .
PHD Virtual did a couple of data protection (backup/restore , BC, DR ) related announcements (here and here ). Speaking of backup/restore and data protection, if you are at VMworld on Tuesday August 27th around 1:30PM, stop by the Dell booth where I will be presenting, and stay tuned for more info on some things we are going to announce at that time.
In case you missed it, Imation who bought Nexsan earlier this year last week announced their new unified NST6000 series of storage systems. The NST6000 storage solutions support Fibre Channel (FC) and iSCSI for block along with NFS, CIFS/SMB and FTP for file access from virtual and physical servers.
Emulex announced some new 16Gb Fibre Channel (e.g. 16GFC) aka what Brocade wants you to refer to as Gen 5 converged and multi-port adapters. I wonder how many still remember or would rather forget how many ASIC and adapter gens from various vendors occurred just at 1Gb Fibre Channel?
Proximal announced V2.0 of AutoCache 2.0 with role based administration, multi-hypervisor support (a growing trend beyond just a VMware focus) and more vCenter/vSphere integration. This is on the heels of last week’s FusionIO powered IBM Flash Cache Storage Accelerator (FCSA ) announcement, along with others such as EMC , Infinio, Intel, NetApp, Pernix, SanDisk (Flashsoft) to name a few.
Mellanox (VMworld booth #2005), you know, the Infinaband folks who also have some Ethernet (which also includes Fibre Channel over Ethernet) technology did a series of announcements today with various PCIe nand flash SSD card vendors. The common theme with the various vendors including Micron (Booth #1635) and LSI is in support of VMware virtual servers using iSER or iSCSI over RDMA (Remote Direct Memory Access). RDMA or server to server direct memory access (what some of you might know as remote memory mapped IO or channel to channel C2C) enables very fast low server to server data movement such as in a VMware cluster. Check out Mellanox and their 40Gb Ethernet along with Infinaband among other solutions if you are into server, storage i/o and general networking, along with their partners. Need or want to learn more about networking with your servers and storage check out Cloud and Virtual Data Storage Networking and Resilient Storage Networking .
Rest assured there are many more announcements and updates to come this week, and in the weeks to follow…
Ok, nuff said (for now).
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
As I often say, the best server storage I/O or IOP is the one that you do not have to do. The second best storage I/O or IOP is the one with least impact or that can be done in a cost-effective way. Likewise the question is not if solid-state device (SSD) including nand flash are in your future, rather when, where, why, with what, how much along with from whom. Also location matters when it comes to SSD including nand flash with different environments and applications leveraging different placement (locality) options, not to mention how much performance do you need vs. want?
As part of their $1 billion USD (to be spent over three years, or $333.3333 million per year) flash ahead initiative IBM has announced their Flash Cache Storage Accelerator (FCSA) server software. While IBM did not use the term, (congratulations and thank you btw) some creative marketer might want to try calling this Software Defined Cache (SDC) or Software Defined SSD (SDSSD) which if that occurs, apologies in advance ;). Keep in mind that it was about a year ago this time when IBM announced that they were acquiring SSD industry veteran Texas Memory Systems (TMS).
With this announcement of FCSA slated for customer general availability by end of August, IBM joins EMC and NetApp among other storage systems vendors who developed their own, or have collaborated on server-side IO optimization and cache software. Some of the other startup and established vendors who have IO optimization, performance acceleration and caching software include DataRam (Ramdisk), FusionIO, Infinio (NFS for VMware), Pernix (block for VMware), Proximal and SANdisk (bought flashsoft) among others.
Read more about IBM Flash Cache Software (FCSA) including various questions and perspectives in part two of this two-part post located here.
Ok, nuff said (for now)
Cheers
Gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
This is the second in a two-part post series on IBM’s Flash Cache Storage Accelerator (FCSA) for Solid State Device (SSD) storage announced today. You can view part I of the IBM FCSA announcement synopsis here.
What is FCSA?
FCSA is a server-side storage I/O or IOP caching software tool that makes use of local (server-side) nand flash SSD (PCIe cards or drives). As a cache tool (view IBM flash site here) FCSA provides persistent read caching on IBM servers (xSeries, Flex and Blade x86 based systems) with write through cache (e.g. data cached for later reads) while write data is written directly to block attached storage including SANs. back-end storage can be iSCSI, SAS, FC or FCoE based block systems from IBM or others including all SSD, hybrid SSD or traditional HDD based solutions from IBM and others.
How is this different from just using a dedicated PCIe nand flash SSD card?
FCSA complements those by using them as a persistent storage to cache storage I/O reads to boost performance. By using the PCIe nand flash card or SSD drives, FCSA and other storage I/O cache optimization tools free up valuable server-side DRAM from having to be used as a read cache on the servers. On the other hand, caching tools such as FCSA also keep local cached reads closer to the applications on the servers (e.g. locality of reference) reducing the impact on backed shared block storage systems.
What is FCSA for?
With storage I/O or IOPS and application performance in general, location matters due to locality of reference hence the need for using different approaches for various environments. IBM FCSA is a storage I/O caching software technology that reduces the impact of applications having to do random read operations. In addition to caching reads, FCSA also has a write-through cache, which means that while data written to back-end block storage including on iSCSI, SAS, FC or FCoE based storage (IBM or other vendors), a copy of the data is cached for later reads. Thus while the best storage I/O is the one that does not have to be done (e.g. can be resolved from cache), the second best would be writes that go to a storage system that are not competing with read requests (handled via cache).
Who else is doing this?
This is similar to what EMC initially announced and released in February 2012 with VFcache now renamed to be XtremSW along with other caching and IO optimization software from others (e.g. SANdisk, Proximal and Pernix among others.
Does this replace IBM EasyTier?
Simple answer is no, one is for tiering (e.g. EasyTier), the other is for IO caching and optimization (e.g. FCSA).
Does this replace or compete with other IBM SSD technologies?
With anything, it is possible to find a way to make or view it as competitive. However in general FCSA complements other IBM storage I/O optimization and management software tools such as EasyTier as well as leverage and coexist with their various SSD products (from PCIe cards to drives to drive shelves to all SSD and hybrid SSD solutions).
How does FCSA work?
The FCSA software works in either a physical machine (PM) bare metal mode with Microsoft Windows operating systems (OS) such as Server 2008, 2012 among others. There is also *nix support for RedHat Linux, along with in a VMware virtual machine (VM) environment. In a VMware environment High Availability (HA), DRS and VMotion services and capabilities are supported. Hopefully it will be sooner vs. later that we hear IBM do a follow-up announcement (pure speculation and wishful thinking) on more hypervisors (e.g. Hyper-V, Xen, KVM) support along with Centos, Ubuntu or Power based systems including IBM pSeries. Read more about IBM Pure and Flex systems here.
What about server CPU and DRAM overhead?
As should be expected, a minimal amount of server DRAM (e.g. main memory) and CPU processing cycles are used to support the FCSA software and its drivers. Note the reason I say as should be expected is how you can have software running on a server doing any type of work that does not need some amount of DRAM and processing cycles. Granted some vendors will try to spin and say that there is no server-side DRAM or CPU consumed which would be true if they are completely external to the server (VM or PM). The important thing is to understand how much of an impact in terms of CPU along with DRAM consumed along with their corresponding effectiveness benefit that are derived.
Does FCSA work with NAS (NFS or CIFS) back-end storage?
No this is a server-side block only cache solution. However having said that, if your applications or server are presenting shared storage to others (e.g. out the front-end) as NAS (NFS, CIFS, HDFS) using block storage (back-end), then FCSA can cache the storage I/O going to those back-end block devices.
Is this an appliance?
Short and simple answer is no, however I would not be surprised to hear some creative software defined marketer try to spin it as a flash cache software appliance. What this means is that FCSA is simply IO and storage optimization software for caching to boost read performance for VM and PM servers.
What is this hardware or storage agnostic stuff mean?
Simple, it means that FCSA can work with various nand flash PCIe cards or flash SSD drives installed in servers, as well as with various back-end block storage including SAN from IBM or others. This includes being able to use block storage using iSCSI, SAS, FC or FCoE attached storage.
What is the difference between Easytier and FCSA?
Simple, FCSA is providing read acceleration via caching which in turn should offload some reads from affecting storage systems so that they can focus on handling writes or read ahead operations. Easytier on the other hand is for as its name implies tiering or movement of data in a more deterministic fashion.
How do you get FCSA?
It is software that you buy from IBM that runs on an IBM x86 based server. It is licensed on a per server basis including one-year service and support. IBM has also indicated that they have volume or multiple servers based licensing options.
Does this mean IBM is competing with other software based IO optimization and cache tool vendors?
IBM is focusing on selling and adding value to their server solutions. Thus while you can buy the software from IBM for their servers (e.g. no bundling required), you cannot buy the software to run on your AMD/Seamicro, Cisco (including EMC/VCE and NetApp) , Dell, Fujitsu, HDS, HP, Lenovo, Oracle, SuperMicro among other vendors servers.
Will this work on non-IBM servers?
IBM is only supporting FCSA on IBM x86 based servers; however, you can buy the software without having to buy a solution bundle (e.g. servers or storage).
What is this Cooperative Caching stuff?
Cooperative caching takes the next step from simple read cache with write-through to also support chance coherency in a shared environment, as well as leverage tighter application or guest operating system and storage system integration. For example, applications can work with storage systems to make intelligent predictive informed decisions on what to pre-fetch or read ahead and cached, as well as enable cache warming on restart. Another example is where in a shared storage environment if one server makes a change to a shared LUN or volume that the local server-side caches are also updated to prevent stale or inconsistent reads from occurring.
Can FCSA use multiple nand flash SSD devices on the same server?
Yes, IBM FCSA supports use of multiple server-side PCIe and or drive based SSD devices.
How is cache coherency maintained including during a reboot?
While data stored in the nand flash SSD device is persistent, it’s up to the server and applications working with the storage systems to decide if there is coherent or stale data that needs to be refreshed. Likewise, since FCSA is server-side and back-end storage system or SAN agnostic, without cooperative caching it will not know if the underlying data for a storage volume changed without being notified from another server that modified it. Thus if using shared back-end including SAN storage, do your due diligence to make sure multi-host access to the same LUN’s or volumes is being coordinated with some server-side software to support cache coherency, something that would apply to all vendors.
What about cache warming or reloading of the read cache?
Some vendors who have tightly interested caching software and storage systems, something IBM refers to as cooperative caching that can have the ability to re-warm the cache. With solutions that support cache re-warming, the cache software and storage systems work together to main cache coherency while pre-loading data from the underlying storage system based on hot bands or other profiles and experience. As of this announcement, FCSA does not support cache warming on its own.
Does IBM have service or tools to complement FCSA?
Yes, IBM has an assessment, profile and planning tool that are available on a free consultation services basis with a technician to check your environment. Of course, the next logical step would be for IBM to make the tool available via free download or on some other basis as well.
Do I recommend and have I tried FCSA?
On paper, or WebEx, YouTube or other venue FCSA looks interesting and capable, a good fit for some environments particular if IBM server-based. However since my PM and VMware VM based servers are from other vendors, along with the fact that FCSA only runs on IBM servers, have not actually given it a hands on test drive yet. Thus if you are looking at storage I/O optimization and caching software tools for your VM or PM environment, checkout IBM FCSA to see if it meets your needs.
It is great to see server and storage systems vendors add value to their solutions with I/O and performance optimization as well as caching software tools. However, I am also concerned with the growing numbers of different software tools that only work with one vendor’s servers or storage systems, or at least are supported as such.
This reminds me of a time not all that long ago (ok, for some longer than others) when we had a proliferation of different host bus adapter (HBA) driver and pathing drivers from various vendors. The result is a hodge podge (a technical term) of software running on different operating systems, hypervisors, PM’s, VMs, and storage systems, all of which need to be managed. On the other hand, for the time being perhaps the benefit will outweigh the pain of having different tools. That is where there are options from server-side vendor centric, storage system focused, or third-party software tool providers.
Another consideration is that some tools work in VMware environments; others support multiple hypervisors while others also support bare metal servers or PMs. Which applies to your environment will of course depend. After all, if you are an all VMware environment given that many of the caching tools tend to be VMware focused, that gives more options vs. for those who are still predominately PM environments.
Ok, nuff said (for now)
Cheers
Gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
Often what’s assumed is not always the case. For example in along with around server, storage and IO networking circles including virtual as well as cloud environments terms such as nand (Negated AND or NOT And) flash memory aka (Solid State Device or SSD), DRAM (Dynamic Random Access Memory), DDR3 (Double Data Rate 3) not to mention DIMM (Dual Inline Memory Module) get tossed around with the assumption everybody must know what they mean.
On the other hand, I find plenty of people who are not sure what those among other terms or things are, sometimes they are even embarrassed to ask, particular if they are a self-proclaimed expert.
So for those who need a refresh or primer, here you go, an excerpt from Chapter 7 (Servers – Physical, Virtual and Software) from my book "The Green and Virtual Data Center" (CRC Press) available at Amazon.com and other global venues in print and ebook formats.
7.2.2 Memory Computers rely on some form of memory ranging from internal registers, local on-board processor Level 1 (L1) and Level 2 (L2) caches, random accessible memory (RAM), non-volatile RAM (NVRAM) or nand Flash (SSD) along with external disk storage. Memory, which includes external disk storage, is used for storing operating system software along with associated tools or utilities, application programs and data. Main memory or RAM, also known as dynamic RAM (DRAM) chips, is packaged in different ways with a common form being dual inline memory modules (DIMMs) for notebook or laptop, desktop PC and servers. RAM main memory on a server is the fastest form of memory, second only to internal processor or chip based registers, L1, L2 or local memory. RAM and processor based memories are volatile and non-persistent in that when power is removed, the contents of memory are lost. As a result, some form of persistent memory is needed to keep programs and data when power is removed. Read only memory (ROM) and NVRAM are both persistent forms of memory in that their contents are not lost when power is removed. The amount of RAM that can be installed into a server will vary with specific architecture implementation and operating software being used. In addition to memory capacity and packaging format, the speed of memory is also important to be able to move data and programs quickly to avoid internal bottlenecks. Memory bandwidth performance increases with the width of the memory bus in bits and frequency in MHz. For example, moving 8 bytes on a 64 bit buss in parallel at the same time at 100MHz provides a theoretical 800MByte/sec speed. To improve availability and increase the level of persistence, some servers include battery backed up RAM or cache to protect data in the event of a power loss. Another technique to protect memory data on some servers is memory mirroring where twice the amount of memory is installed and divided into two groups. Each group of memory has a copy of data being stored so that in the event of a memory failure beyond those correctable with standard parity and error correction code (ECC) no data is lost. In addition to being fast, RAM based memories are also more expensive and used in smaller quantities compared to external persistent memories such as magnetic hard disk drives, magnetic tape or optical based memory medias. The above shows a tiered memory model that may look familiar as the bottom part is often expanded to show tiered storage. At the top of the memory pyramid is high-speed processor memory followed by RAM, ROM, NVRAM and FLASH along with many forms of external memory commonly called storage. More detail about tiered storage is covered in chapter 8 (Data Storage – Data Storage – Disk, Tape, Optical, and Memory). In addition to being slower and lower cost than RAM based memories, disk storage along with NVRAM and FLASH based memory devices are also persistent. By being persistent, when power is removed, data is retained on the storage or memory device. Also shown in the above figure is that on a relative basis, less energy is used for power storage or memory at the bottom of the pyramid than for upper levels where performance increases. From a PCFE (Power, Cooling, Floor space, Economic) perspective, balancing memory and storage performance, availability, capacity and energy to a given function, quality of service and service level objective for a given cost needs to be kept in perspective and not considering simply the lowest cost for the most amount of memory or storage. In addition to gauging memory on capacity, other metrics include percent used, operating system page faults and page read/write operations along with memory swap activity as well memory errors. Base 2 versus base 10 numbering systems can account for some storage capacity that appears to “missing” when real storage is compared to what is expected to be seen. Disk drive manufacturers use base 10 (decimal) to count bytes of data while memory chip, server and operating system vendors typically use base 2 (binary) to count bytes of data. This has led to confusion when comparing a disk drive base 10 GB with a chip memory base 2 GB of memory capacity, such as 1,000,000,000 (10^9) bytes versus 1,073,741,824 (2^30) bytes. Nomenclature based on the International System of Units uses MiB, GiB and TiB to denote million, billion and trillion bytes for base 2 numbering with base 10 using MB, TB and GB . Most vendors do document how many bytes, sometimes in both base 2 and base 10, as well as the number of 512 byte sectors supported on their storage devices and storage systems, though it might be in the small print. |
Related more reading:
How much storage performance do you want vs. need?
Can RAID extend the life of nand flash SSD?
Can we get a side of context with them IOPS and other storage metrics?
SSD & Real Estate: Location, Location, Location
What is the best kind of IO? The one you do not have to do
SSD, flash and DRAM, DejaVu or something new?
Ok, nuff said (for now).
Cheers
Gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier).
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
Whats the best server storage I/O network metric or benchmark? It depends as there needs to be some context with them IOPS and other server storage I/O metrics that matter.
There is an old saying that the best I/O (Input/Output) is the one that you do not have to do.
In the meantime, let’s get a side of some context with them IOPS from vendors, marketers and their pundits who are tossing them around for server, storage and IO metrics that matter.
The good news is that people are beginning to discuss storage beyond space capacity and cost per GByte, TByte or PByte for both DRAM or nand flash Solid State Devices (SSD), Hard Disk Drives (HDD) along with Hybrid HDD (HHDD) and Solid State Hybrid Drive (SSHD) based solutions. This applies to traditional enterprise or SMB IT data center with physical, virtual or cloud based infrastructures.
This is good because it expands the conversation beyond just cost for space capacity into other aspects including performance (IOPS, latency, bandwidth) for various workload scenarios along with availability, energy effective and management.
The catch is that IOPS while part of the equation are just one aspect of performance and by themselves without context, may have little meaning if not misleading in some situations.
Granted it can be entertaining, fun to talk about or simply make good press copy for a million IOPS. IOPS vary in size depending on the type of work being done, not to mention reads or writes, random and sequential which also have a bearing on data throughout or bandwidth (Mbytes per second) along with response time. Not to mention block, file, object or blob as well as table.
However, are those million IOP’s applicable to your environment or needs?
Likewise, what do those million or more IOPS represent about type of work being done? For example, are they small 64 byte or large 64 Kbyte sized, random or sequential, cached reads or lazy writes (deferred or buffered) on a SSD or HDD?
How about the response time or latency for achieving them IOPS?
In other words, what is the context of those metrics and why do they matter?
Click on image to view more metrics that matter including IOP’s for HDD and SSD’s
Metrics that matter give context for example IO sizes closer to what your real needs are, reads and writes, mixed workloads, random or sequential, sustained or bursty, in other words, real world reflective.
As with any benchmark take them with a grain (or more) of salt, they key is use them as an indicator then align to your needs. The tool or technology should work for you, not the other way around.
Here are some examples of context that can be added to help make IOP’s and other metrics matter:
The above are just a sampling and not all may be relevant to your particular needs, however they help to put IOP’s into more contexts. Another consideration around IOPS are the configuration of the environment, from an actual running application using some measurement tool, or are they generated from a workload tool such as IOmeter, IOrate, VDbench among others.
Sure, there are more contexts and information that would be interesting as well, however learning to walk before running will help prevent falling down.
Some vendors are doing a good job of going for out of this world record-setting marketing hero numbers.
Meanwhile other vendors are doing a good job of adding context to their IOP or response time or bandwidth among other metrics that matter. There is a mix of startup and established that give context with their IOP’s or other metrics, likewise size or age does not seem to matter for those who lack context.
Some vendors may not offer metrics or information publicly, so fine, go under NDA to learn more and see if the results are applicable to your environments.
Likewise, if they do not want to provide the context, then ask some tough yet fair questions to decide if their solution is applicable for your needs.
View additional NAS, NVMe, SSD, NVM, SCM, Data Infrastructure and HDD related topics via the following links.
Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.
What this means is let us start putting and asking for metrics that matter such as IOP’s with context.
If you have a great IOP metric, if you want it to matter than include some context such as what size (e.g. 4K, 8K, 16K, 32K, etc.), percentage of reads vs. writes, latency or response time, random or sequential.
IMHO the most interesting or applicable metrics that matter are those relevant to your environment and application. For example if your main application that needs SSD does about 75% reads (random) and 25% writes (sequential) with an average size of 32K, while fun to hear about, how relevant is a million 64 byte read IOPS? Likewise when looking at IOPS, pay attention to the latency, particular if SSD or performance is your main concern.
Get in the habit of asking or telling vendors or their surrogates to provide some context with them metrics if you want them to matter.
So how about some context around them IOP’s (or latency and bandwidth or availability for that matter)?
Ok, nuff said, for now.
Gs
Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.
Hitachi Data Systems (HDS) announced today several enhancements to their data storage and unified compute portfolio as part of their Maximize I.T. initiative.
As part of setting the stage for this announcement, HDS has presented the following strategy vision as part their vision for IT transformation and cloud computing.
This announcement builds on earlier ones around HDS Unified Storage (HUS) primary storage using nand flash MLC Solid State Devices (SSD) and Hard Disk Drives (HDD’s), along with unified block and file (NAS), as well Unified Compute Platform (UCP) also known as converged compute, networking, storage and software. These enhancements follow recent updates to the HDS Content Platform (HCP) for object, file and content storage.
There are three main focus areas of the announcement:
The question should not be if SSD is in your future, rather when, where, with what and how much will be needed.
As part of this announcement, HDS is releasing an all flash SSD based HUS enterprise storage system. Similar to what other vendors have done, HDS is attaching flash SSD storage to their HUS systems in place of HDD’s. Hitachi has developed their own SSD module announced in 2012 (read more here). The HDS SSD module use Multi Level Cell (MLC) nand flash chips (dies) that now supports 1.6TB of storage space capacity unit. This is different from other vendors who either use nand flash SSD drive form factor devices (e.g. Intel, Micron, Samsung, SANdisk, Seagate, STEC (now WD), WD among others) or, PCIe form factor cards (e.g. FusionIO, Intel, LSI, Micron, Virident among others) or, attach a third-party external SSD device (e.g. IBM/TMS, Violin, Whiptail etc.).
Like some other vendors, HDS has also done more than simply attach a SSD (drive, PCIe card, or external device) to their storage systems calling it an integrated solution. What this means is that HDS has implemented software or firmware changes into their storage systems to manage durability and extend flash duty cycles caused by program erase (P/E) cycle wear. In addition HDS has implemented performance optimization in their storage systems to leverage the faster SSD modules, after all, faster storage media or devices need fast storage systems or controllers.
While the new all flash storage system can be initially bought with just SSD, similar to other hybrid storage solutions, hard disk drives (HDD’s) can also be installed. For enabling full performance at low latency, HDS is addressing both the flash SSD modules as well as the storage systems they attach to including back-end, front-end and caching in-between.
The release enables 500,000 or half a million IOPS (no IOP size, reads or writes, random or sequential. Future firmware (non-disrupted) to enable higher performance that HDS is claiming will be 1,000,000 IOPS at under a millisecond) were indicated.
In addition to future performance improvements, HDS is also indicating increased storage space capacity of its MLC flash SSD modules (1.6TB today). Using 12 modules (1.6TB each), 154TB of flash SSD can be placed in a single rack.
HUS unified NAS file system and gateway (BlueArc based) enhancements include:
As part of this announcement, HDS is enhancing the Unified Compute Platform (UCP) offerings. HDS re-entered the compute market in 2012 joining other vendors offering unified compute, storage and networking solutions. The HDS converged data infrastructure competes with AMD (Seamicro) SM15000, Dell vStart and VRTX (for lower end market), EMC and VCE vBlock, NetApp FlexPod along with those from HP (or Moonshot micro servers), IBM Puresystems, Oracle and others.
UCP Pro for VMware vSphere
UCP Select for Microsoft Private Cloud
UCP Select for Oracle RAC
UCP Select for SAP HANA
With these announcements HDS is extending its storage centric hardware, software and services solution portfolio for block, file and object access across different usage tiers (systems, applications, mediums). HDS is also expanding their converged unified compute platforms to stay competitive with others including Dell, EMC, Fujitsu, HP, IBM, NEC, NetApp and Oracle among others. For environments with HDS storage looking for converged solutions to support VMware, Microsoft Hyper-V, Oracle or SAP HANA these UCP systems are worth checking out as part of evaluating vendor offerings. Likewise for those who have HDS storage exploring SSD offerings, these announcements give opportunities to enable consolidation as do the unified file (NAS) offerings.
Note that now HDS does not have a public formalized message or story around PCIe flash cards, however they have relationships with various vendors as part of their UCP offerings.
Overall a good set of incremental enhancements for HDS to stay competitive and leverage their field proven capabilities including management software tools.
Ok, nuff said
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
IBM today announced that they are acquiring privately held Dallas Texas-based Softlayer and Infrastructure as a Service (IaaS) provider.
IBM is referring to this as Cloud without Compromise (read more about clouds, conversations and confidence here).
It’s about the management, flexibly, scale up, out and down, agility and valueware.
Is this IBM’s new software defined data center (SDDC) or software defined infrastructure (SDI) or software defined management (SDM), software defined cloud (SDC) or software defined storage (SDS) play?
This is more than a software defined marketing or software defined buzzword announcement.
If your view of software define ties into the theme of leveraging, unleashing resources, enablement, flexibility, agility of hardware, software or services, then you may see Softlayer as part of a software defined infrastructure.
On the other hand, if your views or opinions of what is or is not software defined align with a specific vendor, product, protocol, model or punditry then you may not agree, particular if it is in opposition to anything IBM.
During today’s announcement briefing call with analysts there was a noticeable absence of software defined buzz talk which given its hype and usage lately, was a refreshing welcome relief. So with that, lets set the software defined conversation aside (for now).
Softlayer provide software and services to support both SMB, SME and other environments with bare metal (think traditional hosted servers), along with multi-tenant (shared) cloud virtual public and private cloud service offerings.
Softlayer supports various applications, environments from little data processing to big data analytics to little data processing, from social to mobile to legacy. This includes those app’s or environments that were born in the cloud, or legacy environments looking to leverage cloud in a complimentary way.
Some more information about Softlayer includes:
Softlayer will report into IBM Global Technology Services (GTS) complimenting existing capabilities which includes ten cloud computing centers on five continents. IBM has created a new Cloud Services Division and expects cloud revenues could be $7 billion annually by the end of 2015. Amazon Web Services (AWS) is estimated to hit about $3.8 Billion by end of 2013. Note that in 2012 AWS target available market was estimated to be about $11 Billion which should become larger moving forward. Rackspace by comparison had recent earning announcements on May 8 2013 of $362 Million with most that being hosting vs. cloud services. That works out to an annualized estimated run rate of $1.448 Billion (or better depending on growth).
I mention AWS and Rackspace to illustrate the growth potential for IBM and Softlayer to discuss the needs of both cloud services customers such as those who use AWS (among other providers), as well as bare metal or hosting or dedicated servers such as with Rackspace among others.
What is not clear at this time is if IBM is combing traditional hosting, managed services, new offerings, products and services in that $7 billion number. In other words if the $7 billion represents what the revenues of the new Cloud Services Division independent of other GTS or legacy offerings as well as excluding hardware, software products from STG (Systems Technology Group) among others, that would be impressive and a challenge to the likes of AWS.
IBM has indicated that it will leverage its existing Systems Technology Group (STG) portfolio of servers and storage extending the capabilities of Softlayer. While currently x86 based, one could expect IBM to leverage and add support for their Power systems line of processors and servers, Puresystems, as well as storage such as XIV or V7000 among others for tier 1 needs.
Some more notes:
This is a good and big move for IBM to add value and leverage their current portfolios of both services, as well as products and technologies. However it is more than just adding value or finding new routes to markets for those goods and services, it’s also about enablement IBM has long been in the services including managed services, out or in sourcing and hosting business. This can be seen as another incremental evolution of those offerings to both existing IBM enterprise customers, as well to reach new, emerging along with SMB or SME’s that tend to grow up and become larger consumers of information and data infrastructure services.
Further this helps to add some product and meaning around the IBM Smart Cloud initiatives and programs (not that there was not before) giving customers, partners and resellers something tangible to see, feel, look at, touch and gain experience not to mention confidence with clouds.
On the other hand, is IBM signaling that they want more of the growing business that AWS has been realizing, not to mention Microsoft Azure, Rackspace, Centurylink/Savvis, Verizon/Terremark, CSC, HP Cloud, Cloudsigma, Bluehost among many others (if I missed you or your favorite provider, feel free to add it to the comments section). This also gets IBM added Devops exposure something that Softlayer practices, as well as a Openstack play, not to mention cloud, software defined, virtual, big data, little data, analytics and many other buzzword bingo terms.
Congratulations to both IBM and the Softlayer folks, now lets see some execution to watch how this unfolds.
Ok, nuff said.
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
Join me on Thursday May 30, 2013 at Noon ET (9AM PT) for a live web chat at the 21st Century IT (21cit) site (click here to register, sign-up, or view earlier posts). This will be an online web chat format interactive conversation so if you are not able to attend, you can visit at your convenience to view and give your questions along with comments. I have done several of these web chats with 21cit as well as other venues that are a lot of fun and engaging (time flies by fast).
For those not familiar, 21cIT is part of the Desum/UBM family of sites including Internet Evolution, SMB Authority, and Enterprise Efficiency among others that I do article posts, videos and live chats for.
I like these types of sites in that while they have a sponsor, the content is generally kept separate between those of editors and contributors like myself and the vendor supplied material. In other words I coordinate with the site editors on what topics I feel like writing (or doing videos) about that align with the given sites focus and themes as opposed to following and advertorial calendar script.
During this industry trends perspective web chat, one of the topics and themes planned for discussion include software defined storage (SDS). View a recent video blog post I did here about SDS. In addition to SDS, Solid State Devices (SSD) including nand flash, cloud, virtualization, object, backup and data protection, performance, management tools among others are topics that will be put out on the virtual discussion table.
Following are some examples of recent and earlier industry trends perspectives posts that I have done over at 21cit:
Video: And Now, Software-Defined Storage!
There are many different views on what is or is not “software-defined” with products, protocols, preferences and even press releases. Check out the video and comments here.
Big Data and the Boston Marathon Investigation
How the human face of big-data will help investigators piece together all the evidence in the Boston bombing tragedy and bring those responsible to justice. Check out the post and comments here.
Don’t Use New Technologies in Old Ways
You can add new technologies to your data center infrastructure, but you won’t get the full benefit unless you update your approach with people, processes, and policies. Check out the post and comments here.
Don’t Let Clouds Scare You, Be Prepared
The idea of moving to cloud computing and cloud services can be scary, but it doesn’t have to be so if you prepare as you would for implementing any other IT tool. Check out the post and comments here.
Storage and IO trends for 2013 (& Beyond)
Efficiency, new media, data protection, and management are some of the keywords for the storage sector in 2013. Check out these and other trends, predictions along with comments here.
SSD and Real Estate: Location, Location, Location
You might be surprised how many similarities between buying real estate and buying SSDs.
Location matters and it’s not if, rather when, where, why and how you will be using SSD including nand flash in the future, read more and view comments here.
Everything Is Not Equal in the Data center, Part 3
Here are steps you can take to give the right type of backup and protection to data and solutions, depending on the risks and scenarios they face. The result? Savings and efficiencies. Read more and view comments here.
Everything Is Not Equal in the Data center, Part 2
Your data center’s operations can be affected at various levels, by multiple factors, in a number of degrees. And, therefore, each scenario requires different responses. Read more and view comments here.
Everything Is Not Equal in the Data center, Part 1
It pays to check your data center Different components need different levels of security, storage, and availability. Read more and view comments here.
Data Protection Modernizing: More Than Buzzword Bingo
IT professionals and solution providers should put technologies such as disk based backup, dedupe, cloud, and data protection management tools as assets and resources to make sure they receive necessary funding and buy in. Read more and view comments here.
Don’t Take Your Server & Storage IO Pathing Software for Granted
Path managers are valuable resources. They will become even more useful as companies continue to carry out cloud and virtualization solutions. Read more and view comments here.
SSD Is in Your Future: Where, When & With What Are the Questions
During EMC World 2012, EMC (as have other vendors) made many announcements around flash solid-state devices (SSDs), underscoring the importance of SSDs to organizations future storage needs. Read more here about why SSD is in your future along with view comments.
Changing Life cycles and Data Footprint Reduction (DFR), Part 2
In the second part of this series, the ABCDs (Archive, Backup modernize, Compression, Dedupe and data management, storage tiering) of data footprint reduction, as well as SLOs, RTOs, and RPOs are discussed. Read more and view comments here.
Changing Life cycles and Data Footprint Reduction (DFR), Part 1
Web 2.0 and related data needs to stay online and readily accessible, creating storage challenges for many organizations that want to cut their data footprint. Read more and view comments here.
No Such Thing as an Information Recession
Data, even older information, must be protected and made accessible cost-effectively. Not to mention that people and data are living longer as well as getting larger. Read more and view comments here.
These real-time, industry trends perspective interactive chats at 21cit are open forum format (however be polite and civil) as well as non vendor sales or marketing pitches. If you have specific questions you ‘d like to ask or points of view to express, click here and post them in the chat room at any time (before, during or after).
Mark your calendar for this event live Thursday, May 30, at noon ET or visit after the fact.
Ok, nuff said (for now)
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
Updated 2/10/2018
This is the second post of a two-part series looking at storage performance, specifically in the context of drive or device (e.g. mediums) characteristics of How many IOPS can a HDD HHDD SSD do with VMware. In the first post the focus was around putting some context around drive or device performance with the second part looking at some workload characteristics (e.g. benchmarks).
A common question is how many IOPS (IO Operations Per Second) can a storage device or system do?
The answer is or should be it depends.
Here are some examples to give you some more insight.
For example, the following shows how IOPS vary by changing the percent of reads, writes, random and sequential for a 4K (4,096 bytes or 4 KBytes) IO size with each test step (4 minutes each).
IO Size for test | Workload Pattern of test | Avg. Resp (R+W) ms | Avg. IOP Sec (R+W) | Bandwidth KB Sec (R+W) |
4KB | 100% Seq 100% Read | 0.0 | 29,736 | 118,944 |
4KB | 60% Seq 100% Read | 4.2 | 236 | 947 |
4KB | 30% Seq 100% Read | 7.1 | 140 | 563 |
4KB | 0% Seq 100% Read | 10.0 | 100 | 400 |
4KB | 100% Seq 60% Read | 3.4 | 293 | 1,174 |
4KB | 60% Seq 60% Read | 7.2 | 138 | 554 |
4KB | 30% Seq 60% Read | 9.1 | 109 | 439 |
4KB | 0% Seq 60% Read | 10.9 | 91 | 366 |
4KB | 100% Seq 30% Read | 5.9 | 168 | 675 |
4KB | 60% Seq 30% Read | 9.1 | 109 | 439 |
4KB | 30% Seq 30% Read | 10.7 | 93 | 373 |
4KB | 0% Seq 30% Read | 11.5 | 86 | 346 |
4KB | 100% Seq 0% Read | 8.4 | 118 | 474 |
4KB | 60% Seq 0% Read | 13.0 | 76 | 307 |
4KB | 30% Seq 0% Read | 11.6 | 86 | 344 |
4KB | 0% Seq 0% Read | 12.1 | 82 | 330 |
Dell/Western Digital (WD) 1TB 7200 RPM SATA HDD (Raw IO) thread count 1 4K IO size
In the above example the drive is a 1TB 7200 RPM 3.5 inch Dell (Western Digital) 3Gb SATA device doing raw (non file system) IO. Note the high IOP rate with 100 percent sequential reads and a small IO size which might be a result of locality of reference due to drive level cache or buffering.
Some drives have larger buffers than others from a couple to 16MB (or more) of DRAM that can be used for read ahead caching. Note that this level of cache is independent of a storage system, RAID adapter or controller or other forms and levels of buffering.
Does this mean you can expect or plan on getting those levels of performance?
I would not make that assumption, and thus this serves as an example of using metrics like these in the proper context.
Building off of the previous example, the following is using the same drive however with a 16K IO size.
IO Size for test | Workload Pattern of test | Avg. Resp (R+W) ms | Avg. IOP Sec (R+W) | Bandwidth KB Sec (R+W) |
16KB | 100% Seq 100% Read | 0.1 | 7,658 | 122,537 |
16KB | 60% Seq 100% Read | 4.7 | 210 | 3,370 |
16KB | 30% Seq 100% Read | 7.7 | 130 | 2,080 |
16KB | 0% Seq 100% Read | 10.1 | 98 | 1,580 |
16KB | 100% Seq 60% Read | 3.5 | 282 | 4,522 |
16KB | 60% Seq 60% Read | 7.7 | 130 | 2,090 |
16KB | 30% Seq 60% Read | 9.3 | 107 | 1,715 |
16KB | 0% Seq 60% Read | 11.1 | 90 | 1,443 |
16KB | 100% Seq 30% Read | 6.0 | 165 | 2,644 |
16KB | 60% Seq 30% Read | 9.2 | 109 | 1,745 |
16KB | 30% Seq 30% Read | 11.0 | 90 | 1,450 |
16KB | 0% Seq 30% Read | 11.7 | 85 | 1,364 |
16KB | 100% Seq 0% Read | 8.5 | 117 | 1,874 |
16KB | 60% Seq 0% Read | 10.9 | 92 | 1,472 |
16KB | 30% Seq 0% Read | 11.8 | 84 | 1,353 |
16KB | 0% Seq 0% Read | 12.2 | 81 | 1,310 |
Dell/Western Digital (WD) 1TB 7200 RPM SATA HDD (Raw IO) thread count 1 16K IO size
The previous two examples are excerpts of a series of workload simulation tests (ok, you can call them benchmarks) that I have done to collect information, as well as try some different things out.
The following is an example of the summary for each test output that includes the IO size, workload pattern (reads, writes, random, sequential), duration for each workload step, totals for reads and writes, along with averages including IOP’s, bandwidth and latency or response time.
Want to see more numbers, speeds and feeds, check out the following table which will be updated with extra results as they become available.
Device | Vendor | Make | Model | Form Factor | Capacity | Interface | RPM Speed | Raw Test Result |
HDD | HGST | Desktop | HK250-160 | 2.5 | 160GB | SATA | 5.4K | |
HDD | Seagate | Mobile | ST2000LM003 | 2.5 | 2TB | SATA | 5.4K | |
HDD | Fujitsu | Desktop | MHWZ160BH | 2.5 | 160GB | SATA | 7.2K | |
HDD | Seagate | Momentus | ST9160823AS | 2.5 | 160GB | SATA | 7.2K | |
HDD | Seagate | MomentusXT | ST95005620AS | 2.5 | 500GB | SATA | 7.2K(1) | |
HDD | Seagate | Barracuda | ST3500320AS | 3.5 | 500GB | SATA | 7.2K | |
HDD | WD/Dell | Enterprise | WD1003FBYX | 3.5 | 1TB | SATA | 7.2K | |
HDD | Seagate | Barracuda | ST3000DM01 | 3.5 | 3TB | SATA | 7.2K | |
HDD | Seagate | Desktop | ST4000DM000 | 3.5 | 4TB | SATA | HDD | |
HDD | Seagate | Capacity | ST6000NM00 | 3.5 | 6TB | SATA | HDD | |
HDD | Seagate | Capacity | ST6000NM00 | 3.5 | 6TB | 12GSAS | HDD | |
HDD | Seagate | Savio 10K.3 | ST9300603SS | 2.5 | 300GB | SAS | 10K | |
HDD | Seagate | Cheetah | ST3146855SS | 3.5 | 146GB | SAS | 15K | |
HDD | Seagate | Savio 15K.2 | ST9146852SS | 2.5 | 146GB | SAS | 15K | |
HDD | Seagate | Ent. 15K | ST600MP0003 | 2.5 | 600GB | SAS | 15K | |
SSHD | Seagate | Ent. Turbo | ST600MX0004 | 2.5 | 600GB | SAS | SSHD | |
SSD | Samsung | 840 PRo | MZ-7PD256 | 2.5 | 256GB | SATA | SSD | |
HDD | Seagate | 600 SSD | ST480HM000 | 2.5 | 480GB | SATA | SSD | |
SSD | Seagate | 1200 SSD | ST400FM0073 | 2.5 | 400GB | 12GSAS | SSD | |
Performance characteristics 1 worker (thread count) for RAW IO (non-file system)
Note: (1) Seagate Momentus XT is a Hybrid Hard Disk Drive (HHDD) based on a 7.2K 2.5 HDD with SLC nand flash integrated for read buffer in addition to normal DRAM buffer. This model is a XT I (4GB SLC nand flash), may add an XT II (8GB SLC nand flash) at some future time.
As a starting point, these results are raw IO with file system based information to be added soon along with more devices. These results are for tests with one worker or thread count, other results will be added with such as 16 workers or thread counts to show how those differ.
The above results include all reads, all writes, mix of reads and writes, along with all random, sequential and mixed for each IO size. IO sizes include 4K, 8K, 16K, 32K, 64K, 128K, 256K, 512K, 1024K and 2048K. As with any workload simulation, benchmark or comparison test, take these results with a grain of salt as your mileage can and will vary. For example you will see some what I consider very high IO rates with sequential reads even without file system buffering. These results might be due to locality of reference of IO’s being resolved out of the drives DRAM cache (read ahead) which vary in size for different devices. Use the vendor model numbers in the table above to check the manufactures specs on drive DRAM and other attributes.
If you are used to seeing 4K or 8K and wonder why anybody would be interested in some of the larger sizes take a look at big fast data or cloud and object storage. For some of those applications 2048K may not seem all that big. Likewise if you are used to the larger sizes, there are still applications doing smaller sizes. Sorry for those who like 512 byte or smaller IO’s as they are not included. Note that for all of these unless indicated a 512 byte standard sector or drive format is used as opposed to emerging Advanced Format (AF) 4KB sector or block size. Watch for some more drive and device types to be added to the above, along with results for more workers or thread counts, along with file system and other scenarios.
The above performance results were generated on Ubuntu 12.04 (since upgraded to 14.04 which was hosted on a VMware vSphere 5.1 (upgraded to 5.5U2) purchased version (you can get the ESXi free version here) with vCenter enabled system. I also have VMware workstation installed on some of my Windows-based laptops for doing preliminary testing of scripts and other activity prior to running them on the larger server-based VMware environment. Other VMware tools include vCenter Converter, vSphere Client and CLI. Note that other guest virtual machines (VMs) were idle during the tests (e.g. other guest VMs were quiet). You may experience different results if you ran Ubuntu native on a physical machine or with different adapters, processors and device configurations among many other variables (that was a disclaimer btw ;) ).
All of the devices (HDD, HHDD, SSD’s including those not shown or published yet) were Raw Device Mapped (RDM) to the Ubuntu VM bypassing VMware file system.
| Example of creating an RDM for local SAS or SATA direct attached device. vmkfstools -z /vmfs/devices/disks/naa.600605b0005f125018e923064cc17e7c /vmfs/volumes/dat1/RDM_ST1500Z110S6M5.vmdk The above uses the drives address (find by doing a ls -l /dev/disks via VMware shell command line) to then create a vmdk container stored in a dat. Note that the RDM being created does not actually store data in the .vmdk, it’s there for VMware management operations. |
If you are not familiar with how to create a RDM of a local SAS or SATA device, check out this post to learn how.This is important to note in that while VMware was used as a platform to support the guest operating systems (e.g. Ubuntu or Windows), the real devices are not being mapped through or via VMware virtual drives.
The above shows examples of RDM SAS and SATA devices along with other VMware devices and dats. In the next figure is an example of a workload being run in the test environment.
One of the advantages of using VMware (or other hypervisor) with RDM’s is that I can quickly define via software commands where a device gets attached to different operating systems (e.g. the other aspect of software defined storage). This means that after a test run, I can quickly simply shutdown Ubuntu, remove the RDM device from that guests settings, move the device just tested to a Windows guest if needed and restart those VMs. All of that from where ever I happen to be working from without physically changing things or dealing with multi-boot or cabling issues.
View additional NAS, NVMe, SSD, NVM, SCM, Data Infrastructure and HDD related topics via the following links.
Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.
That depends, however have a look at the above information and results.
Check back from time to time here to see what is new or has been added including more drives, devices and other related themes.
Ok, nuff said, for now.
Gs
Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.
Updated 2/10/2018
A common question I run across is how many I/O iopsS can flash SSD or HDD storage device or system do or give.
The answer is or should be it depends.
This is the first of a two-part series looking at storage performance, and in context specifically around drive or device (e.g. mediums) characteristics across HDD, HHDD and SSD that can be found in cloud, virtual, and legacy environments. In this first part the focus is around putting some context around drive or device performance with the second part looking at some workload characteristics (e.g. benchmarks).
What about cloud, tape summit resources, storage systems or appliance?
Lets leave those for a different discussion at another time.
Part of my interest in tools, metrics that matter, measurements, analyst, forecasting ties back to having been a server, storage and IO performance and capacity planning analyst when I worked in IT. Another aspect ties back to also having been a sys admin as well as business applications developer when on the IT customer side of things. This was followed by switching over to the vendor world involved with among other things competitive positioning, customer design configuration, validation, simulation and benchmarking HDD and SSD based solutions (e.g. life before becoming an analyst and advisory consultant).
Btw, if you happen to be interested in learn more about server, storage and IO performance and capacity planning, check out my first book Resilient Storage Networks (Elsevier) that has a bit of information on it. There is also coverage of metrics and planning in my two other books The Green and Virtual Data Center (CRC Press) and Cloud and Virtual Data Storage Networking (CRC Press). I have some copies of Resilient Storage Networks available at a special reader or viewer rate (essentially shipping and handling). If interested drop me a note and can fill you in on the details.
There are many rules of thumb (RUT) when it comes to metrics that matter such as IOPS, some that are older while others may be guess or measured in different ways. However the answer is that it depends on many things ranging from if a standalone hard disk drive (HDD), Hybrid HDD (HHDD), Solid State Device (SSD) or if attached to a storage system, appliance, or RAID adapter card among others.
Various HDD, HHDD and SSD’s
Even if just looking at a HDD, there are many variables ranging from the rotational speed or Revolutions Per Minute (RPM), interface including 1.5Gb, 3.0Gb, 6Gb or 12Gb SAS or SATA or 4Gb Fibre Channel. If simply using a RUT or number based on RPM can cause issues particular with 2.5 vs. 3.5 or enterprise and desktop. For example, some current generation 10K 2.5 HDD can deliver the same or better performance than an older generation 3.5 15K. Other drive factors (see this link for HDD fundamentals) including physical size such as 3.5 inch or 2.5 inch small form factor (SFF), enterprise or desktop or consumer, amount of drive level cache (DRAM). Space capacity of a drive can also have an impact such as if all or just a portion of a large or small capacity devices is used. Not to mention what the drive is attached to ranging from in internal SAS or SATA drive bay, USB port, or a HBA or RAID adapter card or in a storage system.
HDD fundamentals
How about benchmark and performance for marketing or comparison tricks including delayed, deferred or asynchronous writes vs. synchronous or actually committed data to devices? Lets not forget about short stroking (only using a portion of a drive for better IOP’s) or even long stroking (to get better bandwidth leveraging spiral transfers) among others.
Almost forgot, there are also thick, standard, thin and ultra thin drives in 2.5 and 3.5 inch form factors. What’s the difference? The number of platters and read write heads. Look at the following image showing various thickness 2.5 inch drives that have various numbers of platters to increase space capacity in a given density. Want to take a wild guess as to which one has the most space capacity in a given footprint? Also want to guess which type I use for removable disk based archives along with for onsite disk based backup targets (compliments my offsite cloud backups)?
Thick, thin and ultra thin devices
Beyond physical and configuration items, then there are logical configuration including the type of workload, large or small IOPS, random, sequential, reads, writes or mixed (various random, sequential, read, write, large and small IO). Other considerations include file system or raw device, number of workers or concurrent IO threads, size of the target storage space area to decide impact of any locality of reference or buffering. Some other items include how long the test or workload simulation ran for, was the device new or worn in before use among other items.
Then there are the various tools for generating IO’s or workloads along with recording metrics such as reads, writes, response time and other information. Some examples (mix of free or for fee) include Bonnie, Iometer, Iorate, IOzone, Vdbench, TPC, SPC, Microsoft ESRP, SPEC and netmist, Swifttest, Vmark, DVDstore and PCmark 7 among many others. Some are focused just on the storage system and IO path while others are application specific thus exercising servers, storage and IO paths.
Server, storage and IO performance toolbox
Having used Iometer since the late 90s, it has its place and is popular given its ease of use. Iometer is also long in the tooth and has its limits including not much if any new development, never the less, I have it in the toolbox. I also have Futremark PCmark 7 (full version) which turns out has some interesting abilities to do more than exercise an entire Windows PC. For example PCmark can use a secondary drive for doing IO to.
PCmark can be handy for spinning up with VMware (or other tools) lots of virtual Windows systems pointing to a NAS or other shared storage device doing real world type activity. Something that could be handy for testing or stressing virtual desktop infrastructures (VDI) along with other storage systems, servers and solutions. I also have Vdbench among others tools in the toolbox including Iorate which was used to drive the workloads shown below.
What I look for in a tool are how extensible are the scripting capabilities to define various workloads along with capabilities of the test engine. A nice GUI is handy which makes Iometer popular and yes there are script capabilities with Iometer. That is also where Iometer is long in the tooth compared to some of the newer generation of tools that have more emphasis on extensibility vs. ease of use interfaces. This also assumes knowing what workloads to generate vs. simply kicking off some IOPs using default settings to see what happens.
Another handy tool is for recording what’s going on with a running system including IO’s, reads, writes, bandwidth or transfers, random and sequential among other things. This is where when needed I turn to something like HiMon from HyperIO, if you have not tried it, get in touch with Tom West over at HyperIO and tell him StorageIO sent you to get a demo or trial. HiMon is what I used for doing start, stop and boot among other testing being able to see IO’s at the Windows file system level (or below) including very early in the boot or shutdown phase.
Here is a link to some other things I did awhile back with HiMon to profile some Windows and VDI activity test profiling.
The one that meets your needs, usually your applications or something as close as possible to it.
Various 2.5 and 3.5 inch HDD, HHDD, SSD with different performance
View additional NAS, NVMe, SSD, NVM, SCM, Data Infrastructure and HDD related topics via the following links.
Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.
That depends, however continue reading part II of this series to see some results for various types of drives and workloads.
Ok, nuff said, for now.
Gs
Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.
Part one of this two-part post provided a summary of today’s EMC (@EMCflash) announcement around XtremIO and renaming VFCache to XtremSF and associated software as XtremSW.
Synopsis of announcement
Now lets take a closer look at what was announced along with what it means in terms of Industry Trends and Perspectives.
XtremIO has been in customer beta for some time and now those along with some other early customers are able to acquire the product. In addition, EMC is opening up XtremIO to more prospective customers (Directed Availability) who have requirements or needs that line up with the products target market capabilities.
What this means is that XtremIO is not being simply put out into the general product population for broad distribution. Instead, it is being put into a controlled release (Directed Availability) to help customers, partners and EMC sales decide where best to use it and thus risk revenue prevention in other areas. The criteria or target opportunity (at least initially) are little-data applications including OLTP, server virtualization (where aggregation can cause aggravation) along with virtual desktop or VDI. In other words, many of the traditional or legacy IOP focused SSD opportunities.
In addition to XtremIO EMC has renamed their VFCache PCIe flash SSD cards (Launched February 2012) to XtremSF along with new models with both SLC and MLC nand flash. Also as part of today’s announcement EMC is renaming the cache software for XtremSF (e.g. VFCache) to be known as XtremSW. Now if that did not prompt the question of if you can now buy XtremSF as a target mode only card without the cache software the answer is yes.
What is XtremIO?
It is a new all flash SSD storage array. XtremIO is a Cluster, grid or collection of nodes called bricks with linear performance scaling providing block based all flash SSD storage. Data services consists of data footprint reduction (DFR) including inline global (across all nodes or bricks) dedupe on 4Kbyte chunks along with thin provisioning. Global dedupe is done on ingest using a combination of flash buffered meta-data (tables, index or dictionary) of what has been seen before along with multi-threaded software to leverage multi-core processors. Using the global dedupe at ingest; only new unique data is saved based on 4 Kbyte chunks.
Performance per EMC scales from one single node to more second node or a fourth node. Note: architecturally more nodes can be added with EMC indicating added models will be available in the future.
In addition to DFR, other data services including writable snapshots, and auto-load balancing when new bricks are added. Note that in a normal running XtremIO, data is automatically spread across the nodes for both performance and resiliency. Data only needs to be moved or load-balanced in the background when new bricks are added. Instant copy snapshots are supported along with writable snapshots. Currently replication is done via external EMC products such as VPLEX or RecoverPoint with statement of directions (SOD) for future enhancements.
Additional attributes of XtremIO include:
In addition to 4K being a commonly used or referred to IO size, it is also the same size as the new industry standard Advanced Format (AF). Today the standard storage block, page or sector size is 512 bytes however AF moves that to a larger 4,096 bytes (e.g. 4KB) to closer align with larger IO sizes. Note that many HDD’s and some SSD’s today support AF and provide 512 byte emulation modes for compatibility.
What is XtremSF?
VFCache is renamed XtremSF with new models using eMLC as companion to existing SLC PCIe cards and blade server mezzanine cards. EMC is emphasizing performance metrics that matter including IOPs that are relative to customer workloads such as 4K, 8K or larger with mix of reads and writes with low latency. In addition to IOPs with latency, size along with reads or writes for little data, EMC is also showing bandwidth or throughput numbers for big-data and big-bandwidth.
Model | Capacity | Read Transfer GB/sec | Write Transfer GB/sec | Random 4K Read (IOPS) | Random 4K Write (IOPS) | Random 4K Mixed ( IOPS) | Read latency (usec) | Write latency (usec) |
2200 (eMLC) | 2.2 TB | 2.47 | 1.1 | 343K | 105K | 206K | 87us | 30us |
700 (SLC) | 700 GB | 2.9 | 1.8 | 712K | 197K | 411K | 50us | 13us |
550 (eMLC) | 550 GB | 1.36 | 512 MB/s | 174K | 49K | 96K | 87us | 37us |
350 (SLC) | 350 GB | 2.9 | 756 MB/s | 715K | 95K | 267K | 50us | 13us |
Sampling of SLC and eMLC XtremSF PCIe SSD cards performance characteristics (via EMC) including latency measured in microseconds). Note performance differences due to some cards being based on SLC and others on eMLC.
Additional attributes, some new and some previously announced include:
What is XtremSW?
Server side flash software (not to be confused with FAST) for using XtremSF as a tier 0 (server-side) ssd cache or target. In target mode the XtremSF functions as a high performance persistent local dedicated direct attached storage (DAS) device. Cache mode enables frequently accessed data to be kept close to the applications off-loading underlying storage systems to be more effectively used. The XtremSW complements back-end storage systems for data protection and persistence along with investment protection of those assets.
What this all means
SSD is in your future, question is where, when and with what.
Why not just use SSD (DRAM and or nand flash) everywhere?
Keep in mind that in the data center (traditional, virtual or cloud) everything is not the same. Thus the simple answer is that there is not enough of it available at a low enough price point (think closer to Hard Disk Drives (HDD) costs) to fit into customers budget. Sure SSDs provide better performance and productivity benefits, however while there is no such thing as a data or information recession, there are budget constraints.
Another reason why SSD cant simply be used everywhere are physical (and logical) constraints such as amount of memory a server can directly access, or current DDR3 DIMMs (this could change with DDR4 according to Micron) can only address and work with DRAM, PCIe bus physical slot space, operating and hypervisor addressing limits among others.
If SSD (DRAM and or nand flash) were priced were priced low enough (e.g. much closer to HDDs) and available SSD including both DRAM and nand flash (SLC, MLC, eMLC, TLC, etc) along with emerging Phase Change Memory (PCM) are at the convergence of traditional memory and data storage. While some storage (or server) professionals may not agree, storage is an extension of memory and thus part of the traditional server and storage memory hierarchy shown below.
This brings up the locality of reference topic also shown in the following figure where the best IO is the one that does not have to be done. The second best is the one that can be done closest to application to a given level of service. Locality of reference which is important for servers and storage systems including caching refers to how close frequently accessed data is to where it is needed. For some applications this means as much DRAM main memory in a server as possible either clustered, with battery backup or other data persistency protection including onboard HDD or SSD (e.g. towards the top of the hierarchy).
There are other applications where localized SSD (DRAM or nand flash) are a benefit to compliment main memory or as a persistent cache and target such as PCIe cards or SAS and SATA drives. Further down the stack and for housing larger amounts of storage with performance (reads or writes, random or sequential) along with data services is where all SSD and hybrid (mix of SSD and HDD) fit. Even further down the stack and for a broader segment is where cloud storage services based on SSD such as those from Rackspace (Cloud Block Storage with SSD) and Amazon (provisioned IOPS for EBS) have a play. Lets not forget about SSD in laptop, tablets and workstations, for example I have a Samsung model 830 in my Lenvo X1.
Some general industry trends include:
Related items about nand flash, SSD and metrics related themes:
Some additional thoughts and perspectives
Does this mean traditional storage arrays are now dead?
IMHO, no, there will be some cannibalization of existing storage systems by XtremIO within EMC customers or prospects if not managed, as well as via those from others. Keep in mind that recently EMC announced enhancements to their VMAX including entry-level options for service providers. Some new opportunities opened up will be where traditional all SSD (flash or dram) systems have historically had success.
Traditional SSD and new dedicated SSD systems include Texas Memory Systems (TMS) bought by IBM in 2012, and the recently announced NetApp EF540 (and future FlashRay) along with startups Solidfire, Violin, Whiptail among others. There will be environments where XtremIO may take care of all storage needs for a customer or specific application or piece of it. Then there will be other situations where XtremIO will go-exist with EMC or other vendor’s storage solutions as part of a data infrastructure.
Who will EMC be competing against with XtremIO?
Certainly the startups or smaller players such as Violin, Whiptail, Purestorage, Solidfire along with IBM/TMS and NetApp EF540 (eventually FlashRay as well) among others.
There will also be some competition with other hybrid storage array vendors that have a mix of HDD and SSD. XtremIO will also compete in some situations on its own vs. other PCIe flash target and cache cards such as FusionIO, however for the most part those will up against XtremSF and XtremSW.
Why the slow or “Directed Availability” rollout?
Why not? By taking a controlled rollout selecting and qualifying customers for XtremIO, EMC gets to manage how the product goes out into production and control how it is used to increase chances of success. Unlike a startup that would be forced to try to put their new technology anywhere, EMC has the luxury of selecting where it goes, not to mention needing to avoid introducing a revenue prevention play for its other products.
Overall, I give an Atta boy and Atta girl to the EMC crew for a Product Defined Announcement (PDA) extending their flash portfolio to complement their different customers and prospects various environment needs. Now watch EMC, NetApp and others step up their flash dance moves to see who will out flash the others in the eXtreme flash games, not to mention emerging software defined marketing moves (SDMM) ;) .
Ok, nuff said.
Cheers Gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
EMC (@EMCflash) today announced some new, enhanced, renamed and a rebrand flash solid-state device (SSD) storage portfolio around theme of XtremIO. XtremIO was the startup company with a new all flash SSD storage array that EMC announced they were buying in May 2012. Since that announcement, Project “X” has been used when referring to the product now known as XtremIO (e.g. all flash new storage array).
Synopsis of announcement
What was previously announced:
Overall, I give an Atta boy and Atta girl to the EMC crew for a Product Defined Announcement (PDA) extending their flash portfolio to complement their different customers and prospects various environment needs. Now let us sit back and watch EMC, NetApp and others step up their flash dance moves to see who will out flash the others in the eXtreme flash games, including software defined storage, software defined data centers, software defined flash, and software defined cache.
Related items about nand flash and metrics related themes:
Read more about XtremIO, XtremSF, XtremSW and flash related items here in part II of this post.
Ok, nuff said (for now).
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
Eric Siebert (@Ericsiebert) author of the book Maximum vSphere (get your copy on Amazon.com here) has opened up voting for the annual top vBlog over at his site (vSphere-land).
While there is a focus on VMware and Virtualization blogs, there are also other categories such as Storage, Scripting, pod casting as well as independent for the non vendors and VARs.
It is an honor to be included in the polling along with my many 2012 fellow vExperts on the list.
Last year I made Eric’s 2012 top 50 list as well as appearing in the storage and some other categories in those rankings (thanks to all who voted last year).
This year I forgot to nominate myself (it’s a self nomination process) so while I am not on the storage, independent bloggers, pod cast sub-categories, I am however included in the general voting having made the top 50 list last year (#46).
A summary of Eric’s recommended voting criteria vs. basic popularity are:
Voting is now open (click here on the vote image) and closes on March 1, 2013 so if you read this or any of my other posts, comments and content or listen to our new pod casts at storageio.tv (also on iTunes).
Thank you in advance for your continued support and watch for more posts, comments, perspectives and pod casts about data and information infrastructure topics, trends, tools and techniques including servers, storage, IO networking, cloud, virtualization, backup/recovery, BC, DR and data protection along with big and little data (among other things).
Ok, nuff said.
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
