Tag: Data protection

Posted on

Welcome to the Data Protection Diaries

Updated 1/10/2018

Storage I/O trends

Welcome to the Data Protection Diaries

This is a series of posts about data protection which includes security (logical and physical), backup/restore, business continuance (BC), disaster recovery (DR), business resiliency (BR) along with high availability (HA), archiving and related topic themes, technologies and trends.

Think of data protection like protect, preserve and serve information across cloud, virtual and physical environments spanning traditional servers, storage I/O networking along with mobile (ok, some IoT as well), SOHO/SMB to enterprise.

Getting started, taking a step back

Recently I have done a series of webinars and Google+ hangouts as part of the BackupU initiative brought to you by Dell Software (that’s a disclosure btw ;) ) that are vendor and technology neutral. Instead of the usual vendor product or technology focused seminars and events, these are about getting back to the roots, the fundamentals of what to protect when and why, then decide your options as well as different approaches (e.g. what tools to use when).

In addition over the past year (ok, years) I have also been doing other data protection related events, seminars, workshops, articles, tips, posts across cloud, virtual and physical from SOHO/SMB to enterprise. These are in addition to the other data infrastructure server and storage I/O stuff (e.g. SSD, object storage, software defined, big data, little data, buzzword bingo and others).

Keep in mind that in the data center or information factory everything is not the same as there are different applications, threat risk scenarios, availability and durability among other considerations. In this series like the cloud conversations among others, I’m going to be pulling various data protection themes together hopefully to make it easier for others to find, as well as where I know where to get them.

data protection diaries
Some notes for an upcoming post in this series using my Livescribe about data protection

Data protection topics, trends, technologies and related themes

Here are some more posts to checkout pertaining to data protection trends, technologies and perspectives:

Ok, nuff said (for now)

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

Posted on

IT and data center sustainability, the other convergence zone

Storage I/O trends

IT and data center sustainability convergence

Recently Hailey Lynne McKeefry (@HaileyMcK), Editor in Chief over at Data Center Acceleration (@DataAccelerate) reached out for a conversation about well, data center themes and topics. Given Hailey’s background in covering technology as well as business supply chain we somehow ended up talking about business, IT and data center sustainability. Hailey wrote a piece about Driving for Datacenter Sustainability and in addition I was honored to be an invited guest for a live on-line chat yesterday (you can view the conversation here).

Excerpt from Haileys piece:

Too often, sustainability efforts in the datacenter are written off as feel-good, public relations efforts. In reality, green is about economics — and done well, it can save the datacenter tons of cash.

"You mention green, and datacenter managers run or cringe and roll their eyes, because there’s been so much green washing done in the past few years," said Greg Schulz, founder of IT consultancy StorageIO. "It’s really about green economics, though, and getting more work done with the same budget."

Read more of Hailey’s piece here

Many different faces of IT and data center sustainability

Granted, when you here the term sustainability, IT and data centers you may think of different things depending on your view or area of focus.

For some it will be Green or environmental focused such as use of renewable and EH&S themes, recycling among others Related to the previous item some will see sustainability as being tied to energy, either tied to cost, availability/accessibility, standby or alternative and renewable Yet for others, it will mean business continuance (BC), disaster recovery (DR), business resiliency (BR), high availability or reliability availability service (RAS) among others Then the economics concerns of keeping the business running to discuss top and bottom line concerns.

Otoh, if your focus is on one of the above or a subset of one of them, you might not view the other areas as being tied to sustainability.

It data center sustainability

Likewise, you might even want to not be included in another other, let alone share your area with others. For example if your focus is on security you may not want to see or hear that data protection is part of sustainability, not to mention backup/restore, bc, dr and so forth.

Learning, education and knowledge sustainability

Part of sustainability is also continuing to learn about new things not only in your field or focus area, also in adjacent spaces.

Keep in mind that there is more of a data center or information factory than just a building or facility with power, cooling as there are the technologies, tools, people, process, delivery/distribution network, warehouse for storing raw and finished material, metrics and management that all go into delivering the product which is information services.

Hence there are many aspects to IT and data center sustainability and thus think more pragmatically about sustaining information factories, however lets also be realistic and not jump the shark by declaring everything as sustainable ;).

Check out the live talk chat that we had yesterday over at Data Center Acceleration by clicking here.

Some related more reading:
Green IT, Green Gap, Tiered Energy and Green Myths
The new Green IT: Efficient, Effective, Smart and Productive
Saving Money with Green IT: Time To Invest In Information Factories
PUE, Are you Managing Power, Energy or Productivity?
Green IT deferral blamed on economic recession might be result of green gap
IT and storage economics 101, supply and demand
The Green and Virtual Data Center (CRC Press) – Intel Recommended Reading List
Driving for Datacenter Sustainability
Live Chat 01/23: Building the Sustainable Datacenter

Ok, nuff said

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2014

Posted on

Securing your information assets and data, what about your storage?

Storage I/O trends

Securing your information assets and data, what about your storage?

Recently I did a piece over at the site Information Security Buzz title How Secure Is Your Data Storage? that takes a cursory look at securing your digital assets from a storage perspective. Keep in mind that data protection can mean many things to different people from various focus or technology domain perspectives. Likewise there are various threat risks to protect against and, not all of them are head-line news making events.

data protection threat risk scenarios

Protecting data and data protection

Protecting your data or data protection is a diverse topic and not exclusive to just backup/restore, business continuance (BC), disaster recovery (DR), high availability (HA), durability, archiving, privacy and compliance (PCI, Hippa, High-tech, Sarbox, etc) or security (logical [encryption, access control, identity management] and physical).

In the broader scope and context of information infrastructures and data infrastructures, think of data protection as part of or enabling protect, process, preserve and serving of information in an effective way that does not introduce complexity or compromise your digital and physical assets.

Following is an excerpt from the piece over at Information Security Buzz:

The usual belief is that information behind firewall’s and on storage attached to servers that have rights access control and find access, all is safe; hence no need to encrypt the real storage device.

There is a couple of other usual comments or statements that people make to me about encrypting storage devices that it is too difficult due to lack of good key management, and the other is that people say the encryption algorithms are no good. Both can be valid points, particular given what we are hearing with the NSA and other government activities. My usual response is a) have spare keys placed in safe trusted locations and b) do you lock the doors and windows on your home as somebody who really wants to get in probably can, hence need for multiple rings of security, however the encryption will deter the casual or more typical adversary.

Click to read more

Additional data protection topics and links

In addition to the above, also check out the following related items on the many difference faces or facets of data protection.

Various StorageIO tips and articles from different venues: Via StateTech Magazine – 5 Tips for Factoring Software into Disaster Recovery Plans and Via the StorageIO fall November 2013 news letter, Cloud and data protection perspectives.

Also via StorageIOblog: Data protection modernization, more than swapping out media and Cloud conversations: Has Nirvanix shutdown caused cloud confidence and data protection concerns? along with In the data center or information factory not everything is the same plus Securing data at rest and fast secure erase with SED’s.

Also check out BackupU (www.software.dell.com/backupU) series of webinars and Google+ hangouts that I’m involved with about modernizing and rethinking data protection. Note that while Dell is the sponsor of these events, they are also vendor and technology neutral, that’s a disclosure btw fwiw ;) ).

Closing perspective, for now…


Only you can prevent data loss as it is a shared responsibility!

Ok, nuff said (for now)

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-201

Posted on

Goodbye 2013, hello 2014, predictions past, present and future

Storage I/O trends

Good by 2013 and hello 2014 along with predictions past, present and future

First, for those who may have missed this, thanks to all who helped make 2013 a great year!

2013 season greetings

Looking back at 2013 I saw a continued trend of more vendors and their media public relations (PR) people reaching out to have their predictions placed in articles, posts, columns or trends perspectives pieces.

Hmm, maybe a new trend is predictions selfies? ;)

Not to worry, this is not a wrapper piece for a bunch of those pitched and placed predictions requests that I received in 2013 as those have been saved for a rainy or dull day when we need to have some fun ;) .

What about 2013 server storage I/O networking, cloud, virtual and physical?

2013 end up with some end of year spree’s including Avago acquiring storage I/O and networking vendor LSI for about $6.6B USD (e.g. SSD cards, RAID cards, cache cards, HBA’s (Host Bus Adapters), chips and other items) along with Seagate buying Xyratex for about $374M USD (a Seagate suppliers and a customer partner).

Xyratex is known by some for making the storage enclosures that house hard disk drive (HDD’s) and Solid State Device (SSD) drives that are used by many well-known, and some not so well-known systems and solution vendors. Xyratex also has other pieces of their business such as appliances that combine their storage enclosures for HDD and SSD’s along with server boards, along with a software group focus on High Performance Compute (HPC) Lustre. There is another part of the Xyratex business that is not as well-known which is the test equipment used by disk drive manufacturers such as Seagate as part of their manufacturing process. Thus the Seagate acquisition moves them up market with more integrated solutions to offer to their (e.g. Seagate and Xyratex) joint customers, as well as streamline their own supply chain and costs (not to mention sell equipment to the other remaining drive manufactures WD and Toshiba).

Storage I/O trends

Other 2013 acquisitions included (Whiptail by Cisco, Virident by WD (who also bought several other companies), Softlayer by IBM) along with various mergers, company launches, company shutdowns (cloud storage Nirvanix and SSD maker OCZ bankruptcy filing), and IPO’s (some did well like Nimble while Violin not so well), while earlier high-flying industry darlings such as FusionIO are now the high-flung darling targets of the shareholder sock lawsuit attorneys.

2013 also saw the end of SNW (Storage Network World), jointly produced by SNIA and Computerworld Storage in the US after more than a decade. Some perspectives from the last US SNW held October 2013 can be found in the Fall 2013 StorageIO Update Newsletter here, granted those were before the event was formal announced as being terminated.

Speaking of events, check out the November 2013 StorageIO Update Newsletter here for perspectives from attending the Amazon Web Services (AWS) re:Invent conference which joins VMworld, EMCworld and a bunch of other vendor world events.

Lets also not forget Dell buying itself in 2013.

StorageIO in the news

Click on the following links read (and here) more about various 2013 industry perspectives trends commentary of mine in various venues, along with tips, articles, newsletters, events, pod cast, videos and other items.

What about 2014?

Perhaps 2014 will build on the 2013 momentum of the annual rights of pages refereed to as making meaningless future year trends and predictions as being passe?

Not that there is anything wrong with making predictions for the coming year, particular if they actually have some relevance, practicality not to mention track record.

However that past few years seems to have resulted in press releases along with product (or services) plugs being masked as predictions, or simply making the same predictions for the coming year that did not come to be for the earlier year (or the one before that or before that and so forth).

On the other hand, from an entertainment perspective, perhaps that’s where we will see annual predictions finally get classified and put into perspectives as being just that.

Storage I/O trends

Now for those who still cling to as well as look forward to annual predictions, ok, simple, we will continue in 2014 (and beyond) from where we left off in 2013 (and 2012 and earlier) meaning more (or continued):

  • Software defined "x" (replace "x" with your favorite topic) industry discussion adoption yet customer adoption or deployment question conversations.

  • Cloud conversations shifted from lets all go to the cloud as the new shiny technology to questioning the security, privacy, stability, vendor or service viability not to mention other common sense concerns that should have been discussed or looked into earlier. I have also heard from people who say Amazon (as well as Verizon, Microsoft, Blue host, Google, Nirvanix, Yahoo and the list goes on) outages are bad for the image of clouds as they shake people’s confidences. IMHO people confidence needs to be shaken to that of having some common sense around clouds including don’t be scared, be ready, do your homework and basic due diligence. This means cloud conversations over concerns set the stage for increased awareness into decision-making, usage, deployment and best practices (all of which are good things for continued cloud deployments). However if some vendors or pundits feel that people having basic cloud concerns that can be addressed is not good for their products or services, I would like to talk with them because they may be missing an opportunity to create long-term confidence with their customers or prospects.

  • VDI as a technology being deployed continues to grow (e.g. customer adoption) while the industry adoption (buzz or what’s being talked about) has slowed a bit which makes sense as vendors jump from one bandwagon to the new software defined bandwagon.

  • Continued awareness around modernizing data protection including backup/restore, business continuance (BC), disaster recovery (DR), high availability, archiving and security means more than simply swapping out old technology for new, yet using it in old ways. After all, in the data center and information factory not everything is the same. Speaking of data protection, check out the series of technology neutral webcast and video chats that started last fall as part of BackupU brought to you by Dell. Even though Dell is the sponsor of the series (that’s a disclosure btw ;) ) the focus of the sessions is on how to use different tools, technologies and techniques in new ways as well as having the right tools for different tasks. Check out the information as well as register to get a free Data Protection chapter download from my book Cloud and Virtual Data Storage Networking (CRC Press) at the BackupU site as well as attend upcoming events.

  • The nand flash solid state devices (SSD) cash-dash (and shakeout) continues with some acquisitions and IPO’s, as well as disappearances of some weaker vendors, while appearance of some new. SSD is showing that it is real in several ways (despite myths, fud and hype some of which gets clarified here) ranging from some past IPO vendors (e.g. FusiuonIO) seeing exit of their CEO and founders while their stock plummets and arrival of shareholder investor lawsuits, to Violins ho-hum IPO. What this means is that the market is real, it has a very bright future, however there is also a correction occurring showing that reality may be settling in for the long run (e.g. next couple of decades) vs. SSD being in the realm of unicorns.

    • Storage I/O trends

  • Internet of Things (IoT) and Internet of Devices (IoD) may give some relief for Big Data, BYOD, VDI, Software Defined and Cloud among others that need a rest after they busy usage that past few years. On the other hand, expect enhanced use of earlier buzzwords combined with IoT and IOD. Of course that also means plenty of questions around what is and is not IoD along with IoT and if actually relevant to what you are doing.

  • Also in 2014 some will discover storage and related data infrastructure topics or some new product / service thus having a revolutionary experience that storage is now exciting while others will have a DejaVu moment that it has been exciting for the past several years if not decades.

  • More big data buzz as well as realization by some that a pragmatic approach opens up a bigger broader market, not to mention customers more likely to realize they have more in common with big data than it simply being something new forcing them to move cautiously.

  • To say that OpenStack and related technologies will continue to gain both industry and customer adoption (and deployment) status building off of 2013 in 2014 would be an understatement, not to mention too easy to say, or leave out.

  • While SSD’s continue to gain in deployment, after the question is not if, rather when, where, with what and how much nand flash SSD is in your future, HDD’s continue to evolve for physical, virtual and cloud environments. This also includes Seagate announcing a new (Kinetic) Ethernet attached HDD (note that this is not a NAS or iSCSI device) that uses a new key value object storage API for storing content data (more on this in 2014).

  • This also means realizing that large amounts of little data can result in back logs of lots of big data, and that big data is growing into very fast big data, not to mention realization by some that HDFS is just another distributed file system that happens to work with Hadoop.

  • SOHO’s and lower end of SMB begin to get more respect (and not just during the week of Consumer Electronic Show – CES).

  • Realization that there is a difference between Industry Adoption and Customer Deployment, not to mention industry buzz and traction vs. customer adoption.

server storage I/O trends

What about beyond 2014?

That’s easy, many of the predictions and prophecies that you hear about for the coming year have also been pitched in prior years, so it only makes sense that some of those will be part of the future.

  • If you have seen or experienced something you are more likely to have DejaVu.
  • Otoh if you have not seen or experienced something you are more likely to have a new and revolutionary moment!
  • Start using new (and old) things in new ways vs. simply using new things in old ways.
  • Barrier to technology convergence, not to mention new technology adoption is often people or their organizations.
  • Convergence is still around, cloud conversations around concerns get addressed leading to continued confidence for some.
  • Realization that data infrastructure span servers, storage I/O networking, cloud, virtual, physical, hardware, software and services.
  • That you can not have software defined without hardware and hardware defined requires software.
  • And it is time for me to get a new book project (or two) completed in addition to helping others with what they are working on, more on this in the months to come…

Here’s my point

The late Jim Morrison of the Doors said "There are things known and things unknown and in between are the doors.".

The doors via Amazon.com
Above image and link via Amazon.com

Hence there is what we know about 2013 or will learn about the past in the future, then there is what will be in 2014 as well as beyond, hence lets step through some doors and see what will be. This means learn and leverage lessons from the past to avoid making the same or similar mistakes in the future, however doing so while looking forward without a death grip clinging to the past.

Needless to say there will be more to review, preview and discuss throughout the coming year and beyond as we go from what is unknown through doors and learn about the known.

Thanks to all who made 2013 a great year, best wishes to all, look forward to seeing and hearing from you in 2014!

Ok, nuff said (for now)

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

Posted on

November 2013 Server and StorageIO Update Newsletter & AWS reinvent info


November 2013 Server and StorageIO Update Newsletter & AWS reinvent info

Welcome to the November 2013 edition of the StorageIO Update (newsletter) containing trends perspectives on cloud, virtualization and data infrastructure topics. Fall (here in North America) has been busy with in-person, on-line live and virtual events along with various client projects, research, time in the StorageIO cloud, virtual and physical lab test driving, validating and doing proof of concept research among other tasks. Check out the industry trends perspectives articles, comments and blog posts below that covers some activity over the past month.

Last week I had the chance to attend the second annual AWS re:Invent event in Las Vegas, see my comments, perspectives along with a summary of announcements from that conference below.

Watch for future posts, commentary, perspectives and other information down the road (and in the not so distant future) pertaining to information and data infrastructure topics, themes and trends across cloud, virtual, legacy server, storage, networking, hardware and software. Also check out our backup, restore, BC, DR and archiving (Under the resources section on StorageIO.com) for various presentation, book chapter downloads and other content.

Enjoy this edition of the StorageIO Update newsletter.

Ok, nuff said (for now)

Cheers gs

StorageIO Industry Trends and Perspectives

Industry trends: Amazon Web Services (AWS) re:Invent

Last week I attended the AWS re:Invent event in Las Vegas. This was the second annual AWS re:Invent conference which while having an AWS and cloud theme, it is also what I would describe as a data infrastructure event.

As a data infrastructure event AWS re:Invent spans traditional legacy IT and applications to newly invented, re-written, re-hosted or re-platformed ones from existing and new organizations. By this I mean a mix of traditional IT or enterprise people as well as cloud and virtual geek types (said with affection and all due respect of course) across server (operating system, software and tools), storage (primary, secondary, archive and tools), networking, security, development tools, applications and architecture.

That also means management from application and data protection spanning High Availability (HA), Business Continuance (BC), Disaster Recovery (DR), backup/restore, archiving, security, performance and capacity planning, service management among other related themes across public, private, hybrid and community cloud environments or paradigms. Hmm, I think I know of a book that covers the above and other related topic themes, trends, technologies and best practices called Cloud and Virtual Data Storage Networking (CRC Press) available via Amazon.com in print and Kindle (among other) versions.

During the event AWS announced enhanced and new services including:

  • WorkSpaces (Virtual Desktop Infrastructure – VDI) announced as a new service for cloud based desktops across various client devices including laptops, Kindle Fire, iPad and Android tablets using PCoIP.
  • Kinesis which is a managed service for real-time processing of streaming (e.g. Big) data at scale including ability to collect and process hundreds of GBytes of data per second across hundreds of thousands of data sources. On top of Kinesis you can build your big data applications or conduct analysis to give real-time key performance indicator dashboards, exception and alarm or event notification and other informed decision-making activity.
  • EC2 C3 instances provide Intel Xeon E5 processors and Solid State Device (SSD) based direct attached storage (DAS) like functionality vs. EBS provisioned IOPs for cost-effective storage I/O performance and compute capabilities.
  • Another EC2 enhancement are G2 instance that leverage high performance NVIDIA GRID GPU with 1,536 parallel processing cores. This new instance is well suited for 3D graphics, rendering, streaming video and other related applications that need large-scale parallel or high performance compute (HPC) also known as high productivity compute.
  • Redshift (cloud data warehouse) now supports cross region snapshots for HA, BC and DR purposes.
  • CloudTrail records AWS API calls made via the management console for analytics and logging of API activity.
  • Beta of Trusted Advisor dashboard with cost optimization saving estimates including EBS and provisioned IOPs
  • Relational Database Service (RDS) support for PostgresSQL including multi-AZ deployment.
  • Ability to discover and launch various software from AWS Marketplace via the EC2 Console. The AWS Marketplace for those not familiar with it is a catalog of various software or application titles (over 800 products across 24 categories) including free and commercial licensed solutions that include SAP, Citrix, Lotus Notes/Domino among many others.
  • AppStream is a low latency (STX protocol based) service for streaming resource (e.g. compute, storage or memory) intensive applications and games from AWS cloud to various clients, desktops or mobile devices. This means that the resource intensive functionality can be shifted to the cloud, while providing a low latency (e.g. fast) user experience off-loading the client from having to support increased compute, memory or storage capabilities. Key to AppStream is the ability to stream data in a low-latency manner including over networks normally not suited for high quality or bandwidth intensive applications. IMHO AppStream while focused initially on mobile app’s and gaming, being a bit streaming technology has the potential to be used for other similar functions that can leverage download speed improvements.

    • When I asked an AWS person if or what role AppStream might have or related to WorkSpaces their only response was a large smile and no comment. Does this mean WorkSpaces leverages AppStream? Candidly I don’t know, however if you look deeper into AppStream and expand your horizons, see what you can think up in terms of innovation. Updated 11/21/13 AWS has provided clarification that WorkSpaces is based on PCoIP while AppStream uses the STX protocols.

    Check out AWS Sr. VP Andy Jassy keynote presentation here.

Overall I found the AWS re:Invent event to be a good conference spanning many aspects and areas of focus which means I will be putting it on my must attend list for 2014.

StorageIO Industry Trends and PerspectivesIndustry trends tips, commentary, articles and blog posts
What is being seen, heard and talked about while out and about

The following is a synopsis of some StorageIOblog posts, articles and comments in different venues on various industry trends, perspectives and related themes about clouds, virtualization, data and storage infrastructure topics among related themes.

Storage I/O posts

Recent industry trends, perspectives and commentary by StorageIO Greg Schulz in various venues:

NetworkComputing: Comments on Software-Defined Storage Startups Win Funding

Digistor: Comments on SSD and flash storage
InfoStor: Comments on data backup and virtualization software

ITbusinessEdge: Comments on flash SSD and hybrid storage environments

NetworkComputing: Comments on Hybrid Storage Startup Nimble Storage Files For IPO

InfoStor: Comments on EMC’s Light to Speed: Flash, VNX, and Software-Defined

InfoStor: Data Backup Virtualization Software: Four Solutions

ODSI: Q&A With Greg Schulz – A Quick Roundup of Data Storage Industry

Recent StorageIO Tips and Articles in various venues:

FedTechMagazine: 3 Tips for Maximizing Tiered Hypervisors
InfoStor:
RAID Remains Relevant, Really!

Storage I/O trends

Recent StorageIO blog post:

EMC announces XtremIO General Availability (Part I) – Announcement analysis of the all flash SSD storage system
Part II: EMC announces XtremIO General Availability, speeds and feeds – Part two of two part series with analysis
What does gaining industry traction or adoption mean too you? – There is a difference between buzz and deployment
Fall 2013 (September and October) StorageIO Update Newsletter – In case you missed the fall edition, here it is

StorageIO Industry Trends and Perspectives

Check out our objectstoragecenter.com page where you will find a growing collection of information and links on cloud and object storage themes, technologies and trends.

Server and StorageIO seminars, conferences, web cats, events, activities StorageIO activities (out and about)

Seminars, symposium, conferences, webinars
Live in person and recorded recent and upcoming events

While 2013 is winding down, the StorageIO calendar continues to evolve, here are some recent and upcoming activities.

December 11, 2013 Backup.UData Protection for Cloud 201Backup.U
Google+ hangout
December 3, 2013 Backup.UData Protection for Cloud 101Backup.U
Online Webinar
November 19, 2013 Backup.UData Protection for Virtualization 201Backup.U
Google+ hangout
November 12-13, 2013AWS re:InventAWS re:Invent eventLas Vegas, NV
November 5, 2013 Backup.UData Protection for Virtualization 101Backup.U
Online Webinar
October 22, 2013 Backup.UData Protection for Applications 201Backup.U
Google+ hangout

Click here to view other upcoming along with earlier event activities. Watch for more 2013 events to be added soon to the StorageIO events calendar page. Topics include data protection modernization (backup/restore, HA, BC, DR, archive), data footprint reduction (archive, compression, dedupe), storage optimization, SSD, object storage, server and storage virtualization, big data, little data, cloud and object storage, performance and management trends among others.

Vendors, VAR’s and event organizers, give us a call or send an email to discuss having us involved in your upcoming pod cast, web cast, virtual seminar, conference or other events.

If you missed the Fall (September and October) 2013 StorageIO update newsletter, click here to view that and other previous editions as HTML or PDF versions. Subscribe to this newsletter (and pass it along)

and click here to subscribe to this news letter. View archives of past StorageIO update news letters as well as download PDF versions at: www.storageio.com/newsletter

Ok, nuff said (for now).
Cheers Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved    

Posted on

Fall 2013 StorageIO Update Newsletter

Storage I/O trends

Fall 2013 StorageIO Update Newsletter

Welcome to the Fall 2013 (joint September and October) edition of the StorageIO Update (newsletter) containing trends perspectives on cloud, virtualization and data infrastructure topics. It is fall (at least here in north America) which means conferences, symposium, virtual and physical events, seminars, webinars in addition to normal client project activities. Starting with VMworld back in late August, that event occurred in San Francisco which kicked off the fall (or back to school) season of activity. VMworld was followed with many other events including in-person along with virtual or on-line such as webinars, Google+ hangouts among others, not to mention all the briefings for vendor product announcements and updates. Check out the industry trends perspectives articles, comments and blog posts below that covers some activity over the past few months.

VMworld 2013
Congratulations to VMworld on the 10th anniversary of the event. With the largest installment yet of a VMworld in terms of attendance, there were also many announcements. Here are a synopsis of some of those announcements which of course included plenty of software defined marketing (SDM).

CMG and Storage Performance
During mid-September I was invited to give an industry trends and perspectives presentation to the Storage Performance Council (SPC) board. The SPC board were meeting in the Minneapolis area and I gave a brief talk about Metrics that Matter and importance of context with focus on applications. Speaking of the Minneapolis area, Tom Becchetti (@tbecchetti) organized a great CMG event hosted over at Blue Cross Blue Shield of Minnesota. I gave a discussion around Technolutionary, technology evolution and revolution, using old and new things in new ways.

Check out our backup, restore, BC, DR and archiving (Under the resources section on StorageIO.com) for various presentation, book chapter downloads and other content.

SNW Fall 2013 Long Beach
Talking about traveling, there was a quick trip out to Long Beach for the fall 2013 edition of Storage Networking World (SNW) where I had some good meetings and conversations with those who were actually there. No need to sugar coat it, likewise no need to kick sand in its face. Plain and simple, SNW is not the event it used to be has been a common discussion theme for several years which I had set my expectation accordingly.

Some have asked me why I even spent time, money and resources to attend SNW?

My answer is that I had some meetings to attend to, wanted to see and meet with others who were going to be there, and perhaps even say goodbye to an event that I have been involved with for over a decade.

Does that mean I’m all done with SNW?

Not sure yet as will have to wait and see what SNIA and IDG/Computerworld the event co-owners and producers put together for future events. However there are enough other events and activities to pick up the slack which is part of what has caused the steady decline in events like SNW among others.

Perhaps it is time for SNIA to partner with another adjacent yet like-minded organization such as CMG to collaborate and try doing something like what was done in the early 2000s? That is SNIA providing their own seminars along with others such as myself who involved with both CMG, SNW and SNIA to beef up or set up a storage and I/O focused track at the CMG event.

Beyond those items mentioned above, or in the following section, there are plenty of interesting and exciting things occurring in the background that I cant talk about yet. However watch for future posts, commentary, perspectives and other information down the road (and in the not so distant future).

Enjoy this edition of the StorageIO Update newsletter.

Ok, nuff said (for now)

Cheers gs

StorageIO Industry Trends and PerspectivesIndustry trends perspectives and commentary
What is being seen, heard and talked about while out and about

The following is a synopsis of some StorageIOblog posts, articles and comments in different venues on various industry trends, perspectives and related themes about clouds, virtualization, data and storage infrastructure topics among related themes.

Storage I/O trends

InfoStor: Perspectives on Data Dynamics file migration tool (Read more about StorageX later in this newsletter)
SearchStorage: Perspectives on Data Dynamics resurrects StorageX for file migration
SearchStorage: Perspectives on Cisco buying SSD storage vendor Whiptail

Recent StorageIO Tips and Articles in various venues:

21cIT:  Why You Should Consider Object Storage
InfoStor:  HDDs Are Still Spinning (Rust Never Sleeps)
21cIT:  Object Storage Is in Your Future, Even if You Use Files
21cIT:  Playing the Name Game With Virtual Storage
InfoStor:  Flash Data Storage: Myth vs. Reality
InfoStor:  The Nand Flash Cache SSD Cash Dance
SearchEnterpriseWAN:  Remote Office / ROBO backup and data protection for networking Pro’s
TheVirtualizationPractice:  When and Where to use NAND Flash SSD for Virtual Servers
FedTech:  These Data Center (DCIM) Tools Can Streamline Computing Resources

Storage I/O posts

Recent StorageIO blog post:

Seagate Kinetic Cloud and Object Storage I/O platform (and Ethernet HDD)
Cloud conversations: Has Nirvanix shutdown caused cloud confidence concerns?
Cisco buys Whiptail continuing the SSD storage I/O flash cash cache dash
WD buys nand flash SSD storage I/O cache vendor Virident
EMC New VNX MCx doing more storage I/O work vs. just being more
Is more of something always better? Depends on what you are doing
VMworld 2013 Vmware, server, storage I/O and networking update (Day 1)
EMC ViPR software defined object storage part II

Check out our objectstoragecenter.com page where you will find a growing collection of information and links pertaining to cloud and object storage themes, technologies and trends.

Brouwer Storage Consultancy

StorageIO in Europe (Netherlands)
Spent over a week in the Netherlands where I presented three different seminar workshop sessions organized by Brouwer Storage Consultancy who is celebrating their 10th anniversary in business. These sessions spanned five full days of interactive discussions with an engaged diverse group of attendees in the Nijkerk area who came from across Holland to take part in these workshops.

Congratulations to Gert and Frank Brouwer on their ten years of being in business and best wishes for many more. Fwiw those who are curious StorageIO will be ten years young in business in about two years.

StorageIO Industry Trends and Perspectives

Some observations from while in Europe:

Continued cloud privacy concerns amplified by NSA and suspicion of US-based companies, yet many are not aware of similar concerns of European or UK-based firms from those outside those areas. While there were some cloud concern conversations over the demise of Nirvanix, those seemed less so then in the media or US given that at least in Holland they have seen other cloud and storage as a service firms come and go already. It should be noted that the US has also seen cloud and storage as a service startups come and go, however I think sometimes we or at least the media tends to have a short if not selective memory at times.

In one of our workshops sessions we were talking about service level objectives (SLO), service level agreements (SLA), recovery point objectives (RPO) and recovery time objectives (RTO) among other themes. Somebody mentioned why the focus of time in RPO and questions why not a transactional perspective which I thought was a brilliant question. We had a good conversation in the group and concurred that while RPO is what the industry uses, that there also needs to be a transactional state context tie to what is inferred or assumed with RPO and RTO. Thus the importance of looking beyond just the point in time, however the importance of a transactional context or state, such as not just the time, however to a given transactional point.

Note that transactional could mean database, file system, backup or data protection index or catalog, meta data repository or other entity. This is where some should be jumping up and down like Donkey in Shrek wanting to point out that is exactly what RTO and RPO refer to which would be great. However all to often what is assumed is not conveyed, thus those who don’t know, well, they assume or simply don’t know what others.

StorageIO Industry Trends and Perspectives

Data Dynamics StorageX 7.0 Intelligent Policy Based File Data Migration – There is no such thing as a data or information recession . Likewise, people and data are living longer as well as getting larger. These span various use cases from traditional to personal or at work productivity. From little to big data content, collaboration including file or document sharing to rich media applications all of which are leveraging unstructured data. For example, email, word processing back-office documents, web and text files, presentations (e.g. PowerPoint), photos, audio and video among others. These macro trends result in the continued growth of unstructured Network Attached Storage (NAS) file data.

Thus, a common theme is adding management including automated data movement and migration to carry out structure around unstructured NAS file data. More than a data mover or storage migration tool, Data Dynamics StorageX is a software platform for adding storage management structure around unstructured local and distributed NAS file data. This includes heterogeneous vendor support across different storage system, protocols and tools including Windows CIFS and Unix/Linux NFS.
(Disclosure DataDynamics has been a StorageIO client). Visit Data Dynamics at www.datadynamicsinc.com/

Server and StorageIO seminars, conferences, web cats, events, activities StorageIO activities (out and about)

Seminars, symposium, conferences, webinars
Live in person and recorded recent and upcoming events

Announcing: Backup.U brought to you by Dell

Some on-line (live and recorded) events have include an ongoing series tied to data protection (Backup/restore, HA, BC, DR and Archiving) called Backup.U organized and sponsored by Dell Data Protection Software that you can learn more about at the landing page www.software.dell.com/backupu (more on this in a future post). In addition to data protection, some other events and activities including a BrightTalk webinar on storage I/O and networking for cloud environments (here).

In addition to the above, check out the StorageIO calendar to see more recent and upcoming activities.

Watch for more 2013 events to be added soon to the StorageIO events calendar page. Topics include data protection modernization (backup/restore, HA, BC, DR, archive), data footprint reduction (archive, compression, dedupe), storage optimization, SSD, object storage, server and storage virtualization, big data, little data, cloud and object storage, performance and management trends among others.

Vendors, VAR’s and event organizers, give us a call or send an email to discuss having us involved in your upcoming pod cast, web cast, virtual seminar, conference or other events.

If you missed the Summer (July and August) 2013 StorageIO update newsletter, click here to view that and other previous editions as HTML or PDF versions. Subscribe to this newsletter (and pass it along)

and click here to subscribe to this news letter. View archives of past StorageIO update news letters as well as download PDF versions at: www.storageio.com/newsletter

Ok, nuff said (for now)

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

Posted on

Cloud conversations: Has Nirvanix shutdown caused cloud confidence concerns?

Storage I/O trends

Cloud conversations: Has Nirvanix shutdown caused cloud confidence concerns?

Recently seven plus year old cloud storage startup Nirvanix announced that they were finally shutting down and that customers should move their data.

nirvanix customer message

Nirvanix has also posted an announcement that they have established an agreement with IBM Softlayer (read about that acquisition here) to help customers migrate to those services as well as to those of Amazon Web Services (AWS), (read more about AWS in this primer here), Google and Microsoft Azure.

Cloud customer concerns?

With Nirvanix shutting down there has been plenty of articles, blog posts, twitter tweets and other conversations asking if Clouds are safe.

Btw, here is a link to my ongoing poll where you can cast your vote on what you think about clouds.

IMHO clouds can be safe if used in safe ways which includes knowing and addressing your concerns, not to mention following best practices, some of which pre-date the cloud era, sometimes by a few decades.

Nirvanix Storm Clouds

More on this in a moment, however lets touch base on Nirvanix and why I said they were finally shutting down.

The reason I say finally shutting down is that there were plenty of early warning signs and storm clouds circling Nirvanix for a few years now.

What I mean by this is that in their seven plus years of being in business, there have been more than a few CEO changes, something that is not unheard of.

Likewise there have been some changes to their business model ranging from selling their software as a service to a solution to hosting among others, again, smart startups and establishes organizations will adapt over time.

Nirvanix also invested heavily in marketing, public relations (PR) and analyst relations (AR) to generate buzz along with gaining endorsements as do most startups to get recognition, followings and investors if not real customers on board.

In the case of Nirvanix, the indicator signs mentioned above also included what seemed like a semi-annual if not annual changing of CEOs, marketing and others tying into business model adjustments.

cloud storage

It was only a year or so ago that if you gauged a company health by the PR and AR news or activity and endorsements you would have believed Nirvanix was about to crush Amazon, Rackspace or many others, perhaps some actually did believe that, followed shortly there after by the abrupt departure of their then CEO and marketing team. Thus just as fast as Nirvanix seemed to be the phoenix rising in stardom their aura started to dim again, which could or should have been a warning sign.

This is not to solo out Nirvanix, however given their penchant for marketing and now what appears to some as a sudden collapse or shutdown, they have also become a lightning rod of sort for clouds in general. Given all the hype and fud around clouds when something does happen the distract ors will be quick to jump or pile on to say things like "See, I told you, clouds are bad".

Meanwhile the cloud cheerleaders may go into denial saying there are no problems or issues with clouds, or they may go back into a committee meeting to create a new stack, standard, API set marketing consortium alliance. ;) On the other hand, there are valid concerns with any technology including clouds that in general there are good implementations that can be used the wrong way, or questionable implementations and selections used in what seem like good ways that can go bad.

This is not to say that clouds in general whether as a service, solution or product on a public, private or hybrid bases are any riskier than traditional hardware, software and services. Instead what this should be is a wake up call for people and organizations to review clouds citing their concerns along with revisiting what to do or can be done about them.

Clouds: Being prepared

Ben Woo of Neuralytix posted this question comment to one of the Linked In groups Collateral Considerations If You Were/Are A Nirvanix Customer which I posted some tips and recommendations including:

1) If you have another copy of your data somewhere else (which you should btw), how will your data at Nirvanix be securely erased, and the storage it resides on be safely (and secure) decommissioned?

2) if you do have another copy of your data elsewhere, how current is it, can you bring it up to date from various sources (including update from Nirvanix while they stay online)?

3) Where will you move your data to short or near term, as well as long-term.

4) What changes will you make to your procurement process for cloud services in the future to protect against situations like this happening to you?

5) As part of your plan for putting data into the cloud, refine your strategy for getting it out, moving it to another service or place as well as having an alternate copy somewhere.

Fwiw any data I put into a cloud service there is also another copy somewhere else which even though there is a cost, there is a benefit, The benefit is that ability to decide which to use if needed, as well as having a backup/spare copy.

Storage I/O trends

Cloud Concerns and Confidence

As part of cloud procurement as services or products, the same proper due diligence should occur as if you were buying traditional hardware, software, networking or services. That includes checking out not only the technology, also the companies financial, business records, customer references (both good and not so good or bad ones) to gain confidence. Part of gaining that confidence also involves addressing ahead of time how you will get your data out of or back from that services if needed.

Keep in mind that if your data is very important, are you going to keep it in just one place? For example I have data backed-up as well as archived to cloud providers, however I also have local copies either on-site or off.

Likewise there is data I have local kept at alternate locations including cloud. Sure that is costly, however by not treating all of my data and applications the same, I’m able to balance those costs out, plus use cost advantages of different services as well as on-site to be effective. I may be spending no less on data protection, in fact I’m actually spending a bit more, however I also have more copies and versions of important data and in multiple locations. Data that is not changing often does not get protected as often, however there are multiple copies to meet different needs or threat risks.

Storage I/O trends

Don’t be scared of clouds, be prepared

While some of the other smaller cloud storage vendors will see some new customers, I suspect that near to mid-term, it will be the larger, more established and well funded providers that gain the most from this current situation. Granted some customers are looking for alternatives to the mega cloud providers such as Amazon, Google, HP, IBM, Microsoft and Rackspace among others, however there are a long list of others some of which who are not so well-known that should be such as Centurylink/Savvis, Verizon/Terremark, Sungurd, Dimension Data, Peak, Bluehost, Carbonite, Mozy (owned by EMC), Xerox ACS, Evault (owned by Seagate) not to mention a long list of many others.

Something to be aware of as part of doing your due diligence is determining who or what actually powers a particular cloud service. The larger providers such as Rackspace, Amazon, Microsoft, HP among others have their own infrastructure while some of the smaller service providers may in fact use one of the larger (or even smaller) providers as their real back-end. Hence understanding who is behind a particular cloud service is important to help decide the viability and stability of who it is you are subscribed to or working with.

Something that I have said for the past couple of years and a theme of my book Cloud and Virtual Data Storage Networking (CRC Taylor & Francis) is do not be scared of clouds, however be ready, do your homework.

This also means having cloud concerns is a good thing, again don’t be scared, however find what those concerns are along with if they are major or minor. From that list you can start to decide how or if they can be worked around, as well as be prepared ahead of time should you either need all of your cloud data back quickly, or should that service become un-available.

Also when it comes to clouds, look beyond lowest cost or for free, likewise if something sounds too good to be true, perhaps it is. Instead look for value or how much do you get per what you spend including confidence in the service, service level agreements (SLA), security, and other items.

Keep in mind, only you can prevent data loss either on-site or in the cloud, granted it is a shared responsibility (With a poll).

Additional related cloud conversation items:
Cloud conversations: AWS EBS Optimized Instances
Poll: What Do You Think of IT Clouds?
Cloud conversations: Gaining cloud confidence from insights into AWS outages
Cloud conversations: confidence, certainty and confidentiality
Cloud conversation, Thanks Gartner for saying what has been said
Cloud conversations: AWS EBS, Glacier and S3 overview (Part III)
Cloud conversations: Gaining cloud confidence from insights into AWS outages (Part II)
Don’t Let Clouds Scare You – Be Prepared
Everything Is Not Equal in the Datacenter, Part 3
Amazon cloud storage options enhanced with Glacier
What do VARs and Clouds as well as MSPs have in common?
How many degrees separate you and your information?

Ok, nuff said.

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

Posted on

Fall 2013 Dutch cloud, virtual and storage I/O seminars

Storage I/O trends

Fall 2013 Dutch cloud, virtual and storage I/O seminars

It is that time of the year again when StorageIO will be presenting a series of seminar workshops in the Netherlands on cloud, virtual and data storage networking technologies, trends along with best practice techniques.

Brouwer Storage

StorageIO partners with the independent firm Brouwer Storage Consultancy of Holland who organizes these sessions. These sessions will also mark Brouwer Storage Consultancy celebrating ten years in business along with a long partnership with StorageIO.

Server Storage I/O Backup and Data Protection Cloud and Virtual

The fall 2013 Dutch seminars include coverage of storage I/O networking data protection and related trends topics for cloud and virtual environments. Click on the following links or images to view an abstract of the three sessions including what you will learn, who they are for, buzzwords, themes, topics and technologies that will covered.

Modernizing Data Protection
Moving Beyond Backup and Restore

Storage Industry Trends
What’s News, What’s The Buzz and Hype

Storage Decision Making
Acquisition, Deployment, Day to Day Management

Modern Data Protection
Modern Data Protection
Modern Data Protection
September 30 & October 1
October 2 2013
October 3 and 4 2013

All seminar workshop seminars are presented in a vendor technology neutral including (e.g. these are not vendor marketing sales presentations) providing independent perspectives on industry trends, who is doing what, benefits, caveats of various approaches to addressing data infrastructure and storage challenges. View posts about earlier events here and here.

Storage I/O trends

As part of theme of being vendor and technology neutral, the workshop seminars are held off-site at hotel venues in Nijkerk Netherlands so no need to worry about the sales teams coming in to sell you something during the breaks or lunch which are provided. There are also opportunities throughout the workshops for engagement, discussion and interaction with other attendees that includes your peers from various commercial, government and service providers among others.

Learn more and register for these events by visiting the Brouwer Storage Consultancy website page (here) and calling them at +31-33-246-6825 or via email info@brouwerconsultancy.com.

Storage I/O events

View other upcoming and recent StorageIO activities including live in-person, online web and recorded activities on our events page here, as well as check out our commentary and industry trends perspectives in the news here.

Bitter ballen
Ok, nuff said, I’m already hungry for bitter ballen (see above)!

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

Posted on

Summer 2013 Server and StorageIO Update Newsletter

StorageIO 2013 Summer Newsletter

Cloud, Virtualization, SSD, Data Protection, Storage I/O

Welcome to the Summer 2013 (combined July and August) edition of the StorageIO Update (newsletter) containing trends perspectives on cloud, virtualization and data infrastructure topics.

StorageIO News Letter Image
Summer 2013 News letter

This summer has been far from quiet on the merger and acquisitions (M&E) front with Western Digital (WD) continuing its buying spree including Stec among others. There is the HDS Mid Summer Storage and Converged Compute Enhancements and EMC Evolves Enterprise Data Protection with Enhancements (Part I and Part II).

With VMworld just around the corner along with many other upcoming events, watch for more announcements to be covered in future editions and on StorageIOblog as we move into fall.

Click on the following links to view the Summer 2013 edition as (HTML sent via Email) version, or PDF versions. Visit the news letter page to view previous editions of the StorageIO Update.

You can subscribe to the news letter by clicking here

Enjoy this edition of the StorageIO Update news letter, let me know your comments and feedback.

Ok Nuff said, for now

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

Posted on

Can we get a side of context with them IOPS server storage metrics?

Can we get a side of context with them server storage metrics?

Whats the best server storage I/O network metric or benchmark? It depends as there needs to be some context with them IOPS and other server storage I/O metrics that matter.

There is an old saying that the best I/O (Input/Output) is the one that you do not have to do.

In the meantime, let’s get a side of some context with them IOPS from vendors, marketers and their pundits who are tossing them around for server, storage and IO metrics that matter.

Expanding the conversation, the need for more context

The good news is that people are beginning to discuss storage beyond space capacity and cost per GByte, TByte or PByte for both DRAM or nand flash Solid State Devices (SSD), Hard Disk Drives (HDD) along with Hybrid HDD (HHDD) and Solid State Hybrid Drive (SSHD) based solutions. This applies to traditional enterprise or SMB IT data center with physical, virtual or cloud based infrastructures.

hdd and ssd iops

This is good because it expands the conversation beyond just cost for space capacity into other aspects including performance (IOPS, latency, bandwidth) for various workload scenarios along with availability, energy effective and management.

Adding a side of context

The catch is that IOPS while part of the equation are just one aspect of performance and by themselves without context, may have little meaning if not misleading in some situations.

Granted it can be entertaining, fun to talk about or simply make good press copy for a million IOPS. IOPS vary in size depending on the type of work being done, not to mention reads or writes, random and sequential which also have a bearing on data throughout or bandwidth (Mbytes per second) along with response time. Not to mention block, file, object or blob as well as table.

However, are those million IOP’s applicable to your environment or needs?

Likewise, what do those million or more IOPS represent about type of work being done? For example, are they small 64 byte or large 64 Kbyte sized, random or sequential, cached reads or lazy writes (deferred or buffered) on a SSD or HDD?

How about the response time or latency for achieving them IOPS?

In other words, what is the context of those metrics and why do they matter?

storage i/o iops
Click on image to view more metrics that matter including IOP’s for HDD and SSD’s

Metrics that matter give context for example IO sizes closer to what your real needs are, reads and writes, mixed workloads, random or sequential, sustained or bursty, in other words, real world reflective.

As with any benchmark take them with a grain (or more) of salt, they key is use them as an indicator then align to your needs. The tool or technology should work for you, not the other way around.

Here are some examples of context that can be added to help make IOP’s and other metrics matter:

  • What is the IOP size, are they 512 byte (or smaller) vs. 4K bytes (or larger)?
  • Are they reads, writes, random, sequential or mixed and what percentage?
  • How was the storage configured including RAID, replication, erasure or dispersal codes?
  • Then there is the latency or response time and IO queue depths for the given number of IOPS.
  • Let us not forget if the storage systems (and servers) were busy with other work or not.
  • If there is a cost per IOP, is that list price or discount (hint, if discount start negotiations from there)
  • What was the number of threads or workers, along with how many servers?
  • What tool was used, its configuration, as well as raw or cooked (aka file system) IO?
  • Was the IOP’s number with one worker or multiple workers on a single or multiple servers?
  • Did the IOP’s number come from a single storage system or total of multiple systems?
  • Fast storage needs fast serves and networks, what was their configuration?
  • Was the performance a short burst, or long sustained period?
  • What was the size of the test data used; did it all fit into cache?
  • Were short stroking for IOPS or long stroking for bandwidth techniques used?
  • Data footprint reduction (DFR) techniques (thin provisioned, compression or dedupe) used?
  • Were write data committed synchronously to storage, or deferred (aka lazy writes used)?

The above are just a sampling and not all may be relevant to your particular needs, however they help to put IOP’s into more contexts. Another consideration around IOPS are the configuration of the environment, from an actual running application using some measurement tool, or are they generated from a workload tool such as IOmeter, IOrate, VDbench among others.

Sure, there are more contexts and information that would be interesting as well, however learning to walk before running will help prevent falling down.

Storage I/O trends

Does size or age of vendors make a difference when it comes to context?

Some vendors are doing a good job of going for out of this world record-setting marketing hero numbers.

Meanwhile other vendors are doing a good job of adding context to their IOP or response time or bandwidth among other metrics that matter. There is a mix of startup and established that give context with their IOP’s or other metrics, likewise size or age does not seem to matter for those who lack context.

Some vendors may not offer metrics or information publicly, so fine, go under NDA to learn more and see if the results are applicable to your environments.

Likewise, if they do not want to provide the context, then ask some tough yet fair questions to decide if their solution is applicable for your needs.

Storage I/O trends

Where To Learn More

View additional NAS, NVMe, SSD, NVM, SCM, Data Infrastructure and HDD related topics via the following links.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

What this means is let us start putting and asking for metrics that matter such as IOP’s with context.

If you have a great IOP metric, if you want it to matter than include some context such as what size (e.g. 4K, 8K, 16K, 32K, etc.), percentage of reads vs. writes, latency or response time, random or sequential.

IMHO the most interesting or applicable metrics that matter are those relevant to your environment and application. For example if your main application that needs SSD does about 75% reads (random) and 25% writes (sequential) with an average size of 32K, while fun to hear about, how relevant is a million 64 byte read IOPS? Likewise when looking at IOPS, pay attention to the latency, particular if SSD or performance is your main concern.

Get in the habit of asking or telling vendors or their surrogates to provide some context with them metrics if you want them to matter.

So how about some context around them IOP’s (or latency and bandwidth or availability for that matter)?

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Part II: EMC Evolves Enterprise Data Protection with Enhancements

Storage I/O trends

This is the second part of a two-part series on recent EMC backup and data protection announcements. Read part I here.

What about the products, what’s new?

In addition to articulating their strategy for modernizing data protection (covered in part I here), EMC announced enhancements to Avamar, Data Domain, Mozy and Networker.

Data protection storage systems (e.g. Data Domain)

Building off of previously announced Backup Recovery Solutions (BRS) including Data Domain operating system storage software enhancements, EMC is adding more application and software integration along with new platform (systems) support.

Data Domain (e.g. Protection Storage) enhancements include:

  • Application integration with Oracle, SAP HANA for big data backup and archiving
  • New Data Domain protection storage system models
  • Data in place upgrades of storage controllers
  • Extended Retention now available on added models
  • SAP HANA Studio backup integration via NFS
  • Boost for Oracle RMAN, native SAP tools and replication integration
  • Support for backing up and protecting Oracle Exadata
  • SAP (non HANA) support both on SAP and Oracle

Data in place upgrades of controllers for 4200 series models on up (previously available on some larger models). This means that controllers can be upgraded with data remaining in place as opposed to a lengthy data migration.

Extended Retention facility is a zero cost license that enables more disk drive shelves to be attached to supported Data Domain systems. Thus there is a not a license fee, however you do pay for the storage shelves and drives to increase the available storage capacity. Note that this feature increases the storage capacity by adding more disk drives and does not increase the performance of the Data Domain system. Extended Retention has been available in the past however is now supported via more platform models. The extra storage capacity is essentially placed into a different tier that an archive policy can then migrate data into.

Boost for accelerating data movement to and from Data Domain systems is only available using Fibre Channel. When asked about FC over Ethernet (FCoE) or iSCSI EMC indicated its customers are not asking for this ability yet. This has me wondering if it is that the current customer focus is around FC, or if those customers are not yet ready for iSCSI or FCoE, or, if there were iSCSI or FCoE support, more customers would ask for it?

With the new Data Domain protection storage systems EMC is claiming up to:

  • 4x faster performance than earlier models
  • 10x more scalable and 3x more backup/archive streams
  • 38 percent lower cost per GB based on holding price points and applying improvements


EMC Data Domain data protection storage platform family


Data Domain supporting both backup and archive

Expanding Data Domain from backup to archive

EMC continues to evolve the Data Domain platform from just being a backup target platform with dedupe and replication to a multi-function, multi-role solution. In other words, one platform with many uses. This is an example of using one tool or technology for different purposes such as backup and archiving, however with separate polices. Here is a link to a video where I discuss using common tools for backup and archiving, however with separate polices. In the above figure EMC Data Domain is shown as being used for backup along with storage tiering and archiving (file, email, Sharepoint, content management and databases among other workloads).


EMC Data Domain supporting different functions and workloads

Also shown are various tools from other vendors such as Commvault Simpana that can be used as both a backup or archiving tool with Data Domain as a target. Likewise Dell products acquired via the Quest acquisition are shown along with those from IBM (e.g. Tivoli), FileTek among others. Note that if you are a competitor of EMC or simply a fan of other technology you might come to the conclusion that the above may not be different from others. Then again others who are not articulating their version or vision of something like the above figure probably should be also stating the obvious vs. arguing they did it first.

Data source integration (aka data protection software tools)

It seems like just yesterday that EMC acquired Avamar (2006) and NetWorker aka Legato (2003), not to mention Mozy (2007) or Dantz (Retrospect, since divested) in 2004. With the exception of Dantz (Retrospect) which is now back in the hands of its original developers, EMC continues to enhance and evolve Avamar, Mozy and NetWorker including with this announcement.

General Avamar 7 and Networker 8.1 enhancements include:

  • Deeper integration with primary storage and protection storage tiers
  • Optimization for VMware vSphere virtual server environments
  • Improved visibility and control for data protection of enterprise applications

Additional Avamar 7 enhancements include:

  • More Data Domain integration and leveraging as a repository (since Avamar 6)
  • NAS file systems with NDMP accelerator access (EMC Isilon & Celera, NetApp)
  • Data Domain Boost enhancements for faster backup / recovery
  • Application integration with IBM (DB2 and Notes), Microsoft (Exchange, Hyper-V images, Sharepoint, SQL Server), Oracle, SAP, Sybase, VMware images

Note that Avamar dat is still used mainly for ROBO and desktop, laptop type backup scenarios that do not yet support Data Domain. Also see Mozy enhancements below).

Avamar supports VMware vSphere virtual server environments using granular change block tracking (CBT) technology as well as image level backup and recovery with vSphere plugins. This includes an Instant Access recovery when images are stored on Data Domain storage.

Instant Access enables a VM that has been protected using Avamar image level technology on Data Domain to be booted via an NFS VMware Dat. VMware sees the VM and is able to power it on and boot directly from the Data Domain via the NFS Dat. Once the VM is active, it can be Storage vMotion to a production storage VMware Dat while active (e.g. running) for recovery on the fly capabilities.


Instant Access to a VM on Data Domain storage

EMC NetWorker 8.1 enhancements include:

  • Enhanced visibility and control for owners of data
  • Collaborative protection for Oracle environments
  • Synchronize backup and data protection between DBA and Backup admin’s
  • Oracle DBAs use native tools (e.g. RMAN)
  • Backup admin implements organizations SLA’s (e.g. using Networker)
  • Deeper integration with EMC primary storage (e.g. VMAX, VNX, etc)
  • Isilon integration support
  • Snapshot management (VMAX, VNX, RecoverPoint)
  • Automation and wizards for integration, discovery, simplified management
  • Policy-based management, fast recovery from snapshots
  • Integrating snapshots into and as part of data protection strategy. Note that this is more than basic snapshot management as there is also the ability to roll over a snapshot into a Data Domain protection storage tier.
  • Deeper integration with Data Domain protection storage tier
  • Data Domain Boost over Fibre Channel for faster backups and restores
  • Data Domain Virtual Synthetics to cut impact of full backups
  • Integration with Avamar for managing image level backup recovery (Avamar services embedded as part of NetWorker)
  • vSphere Web Client enabling self-service recovery of VMware images
  • Newly created VMs inherit backup polices automatically

Mozy is being positioned for enterprise remote office branch office (ROBO) or distributed private cloud where Avamar, NetWorker or Data Domain solutions are not as applicable. EMC has mentioned that they have over 800 enterprises using Mozy for desktop, laptop, ROBO and mobile data protection. Note that this is a different target market than the Mozy consumer product focused which also addresses smaller SMBs and SOHOs (Small Office Home Offices).

EMC Mozy enhancements to be more enterprise grade:

  • Simplified management services and integration
  • Active Directory (AD) for Microsoft environments
  • New storage pools (multiple types of pools) vs. dedicated storage per client
  • Keyless activation for faster provisioning of backup clients

Note that EMC enhanced earlier this year Data Protection Advisor (DPA) with version 6.0.

What does this all mean?

Storage I/O trends

Data protection and backup discussions often focus around tape summit resources or cloud arguments, although this is changing. What is changing is growing awareness and discussion around how data protection storage mediums, systems and services are used along with the associated software management tools.

Some will say backup is broke often pointing a finger at a media or medium (e.g. tape and disk) about what is wrong. Granted in some environments the target medium (or media) destination is an easy culprit to point a finger to as the problem (e.g. the usual tape sucks or is dead) mantra. However, for many environments while there can be issues, it is more often than not the media, medium, device or target storage system that is broke, instead how it is being used or abused.

This means revisiting how tools are used along with media or storage systems allocated, used and retained with respect to different threat risk scenarios. After all, not everything is the same in the data center or information factory.

Thus modernizing data protection is more than swapping media or mediums including types of storage system from one to another. It is also more than swapping out one backup or data protection tool for another. Modernizing data protection means rethinking what different applications and data need to be protected against various threat risks.

Storage I/O trends

What this has to do with today’s announcement is that EMC is among others in the industry moving towards a holistic data protection modernizing thought model.

In my opinion what you are seeing out of EMC and some others is taking that step back and expanding the data protection conversation to revisit, rethink why, how, where, when and by whom applications and information get protected.

This announcement also ties into finding and removing costs vs. simply cutting cost at the cost of something elsewhere (e.g. service levels, performance, availability). In other words, finding and removing complexities or overhead associated with data protection while making it more effective.

Some closing points, thoughts and more links:

There is no such thing as a data or information recession
People and data are living longer while getting larger
Not everything is the same in the data center or information factory
Rethink data protection including when, why, how, where, with what and by whom
There is little data, big data, very big data and big fast data
Data protection modernization is more than playing buzzword bingo
Avoid using new technology in old ways
Data footprint reduction (DFR) can be help counter changing data life-cycle patterns
EMC continues to leverage Avamar while keeping Networker relevant
Data Domain evolving for both backup and archiving as an example of tool for multiple uses

Ok, nuff said (for now).

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

Posted on

EMC Evolves Enterprise Data Protection with Enhancements (Part I)

Storage I/O trends

A couple of months ago at EMCworld there were announcements around ViPR, Pivotal along with trust and clouds among other topics. During the recent EMCworld event there were some questions among attendees what about backup and data protection announcements (or lack there of)?

Modernizing Data Protection

Today EMC announced enhancements to its Backup Recovery Solutions (BRS) portfolio (@EMCBackup) that continue to enable information and applications data protection modernizing including Avamar, Data Domain, Mozy and Networker.

Keep in mind you can’t go forward if you can’t go back, which means if you do not have good data protection to go to, you can’t go forward with your information.

EMC Modern Data Protection Announcements

As part of their Backup to the Future event, EMC announced the following:

  • New generation of data protection products and technologies
  • Data Domain systems: enhanced application integration for backup and archive
  • Data protection suite tools Avamar 7 and Networker 8.1
  • Enhanced Cloud backup capabilities for the Mozy service
  • Paradigm shift as part of data protection modernizing including revisiting why, when, where, how, with what and by whom data protection is accomplished.

What did EMC announce for data protection modernization?

While much of the EMC data protection announcement is around product, there is also the aspect of rethinking data protection. This means looking at data protection modernization beyond swapping out media (e.g. tape for disk, disk for cloud) or one backup software tool for another. Instead, revisiting why data protection needs to be accomplished, by whom, how to remove complexity and cost, enable agility and flexibility. This also means enabling data protection to be used or consumed as a service in traditional, virtual and private or hybrid cloud environments.

EMC uses as an example (what they refer to as Accidental Architecture) of how there are different group and areas of focus, along with silos associated with data protection. These groups span virtual, applications, database, server, storage among others.

The results are silos that need to be transformed in part using new technology in new ways, as well as addressing a barrier to IT convergence (people and processes). The theme behind EMC data protection strategy is to enable the needs and requirements of various groups (servers, applications, database, compliance, storage, BC and DR) while removing complexity.

Moving from Silos of data protection to a converged service enabled model

Three data protection and backup focus areas

This sets the stage for the three components for enabling a converged data protection model that can be consumed or used as a service in traditional, virtual and private cloud environments.


EMC three components of modernized data protection (EMC Future Backup)

The three main components (and their associated solutions) of EMC BRS strategy are:

  • Data management services: Policy and storage management, SLA, SLO, monitoring, discovery and analysis. This is where tools such as EMC Data Protection Advisor (aka via WysDM acquisition) fit among others for coordination or orchestration, setting and managing polices along with other activities.
  • Data source integration: Applications, Database, File systems, Operating System, Hypervisors and primary storage systems. This is where data movement tools such as Avamar and Networker among others fit along with interfaces to application tools such as Oracle RMAN.
  • Protection storage: Targets, destination storage system with media or mediums optimized for protecting and preserving data along with enabling data footprint reduction (DFR). DFR includes functionality such as compression and dedupe among others. Example of data protection storage is EMC Data Domain.

Read more about product items announced and what this all means here in the second of this two-part series.

Ok, nuff said (for now).

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

Posted on

Part II: How many IOPS can a HDD HHDD SSD do with VMware?

How many IOPS can a HDD HHDD SSD do with VMware?

server storage data infrastructure i/o iop hdd ssd trends

Updated 2/10/2018

This is the second post of a two-part series looking at storage performance, specifically in the context of drive or device (e.g. mediums) characteristics of How many IOPS can a HDD HHDD SSD do with VMware. In the first post the focus was around putting some context around drive or device performance with the second part looking at some workload characteristics (e.g. benchmarks).

A common question is how many IOPS (IO Operations Per Second) can a storage device or system do?

The answer is or should be it depends.

Here are some examples to give you some more insight.

For example, the following shows how IOPS vary by changing the percent of reads, writes, random and sequential for a 4K (4,096 bytes or 4 KBytes) IO size with each test step (4 minutes each).

IO Size for test
Workload Pattern of test
Avg. Resp (R+W) ms
Avg. IOP Sec (R+W)
Bandwidth KB Sec (R+W)
4KB
100% Seq 100% Read
0.0
29,736
118,944
4KB
60% Seq 100% Read
4.2
236
947
4KB
30% Seq 100% Read
7.1
140
563
4KB
0% Seq 100% Read
10.0
100
400
4KB
100% Seq 60% Read
3.4
293
1,174
4KB
60% Seq 60% Read
7.2
138
554
4KB
30% Seq 60% Read
9.1
109
439
4KB
0% Seq 60% Read
10.9
91
366
4KB
100% Seq 30% Read
5.9
168
675
4KB
60% Seq 30% Read
9.1
109
439
4KB
30% Seq 30% Read
10.7
93
373
4KB
0% Seq 30% Read
11.5
86
346
4KB
100% Seq 0% Read
8.4
118
474
4KB
60% Seq 0% Read
13.0
76
307
4KB
30% Seq 0% Read
11.6
86
344
4KB
0% Seq 0% Read
12.1
82
330

Dell/Western Digital (WD) 1TB 7200 RPM SATA HDD (Raw IO) thread count 1 4K IO size

In the above example the drive is a 1TB 7200 RPM 3.5 inch Dell (Western Digital) 3Gb SATA device doing raw (non file system) IO. Note the high IOP rate with 100 percent sequential reads and a small IO size which might be a result of locality of reference due to drive level cache or buffering.

Some drives have larger buffers than others from a couple to 16MB (or more) of DRAM that can be used for read ahead caching. Note that this level of cache is independent of a storage system, RAID adapter or controller or other forms and levels of buffering.

Does this mean you can expect or plan on getting those levels of performance?

I would not make that assumption, and thus this serves as an example of using metrics like these in the proper context.

Building off of the previous example, the following is using the same drive however with a 16K IO size.

IO Size for test
Workload Pattern of test
Avg. Resp (R+W) ms
Avg. IOP Sec (R+W)
Bandwidth KB Sec (R+W)
16KB
100% Seq 100% Read
0.1
7,658
122,537
16KB
60% Seq 100% Read
4.7
210
3,370
16KB
30% Seq 100% Read
7.7
130
2,080
16KB
0% Seq 100% Read
10.1
98
1,580
16KB
100% Seq 60% Read
3.5
282
4,522
16KB
60% Seq 60% Read
7.7
130
2,090
16KB
30% Seq 60% Read
9.3
107
1,715
16KB
0% Seq 60% Read
11.1
90
1,443
16KB
100% Seq 30% Read
6.0
165
2,644
16KB
60% Seq 30% Read
9.2
109
1,745
16KB
30% Seq 30% Read
11.0
90
1,450
16KB
0% Seq 30% Read
11.7
85
1,364
16KB
100% Seq 0% Read
8.5
117
1,874
16KB
60% Seq 0% Read
10.9
92
1,472
16KB
30% Seq 0% Read
11.8
84
1,353
16KB
0% Seq 0% Read
12.2
81
1,310

Dell/Western Digital (WD) 1TB 7200 RPM SATA HDD (Raw IO) thread count 1 16K IO size

The previous two examples are excerpts of a series of workload simulation tests (ok, you can call them benchmarks) that I have done to collect information, as well as try some different things out.

The following is an example of the summary for each test output that includes the IO size, workload pattern (reads, writes, random, sequential), duration for each workload step, totals for reads and writes, along with averages including IOP’s, bandwidth and latency or response time.

disk iops

Want to see more numbers, speeds and feeds, check out the following table which will be updated with extra results as they become available.

Device
Vendor
Make

Model

Form Factor
Capacity
Interface
RPM Speed
Raw
Test Result
HDD
HGST
Desktop
HK250-160
2.5
160GB
SATA
5.4K
HDD
Seagate
Mobile
ST2000LM003
2.5
2TB
SATA
5.4K
HDD
Fujitsu
Desktop
MHWZ160BH
2.5
160GB
SATA
7.2K
HDD
Seagate
Momentus
ST9160823AS
2.5
160GB
SATA
7.2K
HDD
Seagate
MomentusXT
ST95005620AS
2.5
500GB
SATA
7.2K(1)
HDD
Seagate
Barracuda
ST3500320AS
3.5
500GB
SATA
7.2K
HDD
WD/Dell
Enterprise
WD1003FBYX
3.5
1TB
SATA
7.2K
HDD
Seagate
Barracuda
ST3000DM01
3.5
3TB
SATA
7.2K
HDD
Seagate
Desktop
ST4000DM000
3.5
4TB
SATA
HDD
HDD
Seagate
Capacity
ST6000NM00
3.5
6TB
SATA
HDD
HDD
Seagate
Capacity
ST6000NM00
3.5
6TB
12GSAS
HDD
HDD
Seagate
Savio 10K.3
ST9300603SS
2.5
300GB
SAS
10K
HDD
Seagate
Cheetah
ST3146855SS
3.5
146GB
SAS
15K
HDD
Seagate
Savio 15K.2
ST9146852SS
2.5
146GB
SAS
15K
HDD
Seagate
Ent. 15K
ST600MP0003
2.5
600GB
SAS
15K
SSHD
Seagate
Ent. Turbo
ST600MX0004
2.5
600GB
SAS
SSHD
SSD
Samsung
840 PRo
MZ-7PD256
2.5
256GB
SATA
SSD
HDD
Seagate
600 SSD
ST480HM000
2.5
480GB
SATA
SSD
SSD
Seagate
1200 SSD
ST400FM0073
2.5
400GB
12GSAS
SSD

Performance characteristics 1 worker (thread count) for RAW IO (non-file system)

Note: (1) Seagate Momentus XT is a Hybrid Hard Disk Drive (HHDD) based on a 7.2K 2.5 HDD with SLC nand flash integrated for read buffer in addition to normal DRAM buffer. This model is a XT I (4GB SLC nand flash), may add an XT II (8GB SLC nand flash) at some future time.

As a starting point, these results are raw IO with file system based information to be added soon along with more devices. These results are for tests with one worker or thread count, other results will be added with such as 16 workers or thread counts to show how those differ.

The above results include all reads, all writes, mix of reads and writes, along with all random, sequential and mixed for each IO size. IO sizes include 4K, 8K, 16K, 32K, 64K, 128K, 256K, 512K, 1024K and 2048K. As with any workload simulation, benchmark or comparison test, take these results with a grain of salt as your mileage can and will vary. For example you will see some what I consider very high IO rates with sequential reads even without file system buffering. These results might be due to locality of reference of IO’s being resolved out of the drives DRAM cache (read ahead) which vary in size for different devices. Use the vendor model numbers in the table above to check the manufactures specs on drive DRAM and other attributes.

If you are used to seeing 4K or 8K and wonder why anybody would be interested in some of the larger sizes take a look at big fast data or cloud and object storage. For some of those applications 2048K may not seem all that big. Likewise if you are used to the larger sizes, there are still applications doing smaller sizes. Sorry for those who like 512 byte or smaller IO’s as they are not included. Note that for all of these unless indicated a 512 byte standard sector or drive format is used as opposed to emerging Advanced Format (AF) 4KB sector or block size. Watch for some more drive and device types to be added to the above, along with results for more workers or thread counts, along with file system and other scenarios.

Using VMware as part of a Server, Storage and IO (aka StorageIO) test platform

vmware vexpert

The above performance results were generated on Ubuntu 12.04 (since upgraded to 14.04 which was hosted on a VMware vSphere 5.1 (upgraded to 5.5U2) purchased version (you can get the ESXi free version here) with vCenter enabled system. I also have VMware workstation installed on some of my Windows-based laptops for doing preliminary testing of scripts and other activity prior to running them on the larger server-based VMware environment. Other VMware tools include vCenter Converter, vSphere Client and CLI. Note that other guest virtual machines (VMs) were idle during the tests (e.g. other guest VMs were quiet). You may experience different results if you ran Ubuntu native on a physical machine or with different adapters, processors and device configurations among many other variables (that was a disclaimer btw ;) ).

Storage I/O trends

All of the devices (HDD, HHDD, SSD’s including those not shown or published yet) were Raw Device Mapped (RDM) to the Ubuntu VM bypassing VMware file system.

Example of creating an RDM for local SAS or SATA direct attached device.

vmkfstools -z /vmfs/devices/disks/naa.600605b0005f125018e923064cc17e7c /vmfs/volumes/dat1/RDM_ST1500Z110S6M5.vmdk

The above uses the drives address (find by doing a ls -l /dev/disks via VMware shell command line) to then create a vmdk container stored in a dat. Note that the RDM being created does not actually store data in the .vmdk, it’s there for VMware management operations.

If you are not familiar with how to create a RDM of a local SAS or SATA device, check out this post to learn how.This is important to note in that while VMware was used as a platform to support the guest operating systems (e.g. Ubuntu or Windows), the real devices are not being mapped through or via VMware virtual drives.

vmware iops

The above shows examples of RDM SAS and SATA devices along with other VMware devices and dats. In the next figure is an example of a workload being run in the test environment.

vmware iops

One of the advantages of using VMware (or other hypervisor) with RDM’s is that I can quickly define via software commands where a device gets attached to different operating systems (e.g. the other aspect of software defined storage). This means that after a test run, I can quickly simply shutdown Ubuntu, remove the RDM device from that guests settings, move the device just tested to a Windows guest if needed and restart those VMs. All of that from where ever I happen to be working from without physically changing things or dealing with multi-boot or cabling issues.

Where To Learn More

View additional NAS, NVMe, SSD, NVM, SCM, Data Infrastructure and HDD related topics via the following links.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

So how many IOPs can a device do?

That depends, however have a look at the above information and results.

Check back from time to time here to see what is new or has been added including more drives, devices and other related themes.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

How many I/O iops can flash SSD or HDD do?

How many i/o iops can flash ssd or hdd do with vmware?

sddc data infrastructure Storage I/O ssd trends

Updated 2/10/2018

A common question I run across is how many I/O iopsS can flash SSD or HDD storage device or system do or give.

The answer is or should be it depends.

This is the first of a two-part series looking at storage performance, and in context specifically around drive or device (e.g. mediums) characteristics across HDD, HHDD and SSD that can be found in cloud, virtual, and legacy environments. In this first part the focus is around putting some context around drive or device performance with the second part looking at some workload characteristics (e.g. benchmarks).

What about cloud, tape summit resources, storage systems or appliance?

Lets leave those for a different discussion at another time.

Getting started

Part of my interest in tools, metrics that matter, measurements, analyst, forecasting ties back to having been a server, storage and IO performance and capacity planning analyst when I worked in IT. Another aspect ties back to also having been a sys admin as well as business applications developer when on the IT customer side of things. This was followed by switching over to the vendor world involved with among other things competitive positioning, customer design configuration, validation, simulation and benchmarking HDD and SSD based solutions (e.g. life before becoming an analyst and advisory consultant).

Btw, if you happen to be interested in learn more about server, storage and IO performance and capacity planning, check out my first book Resilient Storage Networks (Elsevier) that has a bit of information on it. There is also coverage of metrics and planning in my two other books The Green and Virtual Data Center (CRC Press) and Cloud and Virtual Data Storage Networking (CRC Press). I have some copies of Resilient Storage Networks available at a special reader or viewer rate (essentially shipping and handling). If interested drop me a note and can fill you in on the details.

There are many rules of thumb (RUT) when it comes to metrics that matter such as IOPS, some that are older while others may be guess or measured in different ways. However the answer is that it depends on many things ranging from if a standalone hard disk drive (HDD), Hybrid HDD (HHDD), Solid State Device (SSD) or if attached to a storage system, appliance, or RAID adapter card among others.

Taking a step back, the big picture

hdd image
Various HDD, HHDD and SSD’s

Server, storage and I/O performance and benchmark fundamentals

Even if just looking at a HDD, there are many variables ranging from the rotational speed or Revolutions Per Minute (RPM), interface including 1.5Gb, 3.0Gb, 6Gb or 12Gb SAS or SATA or 4Gb Fibre Channel. If simply using a RUT or number based on RPM can cause issues particular with 2.5 vs. 3.5 or enterprise and desktop. For example, some current generation 10K 2.5 HDD can deliver the same or better performance than an older generation 3.5 15K. Other drive factors (see this link for HDD fundamentals) including physical size such as 3.5 inch or 2.5 inch small form factor (SFF), enterprise or desktop or consumer, amount of drive level cache (DRAM). Space capacity of a drive can also have an impact such as if all or just a portion of a large or small capacity devices is used. Not to mention what the drive is attached to ranging from in internal SAS or SATA drive bay, USB port, or a HBA or RAID adapter card or in a storage system.

disk iops
HDD fundamentals

How about benchmark and performance for marketing or comparison tricks including delayed, deferred or asynchronous writes vs. synchronous or actually committed data to devices? Lets not forget about short stroking (only using a portion of a drive for better IOP’s) or even long stroking (to get better bandwidth leveraging spiral transfers) among others.

Almost forgot, there are also thick, standard, thin and ultra thin drives in 2.5 and 3.5 inch form factors. What’s the difference? The number of platters and read write heads. Look at the following image showing various thickness 2.5 inch drives that have various numbers of platters to increase space capacity in a given density. Want to take a wild guess as to which one has the most space capacity in a given footprint? Also want to guess which type I use for removable disk based archives along with for onsite disk based backup targets (compliments my offsite cloud backups)?

types of disks
Thick, thin and ultra thin devices

Beyond physical and configuration items, then there are logical configuration including the type of workload, large or small IOPS, random, sequential, reads, writes or mixed (various random, sequential, read, write, large and small IO). Other considerations include file system or raw device, number of workers or concurrent IO threads, size of the target storage space area to decide impact of any locality of reference or buffering. Some other items include how long the test or workload simulation ran for, was the device new or worn in before use among other items.

Tools and the performance toolbox

Then there are the various tools for generating IO’s or workloads along with recording metrics such as reads, writes, response time and other information. Some examples (mix of free or for fee) include Bonnie, Iometer, Iorate, IOzone, Vdbench, TPC, SPC, Microsoft ESRP, SPEC and netmist, Swifttest, Vmark, DVDstore and PCmark 7 among many others. Some are focused just on the storage system and IO path while others are application specific thus exercising servers, storage and IO paths.

performance tools
Server, storage and IO performance toolbox

Having used Iometer since the late 90s, it has its place and is popular given its ease of use. Iometer is also long in the tooth and has its limits including not much if any new development, never the less, I have it in the toolbox. I also have Futremark PCmark 7 (full version) which turns out has some interesting abilities to do more than exercise an entire Windows PC. For example PCmark can use a secondary drive for doing IO to.

PCmark can be handy for spinning up with VMware (or other tools) lots of virtual Windows systems pointing to a NAS or other shared storage device doing real world type activity. Something that could be handy for testing or stressing virtual desktop infrastructures (VDI) along with other storage systems, servers and solutions. I also have Vdbench among others tools in the toolbox including Iorate which was used to drive the workloads shown below.

What I look for in a tool are how extensible are the scripting capabilities to define various workloads along with capabilities of the test engine. A nice GUI is handy which makes Iometer popular and yes there are script capabilities with Iometer. That is also where Iometer is long in the tooth compared to some of the newer generation of tools that have more emphasis on extensibility vs. ease of use interfaces. This also assumes knowing what workloads to generate vs. simply kicking off some IOPs using default settings to see what happens.

Another handy tool is for recording what’s going on with a running system including IO’s, reads, writes, bandwidth or transfers, random and sequential among other things. This is where when needed I turn to something like HiMon from HyperIO, if you have not tried it, get in touch with Tom West over at HyperIO and tell him StorageIO sent you to get a demo or trial. HiMon is what I used for doing start, stop and boot among other testing being able to see IO’s at the Windows file system level (or below) including very early in the boot or shutdown phase.

Here is a link to some other things I did awhile back with HiMon to profile some Windows and VDI activity test profiling.

What’s the best tool or benchmark or workload generator?

The one that meets your needs, usually your applications or something as close as possible to it.

disk iops
Various 2.5 and 3.5 inch HDD, HHDD, SSD with different performance

Where To Learn More

View additional NAS, NVMe, SSD, NVM, SCM, Data Infrastructure and HDD related topics via the following links.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

That depends, however continue reading part II of this series to see some results for various types of drives and workloads.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.