Category: Education

Posted on

Back To Software Defined Data Infrastructure School, Webinar and Fall 2016 events

Software Defined Data Infrastructure Webinars and Fall 2016 events

server storage I/O trends

Its September and that means back to school time, and not just for the kids. Here is the preliminary Server StorageIO fall 2016 back to school, webinar and event activities covering software defined data center, data infrastructure, virtual, cloud, containers, converged, hyper-converged server, storage, I/O network, performance and data protection among other topics.

December 7, 2016 – Webinar 11AM PT – BrightTalk
Hyper-Converged Infrastructure Decision Making

Are Converged Infrastructures (CI), Hyper-Converged Infrastructures (HCI), Cluster in Box or Cloud in Box (CiB) solutions for you? The answer is it depends on what your needs, requirements, application among other criteria are. In addition are you focused on a particular technology solution or architecture approach, or, looking for something that adapts to your needs? Join us in this discussion exploring your options for different scenarios as we look beyond they hype including to next wave of hyper-scale converged along with applicable decision-making criteria. Topics include:

– Data Infrastructures exist to support applications and their underlying resource needs
– What are your application and environment needs along with other objectives
– Explore various approaches for hyper-small and hyper-large environments
– What are you converging, hardware, hypervisors, management or something else?
– Does HCI mean hyper-vendor-lock-in, if so, is that a bad thing?
– When, where, why and how to use different scenarios

November 23, 2016 – Webinar 10AM PT BrightTalk
BCDR and Cloud Backup Software Defined Data Infrastructures (SDDI) and Data Protection

The answer is BCDR and Cloud Backup, however what was the question? Besides how to protect preserve and secure your data, applications along with data Infrastructures against various threat risk issues, what are some other common questions? For example how to modernize, rethink, re-architect, use new and old things in new ways, these and other topics, techniques, trends, tools have a common theme of BCDR and Cloud Backup. Join us in this discussion exploring your options for protecting data, applications and your data Infrastructures spanning legacy, software-defined virtual and cloud environments. Topics include:

– Data Infrastructures exist to support applications and their underlying resource needs
– Various cloud storage options to meet different application PACE needs
– Do clouds need to be backed-up or protected?
– How to leverage clouds for various data protection objectives
– When, where, why and how to use different scenarios

November 23, 2016 – Webinar 9AM PT – BrightTalk
Cloud Storage – Hybrid and Software Defined Data Infrastructures (SDDI)

You have been told, or determined that you need (or want) to use cloud storage, ok, now what? What type of cloud storage do you need or want, or do you simply want cloud storage? However, what are your options as well as application requirements including Performance, Availability, Capacity and Economics (PACE) along with access or interfaces? Where are your applications and where will they be located? What are your objectives for using cloud storage or is it simply you have heard or told its cheaper. Join us in this discussion exploring your options, considerations for cloud storage decision-making. Topics include:

– Data Infrastructures exist to support applications and their underlying resource needs
– Various cloud storage options to meet different application PACE needs
– Storage for primary, secondary, performance, availability, capacity, backup, archiving
– Public, private and hybrid cloud storage options from block, file, object to application service
– When, where, why and how to use cloud storage for different scenarios

November 22, 2016 – Webinar 10AM PT – BrightTalk
Cloud Infrastructure Hybrid and Software Defined Data Infrastructures (SDDI)

At the core of cloud (public, private, hybrid) next generation data centers are software defined data infrastructures that exist to protect, preserve and serve applications, data along with their resulting information services. Software defined data infrastructure core components include hardware, software servers and storage configured (defined) to provide various services enabling application Performance Availability Capacity and Economics (PACE). Just as there are different types of environments, applications along with workloads various options, technologies as well as techniques exist for cloud services (and underlying data infrastructures). Join us in this session to discuss trends, technologies, tools, techniques and services options for cloud infrastructures. Topics include:

– Data Infrastructures exist to support applications and their underlying resource needs
– Software Defined Infrastructures (SDDI) are what enable Software Defined Data Centers and clouds
– Various types of clouds along with cloud services that determine how resources get defined
– When, where, why and how to use cloud Infrastructures along with associated resources

October 27, 2016 – Webinar 10AM PT – Virtual Instruments
The Value of Infrastructure Insight

This webinar looks at the value of data center infrastructure insight both as a technology as well as a business productivity enabler. Besides productivity, having insight into how data infrastructure resources (servers, storage, networks, system software) are used, enables informed analysis, troubleshooting, planning, forecasting as well as cost-effective decision-making. In other words, data center infrastructure insight, based on infrastructure performance analytics, enables you to avoid flying blind, having situational awareness for proactive Information Technology (IT) management. Your return on innovation is increased, and leveraging insight awareness along with metrics that matter drives return on investment (ROI) along with enhanced service delivery.

October 20, 2016 – Webinar 9AM PT – BrightTalk
Next-Gen Data Centers Software Defined Data Infrastructures (SDDI) including Servers, Storage and Virtualizations

At the core of next generation data centers are software defined data infrastructures that enable, protect, preserve and serve applications, data along with their resulting information services. Software defined data infrastructure core components include hardware, software servers and storage configured (defined) to provide various services enabling application Performance Availability Capacity and Economics (PACE). Just as there are different types of environments, applications along with workloads various options, technologies as well as techniques exist for virtual servers and storage. Join us in this session to discuss trends, technologies, tools, techniques and services around storage and virtualization for today, tomorrow, and in the years to come. Topics include:

– Data Infrastructures exist to support applications and their underlying resource needs
– Software Defined Infrastructures (SDDI) are what enable Software Defined Data Centers
– Server and Storage Virtualization better together, with and without CI/HCI
– Many different facets (types) of Server virtualization and virtual storage
– When, where, why and how to use storage virtualization and virtual storage

September 20, 2016 – Webinar 8AM PT – BrightTalk
Software Defined Data Infrastructures (SDDI) Enabling Software Defined Data Centers – Part of Software-Defined Storage summit

Data Infrastructures exist to support applications and their underlying resource needs. Software-Defined Infrastructures (SDI) are what enable Software-Defined Data Centers, and at the heart of a SDI is storage that is software-defined. This spans cloud, virtual and physical storage and is at the focal point of today. Join us in this session to discuss trends, technologies, tools, techniques and services around SDI and SDDC- today, tomorrow, and in the years to come.

September 13, 2016 – Webinar 11AM PT – Redmond Magazine and
Dell Software
Windows Server 2016 and Active Directory
Whats New and How to Plan for Migration

Windows Server 2016 is expected to GA this fall and is a modernized version of the Microsoft operating system that includes new capabilities such as Active Directory (AD) enhancements. AD is critical to organizational operations providing control and secure access to data, networks, servers, storage and more from physical, virtual and cloud (public and hybrid). But over time, organizations along with their associated IT infrastructures have evolved due to mergers, acquisitions, restructuring and general growth. As a result, yesterday’s AD deployments may look like they did in the past while using new technology (e.g. in old ways). Now is the time to start planning for how you will optimize your AD environment using new tools and technologies such as those in Windows Server 2016 and AD in new ways. Optimizing AD means having a new design, performing cleanup and restructuring prior to migration vs. simply moving what you have. Join us for this interactive webinar to begin planning your journey to Windows Server 2016 and a new optimized AD deployment that is flexible, scalable and elastic, and enables resilient infrastructures. You will learn:

  • What’s new in Windows Server 2016 and how it impacts your AD
  • Why an optimized AD is critical for IT environments moving forward
  • How to gain insight into your current AD environment
  • AD restructuring planning considerations

September 8, 2016 – Webinar 11AM PT (Watch on Demand) – Redmond Magazine, Acronis and Unitrends
Data Protection for Modern Microsoft Environments

Your organization’s business depends on modern Microsoft® environments — Microsoft Azure and new versions of Windows Server 2016, Microsoft Hyper-V with RCT, and business applications — and you need a data protection solution that keeps pace with Microsoft technologies. If you lose mission-critical data, it can cost you $100,000 or more for a single hour of downtime. Join our webinar and learn how different data protection solutions can protect your Microsoft environment, whether you store data on company premises, at remote locations, in private and public clouds, and on mobile devices.

Where To Learn More

What This All Means

Its back to school and learning time, join me on these and other upcoming event activities.

Ok, nuff said, for now…

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved

Posted on

Like an Eagle Standing Tall and Proud, 9/11 2001 Never Forget

Like an Eagle Standing Tall and Proud, 9/11 2001 Never Forget

eagle standing tall server storage I/O trends

Updated 1/28/2018

A Eagle Standing Tall and Proud

Like an Eagle Standing Tall and Proud

On this 15th anniversary of the September 11 (e.g. 9/11), 2001 terror attacks on the United States, instead of a lot of words, photos, videos, links and other items, standing tall and proud, never forget, always remember that fateful day (along with December 7, 1941) among others.

My wife took the following photo (the flag was added later) of two adult bald eagles next to our backyard on a windy evening a few weeks ago. Something about the photo caught my eye, perhaps its the focus and look of the eagles or how they are standing tall together having each others backs.

A picture is worth a hundred or thousands of words, photos, videos or links

Standing Tall and Proud

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

Never forget, always remember, like the bald eagles in the photo united standing tall and proud.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Server StorageIO August 2016 Update Newsletter

Volume 16, Issue VIII

Hello and welcome to this August 2016 Server StorageIO update newsletter.

In This Issue

  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts
  • Events and Webinars
  • Industry Activity Trends
  • Resources and Links

    • Enjoy this shortened summer edition of the Server StorageIO update newsletter.

    Cheers GS

    Industry Activity Trends

    With VMworld coming up this week, rest assured, there will be plenty to talk about and discuss in the following weeks. However for now, here are a few things from this past month.

    At Flash Memory Summit (FMS) which is more of a component, vendor to vendor industry type event, there was buzz about analytics, however what was shown as analytics tended to be Iometer. Hmmm, more on that in a future post. However something else at FMS besides variations of Non-Volatile Memory (NVM) including SSD, NAND, Flash, Storage Class Memory (SCM) such as 3D XPoint (among its new marketing names) along with NVM Expres (NVMe) was NVMe over Fabric.

    This includes NVMe over RoCE (RDMA over Converged Ethernet) which can be implemented on some 10 Gb (and faster) Ethernet adapters as well as some InfiniBand adapters from Mellanox among others. Another variation is Fibre Channel NVMe (FC-NVMe) where the NVMe protocol command set is transported as a Upper Level Protocol (ULP) over FC. This is similar to how the SCSI command set is implemented on FC (e.g. SCSI_FCP or FCP) which means NVMe can be seen as a competing protocol to FCP (which it will or could be). Naturally not to be left out, some of the marketers have already started with Persistent Memory over Fabric among other variations of Non- Ephemeral Memory over Fabrics. More on NVM, NVMe and fabrics in future posts, commentary and newsletter.

    Some other buzzword topics regaining mention (or perhaps for the first time for some) includes
    FedRAMP, Authority To Operate (ATO) clouds for Government entities, and FISMA among others. Many service providers, cloud and hosting providers from large AWS and Azure to smaller Blackmesh have added FedRAMP and other options in addition to traditional, DevOps.

    Some of you may recall me mentioning ScaleMP in the past which is a technology for aggregating multiple compute servers including processors and memory into a converged resource pool. Think the opposite of a hypervisor that divides up resources to support consolidation. In other words, where you need to scale up without complexity of clustering or to avoid having to change and partition your software applications. In addition to ScaleMP, a newer hardware agnostic startup to check out is Tidal Scale.

    On the merger and acquisition front, the Dell / EMC deal is moving forward expected to close soon, perhaps by time or before you read this. In other news, HPE announced that it is buying SGI to gain access to a larger part of the traditional legacy big data Super Compute and High Performance Compute (HPC) market. One of the SGI diamonds in the rough that if you are not aware, is DMF for data management. HPE and Dropbox also announced a partnership deal earlier this summer.

    That’s all for now, time to pack my bags and off to Las Vegas for VMworld 2016.

    Ok, nuff said, for now…

     

    StorageIOblog Posts

    Recent and popular Server StorageIOblog posts include:

    View other recent as well as past StorageIOblog posts here

     

    StorageIO Commentary in the news

    Recent Server StorageIO industry trends perspectives commentary in the news.

    Via FutureReadyOEM Q&A: when to implement ultra-dense storage
    Via EnterpriseStorageForum Comments on Top 10 Enterprise SSD Market Trends
    Via SearchStorage Comments on NAS system buying decisions
    Via EnterpriseStorageForum Comments on Cloud Storage Pros and Cons
    EnterpriseStorageForum Comments on Top 10 Enterprise SSD Market Trends

    View more Server, Storage and I/O hardware as well as software trends comments here

     

    StorageIO Tips and Articles

    Recent and past Server StorageIO articles appearing in different venues include:

    Via Iron Mountain Preventing Unexpected Disasters: IT and Data Infrastructure
    Via FutureReadyOEM Q&A: When to implement ultra-dense storage servers
    Via Micron Blog Whats next for NVMe and your Data Center – Preparing for Tomorrow
    Redmond Magazine: Trends – Evolving from Data Protection to Data Resiliency
    IronMountain: 5 Noteworthy Data Privacy Trends From 2015
    InfoStor: Data Protection Gaps, Some Good, Some Not So Good
    Virtual Blocks (VMware Blogs): EVO:RAIL ? When And Where To Use It?

    Check out these resources techniques, trends as well as tools. View more tips and articles here

    StorageIO Webinars and Industry Events

    December 7: BrightTalk Webinar – Hyper-Converged Infrastructure (HCI) Webinar 11AM PT

    November 23: BrightTalk Webinar – BCDR and Cloud Backup – Software Defined Data Infrastructures (SDDI) and Data Protection – 10AM PT

    November 23: BrightTalk Webinar – Cloud Storage – Hybrid and Software Defined Data Infrastructures (SDDI) – 9AM PT

    November 22: BrightTalk Webinar – Cloud Infrastructure – Hybrid and Software Defined Data Infrastructures (SDDI) – 10AM PT

    October 20: BrightTalk Webinar – Next-Gen Data Centers – Software Defined Data Infrastructures (SDDI) including Servers, Storage and Virtualizations – 9AM PT

    September 29: TBA Webinar – 10AM PT

    September 27-28 – NetApp – Las Vegas

    September 20: BrightTalk Webinar – Software Defined Data Infrastructures (SDDI) Enabling Software Defined Data Centers – Part of Software-Defined Storage summit – 8AM PT

    September 13: Redmond Webinar – Windows Server 2016 and Active Directory What’s New and How to Plan for Migration – 11AM PT

    September 8: Redmond Webinar – Data Protection for Modern Microsoft Environments – 11AM PT

    August 29-31: VMworld Las Vegas

    August 25 – MSP CMG – The Answer is Software Defined – What was the question?

    August 16: BrightTalk Webinar Software Defined Data Centers (SDDC) are in your future (if not already) – Part of Enterprise Software and Infrastructure summit 8AM PT

    August 10-11 Flash Memory Summit (Panel discussion August 11th) – NVMe over Fabric

    See more webinars and other activities on the Server StorageIO Events page here.

     

    Server StorageIO Industry Resources and Links

    Check out these useful links and pages:

    storageio.com/links – Various industry links (over 1,000 with more to be added soon)
    objectstoragecenter.com – Cloud and object storage topics, tips and news items
    storageioblog.com/data-protection-diaries-main/ – Various data protection items and topics
    thenvmeplace.com – Focus on NVMe trends and technologies
    thessdplace.com – NVM and Solid State Disk topics, tips and techniques
    storageio.com/performance.com – Various server, storage and I/O performance and benchmarking

    Ok, nuff said

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Happy 20th Birthday Microsoft Windows Server, get ready for Windows Server 2016

    Happy 20th Birthday Windows Server, ready for Server 2016?

    server storage I/O trends

    In case you have not heard, Microsoft is celebrating the 20th birthday (or anniversary) of Windows Server.

    Microsoft has a nice site with info graphics and timelines of where Windows Server has been and accomplished over the past 20 years (view here).

    Some of you may remember from 20 years ago Windows Server with a different name aka Windows NT Server. Back in the day, if you recall (or read), server requirements were more in the 33 MHz vs. 3.3GHz range, 32MB of RAM Memory vs. 32GB to 320GB, 150MB HDD vs. 150GB SSD or 1.5TB HDD.

    Keep in mind that 20 years ago Linux was a relative new thing with Red Hat not yet quite household or more specific enterprise name. The various Unix (e.g. IBM AIX, HP HP-UX, Sun Solaris, DEC Unix and Ultrix among many others) were still dominate, OS2 had peaked or close to, among others. Virtual Machines were Logical Partitions (LPAR) on Mainframes along with virtual PCs software and hardware assist boards.

    IMHO there is no coincidence of Microsoft celebrating 20 years of WIndows Server going into the fall of 2016 and the upcoming release of Server 2016.

    What’s New in Server 2016 (TP5)?

    If you have not done so, check out the latest Tech Preview 5 (TP5) of Windows Server 2016 (get the bits e.g. software here to try) which includes Storage Spaces Direct (S2D) that leverages internal PCIe and drive format SSD (NVMe, SAS, SATA) along with HDDs (SAS, SATA) for creating local and scale-out converged (desegregated) and hyper-converged (aggregated) solutions. In addition to S2D there is Storage Replica (SR) which is replication of local storage part of S2D (not to be confused with DFS or other replication).

    Other enhancements include ReFS as the default file system instead of NTFS (don’t worry, NTFS like FAT does not go away yet). There are enhancements to Hyper-V including VM shielding, hot-plug virtual network adapters, enhanced Linux support and fail over priorities among others. Other enhancements include updates for AD including improved integration with on-premises as well as Azure AD for hybrid environments, PowerShell updates, Docker management including Linux (via Hyper-V) and Windows via Nano) container engines.

    Microsoft Windows Server 2016 TP5

    Speaking of Nano, if you had not heard, this is a new very light weight Windows Kernel that removes 32 bit WOW and GUI support. The result is that Nano is a very small physical (under 1GB image instance size) using less disk, less memory and less CPU to do a given amount of work, oh, and boots super fast, even without SSD. By not having all the 32 bit and GUI overhead, the intent with Nano is there should be fewer updates and maintenance tasks to do, while enabling Windows containers for SQL Server and other applications.

    In addition to PowerShell, AD and other management enhancements, Windows Server 2016 (TP5) also enables bridging two worlds e.g. traditional on-premises (or cloud) based Windows Server and Public Cloud (e.g. Azure) and Private or Hybrid including Azure Stack. Note that if you have not heard of Azure Stack and are looking at cloud stacks such as OpenStack, do your due diligence and at least familiarize yourself with Azure Stack.

    View more about WIndows 2016 TP5 enhancements here.

    Where To Learn More

    What This All Means

    Congratulations Microsoft and Windows Server on 20th birthday (anniversary) you have come a long way.

    With the new features and functionality in Windows Server 2016, looks like there is still a good future for the software defined server.

    Ok, nuff said, for now…

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved

    Posted on

    Get in the NVMe SSD game (if you are not already) with Intel 750 among others

    Get in the NVMe SSD game (if you are not already)

    Get in the NVMe SSD game. If you are or have talked the Non Volatile Memory (NVM) and NVM Express (NVMe) flash SSD talk, however have not yet done the walk, there are some good opportunities to do so.

    One example how to start walking the talk is get something like the Intel 750 400GB NVMe PCIe Add in Card (AiC) which you should be able to find on different venues including Amazon for $400 USD or less (recently saw the 400GB version for about $309 USD delivered).

    Of course the question might then be what to do with a NVMe device when you get it which besides running your applications on it, you can find some benchmark and workload generator tools, scripts and examples here.

    NVMe ssd storage
    Various NVM flash SSD including NVMe devices

    Sure there are newer, more expensive, higher capacity cards, some are general availability (GA) while others are on early preview or evaluation basis.

    However a good value, cost-effective way to get into the NVMe game, or expand your footprint are devices such as the Intel 750.

    I have some of these cards and they have been great for use with various workloads. The Intel 750 cards are plug and play in various systems including my TS140s among larger robust servers running Windows Server (2012 R2, 2016 TP5), Ubuntu (various) and VMware ESXi (various).

    There are in the box drivers for these Intel 750 cards, however to get a little more performance recommend visiting the Intel site, get the SSD tools, update the firmware on the cards if needed, as well as try the Intel vs. native drivers.

    There are many vendors with NVMe PCIe cards, as well as U.2/SFF 8639 drives and M.2 mini-cards including Intel, Micron, Seagate, WD among many others

    Where To Learn More

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    What This All Means

    Time to get in the NVMe game if you have not already done so moving from talking the talk to walking the talk. With cost of early generation NVMe devices such as the Intel 750 among others there are great opportunities to get started. On the other hand if you are already using NVMe, there are some good opportunities to add extra NVMe based SSD to other servers or systems.

    Speaking of NVMe talk, join me and others on a panel (Session 303) moderated by Dr. Jay Metz (@drjmetz) August 11th at Flash Memory Summit where we will discuss NVMe including NVMe over fabric networks (e.g. moving beyond the PCIe bus). Also keep in mind, with NVMe the question is not if, rather when, where, why, how with what and how to coexist and leverage existing technologies, hardware, software and people skill sets.

    Ok, nuff said, for now.

    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

    Posted on

    Server storage I/O performance benchmark workload scripts Part I

    Server storage I/O performance benchmark workload scripts Part I

    Server storage I/O performance benchmark workload scripts

    Update 1/28/2018

    This is part one of a two-part series of posts about Server storage I/O performance benchmark workload tools and scripts. View part II here which includes the workload scripts and where to view sample results.

    There are various tools and workloads for server I/O benchmark testing, validation and exercising different storage devices (or systems and appliances) such as Non-Volatile Memory (NVM) flash Solid State Devices (SSDs) or Hard Disk Drives (HDD) among others.

    NVMe ssd storage
    Various NVM flash SSD including NVMe devices

    For example, lets say you have an SSD such as an Intel 750 (here, here, and here) or some other vendors NVMe PCIe Add in Card (AiC) installed into a Microsoft Windows server and would like to see how it compares with expected results. The following scripts allow you to validate your system with those of others running the same workload, granted of course your mileage (performance) may vary.

    server storage I/O SCM NVM SSD performance

    Why Your Performance May Vary

    Reasons you performance may vary include among others:

    • GHz Speed of your server, number of sockets, cores
    • Amount of main DRAM memory
    • Number, type and speed of PCIe slots
    • Speed of storage device and any adapters
    • Device drivers and firmware of storage devices and adapters
    • Server power mode setting (e.g. low or balanced power vs. high-performance)
    • Other workload running on system and device under test
    • Solar flares (kp-index) among other urban (or real) myths and issues
    • Typos or misconfiguration of workload test scripts
    • Test server, storage, I/O device, software and workload configuration
    • Versions of test software tools among others

    Windows Power (and performance) Settings

    Some things are assumed or taken for granted that everybody knows and does, however sometimes the obvious needs to be stated or re-stated. An example is remembering to check your server power management settings to see if they are in energy efficiency power savings mode, or, in high-performance mode. Note that if your focus is on getting the best possible performance for effective productivity, then you want to be in high performance mode. On the other hand if performance is not your main concern, instead a focus on energy avoidance, then low power mode, or perhaps balanced.

    For Microsoft Windows Servers, Desktop Workstations, Laptops and Tablets you can adjust power settings via control panel and GUI as well as command line or Powershell. From command line (privileged or administrator) the following are used for setting balanced or high-performance power settings.

    Balanced

    powercfg.exe /setactive 381b4222-f694-41f0-9685-ff5bb260df2e

    High Performance

    powercfg.exe /setactive 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c

    From Powershell the following set balanced or high-performance.

    Balanced
    PowerCfg -SetActive "381b4222-f694-41f0-9685-ff5bb260df2e"

    High Performance
    PowerCfg -SetActive "8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c"

    Note that you can list Windows power management settings using powercfg -LIST and powercfg -QUERY

    server storage I/O power management

    Btw, if you have not already done so, enable Windows disk (HDD and SSD) performance counters so that they appear via Task Manager by entering from a command prompt:

    diskperf -y

    Workload (Benchmark) Simulation Test Tools Used

    There are many tools (see storageio.com/performance) that can be used for creating and running workloads just as there are various application server I/O characteristics. Different server I/O and application performance attributes include among others read vs. write, random vs. sequential, large vs. small, long vs. short stride, burst vs. sustain, cache and non-cache friendly, activity vs. data movement vs. latency vs. CPU usage among others. Likewise the number of workers, jobs, threads, outstanding and overlapped I/O among other configuration settings can have an impact on workload and results.

    The four free tools that I’m using with this set of scripts are:

    • Microsoft Diskspd (free), get the tool and bits here or here (open source), learn more about Diskspd here.
    • FIO.exe (free), get the tool and bits here or here among other venues.
    • Vdbench (free with registration), get the tool and bits here or here among other venues.
    • Iometer (free), get the tool and bits here among other venues.

    Notice: While best effort has been made to verify the above links, they may change over time and you are responsible for verifying the safety of links and your downloads.

    Where To Learn More

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    What This All Means

    Remember, everything is not the same in the data center or with data infrastructures that support different applications.

    While some tools are more robust or better than others for different things, ultimately it’s usually not the tool that results in a bad benchmark or comparison, it’s the configuration or lack of including workload settings that are not relevant or applicable. The best benchmark, workload or simulation is your own application. Second best is one that closely resembles your application workload characteristics. A bad benchmark is one that has no relevance to your environment, application use scenario. Take and treat all benchmark or workload simulation results with a grain of salt as something to compare, contrast or make reference to in the proper context. Read part two of this post series to view test tool workload scripts along with sample results.

    Ok, nuff said, for now.

    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

    Posted on

    Part II – Some server storage I/O workload scripts and results

    Part II – Some server storage I/O workload scripts and results

    server storage I/O trends

    Updated 1/28/2018

    This is the second in a two part series of posts pertaining to using some common server storage I/O workload benchmark tools and scripts. View part I here which includes overview, background and information about the tools used and related topics.

    NVMe ssd storage
    Various NVM flash SSD including NVMe devices

    Following are some server I/O benchmark workload scripts to exercise various storage devices such as Non-Volatile Memory (NVM) flash Solid State Devices (SSDs) or Hard Disk Drives (HDD) among others.

    The Workloads

    Some ways that can impact the workload performance results besides changing the I/O size, read write, random sequential mix is the number of threads, workers and jobs. Note that in the workload steps, the larger 1MB and sequential scenarios have fewer threads, workers vs. the smaller IOP or activity focused workloads. Too many threads or workers can cause overhead and you will reach a point of diminishing return at some point. Likewise too few and you will not drive the system under test (SUT) or device under test (DUT) to its full potential. If you are not sure how many threads or workers to use, run some short calibration tests to see the results before doing a large, longer test.

    Keep in mind that the best benchmark or workload is your own application running with similar load to what you would see in real world, along with applicable features, configuration and functionality enabled. The second best would be those that closely resemble your workload characteristics and that are relevant.

    The following workloads involved a system test initiator (STI) server driving workload using the different tools as well as scripts shown. The STI sends the workload to a SUT or DUT that can be a single drive, card or multiple devices, storage system or appliance. Warning: The following workload tests does both reads and writes which can be destructive to your device under test. Exercise caution on the device and file name specified to avoid causing a problem that might result in you testing your backup / recovery process. Likewise no warranty is given, implied or made for these scripts or their use or results, they are simply supplied as is for your reference.

    The four free tools that I’m using with this set of scripts are:

    • Microsoft Diskspd (free), get the tool and bits here or here (open source), learn more about Diskspd here.
    • FIO.exe (free), get the tool and bits here or here among other venues.
    • Vdbench (free with registration), get the tool and bits here or here among other venues.
    • Iometer (free), get the tool and bits here among other venues.

    Notice: While best effort has been made to verify the above links, they may change over time and you are responsible for verifying the safety of links and your downloads

    Microsoft Diskspd workloads

    Note that a 300GB size file named iobw.tst on device N: is being used for performing read and write I/Os to. There are 160 threads, I/O size of 4KB and 8KB varying from 100% Read (0% write), 70% Read (30% write) and 0% Read (100% write) with random (seek) and no hardware or software cache. Also specified are to collect latency statistics, a 30 second warm up ramp up time, and a quick 5 minute duration (test time). 5 minutes is a quick test for calibration, verify your environment however relatively short for a real test which should be in the hours or more depending on your needs.

    Note that the output results are put into a file with a name describing the test tool, workload and other useful information such as date and time. You may also want to specify a different directory where output files are placed.

    diskspd.exe -c300G -o160 -t160 -b4K -w0 -W30 -d300 -h -fr  N:iobw.tst -L  > DiskSPD_300G_4KRan100Read_160x160_072416_8AM.txt
diskspd.exe -c300G -o160 -t160 -b4K -w30 -W30 -d300 -h -fr  N:iobw.tst -L  > DiskSPD_300G_4KRan70Read_160x160_072416_8AM.txt
diskspd.exe -c300G -o160 -t160 -b4K -w100 -W30 -d300 -h -fr  N:iobw.tst -L  > DiskSPD_300G_4KRan0Read_160x160_072416_8AM.txt
diskspd.exe -c300G -o160 -t160 -b8K -w0 -W30 -d300 -h -fr  N:iobw.tst -L  > DiskSPD_300G_8KRan100Read_160x160_072416_8AM.txt
diskspd.exe -c300G -o160 -t160 -b8K -w30 -W30 -d300 -h -fr  N:iobw.tst -L  > DiskSPD_300G_8KRan70Read_160x160_072416_8AM.txt
diskspd.exe -c300G -o160 -t160 -b8K -w100 -W30 -d300 -h -fr  N:iobw.tst -L  > DiskSPD_300G_8KRan0Read_160x160_072416_8AM.txt

    The following Diskspd tests use similar settings as above, however instead of random, sequential is specified, threads and outstanding I/Os are reduced while I/O size is set to 1MB, then 8KB, with 100% read and 100% write scenarios. The -t specifies the number of threads and -o number of outstanding I/Os per thread.

    diskspd.exe -c300G -o32 -t132 -b1M -w0 -W30 -d300 -h -si  N:iobw.tst -L  > DiskSPD_300G_1MSeq100Read_32x32_072416_8AM.txt
diskspd.exe -c300G -o32 -t132 -b1M -w100 -W30 -d300 -h -si  N:iobw.tst -L  > DiskSPD_300G_1MSeq0Read_32x32_072416_8AM.txt
diskspd.exe -c300G -o160 -t160 -b8K -w0 -W30 -d300 -h -si  N:iobw.tst -L  > DiskSPD_300G_8KSeq100Read_32x32_072416_8AM.txt
diskspd.exe -c300G -o160 -t160 -b8K -w100 -W30 -d300 -h -si  N:iobw.tst -L  > DiskSPD_300G_8KSeq0Read_32x32_072416_8AM.txt

    Fio.exe workloads

    Next are the fio workloads similar to those run using Diskspd except the sequential scenarios are skipped.

    fio --filename=N\:\iobw.tst --filesize=300000M --direct=1  --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=windowsaio  --ba=4k --bs=4k --rwmixread=100 --iodepth=32 --numjobs=5 --exitall --time_based  --ramp_time=30 --runtime=300 --group_reporting --name=xxx  --output=FIO_300000M_4KRan100Read_5x32_072416_8AM.txt
fio --filename=N\:\iobw.tst --filesize=300000M --direct=1  --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=windowsaio  --ba=4k --bs=4k --rwmixread=70 --iodepth=32 --numjobs=5 --exitall --time_based  --ramp_time=30 --runtime=300 --group_reporting --name=xxx  --output=FIO_300000M_4KRan70Read_5x32_072416_8AM.txt
fio --filename=N\:\iobw.tst --filesize=300000M --direct=1  --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=windowsaio  --ba=4k --bs=4k --rwmixread=0 --iodepth=32 --numjobs=5 --exitall --time_based  --ramp_time=30 --runtime=300 --group_reporting --name=xxx  --output=FIO_300000M_4KRan0Read_5x32_072416_8AM.txt
fio --filename=N\:\iobw.tst --filesize=300000M --direct=1  --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=windowsaio  --ba=8k --bs=8k --rwmixread=100 --iodepth=32 --numjobs=5 --exitall --time_based  --ramp_time=30 --runtime=300 --group_reporting --name=xxx  --output=FIO_300000M_8KRan100Read_5x32_072416_8AM.txt
fio --filename=N\:\iobw.tst --filesize=300000M --direct=1  --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=windowsaio  --ba=8k --bs=8k --rwmixread=70 --iodepth=32 --numjobs=5 --exitall --time_based  --ramp_time=30 --runtime=300 --group_reporting --name=xxx  --output=FIO_300000M_8KRan70Read_5x32_072416_8AM.txt
fio --filename=N\:\iobw.tst --filesize=300000M --direct=1  --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=windowsaio  --ba=8k --bs=8k --rwmixread=0 --iodepth=32 --numjobs=5 --exitall --time_based  --ramp_time=30 --runtime=300 --group_reporting --name=xxx  --output=FIO_300000M_8KRan0Read_5x32_072416_8AM.txt

    Vdbench workloads

    Next are the Vdbench workloads similar to those used with the Microsoft Diskspd scenarios. In addition to making sure Vdbench is installed and working, you will need to create a text file called seqrxx.txt containing the following:

    hd=localhost,jvms=!jvmn
    
sd=sd1,lun=!drivename,openflags=directio,size=!dsize
    
wd=mix,sd=sd1
    
rd=!jobname,wd=mix,elapsed=!etime,interval=!itime,iorate=max,forthreads=(!tthreads),forxfersize=(!worktbd),forseekpct=(!workseek),forrdpct=(!workread),openflags=directio

    The following are the commands that call the Vdbench script file. Note Vdbench puts output files (yes, plural there are many results) in a output folder.

    vdbench -f seqrxx.txt dsize=300G  tthreads=160 jvmn=64 worktbd=4k workseek=100 workread=100 jobname=NVME etime=300 itime=30 drivename="\\.\N:\iobw.tst" -o  vdbench_NNVMe_300GB_64JVM_160TH_4K100Ran100Read_0726166AM
vdbench -f seqrxx.txt dsize=300G  tthreads=160 jvmn=64 worktbd=4k workseek=100 workread=70 jobname=NVME etime=300 itime=30 drivename="\\.\N:\iobw.tst" -o vdbench_NNVMe_300GB_64JVM_160TH_4K100Ran70Read_072416_8AM
vdbench -f seqrxx.txt dsize=300G  tthreads=160 jvmn=64 worktbd=4k workseek=100 workread=0 jobname=NVME etime=300 itime=30 drivename="\\.\N:\iobw.tst" -o vdbench_NNVMe_300GB_64JVM_160TH_4K100Ran0Read_072416_8AM
vdbench -f seqrxx.txt dsize=300G  tthreads=160 jvmn=64 worktbd=8k workseek=100 workread=100 jobname=NVME etime=300 itime=30 drivename="\\.\N:\iobw.tst" -o vdbench_NNVMe_300GB_64JVM_160TH_8K100Ran100Read_072416_8AM
vdbench -f seqrxx.txt dsize=300G  tthreads=160 jvmn=64 worktbd=8k workseek=100 workread=70 jobname=NVME etime=300 itime=30 drivename="\\.\N:\iobw.tst" -o vdbench_NNVMe_300GB_64JVM_160TH_8K100Ran70Read_072416_8AM
vdbench -f seqrxx.txt dsize=300G  tthreads=160 jvmn=64 worktbd=8k workseek=100 workread=0 jobname=NVME etime=300 itime=30 drivename="\\.\N:\iobw.tst" -o vdbench_NNVMe_300GB_64JVM_160TH_8K100Seq0Read_072416_8AM
vdbench -f seqrxx.txt dsize=300G  tthreads=160 jvmn=64 worktbd=8k workseek=0 workread=100 jobname=NVME etime=300 itime=30 drivename="\\.\N:\iobw.tst" -o vdbench_NNVMe_300GB_64JVM_160TH_8K100Seq100Read_072416_8AM
vdbench -f seqrxx.txt dsize=300G  tthreads=160 jvmn=64 worktbd=8k workseek=0 workread=70 jobname=NVME etime=300 itime=30 drivename="\\.\N:\iobw.tst" -o vdbench_NNVMe_300GB_64JVM_160TH_8K100Seq70Read_072416_8AM
vdbench -f seqrxx.txt dsize=300G  tthreads=160 jvmn=64 worktbd=8k workseek=0 workread=0 jobname=NVME etime=300 itime=30 drivename="\\.\N:\iobw.tst" -o vdbench_NNVMe_300GB_64JVM_160TH_8K100Seq0Read_072416_8AM
vdbench -f seqrxx.txt dsize=300G  tthreads=32 jvmn=64 worktbd=1M workseek=0 workread=100 jobname=NVME etime=300 itime=30 drivename="\\.\N:\iobw.tst" -o vdbench_NNVMe_300GB_64JVM_32TH_1M100Seq100Read_072416_8AM
vdbench -f seqrxx.txt dsize=300G  tthreads=32 jvmn=64 worktbd=1M workseek=0 workread=0 jobname=NVME etime=300 itime=30 drivename="\\.\N:\iobw.tst" -o vdbench_NNVMe_300GB_64JVM_32TH_1M100Seq0Read_072416_8AM

    Iometer workloads

    Last however not least, lets do an Iometer run. The following command calls an Iometer input file (icf) that you can find here. In that file you will need to make a few changes including the name of the server where Iometer is running, description and device under test address. For example in the icf file change SIOSERVER to the name of the server where you will be running Iometer from. Also change the address for the DUT, for example N: to what ever address, drive, mount point you are using. Also update the description accordingly (e.g. "NVME" to "Your test example".

    Here is the command line to run Iometer specifying an icf and where to put the results in a CSV file that can be imported into Excel or other tools.

    iometer /c  iometer_5work32q_intel_Profile.icf /r iometer_nvmetest_5work32q_072416_8AM.csv

    server storage I/O SCM NVM SSD performance

    What About The Results?

    For context, the following results were run on a Lenovo TS140 (32GB RAM), single socket quad core (3.2GHz) Intel E3-1225 v3 with an Intel NVMe 750 PCIe AiC (Intel SSDPEDMW40). Out of the box Microsoft Windows NVMe drive and controller drivers were used (e.g. 6.3.9600.18203 and 6.3.9600.16421). Operating system is Windows 2012 R2 (bare metal) with NVMe PCIe card formatted with ReFS file system. Workload generator and benchmark driver tools included Microsoft Diskspd version 2.012, Fio.exe version 2.2.3, Vdbench 50403 and Iometer 1.1.0. Note that there are newer versions of the various workload generation tools.

    Example results are located here.

    Where To Learn More

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    What This All Means

    Remember, everything is not the same in the data center or with data infrastructures that support different applications.

    While some tools are more robust or better than others for different things, ultimately its usually not the tool that results in a bad benchmark or comparison, its the configuration or lack of including workload settings that are not relevant or applicable. The best benchmark, workload or simulation is your own application. Second best is one that closely resembles your application workload characteristics. A bad benchmark is one that has no relevance to your environment, application use scenario. Take and treat all benchmark or workload simulation results with a grain of salt as something to compare, contrast or make reference to in the proper context.

    Ok, nuff said, for now.

    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

    Posted on

    12Gb SAS SSD Enabling Server Storage I/O Performance and Effectiveness Webinar

    12Gb SAS SSD Enabling Server Storage I/O Performance and Effectiveness Webinar

    server storage I/O trends

    Non-Volatile Memory (NVM) Solid State Devices (SSDs) including nand flash, DRAM as well as emerging PCM and 3D XPoint as part of Storage Class Memories (SCMs) are in your future. The questions are where, when, for what, how much as well as what form factor packaging, along with server storage I/O interface are applicable for your different applications and data infrastructures.

    server storage I/O SCM NVM SSD performance

    Server storage I/O physical interfaces for access NVM SSDs include PCIe Add in Cards (AiC), M.2 as well as emerging SFF 8639 (e.g. NVMe U2 drive form factor) along with mSATA (e.g. mini PCIe card) in addition to SAS, SATA, USB among others. Protocols include NVM Express (NVMe), SAS, SATA as well as general server storage I/O access of shared storage systems that leverage NVM SSD and SCM technologies.

    To help address the question of which server storage I/O interface is applicable for different environments, I invite you to a webinar on June 22, 2016 at 1PM ET hosted by and compliments of Micron.

    During the webinar myself and Rob Peglarr (@peglarr) of Micron will discuss and answer questions about how 12Gb SAS remains a viable option for attach NVM SSD storage to servers, as well as via storage systems today and into the future. Today’s 12Gb SAS SSDs enable you to leverage your existing knowledge, skill sets, as well as technology to maximize your data infrastructure investments. For servers or storage systems that are PCIe slot constrained, 12Gb SAS enables more SSD including 2.5" form factor multiple TByte capacity devices to be used to boost performance and capacity in a cost as well as energy effective way.

    server storage I/O nvm ssd options

    In addition to Rob Peglarr, we will also be joined by Doug Rollins of Micron (@GreyHairStorage) who will share some technical speeds, feeds, slots and watts information about Micron 12Gb SAS SSDs that can scale into the TBs in capacity per device.

    Here’s the synopsis from the Micron information page for this webinar.

    Don’t let old, slow SAS HDDs drag down your data center

    Modernize it by upgrading your storage from SAS HDDs to SAS SSDs. It’s an easy upgrade that provides a significant boost in performance, longer lasting endurance and nearly 4X the capacity. Flash storage changes how you do business and keeps you competitive.

    We invite you to join Rob Peglar, Greg Schulz, along with Doug Rollins, from Micron’s technical marketing team to learn:

    • Simple solutions to solving the challenges with today’s ever-growing data demands
    • Why SAS—how it continues to fuel the data center
    • HDDs versus SDDs—before and after stories from your peers, including upfront cost savings

    We will also have a live Q&A session so you can talk with the experts. Please register today! If you’re unable to attend the live webinar, we encourage you to register anyway to receive a link to the recorded session, as well as a copy of the presentation.

    Where To Learn More

    What This All Means

    Remember, everything is not the same in the data center or with data infrastructures that support different applications, like there are various NVM SSD options as well as interfaces.

    Join us for this webinar, you can view more information here, as well as register for the event.

    Ok, nuff said, for now…

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved

    Posted on

    2016 Going Dutch Cloud Virtualization Server Storage I/O Seminars

    2016 Going Dutch Cloud Virtualization Server Storage I/O Seminars

    server storage I/O trends

    In June 2016 Brouwer Storage Consultancy is organizing their yearly spring seminar workshops in Nijkerk Holland (south of Amsterdam, near Utrecht and Amersfoort) with myself among others presenting.

    Brouwer Consultancy

    Cloud Virtualization Server Storage I/O Seminars

    For this series of seminar workshops, there are four sessions, two being presented by myself, and two others in conjunction with Reduxio as well as Fujitsu & SJ Solutions.

    Brouwer and Server StorageIO Seminar Sessions

    Agenda, How To Register and Where To Learn More

    The vendor sponsored sessions will consist of about 50% content being independent presented by myself and Gert Brouwer, the balance by the event sponsors as well as their partners. All presentation and associated content including handouts will be in English.

    There will be 4 seminar workshop sessions, two of those are paid sessions dedicated to Greg Schulz and the other two are free (sponsored) sessions where 50% of the content is sponsored (Reduxio, Fujitsu & SJ Solutions) and the other 50% will be independent (Greg Schulz & Gert Brouwer).

    Thursday June 9th – Server StorageIO Trends and Updates

    Server Storage I/O Fundamental Trends V2.016 and Updates. What’s New, What’s the buzz, what you need to know about. From Speeds and Feeds, Slots and Watts to Who’s doing what. Event Location: Golden Tulip Ampt van Nijkerk Hotel, Berencamperweg 4, 3861MC, Nijkerk. Learn more here (PDF abstract and topics to be covered).

    Thursday June 10th – Converged Day

    Converged Day – Moving beyond Hyper-Converged Hype and Server Storage I/O Decision Making Strategies. Event Location: Golden Tulip Ampt van Nijkerk Hotel, Berencamperweg 4, 3861MC, Nijkerk. Learn more here (PDF abstract and topics to be covered).

    Brouwer and Server StorageIO Seminar Sessions De Roode Schuur

    Tuesday June 14th – Round Table Vendor Session with Reduxio

    Symposium Workshop – Round Table Vendor Session with Reduxio – Are some solutions really ‘a Paradigm shift’ or ‘new and revolutionary” as they claim to be, or is it just more of the same (e.g. evolutionary)? – Presentations and discussions led by Greg Schulz (StorageIO), Reduxio and Brouwer Storage Consultancy. (Free, sponsored Session, Access for end-users only). Event Location: Hotel & Gasterij De Roode Schuur, Oude Barneveldseweg 98, 3862PS Nijkerk. Learn more here (PDF abstract and topics to be covered).

    Wednesday June 15th – Software Defined Data Center Symposium Workshop

    Software Defined Data Center Symposium Workshop – Round Table Vendor Session with Fujitsu & SJ Solutions
    With subjects like Openstack, Ceph, distributed object storage, Bigdata, Hyper-Converged Infrastructure (HCI), Converged Infrastructure (CI), Software defined storage (SDS) and Network (SDN and NFV), this round table format workshop seminars explores these and other related topics including what to use when, where, why and how. Presentations by Greg Schulz (StorageIO), SJ Solutions & Fujitsu and Brouwer Storage Consultancy. Event Location: Hotel & Gasterij De Roode Schuur, Oude Barneveldseweg 98, 3862PS Nijkerk. Learn more here (PDF abstract and topics to be covered).

    For more information, abstracts/agenda, registration and the specific locations for all the above events click here.

    Brouwer and Server StorageIO Sessions Ampt van Nijkerk

    What This All Means

    There is a lot of things occurring in the IT industry from physical to software defined clouds, containers and virtualization, nonvolatile memory (NVM) including flash SSD among others. These series of interactive educational workshop seminars converge on Nijkerk Holland combing content discussions from strategy, planning decision making, to what’s new (and old) that can be used in new ways, as well as some trends, speeds and feeds along with practicality for your environment.

    Brouwer Consultancy

    I Look forward to seeing you in Nijkerk and Europe during June 2016, in the meantime, contact Brouwer Storage Consultancy for more information on the above sessions as well as to arrange private discussions or meetings.

    Ok, nuff said, for now…

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved

    Posted on

    Participate in Top vBlog 2016 Voting Now

    Participate in Top vBlog 2016 Voting Now

    server storage I/O trends

    It’s that time of the year when Eric Siebert (@ericsiebert) hosts his annual top virtual Blog (vBlog) voting via his great vsphere-land site (check it out if you are not familiar). The voting is now open until May 27th after which the results tabulated, will be announced.

    While the focus is virtualization, rest assured there are other categories including scripting, storage, independent, new, video and podcast among others. For example my blog is listed under StorageIO (Greg Schulz) and included in storage and independent among some other categories.

    Granted it is an election year here in the US and hopefully those participating in the top vBlog 2016 voting process are doing so based on content vs. simply popularity or what their virtual Popularity Action Committees (vPAC) tells them to do, that is vPACs actually exist or if they are simply vUrban Myths ;). In other words I’m not going to tell you who to vote for, or who I voted for other than that it is based on useful I found those sites and their content contributions.

    Who Is Eligible To Vote

    Anybody can vote, granted you can only vote once. Of course you can get your friends, family, co-workers, sales and marketing department, community or club, customers, clients, basically anything with an IP address and email address in theory including IoT and IoD could vote. However that would be like buying twitter followers, Facebook likes, click for view or pay for view results to game the system which if that is your game, so be it.

    How Did People Get On The List (Ballot)

    Eric puts out a call (tweets, posts here, here and here) that gets amplified for people to submit new blogs to be included, as well as then to self-nominate their site, as well as for what categories. If people do not take the initiative to get on the list, they don’t get included. If the list if important enough to be included on, then it should be important enough to to know or remember to self-nominate to be included.

    I know this from experience in that a few years ago I forgot to nominate my blog in the categories of storage, independent thus was not included in the voting for those categories. However since I had previously notified Eric to include my blog, it was in the general category and thus included. Note to bloggers, if it is important for you to be included, then notify Eric that you should be added to his lists, as well as take the time to nominate yourself to be included in the future. Simply help others help you.

    What Is The Voting Criteria

    Eric for this years top vBlog voting has culled the list to those who besides self nominating in different categories, also had at least 50 posts in the past year.

    In addition, Eric suggests focus on the content, creative and contribution (Longevity, Length, Frequency, Quality) vs. simply being a popularity contest or driven by virtual Popularity Action Committees (e.g. vPAC).

    Following are my paraphrase:

    • Longevity – How long has the blog existed and continued to be maintained vs. one started a long time ago and had not been updated in months or years.
    • Length – Are there lots of very short basically expanded micro twitter posts, recopy press releases or curation of other news, real content and analysis that requires some thought along with creative. These could be short, long or a series of short to medium size posts.
    • Frequency – How often do posts appear, daily, weekly, monthly, yearly. There’s a balance between frequency, length and content along with time effort to create something.
    • Quality – Some can be rehashed with more perspectives, inputs, hints and tips along with analysis, insight or experiences of existing, or new items. The key is what is the value add to the topic, theme or conversation vs. simply reposting or amplifying what’s already out there. In other words, is there new or unique content, perspectives, thought analysis, insight, experiences or simply repeat and amplify those of others.

    Call To Action, Get Out and Vote

    Simple, get out and vote and thanks in advance by using this link to Eric’s site.

    Where To Learn More

    • Voting now open for Top vBlog 2016
    • Link to actual voting page

    What This All Means

    Support and say thanks, give an "atta boy" or "atta girl" to those who take time to create content to share with you on various virtualization related topics from servers, storage, I/O networking, scripting, tools, techniques, clouds, containers and more via blogs, podcast’s and webinars. This includes both the independents like myself and others as well as the vendors, press and media who give the content you consume.

    So take a few moments to jump on over to Eric’s site and cast your vote and if you have found my content to be useful, I humbly appreciate your vote and say thank you for your support, as well as that for others.

    Ok, nuff said and thank you for supporting StorageIOblog.

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved

    Posted on

    Which Enterprise HDD for Content Server Platform

    Which Enterprise HDD to use for a Content Server Platform

    data infrastructure HDD server storage I/O trends

    Updated 1/23/2018

    Which enterprise HDD to use with a content server platform?

    Insight for effective server storage I/O decision making
    Server StorageIO Lab Review

    Which enterprise HDD to use for content servers

    This post is the first in a multi-part series based on a white paper hands-on lab report I did compliments of Equus Computer Systems and Seagate that you can read in PDF form here. The focus is looking at the Equus Computer Systems (www.equuscs.com) converged Content Solution platforms with Seagate Enterprise Hard Disk Drive (HDD’s). I was given the opportunity to do some hands-on testing running different application workloads with a 2U content solution platform along with various Seagate Enterprise 2.5” HDD’s handle different application workloads. This includes Seagate’s Enterprise Performance HDD’s with the enhanced caching feature.

    Issues And Challenges

    Even though Non-Volatile Memory (NVM) including NAND flash solid state devices (SSDs) have become popular storage for use internal as well as external to servers, there remains the need for HDD’s Like many of you who need to make informed server, storage, I/O hardware, software and configuration selection decisions, time is often in short supply.

    A common industry trend is to use SSD and HDD based storage mediums together in hybrid configurations. Another industry trend is that HDD’s continue to be enhanced with larger space capacity in the same or smaller footprint, as well as with performance improvements. Thus, a common challenge is what type of HDD to use for various content and application workloads balancing performance, availability, capacity and economics.

    Content Applications and Servers

    Fast Content Needs Fast Solutions

    An industry and customer trend are that information and data are getting larger, living longer, as well as there is more of it. This ties to the fundamental theme that applications and their underlying hardware platforms exist to process, move, protect, preserve and serve information.

    Content solutions span from video (4K, HD, SD and legacy streaming video, pre-/post-production, and editing), audio, imaging (photo, seismic, energy, healthcare, etc.) to security surveillance (including Intelligent Video Surveillance [ISV] as well as Intelligence Surveillance and Reconnaissance [ISR]). In addition to big fast data, other content solution applications include content distribution network (CDN) and caching, network function virtualization (NFV) and software-defined network (SDN), to cloud and other rich unstructured big fast media data, analytics along with little data (e.g. SQL and NoSQL database, key-value stores, repositories and meta-data) among others.

    Content Solutions And HDD Opportunities

    A common theme with content solutions is that they get defined with some amount of hardware (compute, memory and storage, I/O networking connectivity) as well as some type of content software. Fast content applications need fast software, multi-core processors (compute), large memory (DRAM, NAND flash, SSD and HDD’s) along with fast server storage I/O network connectivity. Content-based applications benefit from having frequently accessed data as close as possible to the application (e.g. locality of reference).

    Content solution and application servers need flexibility regarding compute options (number of sockets, cores, threads), main memory (DRAM DIMMs), PCIe expansion slots, storage slots and other connectivity. An industry trend is leveraging platforms with multi-socket processors, dozens of cores and threads (e.g. logical processors) to support parallel or high-concurrent content applications. These servers have large amounts of local storage space capacity (NAND flash SSD and HDD) and associated I/O performance (PCIe, NVMe, 40 GbE, 10 GbE, 12 Gbps SAS etc.) in addition to using external shared storage (local and cloud).

    Where To Learn More

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    What This All Means

    Fast content applications need fast content and flexible content solution platforms such as those from Equus Computer Systems and HDD’s from Seagate. Key to a successful content application deployment is having the flexibility to hardware define and software defined the platform to meet your needs. Just as there are many different types of content applications along with diverse environments, content solution platforms need to be flexible, scalable and robust, not to mention cost effective.

    Continue reading part two of this multi-part series here where we look at how and what to test as well as project planning.

    Ok, nuff said, for now.

    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

    Posted on

    Part 4 – Which HDD for Content Applications – Database Workloads

    Part 4 – Which HDD for Content Applications – Database Workloads

    data base server storage I/O trends

    Updated 1/23/2018
    Which enterprise HDD to use with a content server platform for database workloads

    Insight for effective server storage I/O decision making
    Server StorageIO Lab Review

    Which enterprise HDD to use for content servers

    This is the fourth in a multi-part series (read part three here) based on a white paper hands-on lab report I did compliments of Servers Direct and Seagate that you can read in PDF form here. The focus is looking at the Servers Direct (www.serversdirect.com) converged Content Solution platforms with Seagate Enterprise Hard Disk Drive (HDD’s). In this post the focus expands to database application workloads that were run to test various HDD’s.

    Database Reads/Writes

    Transaction Processing Council (TPC) TPC-C like workloads were run against the SUT from the STI. These workloads simulated transactional, content management, meta-data and key-value processing. Microsoft SQL Server 2012 was configured and used with databases (each 470GB e.g. scale 6000) created and workload generated by virtual users via Dell Benchmark Factory (running on STI Windows 2012 R2).

    A single SQL Server database instance (8) was used on the SUT, however unique databases were created for each HDD set being tested. Both the main database file (.mdf) and the log file (.ldf) were placed on the same drive set being tested, keep in mind the constraints mentioned above. As time was a constraint, database workloads were run concurrent (9) with each other except for the Enterprise 10K RAID 1 and RAID 10. Workload was run with two 10K HDD’s in a RAID 1 configuration, then another workload run with a four drive RAID 10. In a production environment, ideally the .mdf and .ldf would be placed on separate HDD’s and SSDs.

    To improve cache buffering the SQL Server database instance memory could be increased from 16GB to a larger number that would yield higher TPS numbers. Keep in mind the objective was not to see how fast I could make the databases run, rather how the different drives handled the workload.

    (Note 8) The SQL Server Tempdb was placed on a separate NVMe flash SSD, also the database instance memory size was set to 16GB which was shared by all databases and virtual users accessing it.

    (Note 9) Each user step was run for 90 minutes with a 30 minute warm-up preamble to measure steady-state operation.

    Users

    TPCC Like TPS

    Single Drive Cost per TPS

    Drive Cost per TPS

    Single Drive Cost / Per GB Raw Cap.

    Cost / Per GB Usable (Protected) Cap.

    Drive Cost (Multiple Drives)

    Protect
    Space Over head

    Cost per usable GB per TPS

    Resp. Time (Sec.)

    ENT 15K R1

    1

    23.9

    $24.94

    $49.89

    $0.99

    $0.99

    $1,190

    100%

    $49.89

    0.01

    ENT 10K R1

    1

    23.4

    $37.38

    $74.77

    $0.49

    $0.49

    $1,750

    100%

    $74.77

    0.01

    ENT CAP R1

    1

    16.4

    $24.26

    $48.52

    $0.20

    $0.20

    $ 798

    100%

    $48.52

    0.03

    ENT 10K R10

    1

    23.2

    $37.70

    $150.78

    $0.49

    $0.97

    $3,500

    100%

    $150.78

    0.07

    ENT CAP SWR5

    1

    17.0

    $23.45

    $117.24

    $0.20

    $0.25

    $1,995

    20%

    $117.24

    0.02

    ENT 15K R1

    20

    362.3

    $1.64

    $3.28

    $0.99

    $0.99

    $1,190

    100%

    $3.28

    0.02

    ENT 10K R1

    20

    339.3

    $2.58

    $5.16

    $0.49

    $0.49

    $1,750

    100%

    $5.16

    0.01

    ENT CAP R1

    20

    213.4

    $1.87

    $3.74

    $0.20

    $0.20

    $ 798

    100%

    $3.74

    0.06

    ENT 10K R10

    20

    389.0

    $2.25

    $9.00

    $0.49

    $0.97

    $3,500

    100%

    $9.00

    0.02

    ENT CAP SWR5

    20

    216.8

    $1.84

    $9.20

    $0.20

    $0.25

    $1,995

    20%

    $9.20

    0.06

    ENT 15K R1

    50

    417.3

    $1.43

    $2.85

    $0.99

    $0.99

    $1,190

    100%

    $2.85

    0.08

    ENT 10K R1

    50

    385.8

    $2.27

    $4.54

    $0.49

    $0.49

    $1,750

    100%

    $4.54

    0.09

    ENT CAP R1

    50

    103.5

    $3.85

    $7.71

    $0.20

    $0.20

    $ 798

    100%

    $7.71

    0.45

    ENT 10K R10

    50

    778.3

    $1.12

    $4.50

    $0.49

    $0.97

    $3,500

    100%

    $4.50

    0.03

    ENT CAP SWR5

    50

    109.3

    $3.65

    $18.26

    $0.20

    $0.25

    $1,995

    20%

    $18.26

    0.42

    ENT 15K R1

    100

    190.7

    $3.12

    $6.24

    $0.99

    $0.99

    $1,190

    100%

    $6.24

    0.49

    ENT 10K R1

    100

    175.9

    $4.98

    $9.95

    $0.49

    $0.49

    $1,750

    100%

    $9.95

    0.53

    ENT CAP R1

    100

    59.1

    $6.76

    $13.51

    $0.20

    $0.20

    $ 798

    100%

    $13.51

    1.66

    ENT 10K R10

    100

    560.6

    $1.56

    $6.24

    $0.49

    $0.97

    $3,500

    100%

    $6.24

    0.14

    ENT CAP SWR5

    100

    62.2

    $6.42

    $32.10

    $0.20

    $0.25

    $1,995

    20%

    $32.10

    1.57

    Table-2 TPC-C workload results various number of users across different drive configurations

    Figure-2 shows TPC-C TPS (red dashed line) workload scaling over various number of users (1, 20, 50, and 100) with peak TPS per drive shown. Also shown is the used space capacity (in green), with total raw storage capacity in blue cross hatch. Looking at the multiple metrics in context shows that the 600GB Enterprise 15K HDD with performance enhanced cache is a premium option as an alternative, or, to complement flash SSD solutions.

    database TPCC transactional workloads
    Figure-2 472GB Database TPS scaling along with cost per TPS and storage space used

    In figure-2, the 1.8TB Enterprise 10K HDD with performance enhanced cache while not as fast as the 15K, provides a good balance of performance, space capacity and cost effectiveness. A good use for the 10K drives is where some amount of performance is needed as well as a large amount of storage space for less frequently accessed content.

    A low cost, low performance option would be the 2TB Enterprise Capacity HDD’s that have a good cost per capacity, however lack the performance of the 15K and 10K drives with enhanced performance cache. A four drive RAID 10 along with a five drive software volume (Microsoft WIndows) are also shown. For apples to apples comparison look at costs vs. capacity including number of drives needed for a given level of performance.

    Figure-3 is a variation of figure-2 showing TPC-C TPS (blue bar) and response time (red-dashed line) scaling across 1, 20, 50 and 100 users. Once again the Enterprise 15K with enhanced performance cache feature enabled has good performance in an apples to apples RAID 1 comparison.

    Note that the best performance was with the four drive RAID 10 using 10K HDD’s Given popularity, a four drive RAID 10 configuration with the 10K drives was used. Not surprising the four 10K drives performed better than the RAID 1 15Ks. Also note using five drives in a software spanned volume provides a large amount of storage capacity and good performance however with a larger drive footprint.

    database TPCC transactional workloads scaling
    Figure-3 472GB Database TPS scaling along with response time (latency)

    From a cost per space capacity perspective, the Enterprise Capacity drives have a good cost per GB. A hybrid solution for environment that do not need ultra-high performance would be to pair a small amount of flash SSD (10) (drives or PCIe cards), as well as the 10K and 15K performance enhanced drives with the Enterprise Capacity HDD (11) along with cache or tiering software.

    (Note 10) Refer to Seagate 1200 12 Gbps Enterprise SAS SSD StorageIO lab review

    (Note 11) Refer to Enterprise SSHD and Flash SSD Part of an Enterprise Tiered Storage Strategy

    Where To Learn More

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    What This All Means

    If your environment is using applications that rely on databases, then test resources such as servers, storage, devices using tools that represent your environment. This means moving up the software and technology stack from basic storage I/O benchmark or workload generator tools such as Iometer among others instead using either your own application, or tools that can replay or generate various workloads that represent your environment.

    Continue reading part five in this multi-part series here where the focus shifts to large and small file I/O processing workloads.

    Ok, nuff said, for now.

    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

    Posted on

    Which Enterprise HDD for Content Applications Different File Size Impact

    Which HDD for Content Applications Different File Size Impact

    Different File Size Impact server storage I/O trends

    Updated 1/23/2018

    Which enterprise HDD to use with a content server platform different file size impact.

    Insight for effective server storage I/O decision making
    Server StorageIO Lab Review

    Which enterprise HDD to use for content servers

    This is the fifth in a multi-part series (read part four here) based on a white paper hands-on lab report I did compliments of Servers Direct and Seagate that you can read in PDF form here. The focus is looking at the Servers Direct (www.serversdirect.com) converged Content Solution platforms with Seagate Enterprise Hard Disk Drive (HDD’s). In this post the focus looks at large and small file I/O processing.

    File Performance Activity

    Tip, Content solutions use files in various ways. Use the following to gain perspective how various HDD’s handle workloads similar to your specific needs.

    Two separate file processing workloads were run (12), one with a relative small number of large files, and another with a large number of small files. For the large file processing (table-3), 5 GByte sized files were created and then accessed via 128 Kbyte (128KB) sized I/O over a 10 hour period with 90% read using 64 threads (workers). Large file workload simulates what might be seen with higher definition video, image or other content streaming.

    (Note 12) File processing workloads were run using Vdbench 5.04 and file anchors with sample script configuration below. Instead of vdbench you could also use other tools such as sysbench or fio among others.

    VdbenchFSBigTest.txt
    # Sample script for big files testing
    fsd=fsd1,anchor=H:,depth=1,width=5,files=20,size=5G
    fwd=fwd1,fsd=fsd1,rdpct=90,xfersize=128k,fileselect=random,fileio=random,threads=64
    rd=rd1,fwd=fwd1,fwdrate=max,format=yes,elapsed=10h,interval=30

    vdbench -f VdbenchFSBigTest.txt -m 16 -o Results_FSbig_H_060615

    VdbenchFSSmallTest.txt
    # Sample script for big files testing
    fsd=fsd1,anchor=H:,depth=1,width=64,files=25600,size=16k
    fwd=fwd1,fsd=fsd1,rdpct=90,xfersize=1k,fileselect=random,fileio=random,threads=64
    rd=rd1,fwd=fwd1,fwdrate=max,format=yes,elapsed=10h,interval=30

    vdbench -f VdbenchFSSmallTest.txt -m 16 -o Results_FSsmall_H_060615

    The 10% writes are intended to reflect some update activity for new content or other changes to content. Note that 128KB per second translates to roughly 1 Gbps streaming content such as higher definition video. However 4K video (not optimized) would require a higher speed as well as resulting in larger file sizes. Table-3 shows the performance during the large file access period showing average read /write rates and response time, bandwidth (MBps), average open and close rates with response time.

    Avg. File Read Rate

    Avg. Read Resp. Time
    Sec.

    Avg. File Write Rate

    Avg. Write Resp. Time
    Sec.

    Avg.
    CPU %
    Total

    Avg. CPU % System

    Avg. MBps
    Read

    Avg. MBps
    Write

    ENT 15K R1

    580.7

    107.9

    64.5

    19.7

    52.2

    35.5

    72.6

    8.1

    ENT 10K R1

    455.4

    135.5

    50.6

    44.6

    34.0

    22.7

    56.9

    6.3

    ENT CAP R1

    285.5

    221.9

    31.8

    19.0

    43.9

    28.3

    37.7

    4.0

    ENT 10K R10

    690.9

    87.21

    76.8

    48.6

    35.0

    21.8

    86.4

    9.6

    Table-3 Performance summary for large file access operations (90% read)

    Table-3 shows that for two-drive RAID 1, the Enterprise 15K are the fastest performance, however using a RAID 10 with four 10K HDD’s with enhanced cache features provide a good price, performance and space capacity option. Software RAID was used in this workload test.

    Figure-4 shows the relative performance of various HDD options handling large files, keep in mind that for the response line lower is better, while for the activity rate higher is better.

    large file processing
    Figure-4 Large file processing 90% read, 10% write rate and response time

    In figure-4 you can see the performance in terms of response time (reads larger dashed line, writes smaller dotted line) along with number of file read operations per second (reads solid blue column bar, writes green column bar). Reminder that lower response time, and higher activity rates are better. Performance declines moving from left to right, from 15K to 10K Enterprise Performance with enhanced cache feature to Enterprise Capacity (7.2K), all of which were hardware RAID 1. Also shown is a hardware RAID 10 (four x 10K HDD’s).

    Results in figure-4 above and table-4 below show how various drives can be configured to balance their performance, capacity and costs to meet different needs. Table-4 below shows an analysis looking at average file reads per second (RPS) performance vs. HDD costs, usable capacity and protection level.

    Table-4 is an example of looking at multiple metrics to make informed decisions as to which HDD would be best suited to your specific needs. For example RAID 10 using four 10K drives provides good performance and protection along with large usable space, however that also comes at a budget cost (e.g. price).

    Avg.
    File Reads Per Sec. (RPS)

    Single Drive Cost per RPS

    Multi-Drive Cost per RPS

    Single Drive Cost / Per GB Capacity

    Cost / Per GB Usable (Protected) Cap.

    Drive Cost (Multiple Drives)

    Protection Overhead (Space Capacity for RAID)

    Cost per usable GB per RPS

    Avg. File Read Resp. (Sec.)

    ENT 15K R1

    580.7

    $1.02

    $2.05

    $ 0.99

    $0.99

    $1,190

    100%

    $2.1

    107.9

    ENT 10K R1

    455.5

    1.92

    3.84

    0.49

    0.49

    1,750

    100%

    3.8

    135.5

    ENT CAP R1

    285.5

    1.40

    2.80

    0.20

    0.20

    798

    100%

    2.8

    271.9

    ENT 10K R10

    690.9

    1.27

    5.07

    0.49

    0.97

    3,500

    100%

    5.1

    87.2

    Table-4 Performance, capacity and cost analysis for big file processing

    Small File Size Processing

    To simulate a general file sharing environment, or content streaming with many smaller objects, 1,638,464 16KB sized files were created on each device being tested (table-5). These files were spread across 64 directories (25,600 files each) and accessed via 64 threads (workers) doing 90% reads with a 1KB I/O size over a ten hour time frame. Like the large file test, and database activity, all workloads were run at the same time (e.g. test devices were concurrently busy).

    Avg. File Read Rate

    Avg. Read Resp. Time
    Sec.

    Avg. File Write Rate

    Avg. Write Resp. Time
    Sec.

    Avg.
    CPU %
    Total

    Avg. CPU % System

    Avg. MBps
    Read

    Avg. MBps
    Write

    ENT 15K R1

    3,415.7

    1.5

    379.4

    132.2

    24.9

    19.5

    3.3

    0.4

    ENT 10K R1

    2,203.4

    2.9

    244.7

    172.8

    24.7

    19.3

    2.2

    0.2

    ENT CAP R1

    1,063.1

    12.7

    118.1

    303.3

    24.6

    19.2

    1.1

    0.1

    ENT 10K R10

    4,590.5

    0.7

    509.9

    101.7

    27.7

    22.1

    4.5

    0.5

    Table-5 Performance summary for small sized (16KB) file access operations (90% read)

    Figure-5 shows the relative performance of various HDD options handling large files, keep in mind that for the response line lower is better, while for the activity rate higher is better.

    small file processing
    Figure-5 Small file processing 90% read, 10% write rate and response time

    In figure-5 you can see the performance in terms of response time (reads larger dashed line, writes smaller dotted line) along with number of file read operations per second (reads solid blue column bar, writes green column bar). Reminder that lower response time, and higher activity rates are better. Performance declines moving from left to right, from 15K to 10K Enterprise Performance with enhanced cache feature to Enterprise Capacity (7.2K RPM), all of which were hardware RAID 1. Also shown is a hardware RAID 10 (four x 10K RPM HDD’s) that has higher performance and capacity along with costs (table-5).

    Results in figure-5 above and table-5 below show how various drives can be configured to balance their performance, capacity and costs to meet different needs. Table-6 below shows an analysis looking at average file reads per second (RPS) performance vs. HDD costs, usable capacity and protection level.

    Table-6 is an example of looking at multiple metrics to make informed decisions as to which HDD would be best suited to your specific needs. For example RAID 10 using four 10K drives provides good performance and protection along with large usable space, however that also comes at a budget cost (e.g. price).

    Avg.
    File Reads Per Sec. (RPS)

    Single Drive Cost per RPS

    Multi-Drive Cost per RPS

    Single Drive Cost / Per GB Capacity

    Cost / Per GB Usable (Protected) Cap.

    Drive Cost (Multiple Drives)

    Protection Overhead (Space Capacity for RAID)

    Cost per usable GB per RPS

    Avg. File Read Resp. (Sec.)

    ENT 15K R1

    3,415.7

    $0.17

    $0.35

    $0.99

    $0.99

    $1,190

    100%

    $0.35

    1.51

    ENT 10K R1

    2,203.4

    0.40

    0.79

    0.49

    0.49

    1,750

    100%

    0.79

    2.90

    ENT CAP R1

    1,063.1

    0.38

    0.75

    0.20

    0.20

    798

    100%

    0.75

    12.70

    ENT 10K R10

    4,590.5

    0.19

    0.76

    0.49

    0.97

    3,500

    100%

    0.76

    0.70

    Table-6 Performance, capacity and cost analysis for small file processing

    Looking at the small file processing analysis in table-5 shows that the 15K HDD’s on an apples to apples basis (e.g. same RAID level and number of drives) provide the best performance. However when also factoring in space capacity, performance, different RAID level or other protection schemes along with cost, there are other considerations. On the other hand the Enterprise Capacity 2TB HDD’s have a low cost per capacity, however do not have the performance of other options, assuming your applications need more performance.

    Thus the right HDD for one application may not be the best one for a different scenario as well as multiple metrics as shown in table-5 need to be included in an informed storage decision making process.

    Where To Learn More

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    What This All Means

    File processing are common content applications tasks, some being small, others large or mixed as well as reads and writes. Even if your content environment is using object storage, chances are unless it is a new applications or a gateway exists, you may be using NAS or file based access. Thus the importance of if your applications are doing file based processing, either run your own applications or use tools that can simulate as close as possible to what your environment is doing.

    Continue reading part six in this multi-part series here where the focus is around general I/O including 8KB and 128KB sized IOPs along with associated metrics.

    Ok, nuff said, for now.

    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

    Posted on

    Happy Earth Day 2016 Eliminating Digital and Data e-Waste

    Happy Earth Day 2016 Eliminating Digital and Data e-Waste

    With Earth Day 2016 on April 22, here are some thoughts about electronic waste (e-waste).

    For those involved in data management or data infrastructures, the following are five tips to help cut the overhead and resulting impact of digital e-waste and later physical e-waste. Most conversations involving e-waste focus on the physical aspects from disposing of electronics along with later impacts. While physical e-waste is an important topic, lets expand the conversation including other variations of e-waste including digital. By digital e-waste I’m referring to the use of physical items that end up contributing to traditional e-waste.

    digital and data ewaste

    Digital e-waste ranges from the overhead of keeping extra copies of data that result in an expanding data footprint that in turn requires extra physical resources and their impact. Addressing physical e-waste also means keeping digital (not the physical items) including data waste in perspective.

    Also note that digital or data waste may in fact not be waste per say if it exists as a by-product of making sure applications, data and resulting information are protected, preserved, secured and served for when needed. The warning is what can be done to make sure there are good useful effective and efficient copy data that has a relative low data footprint overhead impact, more on this later.

    Here are six themes to consider to cut the impact without costing or compromising your organization when address e-waste (physical, digital, data).

    1. Understand Digital e-waste

    You might be familiar with the term e-waste (electronic waste), you know, those physical items that get discarded from supporting your digital lifestyle. The reason awareness around e-waste is important is because of the environmental impacts of discarding all those devices. The more known about the issue, impacts, causes and effects helps to drive awareness as well as insight into what can be done to mitigate those items.

    ewaste

    Devices range from smart and dumb cell phones, personal digital assistants (PDAs), tablets, notebook and workstation computers, MP3 devices, cameras, video display monitors along with larger servers, storage and networking technology, not to mention all the other Internet of Things (IoT) and Internet of Device (IoD) items. What’s important to know about physical e-waste is the impact of the various components. You can learn more about physical e-waste impact in general with a web search such as Google e-waste impact.

    2. Reuse, Repurpose, Redeploy, Reconfigure, Re-Tool, Recycle

    Reconfigure and retool where possible by re-driving installing newer, more energy-efficient high-capacity drives, or more performance effective devices. Besides replacing Hard Disk Drives (HDD), Solid State Devices (SSDs), magnetic tape among other mediums, look at the pro’s and con’s of replacing CPU processor sockets, upgrading memory and PCIe I/O cards for networking or storage among other enhancements.

    Pro’s include being able to use the chassis longer reducing amount of physical e-waste, however at some point it can be more cost-effective to do a total replacement. However the longer you can use the asset or device the more that has as a positive benefit to cut e-waste.

    Repurpose, reuse and redeploy assets such as servers, storage and networking devices in a hand me down approach assuming there is a value or benefit in doing so.

    Recycle when done, dispose of the technology properly including for storage secure erase of digital media and later physical handling.

    3. Responsible Recycle and Disposition of technology (including secure digital destruction)

    What are you doing with, how are you disposing of physical items ranging from laptops, workstations, tablets, phones, MP3 players, TVs and monitors, servers, network and storage devices among others when no longer needed?

    Are you securely erasing your digital data on HDDs as well as SSDs or even tape and optical devices before they are disposed of? If not, you should be. For example if you are not yet using or looking at Self Encrypting Drives (SED) including HDDs and SSDs for securing your data, start investigating them. Sure they have a security value proposition for when lost or stolen, however they can also cut the time to secure erase to a given standard from days or hours to minutes or seconds.

    These will become e-waste

    Smart shopping up front, what you want, what you need, how long can you leverage, spend more up front to get something that can last 3-5 years vs. discarding in 1.5-3 years.
    Smart management with insight, know your cost and impacts, not just for PR purpose, for profit and practicality

    4. Plan acquisitions with disposition in mind

    Redesign and design for replacement, maximizing what you have or will acquire, using it for longer time to cut costs, improve productivity (and profitability) while reducing e-waste overhead contributing footprint.

    For example, do you need or want to have the latest in new technology replacing that phone, tablet, watch or other IoT or IoD item as soon as something newer comes along? No worries if you are also doing something responsible with what was new and now old by such as donating or giving it to somebody else who might be able to get a few more years worth of use out of it before it becomes e-waste.

    On the other hand, if you are acquiring technology with a 2-3 year useful life plan, what would it take to upgrade that item to a larger or more robust version using it for 3-5 years. Granted, you might not use it in its primary role for the longer duration, however can it be repurposed for some other uses? Also from a technology acquisition perspective, have a forecast and plan that can help you make smart, informed decisions up front knowing when upgrades or extra resources will be needed to prolong the usefulness of the item.

    Of course you can also simply move everything to the cloud and out-source your e-waste footprint to the vendor, MSP or cloud provider.

    5. Understand Changing Data Value

    Keep in mind that data has either no value, some value or unknown value all of which can change over time. For example some data has value for seconds, minutes or hours and can then be discarded. Other data have some value which can be low or high which determines how as well as when, where and how to protect, preserve, secure and serve it when needed. Then there is data that has an unknown value. However, that can change over time.

    Different and Changing Data Value

    Over time your data may end up having no value meaning it can be discarded, or, it might have some value (low or high) meaning change how it should be protected, preserved, secured and served. Then there is data that may stay in limbo or unknown status indefinitely or until somebody, or some software or via other means decide if it has value or not.

    The point is that to cut digital e-waste is to discard data with no value as soon as possible, protect, preserve, secure and serve data with value appropriately. Likewise, for all of that growing data with an unknown value, rethink how it is protected and stored, all of which has an impact on both physical as well as digital e-waste.

    This means having insight and awareness into your environment, applications, data, settings, configuration and metadata, not only of the space being used, or when it was last updated. Also, look beyond when data was last modified or changed, look at when it was last read or accessed to decide how protected and secured including virus and other scans.

    6. Data Footprint Reduction (DFR)

    Implement data footprint reduction (DFR) to lower overhead impact not only at the target or downstream destination using compression, dedupe and other techniques. Also, move upstream to the source where the problem starts and address it there. Addressing at the source leverages various techniques from Archiving, Backup/Data Protection Modernization (rethinking what saved, when, how often, etc), Cleanup, Compression and Consolidation, Data management, Deletion and Dedupe along with storage tiering, RAID/Parity/Mirroring/Replication/Erasure Code and Advanced Parity/LRC/Forward Error Correction and other technologies.

    For example if you have 10TB of data, how many copies do you have and why, how are those copies protected and what is their overhead. The issue and concern should not be primarily how many copies, rather, if those copies add or give value, then what can you do to keep them while reducing their overhead impact, besides simply trying to compress or dedupe everything. Hint, start exploring copy management as well as revisiting what you protect, when, where, why, how often along with options for implementing DFR as close to the data source as possible, as well as downstream.

    Where To Learn More

    What This All Means

    Gain insight and awareness into what is occurring with physical and digital ewaste side stepping the greenwashing and other activity. Small steps implemented by many will have a big impact. Every bit, byte, block, blob, bucket file or object along with their copies have an impact, hopefully as well as a benefit, a question is how can you reduce the overhead while increasing your return on innovation reducing costs, complexity and overhead while enhancing organization capabilities. There are many techniques, technologies, tools and approaches to apply to various environments, after all, everything is not the same, yet there are similarities. Happy Earth Day 2016 and happy spring to those of you in the northern hemisphere (as well as elsewhere).

    Ok, nuff said, for now

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved