datacenter Archives

March 29, 2018November 26, 2023

AWS Cloud Application Data Protection Webinar

AWS Cloud Application Data Protection Webinar trends

AWS Cloud Application Data Protection Webinar
Date: Tuesday, April 24, 2018 at 11:00am PT / 2:00pm ET

Only YOU can prevent data loss for on-premises, Amazon Web Service (AWS) based cloud, and hybrid applications.

Join me in this free AWS Cloud Application Data Protection Webinar (registration required) sponsored by Veeam produced by Redmond Magazine as we explore issues, trends, tools, best practices and techniques for enabling data protection with AWS technologies.

Attend and learn about:

Application-aware point in time snapshot data protection
Protecting AWS EC2 and on-premises applications (and data)
Leveraging AWS for data protection and recovery
And much more

Where to learn more

Learn more about data protection, software defined data center (SDDC), software defined data infrastructures (SDDI), AWS, cloud and related topics via the following links:

World Backup Day 2018 Data Protection Readiness Reminder
Data Infrastructure Server Storage I/O related Tradecraft Overview
Data Infrastructure Overview, Its What’s Inside of Data Centers
4 3 2 1 and 3 2 1 data protection best practices
Only you can prevent cloud data loss
Cloud conversations: confidence, certainty and confidentiality
Cloud conversations: AWS EBS, Glacier and S3 overview (Part I)
Cloud Conversations AWS Azure Service Maps via Microsoft
AWS S3 Storage Gateway Revisited (Part I)
Cloud Conversations: AWS S3 Cross Region Replication storage enhancements
Cloud conversations: AWS EBS, Glacier and S3 overview (Part II S3)
Microsoft Windows Server 2019 Insiders Preview
March 2018 Server StorageIO Data Infrastructure Update Newsletter
AWS Announces New S3 Cloud Storage Security Encryption Features
All You Need To Know about ROBO Data Protection Backup (free webinar with registration)
Data Protection Recovery Life Post World Backup Day Pre GDPR
Software Defined, Converged Infrastructure (CI), Hyper-Converged Infrastructure (HCI) resources
The SSD Place (SSD, NVM, PM, SCM, Flash, NVMe, 3D XPoint, MRAM and related topics)
The NVMe Place (NVMe related topics, trends, tools, technologies, tip resources)
Data Protection Diaries (Archive, Backup/Restore, BC, BR, DR, HA, RAID/EC/LRC, Replication, Security)
Various Data Infrastructure related events, webinars and other activities
Server StorageIO.tv (various videos and podcasts, fun and for work)

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

What this all means and wrap-up

You can not go forward if you can not go back to a particular point in time (e.g. recovery point objective or RPO). Likewise, if you can not go back to a given RPO, how can you go forward with your business as well as meet your recovery time objective (RTO)? Join us for the live conversation or replay by registering (free) here to learn how to enable AWS Cloud Application Data Protection Webinar, as well as using AWS S3 for on-site, on-premises data protection.

Ok, nuff said, for now.

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

January 25, 2018November 26, 2023

Data Infrastructure Resource Links cloud data protection tradecraft trends

Data Infrastructure Resource Links Server Storage I/O Network

By Greg Schulz – www.storageioblog.com April 28, 2018

Various data infrastructure resource links.

The following are a collection of server storageioblog data infrastructure resource links.

Where to learn more

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

What This All Means

Check out the above links to data infrastructure resource links.

Ok, nuff said, for now.

November 14, 2017November 26, 2023

AWS Announces New S3 Cloud Storage Security Encryption Features

server storage I/O data infrastructure trends

Updated 1/17/2018

Amazon Web Services (AWS) recently announced new Simple Storage Service (S3) e.g. AWS S3 encryption and security enhancements including Default Encryption, Permission Checks, Cross-Region Replication ACL Overwrite, Cross-Region Replication with KMS and Detailed Inventory Report. Another recent announcement by AWS is for PrivateLinks endpoints within a Virtual Private Cloud (VPC).

AWS Dashboard
AWS Service Dashboard

Default Encryption

Extending previous security features, now you can mandate all objects stored in a given S3 bucket be encrypted without specifying a bucket policy that rejects non-encrypted objects. There are three server-side encryption (SSE) options for S3 objects including keys managed by S3, AWS KMS and SSE Customer ( SSE-C) managed keys. These options provide more flexibility as well as control for different environments along with increased granularity. Note that encryption can be forced on all objects in a bucket by specifying a bucket encryption configuration. When an unencrypted object is stored in an encrypted bucket, it will inherit the same encryption as the bucket, or, alternately specified by a PUT required.

AWS S3 Bucket Encryption
AWS S3 Buckets

Permission Checks

There is now an indicator on the S3 console dashboard prominently indicating which S3 buckets are publicly accessible. In the above image, some of my AWS S3 buckets are shown including one that is public facing. Note in the image above how there is a notion next to buckets that are open to public.

Cross-Region Replication ACL Overwrite and KMS

AWS Key Management Service (KMS) keys can be used for encrypting objects. Building on previous cross-region replication capabilities, now when you replicate objects across AWS accounts, a new ACL providing full access to the destination account can be specified.

Detailed Inventory Report

The S3 Inventory report ( which can also be encrypted) now includes the encryption status of each object.

PrivateLink for AWS Services

PrivateLinks enable AWS customers to access services from a VPC without using a public IP as well as traffic not having to go across the internet (e.g. keeps traffic within the AWS network. PrivateLink endpoints appear in Elastic Network Interface (ENI) with private IPs in your VPC and are highly available, resiliency and scalable. Besides scaling and resiliency, PrivateLink eliminates the need for white listing of public IPs as well as managing internet gateway, NAT and firewall proxies to connect to AWS services (Elastic Cloud Compute (EC2), Elastic Load Balancer (ELB), Kinesis Streams, Service Catalog, EC2 Systems Manager). Learn more about AWS PrivateLink for services here including VPC Endpoint Pricing here.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

What This All Means

Common cloud concern considerations include privacy and security. AWS S3 among other industry cloud service and storage providers have had their share of not so pleasant news coverage involving security.

Keep in mind that data protection including security is a shared responsibility (and only you can prevent data loss). This means that the vendor or service provider has to take care of their responsibility making sure their solutions have proper data protection and security features by default, as well as extensions, and making those capabilities known to consumers.

The other part of shared responsibility is that consumers and users of cloud services need to know what the capabilities are, defaults and options as well as when to use various approaches. Ultimately it is up to the user of a cloud service to implement best practices to leverage cloud as well as their own on-premises technologies so that they can support data infrastructure that in turn protect, preserve, secure and serve information (along with their applications and data).

These are good enhancements by AWS to make their S3 cloud storage security encryption features available as well as provide options and awareness for users on how to use those capabilities.

Ok, nuff said, for now.

August 27, 2017November 3, 2024

August 2017 Server StorageIO Data Infrastructure Update Newsletter

Server StorageIO August 2017 Data Infrastructure Update Newsletter

Volume 17, Issue VII (Pre VMworld 2017)

Hello and welcome to the August 2017 issue of the Server StorageIO update newsletter.

Its end of summer season here in north america which means wrapping up holidays, vacations, back to school shopping (and going to school), as well as the start of the fall IT technology conference season. VMworld 2017 USA is this week in Las Vegas and there will be several announcements coming out of that event. Given all of the activity so far this month, I’m going to cover the VMworld and related topics in a special early September issue of this newsletter.

Speaking of VMworld 2017, if you are going to be there in Las Vegas, stop by the book store located in the community village area on Tuesday at 1PM I will be doing a book signing, meet and greet, stop by and say hello.

Thanks to all who participated in the recent thevPad top 100 vBloggers event, I am honored to have StorageIOblog listed in the top 100 vBlogs. Also congratulations to new and returning fellow Microsoft MVPs and VMware vExperts. There is a lot going on in the industry, lets get to it in this Server StorageIO Data Infrastructure Update Newsletter.

Various Events and Webinars
Industry Resources and Links
Connect and Converse With Us
About Us

Enjoy this edition of the Server StorageIO update newsletter (pre VMworld edition).

Cheers GS

Data Infrastructure and IT Industry Activity Trends

Acronis announced True Image 2018 for home based data protection (backup), while Crashplan aka code42 announced they were getting out of the consumer, small office home office (SOHO) backup and data protection space to focus on the enterprise.

Cisco bought software defined storage converged infrastructure software vendor Springpath for about $320M USD. Cisco and Swiftstack (object storage software) also announced interoperability news with the UCS S32600 storage server platform.

GPU vendor NVIDIA announced Quadro Virtual Data Center workstation technology.

Meanwhile ioFABRIC announced their new Vicinity 3.0 software defined management solution.

Microsemi (remember PMC Sierra) announced release of its Flashtec PCIe controllers to help speed adoption deployment of SSDs including NVMe based.

Microsoft bought Cycle Computing to enhance Azure services, while also making Azure Blob storage tiering available as part of an ongoing public preview. For those not aware, Azure Blob is similar to what other services call objects. Get in on the public preview here. For those who live in a hybrid world where your environment and experience include both Windows and Linux, check out Windows Services for Linux here. With this service which can install onto an Windows 10 system along side Win32 (e.g. it co-exists, its not a virtual machine), you can choose from the Windows Store which Linux distro you want (e.g. Centos, Ubuntu, etc).

Need to learn, refresh or simply gain a better understanding of Microsoft PowerShell for software defined management of Windows, Azure and other environments? Check out this great post from Microsoft Blogs.

For those who work in a Windows or Azure environment, here are some useful icons for Powerpoint, Visio, PNG and SVG from Microsoft. With Microsoft Ignite coming up in September, watch for some interesting update enhancements to Windows Server from a server storage I/O perspective.

NextPlatform.com has an interesting article on Exascale Timeline for Storage and I/O systems worth a read. Panzura global name space and scale out software defined storage management software announced mobile client file sharing. After dropping their own cloud business, Verizon is now a virtual network services partner with Amazon.

Over at all flash array (AFA) SSD vendor Pure, revenues are growing closer to an annual $1B USD rate despite loss per share, Pure also announced a change in leadership with current CEO Scott Dietzen stepping aside for Charles Giancarlo to take the lead spot.

VMware has been talking about the continued increase in customer adoption and deployment of VSAN now they are showing they eat their own dog food. Check out this post here from VMware that shows how many and what size VSAN clusters they are using for various internal operations. Also on the VMware storage front, learn more about enhancements for large and small file allocation blocks with vSphere VMFS6.

With all of the pre and post VMworld related announcements, remember to check out the tools available over at the VMware flings site including vSphere HTML5 Web Client, HCIBench, vRealize Operations Export, VisualEsxtop, ESXi Embedded Host Client, VMware OS Optimization Tool and many others. Watch for VMworld coverage in the September newsletter along with posts at www.storageioblog.com

Check out other industry news, comments, trends perspectives here.

Server StorageIO Commentary in the news

Recent Server StorageIO industry trends perspectives commentary in the news.

Via EnterpriseStorageForum: Comments on Who Will Rule the Storage World?
Via InfoGoto: Comments on Google Cloud Platform Gaining Data Storage Momentum
Via InfoGoto: Comments on Singapore High Rise Data Centers
Via InfoGoto: Comments on New Tape Storage Capacity

View more Server, Storage and I/O trends and perspectives comments here

Server StorageIOblog Posts

Recent and popular Server StorageIOblog posts include:

NVMe Wont Replace Flash By Itself They Complement Each Other
There has been some recent industry marketing buzz generated by a startup to get some attention by claiming via a study sponsored by including the startup that Non-Volatile Memory (NVM) Express (NVMe) will replace flash storage. Granted, many IT customers as well as vendors are still confused by NVMe thinking it is a storage medium as opposed to an interface used for accessing fast storage devices such as nand flash among other solid state devices (SSDs). Part of that confusion can be tied to common SSD based devices rely on NVM that are persistent memory retaining data when powered off (unlike the memory in your computer).

Like IT Data Centers Do You Take Trade Show Exhibit Infrastructure For Granted?
Think about this for a moment; do you assume that Information Technology (IT) and Cloud based data centers along with their associated Data Infrastructure supporting various applications will be accessible when needed. Likewise, when you go to a trade show, conference, symposium, user group or another conclave is it assumed that the trade show, exposition (expo), exhibits, booths, stands or demo areas will be ready, waiting and accessible? In case you did not hear, things heated up at the recent Flash Memory Summit (FMS) in Santa Clara where a small fire in one of the exhibt booths prevented the expo hall from opening during the event.

Microsoft Azure Software Defined Data Infrastructure Reference Resources
Need to learn more about Microsoft Azure Cloud Software Defined Data Infrastructure topics including reference architecture among other resources for various application workloads? Microsoft Azure has an architecture and resources page (here) that includes various application workload reference tools.

Chelsio Storage over IP and other Networks Enable Data Infrastructures
Chelsio and Storage over IP (SoIP) continue to enable Data Infrastructures from legacy to software defined virtual, container, cloud as well as converged.

Announcing Software Defined Data Infrastructure Essentials Book by Greg Schulz
Software Defined Data Infrastructure (SDDI) Essentials is now generally available at various global venues in hardcopy, hardback print as well as various electronic versions including via Amazon and CRC Press among others.

Server StorageIO Industry Trends Perspectives Report WekaIO Matrix
Like Data They Protect For Now Quantum Revenues Continue To Grow
Travel Fun Crossword Puzzle For VMworld 2017 Las Vegas
Hot Popular New Trending Data Infrastructure Vendors To Watch

Server StorageIO Data Infrastructure Tips and Articles

Recent Server StorageIO industry trends perspectives commentary in the news.

Via NetworkWorld: Do you have an IT trade craft skills gap?

View more Server, Storage and I/O trends and perspectives comments here

Events and Activities

Recent and upcoming event activities.

Sep. 21, 2017 – MSP CMG – Minneapolis MN
Sep. 20, 2017 – Redmond Data Protection and Backup – Webinar
Sep. 14, 2017 – Fujifilm IT Executive Summit – Seattle WA
Sep. 12, 2017 – SNIA Software Developers Conference (SDC) – Santa Clara CA
Sep. 7, 2017 – WiPro – Planning Your Software Defined Journey – New York City
August 29, 2017 – VMworld – Las Vegas

See more webinars and activities on the Server StorageIO Events page here.

Server StorageIO Industry Resources and Links

Useful links and pages:
Microsoft TechNet – Various Microsoft related from Azure to Docker to Windows
storageio.com/links – Various industry links (over 1,000 with more to be added soon)
objectstoragecenter.com – Cloud and object storage topics, tips and news items
OpenStack.org – Various OpenStack related items
storageio.com/protect – Various data protection items and topics
thenvmeplace.com – Focus on NVMe trends and technologies
thessdplace.com – NVM and Solid State Disk topics, tips and techniques
storageio.com/converge – Various CI, HCI and related SDS topics
storageio.com/performance – Various server, storage and I/O benchmark and tools
VMware Technical Network – Various VMware related items

Ok, nuff said, for now.

Cheers
Gs

Greg Schulz – Multi-year Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

August 26, 2017November 26, 2023

Hot Popular New Trending Data Infrastructure Vendors To Watch

server storage I/O data infrastructure trends

Updated 1/21/2018

A common question I get asked is who are the hot popular new trending data infrastructure vendors to watch. This post looks at some data infrastructure vendors to watch and keep an eye on.

Keep in mind that there is a difference between industry adoption and customer deployment, the former being what the industry (e.g. Vendors, resellers, integrators, investors, consultants, analyst, press, media, analysts, bloggers or other influences) like, want and need to talk about. Then there is customer adoption and deployment which is what is being bought, installed and used.

Some Popular Trending Vendors To Watch

The following is far from an exhaustive list however here are some that come to mind that I’m watching.

Apcera – Enterprise class containers and management tools
AWS – Rolls our new services like a startup with size momentum of a legacy player
Blue Medora – Data Infrastructure insight, software defined management
Broadcom – Avago/LSI, legacy Broadcom, Emulex, Brocade acquisition interesting portfolio
Chelsio – Server, storage and data Infrastructure I/O technologies
Commvault – Data protection and backup solutions
Compuverde – Software defined storage
Data Direct Networks (DDN) – Scale out and high performance storage
Datadog – Software defined management, data infrastructure insight, analytics, reporting
Datrium – Converged software defined data infrastructure solutions
Dell EMC Code – Rexray container persistent storage management
Docker – Container and management tools
E8 Storage – NVMe based storage solutions
Elastifile – Scale out software defined storage and file system
Enmotus – MicroTiering that works with Windows, Linux and various cloud platforms
Everspin – storage class memories and NVDIMM
Excelero – NVMe based storage
Hedvig – Scale out software defined storage
Huawei – While not common in the US, in Europe and elsewhere they are gaining momentum
Intel – Watch what they do with Optane and storage class memories
Kubernetes – Container software defined management
Liqid – Stealth Colorado startup focusing on PCIe fabrics and composable infrastructure
Maxta – Hyper converged infrastructure (HCI) and software defined data infrastructure vendor
Mellanox – While not a startup, keep an eye on what they are doing with their adapters
Micron – Watch what they do with 3D XPoint storage class memory and SSD
Microsoft – Not a startup, however keep an eye on Azure, Azure Stack, Window Server with S2D, ReFS, tiering, CI/HCI as well as Linux services on Windows.
Minio – Software defined storage solutions
NetApp – While FAS/Ontap and Solidfire get the headlines, E series generates revenue, keep an eye on StorageGrid and AltaVault
Neuvector – Container management and security
Noobaa – Software defined storage and more
NVIDA – No longer just another graphics process unit based company
Pivot3 – An original HCI software defined players, granted, some of their competitors might not think so
Pluribus Networks – Software Defined Networks for Software Defined Data Infrastructures
Portwork – Container management and persistent storage
Rozo Systems – Scale out software defined storage and file system
Rubrik – Data Protection software, reminds me of a startup called Commvault 20 years ago.
ScaleMP – Composable scale out software defined servers
Storpool – Scale out software defined storage
Stratoscale – Software defined data infrastructure and hybrid solutions
SUSE – Linux distribution looking to expand their offerings, gain more insight
Tidalscale – Composable software defined data infrastructures
Turbonomic – Software Defined Management, insight, analytics and automation
Ubuntu – Known for their Linux distribution, check out their Metal as a Service (MaaS) technology
Veeam – Data protection and backup solutions
technology
Virtuozzo – Software defined storage and data infrastructure technologies
VMware – AWS, vSAN, NSX, Integrated Containers and much more
WekaIO – Scale out software defined storage and file system

Some Popular Trending Technology Trends

ARM, ASIC, FPGA, GPU servers among others
Converged Infrastructure (CI), Hyper Converged Infrastructure (HCI), Composable Infrastructure
Analytics, reporting, insight, machine learning (ML), artificial intelligence (AI), automation
Software Defined including Cloud, Virtual, Containers, Docker, kubernetes, mesos, serverless, micro services
Data protection, backup/restore, archive, security, business resiliency (BR), business continuance (BC), disaster recovery (DR)
Non-volatile memory (NMV), NVM Express (NVMe), storage class memories (SCM), persistent memory, nand flash, SSD

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

Server StorageIO IT Data Infrastructure Industry Links A to E
Server Storage I/O Benchmarking Performance Resource Tools
What’s a data infrastructure? (Via NetworkWorld)
Software Defined Data Infrastructure Essentials (CRC Press) book companion page (includes various images, sample figures and added content)
Object and Cloud Storage Center (www.objectstoragecenter.com)
www.thessdplace.com – NVM, flash, SSD, SCM and related topics
www.thenvmeplace.com – NVM Express (NVMe) related topics

Various IT and Cloud Infrastructure Layers including Data Infrastructures

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

What This All Means

There are always more hot popular new or trending data infrastructure vendors to watch, which ones are you keeping an eye on?

Ok, nuff said, for now.

August 23, 2017November 19, 2024

Announcing Software Defined Data Infrastructure Essentials Book by Greg Schulz

New SDDI Essentials Book by Greg Schulz of Server StorageIO

Cloud, Converged, Virtual Fundamental Server Storage I/O Tradecraft

server storage I/O data infrastructure trends

Update 1/21/2018
Over the past several months I have posted, commenting, presenting and discussing more about Software Defined Data Infrastructure Essentials aka SDDI or SDDC and SDI. Now it is time to announce my new book (my 4th solo project), Software Defined Data Infrastructure Essentials Book (CRC Press). Software Defined Data Infrastructure Essentials is now generally available at various global venues in hardcopy, hardback print as well as various electronic versions including via Amazon and CRC Press among others. For those attending VMworld 2017 in Las Vegas, I will be doing a book signing, meet and greet at 1PM Tuesday August 29 in the VMworld book store, as well as presenting at various other fall industry events.

Software Defined Data Infrastructure Essentials Book Announcement

(Via Businesswire) Stillwater, Minnesota – August 23, 2017 – Server StorageIO, a leading independent IT industry advisory and consultancy firm, in conjunction with publisher CRC Press, a Taylor and Francis imprint, announced the release and general availability of “Software-Defined Data Infrastructure Essentials,” a new book by Greg Schulz, noted author and Server StorageIO founder.

The Software Defined Data Infrastructure Essentials book covers physical, cloud, converged (and hyper-converged), container, and virtual server storage I/O networking technologies, revealing trends, tools, techniques, and tradecraft skills.

Various IT and Cloud Infrastructure Layers including Data Infrastructures

From cloud web scale to enterprise and small environments, IoT to database, software-defined data center (SDDC) to converged and container servers, flash solid state devices (SSD) to storage and I/O networking,, the book helps develop or refine hardware, software, services and management experiences, providing real-world examples for those involved with or looking to expand their data infrastructure education knowledge and tradecraft skills.

Software Defined Data Infrastructure Essentials book topics include:

Cloud, Converged, Container, and Virtual Server Storage I/O networking
Data protection (archive, availability, backup, BC/DR, snapshot, security)
Block, file, object, structured, unstructured and data value
Analytics, monitoring, reporting, and management metrics
Industry trends, tools, techniques, decision making
Local, remote server, storage and network I/O troubleshooting
Performance, availability, capacity and economics (PACE)

Where To Purchase Your Copy

Order via Amazon.com and CRC Press along with Google Books among other global venues.

What People Are Saying About Software Defined Data Infrastructure Essentials Book

“From CIOs to operations, sales to engineering, this book is a comprehensive reference, a must-read for IT infrastructure professionals, beginners to seasoned experts,” said Tom Becchetti, advisory systems engineer.

“We had a front row seat watching Greg present live in our education workshop seminar sessions for ITC professionals in the Netherlands material that is in this book. We recommend this amazing book to expand your converged and data infrastructure knowledge from beginners to industry veterans.”

Gert and Frank Brouwer – Brouwer Storage Consultancy

“Software-Defined Data Infrastructures provides the foundational building blocks to improve your craft in several areas including applications, clouds, legacy, and more. IT professionals, as well as sales professionals and support personal, stand to gain a great deal by reading this book.”

Mark McSherry- Oracle Regional Sales Manager

“Greg Schulz has provided a complete ‘toolkit’ for storage management along with the background and framework for the storage or data infrastructure professional (or those aspiring to become one).”
Greg Brunton – Experienced Storage and Data Management Professional

“Software-defined data infrastructures are where hardware, software, server, storage, I/O networking and related services converge inside data centers or clouds to protect, preserve, secure and serve applications and data,” said Schulz. “Both readers who are new to data infrastructures and seasoned pros will find this indispensable for gaining and expanding their knowledge.”

More About Software Defined Data Infrastructure Essentials
Software Defined Data Infrastructures (SDDI) Essentials provides fundamental coverage of physical, cloud, converged, and virtual server storage I/O networking technologies, trends, tools, techniques, and tradecraft skills. From webscale, software-defined, containers, database, key-value store, cloud, and enterprise to small or medium-size business, the book is filled with techniques, and tips to help develop or refine your server storage I/O hardware, software, Software Defined Data Centers (SDDC), Software Data Infrastructures (SDI) or Software Defined Anything (SDx) and services skills. Whether you are new to data infrastructures or a seasoned pro, you will find this comprehensive reference indispensable for gaining as well as expanding experience with technologies, tools, techniques, and trends.

This book is the definitive source providing comprehensive coverage about IT and cloud Data Infrastructures for experienced industry experts to beginners. Coverage of topics spans from higher level applications down to components (hardware, software, networks, and services) that get defined to create data infrastructures that support business, web, and other information services. This includes Servers, Storage, I/O Networks, Hardware, Software, Management Tools, Physical, Software Defined Virtual, Cloud, Docker, Containers (Docker and others) as well as Bulk, Block, File, Object, Cloud, Virtual and software defined storage.

Additional topics include Data protection (Availability, Archiving, Resiliency, HA, BC, BR, DR, Backup), Performance and Capacity Planning, Converged Infrastructure (CI), Hyper-Converged, NVM and NVMe Flash SSD, Storage Class Memory (SCM), NVMe over Fabrics, Benchmarking (including metrics matter along with tools), Performance Capacity Planning and much more including whos doing what, how things work, what to use when, where, why along with current and emerging trends.

Book Features

ISBN-13: 978-1498738156
ISBN-10: 149873815X
Hardcover: 672 pages
(Available in Kindle and other electronic formats)
Over 200 illustrations and 70 plus tables
Frequently asked Questions (and answers) along with many tips
Various learning exercises, extensive glossary and appendices
Publisher: Auerbach/CRC Press Publications; 1 edition (June 19, 2017)
Language: English

SDDI and SDDC toolbox

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

Whats a data infrastructure? (Via NetworkWorld)
Software Defined Data Infrastructure Essentials (CRC Press) book companion page (includes various images, sample figures and added content)
Click here to view (PDF) table of contents (TOC)
Click here to view (PDF) Preface, who should read, how organized and related material
Search and see whats inside Software Defined Data Infrastructure Essentials using this link to Google Books
Do you have an IT trade craft skills gap? (Via NetworkWorld)
Ensure your data infrastructure remains available and resilient (Via NetworkWorld)
Data Infrastructure Primer and Overview (Its Whats Inside The Data Center)
Data Infrastructure Server Storage I/O Tradecraft Trends
Data Infrastructure Server Storage I/O related Tradecraft Overview
Some server storage I/O benchmark workload scripts (Part I)
Server Storage I/O Benchmarking Performance Resource Tools
Server Storage I/O Converged (CI) Hyper-converged (HCI) overview
Data Infrastructure industry links page – Various Data Infrastructure related links
Welcome to the Data Protection Diaries – Data protection related topics
Time to restore your backups, do you know where is your data is?
Object and Cloud Storage Center (www.objectstoragecenter.com)
www.thessdplace.com – NVM, flash, SSD, SCM and related topics
www.thenvmeplace.com – NVM Express (NVMe) related topics
Cloud and Virtual Data Storage Networking (CRC Press) “Intel Recommended Reading” book companion page
The Green and Virtual Data Center (CRC Press) “Intel Recommended Reading” book companion page
Resilient Storage Networks – Designing Flexible Scalable Data Infrastructures (Elsevier) book companion page
Other books, articles, tips, blog posts, news, events, webinars, videos by Greg Schulz and Server StorageIO (portfolio page)

Various IT and Cloud Infrastructure Layers including Data Infrastructures

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

What This All Means

Data Infrastructures exist to protect, preserve, secure and serve information along with the applications and data they depend on. With more data being created at a faster rate, along with the size of data becoming larger, increased application functionality to transform data into information means more demands on data infrastructures and their underlying resources.

Software-Defined Data Infrastructure Essentials: Cloud, Converged, and Virtual Fundamental Server Storage I/O Tradecraft is for people who are currently involved with or looking to expand their knowledge and tradecraft skills (experience) of data infrastructures. Software-defined data centers (SDDC), software data infrastructures (SDI), software-defined data infrastructure (SDDI) and traditional data infrastructures are made up of software, hardware, services, and best practices and tools spanning servers, I/O networking, and storage from physical to software-defined virtual, container, and clouds. The role of data infrastructures is to enable and support information technology (IT) and organizational information applications.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Everything is not the same in business, organizations, IT, and in particular servers, storage, and I/O. This means that there are different audiences who will benefit from reading this book. Because everything and everybody is not the same when it comes to server and storage I/O along with associated IT environments and applications, different readers may want to focus on various sections or chapters of this book.

If you are looking to expand your knowledge into an adjacent area or to understand whats under the hood, from converged, hyper-converged to traditional data infrastructures topics, this book is for you. For experienced storage, server, and networking professionals, this book connects the dots as well as provides coverage of virtualization, cloud, and other convergence themes and topics.

This book is also for those who are new or need to learn more about data infrastructure, server, storage, I/O networking, hardware, software, and services. Another audience for this book is experienced IT professionals who are now responsible for or working with data infrastructure components, technologies, tools, and techniques.

Learn more here about Software Defined Data Infrastructure (SDDI) Essentials book along with cloud, converged, and virtual fundamental server storage I/O tradecraft topics, order your copy from Amazon.com or CRC Press here, and thank you in advance for learning more about SDDI and related topics.

Ok, nuff said, for now.

May 29, 2017October 18, 2024

May 2017 Server StorageIO Data Infrastructures Update Newsletter

Volume 17, Issue V

Hello and welcome to the May 2017 issue of the Server StorageIO update newsletter.

Summer officially here in the northern hemisphere is still a few weeks away, however for all practical purposes it has arrived. What this means is that in addition to normal workplace activities and projects, there are plenty of outdoor things (as well as distractions) to attend to.

Over the past several months I have mentioned a new book that is due out this summer and which means it’s getting close to announcement time. The new book title is Software Defined Data Infrastructure Essentials – Cloud, Converged, and Virtual Fundamental Server Storage I/O Tradecraft (CRC PRess/Taylor Francis/Auerbach) that you can learn more about here (with more details being added soon). A common question is will there be electronic versions of the book and the answer is yes (more on this in future newsletter).

Data Infrastructures

Another common question is what is it about, what is a data infrastructure (see this post) and what is tradecraft (see this post). Software-Defined Data Infrastructures Essentials provides fundamental coverage of physical, cloud, converged, and virtual server storage I/O networking technologies, trends, tools, techniques, and tradecraft skills.

Software-Defined Data Infrastructures Essentials provides fundamental coverage of physical, cloud, converged, and virtual server storage I/O networking technologies, trends, tools, techniques, and tradecraft skills. From webscale, software-defined, containers, database, key-value store, cloud, and enterprise to small or medium-size business, the book is filled with techniques, and tips to help develop or refine your server storage I/O hardware, software, and services skills. Whether you are new to data infrastructures or a seasoned pro, you will find this comprehensive reference indispensable for gaining as well as expanding experience with technologies, tools, techniques, and trends.

ISBN-13: 978-1498738156
ISBN-10: 149873815X
Hardcover: 672 pages
Publisher: Auerbach Publications; 1 edition (June 2017)
Language: English

Watch for more news and insight about my new book Software-Defined Data Infrastructure Essentials soon. In the meantime, check out the various items below in this edition of the Server StorageIO Update.

In This Issue

Server StorageIO trends
Server StorageIOblog posts
Server StorageIO News Commentary
Various Events and Webinars

Industry Resources and Links
Connect and Converse With Us
About Us

Enjoy this edition of the Server StorageIO update newsletter.

Cheers GS

Data Infrastructure and IT Industry Activity Trends

Some recent Industry Activities, Trends, News and Announcements include:

Flackbox.com has some new independent (non NetApp produced) learning resources including NetApp simulator eBook and MetroCluster tutorial. Over in the Microsoft world, Thomas Maurer has a good piece about Windows Server build 2017 and all about containers. Microsoft also announced SQL Server 2017 CTP 2.1 is now available. Meanwhile here are some my experiences and thoughts from test driving Microsoft Azure Stack.

Speaking of NetApp among other announcements they released a new version of their StorageGrid object storage software. NVMe activity in the industry (and at customer sites) continues to increase with Cavium Qlogic NVMe over Fabric news, along with Broadcom recent NVMe RAID announcements. Keep in mind that if the answer is NVMe, than what are the questions.

Here is a good summary of the recent OpenStack Boston Summit. Storpool did a momentum announcement which for those of you into software defined storage, add Storpool to your watch list. On the VMware front, check out this vSAN 6.6 demo (video) of stretched cluster via Yellow Bricks.

Check out other industry news, comments, trends perspectives here.

Server StorageIOblog Posts

Recent and popular Server StorageIOblog posts include:

View other recent as well as past StorageIOblog posts here

Server StorageIO Commentary in the news

Recent Server StorageIO industry trends perspectives commentary in the news.

Via EnterpriseStorageForum: What to Do with Legacy Assets in a Flash Storage World
There is still a place for hybrid arrays. A hybrid array is the home run when it comes to leveraging your existing non-flash, non-SSD based assets today.

Via EnterpriseStorageForum: Where All-Flash Storage Makes No Sense
A bit of flash in the right place can go a long way, and everybody can benefit from at least a some of flash somewhere. Some might say the more, the better. But where you have budget constraints that simply prevent you from having more flash for things such as cold, inactive, or seldom access data, you should explore other options.

Via Bitpipe: Changing With the Times – Protecting VMs(PDF)

Via FedTech: Storage Strategies: Agencies Optimize Data Centers by Focusing on Storage

Via SearchCloudStorage: Dell EMC cloud storage strategy needs to cut through fog

Via SearchStorage: Microsemi upgrades controllers based on HPE technology

Via EnterpriseStorageForum: 8 Data Machine Learning and AI Storage Tips

Via SiliconAngle: Dell EMC announces hybrid cloud platform for Azure Stack

View more Server, Storage and I/O trends and perspectives comments here

Events and Activities

Recent and upcoming event activities.

Sep. 13-15, 2017 – Fujifilm IT Executive Summit – Seattle WA

August 28-30, 2017 – VMworld – Las Vegas

Jully 22, 2017 – TBA

June 22, 2017 – Webinar – GDPR and Microsoft Environments

May 11, 2017 – Webinar – Email Archiving, Compliance and Ransomware

See more webinars and activities on the Server StorageIO Events page here.

Server StorageIO Industry Resources and Links

Ok, nuff said, for now.

Cheers
Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Watch for the spring 2017 release of his new book “Software-Defined Data Infrastructure Essentials” (CRC Press).

May 2, 2017December 31, 2022

Broadcom aka Avago aka LSI announces SAS SATA NVMe Adapters with RAID

server storage I/O trends

Broadcom aka Avago aka LSI announces SAS SATA NVMe Adapters with RAID

In case you missed it, Broadcom formerly known as Avago who bought the LSI adapter and RAID card business announced shipping new SAS, SATA and NVMe devices.

While SAS and SATA are well established continuing to be deployed for both HDD as well as flash SSD, NVMe continues to evolve with a bright future. Likewise, while there is a focus on software-defined storage (SDS), software defined data centers (SDDC) and software defined data infrastructures (SDDI) along with advanced parity RAID including erasure codes, object storage among other technologies, there is still a need for adapter cards including traditional RAID.

Keep in mind that while probably not meeting the definition of some software-defined aficionados, the many different variations, permutations along with derivatives of RAID from mirror and replication to basic parity to advanced erasure codes (some based on Reed Solomon aka RAID 2) rely on software. Granted, some of that software is run on regular primary server processors, some on packaged in silicon via ASICs or FPGAs, or System on Chips (SOC), RAID on Chip (RoC) as well as BIOS, firmware, drivers as well as management tools.

For some environments cards such as those announced by Broadcom are used in passthru mode effectively as adapters for attaching SAS, SATA and NVMe storage devices to servers. Those servers may be deployed as converged infrastructures (CI), hyper-converged infrastructures (HCI), Cluster or Cloud in Box (CiB) among other variations. To name names you might find the above (or in the not so distant future) in VMware vSAN or regular vSphere based environments, Microsoft Windows Server, Storage Spaces Direct (S2D) or Azure Stack, OpenStack among other deployments (check your vendors Hardware Compatibility Lists aka HCLs). In some cases these cards may be adapters in passthru mode, or using their RAID (support various by different software stacks). Meanwhile in other environments, the more traditional RAID features are still used spanning Windows to Linux among others.

Who Is Broadcom?

Some of you may know of Broadcom having been around for many years with a focus on networking related technologies. However some may not realize that Avago bought Broadcom and changed their name to Broadcom. Here is a history that includes more recent acquisitions such as Brocade, PLX, Emulex as well as LSI. Some of you may recall Avago buying LSI (the SAS, SATA, PCIe HBA, RAID and components) business not sold to NetApp as part of Engenio. Also recall that Avago sold the LSI flash SSD business unit to Seagate a couple of years ago as part of its streamlining. That’s how we get to where we are at today with Broadcom aka formerly known as Avago who bought the LSI adapter and RAID business announcing new SAS, SATA, NVMe cards.

What Was Announced?

Broadcom has announced cards that are multi-protocol supporting Serial Attached SCSI (SAS), SATA/AHCI as well as NVM Express (NVMe) as basic adapters for attaching storage (HDD, SSD, storage systems) along with optional RAID as well as cache support. These cards can be used in application servers for traditional, as well as virtualized SDDC environments, as well as storage systems or appliances for software-defined storage among other uses. The basic functionality of these cards is to provide high performance (IOPs and other activity, as well as bandwidth) along with low latency combined with data protection as well as dense connectivity.

Specific features include:

Broadcom’s Tri-Mode SerDes Technology enables the operation of NVMe, SAS or SATA devices in a single drive bay, allowing for endless design flexibility.
Management software including LSI Storage Authority (LSA), StorCLI, HII (UEFI)
Optional CacheVault(R) flash cache protection
Physical dimension Low Profile 6.127” x 2.712”
Host bus type x8 lane PCIe Express 3.1
Data transfer rates SAS-3 12Gbs; NVMe up to 8 GT/s PCIe Gen 3
Various OS and hypervisors host platform support
Warranty 3 yrs, free 5×8 phone support, advanced replacement option
RAID levels 0, 1, 5, 6, 10, 50, and 60

Note that some of the specific feature functionality may be available at a later date, check with your preferred vendors HCL

Specification	9480 8i8e	9440 8i	9460 8i	9460 16i
Image
Internal Ports	8		8	16
Internal Connectors	2 x Mini-SAS HD x4 SFF-8643	2 x Mini-SAS HD x4 SFF-8643	2 x Mini-SAS HD x4 SFF-8643	4 Mini-SAS HD x4 SFF-8643
External Ports	8
External Connectors	2 x Mini-SAS HD SFF8644
Cache Protection	CacheVault CVPM05		CacheVault CVPM05	CacheVault CVPM05
Cache Memory	2GB 2133 MHz DDR4 SDRAM		2GB 2133 MHz DDR4 SDRAM	4GB 2133 MHz DDR4 SDRAM
Devices Supported	SAS/SATA: 255, NVMe: 4 x4, up to 24 x2 or x4*	SAS/SATA: 63, NVMe: 4 x4, up to 24 x2 or x4*	SAS/SATA: 255, NVMe: 4 x4, up to 24 x2 or x4*	SAS/SATA: 255, NVMe: 4 x4, up to 24 x2 or x4*
I/O Processors (SAS Controller)	SAS3516 dual-core RAID-on-Chip (ROC)	SAS3408 I/O controller (IOC)	SAS3508 dual-core RAID-on-Chip (ROC)	SAS3516 dual-core RAID-on-Chip (ROC)

In case you need a refresher on SFF cable types, click on the following two images which take you to Amazon.com where you can learn more, as well as order various cable options. PC Pit Stop has a good selection of cables (See other SFF types), connectors and other accessories that I have used, along with those from Amazon.com and others.

Left: SFF 8644 Mini SAS HD (External), Right SFF-8643 Mini SAS HD (internal) Image via Amazon.com

Available via Amazon.com sff 8644 8642 sas mini hd cable
Left: SFF 8643 Mini SAS HD (Internal), Right SFF-8642 SATA with power (internal) Image via Amazon.com

Wait, Doesnt NVMe use PCIe

For those who are not familiar with NVMe and in particular U.2 aka SFF 8639 based devices, physically they look the same (almost) as a SAS device connector. The slight variation is if you look at a SAS drive, there is a small tab to prevent plugging into a SATA port (recall you can plug SATA into SAS. For SAS drives that tab is blank, however on the NVMe 8639 aka U.2 drives (below left) that tab has several connectors which are PCIe x4 (single or dual path).

What this means is that the PCIe x4 bus electrical signals are transferred via a connector, to backplane chassis to 8639 drive slot to the drive. Those same 8639 drive slots can also have a SAS SATA connection using their traditional connectors enabling a converged or hybrid drive slot so to speak. Learn more about NVMe here (If the Answer is NVMe, then what were and are the questions?) as well as at www.thenvmeplace.com.

Left NVMe U.2 drive showing PCIe x4 connectors, right, NVMe U.2 8639 connector

Who Is This For?

These cards are applicable for general purpose IT and other data infrastructure environments in traditional servers among others uses. They are also applicable for systems builders, integrators and OEMs whom you may be buying your current systems from, or future ones.

Where to Learn More

The following are additional resources to learn more about vSAN and related technologies.

Read the Broadcom announcement here
SSD, flash, Non-volatile memory (NVM) storage Trends, Tips & Topics
Get in the NVMe SSD game (if you are not already) with Intel 750 among others
InfoStor – SSD Trends, Tips and Topics
Server and Storage I/O Benchmarking and Performance Resources
If the Answer is NVMe, then what were and are the questions?
Announcing SAS SANs for Dummies book, LSI edition
Thessdplace.com and Thenvmeplace.com
Server StorageIO CI and HCI microsite landing page www.storageio.com/converge

What this all means

Even as the industry continues to talk and move towards more software-defined focus, even for environments that are serverless, there is still need for hardware somewhere. These adapters are a good sign of the continued maturing cycle of NVMe to be well positioned into the next decade and beyond, while also being relevant today. Likewise, even though the future involves NVMe, there is a still a place for SAS along with SATA to coexist in many environments. For some environment there is a need for traditional RAID while for others simply the need for attachment of SAS, SATA and NVMe devices. Overall, a good set of updates, enhancements and new technology for today and tomorrow, now, when do I get some to play with? ;).

Ok, nuff said (for now…).

Cheers
Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Watch for the spring 2017 release of his new book "Software-Defined Data Infrastructure Essentials" (CRC Press).

February 22, 2017November 26, 2023

Data Infrastructure IT Industry Related Resource Links K to O

IT Data Center and Data Infrastructure Industry Resources

Updated 2/20/2018

Following are some useful Data Infrastructure IT Industry Resource Links K to O to cloud, virtual and traditional IT data infrastructure related web sites. The data infrastructure environment (servers, storage, IO and networking, hardware, software, services, virtual, container and cloud) is rapidly changing. You may encounter a missing URL, or a URL that has changed. This list is updated on a regular basis to reflect changes (additions, changes, and retirement).

Disclaimer and note: URL’s submitted for inclusion on this site will be reviewed for consideration and to be in generally accepted good taste in regards to the theme of this site.

Best effort has been made to validate and verify the data infrastructure URLs that appear on this page and web site however they are subject to change. The author and/or maintainer(s) of this page and web site make no endorsement to and assume no responsibility for the URLs and their content that are listed on this page.

Send an email note to info at storageio dot com that includes company name, URL, contact name, title and phone number along with a brief 40 character description to be considered for addition to the above data infrastructure list, or, to be removed. Note that Server StorageIO and UnlimitedIO LLC (e.g. StorageIO) does not sell, trade, barter, borrow or share your contact information per our Privacy and Disclosure policy. View related data infrastructure Server StorageIO content here, and signup for our free newsletter here.

Links A-E
Links F-J
Links K-O
Links P-T
Links U-Z
Other Links

kaminario.com Scale out high performance database server
kanbox.com Chineese cloud provider
kaseya.com IT Infrastructure resource management tools
kashya.com Data Protection Solutions (Bought by EMC)
kastenchase.com Data and information security solutions
kazeon.com eDiscovery, search, indexing, classification (Bought by EMC)
kcura.com eDiscovery tools
Kerstor.com Cloud storage
kickfire.com Business analytics tools
kineticd.com Storage and data protection solutions
Kingston.com Memory manufacturer
KLSecurity.com Data protection solutions
komnetworks.com Archiving solutions
k-par.com Archiving solutions
ksplice.com Linux management tools
Kubisys.com Virtualization management tools
KVH Co. Ltd. AWS connect parter, Hosting/cloud/access services
kvsinc.com Email archiving and compliance software
lacie.com Consumer, SOHO and SMB NAS and DAS storage (Bought by Seagate)
laconicsecurity.com Cloud security and storage
lampertz.com Environmental and data center protection
laurustech.com Managed services provider
lecroy.com Analyzers, probes and diagnostics
lefthandnetworks.com iSCSI clustered storage (Bought by HP)
legato.com Storage management software (Bought by EMC)
Lenovo Servers, storage, workstations
Level 3 Communications AWS connect parter, Hosting/cloud/access services
lexar.com Flash memory
liebert.com Data center power and cooling systems (Bought by Emerson)
Lightower AWS connect parter, Hosting/cloud/access services
lightsand.com SAN connectivity
likewise.com Open backup software for macs/Linux/windows
liquidcomputing.com High density servers
liquidcoolsolutions.com Liquid cooled servers
Liquidware Labs Desktop management solutions
livedrive.com Cloud storage and backup
www.liveoffice.com Cloud and Email hosting
LiveVault.com Online cloud backup service (Bought by IronMountain)
locknet-inc.com Security and network management
Lockstep.com Backup and data protection tools
logicube.com Hard drive copy tools
LoginVSI VDI Testing solutions
losttapes.org Portal pertaining to lost magnetic tape information
lsi.com Storage and networking chips and controllers (Bought by Avago)
lto-technology.com Information about LTO tape media
lucent.com Networking components
lumigent.com Compliance management tools
luminex.com S390 IBM Mainframe to open systems gateways
luxtera.com High speed network and storage interconnects
m5data.com VAR
MagePlace Backup Backup and data protection solutions
Maginatics.com Storage for virtual environments
magma.com PCIe and storage solutions
mainline.com VAR
Maintech Infrastructure services
maintech.com IT Servers and solutions
Maldivica Cloud and object storage gateway
ManageEngine DCIM, data center and networking management
maponics.com Mapping data solutions
MAPR Hadoop and Big Data Tools

maranti.com Storage systems
marconi.com Networking equipment
marvell.com Data storage components
Masergy AWS connect parter, Hosting/cloud/access services
Mathon.com eDiscovery
maxava.com HA tools for IBM iSeries
maxell.com Data storage media
maxim-ic.com Data storage components and solutions
maxiscale.com Web infrastructure scale out file system
Maxta Virtualization storage
maxtor.com Disk drives (Bought by Seagate)
maxxan.com Storage systems (Now Ciphermax)
mcdata.com Switches, WAN gateways, software (Bought by Brocade)
mediagateusa.com HD Video streaming appliance
Meditech.com Medical information technology software
Megaport AWS connect parter, Hosting/cloud/access services
mellanox.com Ethernet and InfiniBand technology
Memeo.com Data and content management tools
Mendocino.com CDP and data protection (Missing in Action)
metalogix.net Sharepoint archiving solutions
methode.com Storage networking transceivers
mezeo.com Cloud storage management tools
micron.com SSD storage solutions
micronet.com Storage solutions
microsoft.com Hyper-V virtualizaiton, Windows Storage Server, Azure Cloud, Storsimple, SkyDrive (aka OneDrive) and more
midwave.com Var
Mimecast email management including archiving
Mimosasystems.com Email protection and archiving (Bought by Iron Mountain)
mindtree.com Testing services
Mirantis OpenStack tools, downloads, service and support
Miray HDClone Storage clone and data protection tools
mirra.com SMB storage sharing
mobiuspartners.com VAR
Modius DCIM power, cooling and monitoring
moka5.com VDI tools
MongoDB Open source document optimized database
Monitis.com IT Monitoring and services (applications, systems, clouds and more)
Monosphere.com Storage planning and usage software (Bought by Quest)
moonwalk.com HSM, ILM and data management software
morphlabs.com Dense converged server platforms
mosaictec.com Value added reseller (VAR)
mosys.com High density memory components
mpstor.com Cloud storage and orchestration software
msiinet.com VAR
mti.com European VAR
mtron.net Solid state storage devices
My Digital SSD SSD solutions
NAKIVO Backup and data protection tools
napatech.com Network tools and analysis
naspa.org System administrator’s user group
nasuni.com Cloud storage access appliance
ncino.com Banking and financial deployment platform
ncipher.com Data protection and security solutions (Bought by Thales)
ncompass-inc.com VAR and services firm
ncr.com Servers and storage solutions
ndci.com Data conversion services
ndmp.org Trade organization for NDMP backup protocol
neartek.com VTL software (Bought by EMC)
nebulassolutions.com Security and data protection
NEC.com Servers and storage
neopathnetworks.com Network file management solutions (Bought by Cisco)
neoscale.com Storage networking security (Assets Bought by nCipher)
nephoscale.com Cloud and object storage
NeptunesCloud.net Cloud IaaS solutions
www.neptuny.com Performance and capacity planning tools
nerc.com North American Electrical Reliability Council
netcelera.com WAN File system caching and acceleration
netcomm.com.au Broadband service provider
netdirectsystems.com VAR
neterion.com High performance iSCSI and Ethernet 10Gb technology Bought by Exar)
netex.com IP compression and channel extension
netezza.com Big Data – Data warehouse storage solutions (Bought by IBM)
Netgear.com SOHO and SMB storage and networking
NetIQ Data center, security, identify and data protection management
netlist.com Memory solutions
netoptics.com Network monitoring
netronome.com Network and I/O optimization technology
netspi.com Network security and digital forensics
networkappliance.com Storage sub-systems, management software
networkgeneral.com Network and application monitoring
networkinstruments.com Network test and diagnostic and performance monitoring
netwrix.com Enterprise systems management tools
netxen.com High performance 10Gb Ethernet chips and NICs
neuwingenergy.com Energy management organization
neuxpower.com File compression and data reduction
Neverfailgroup.com HA software
nevex.com Caching and application acceleration
newboundary.com IT policy management and IRM tools
newisys.com Storage enclosures
newrelic.com Web, cloud and application management
nexenta.com ZFS based storage management solutions
nexgenstorage.com Storage with PCIe flash card
nexsan.com SAS/SATA and MAID storage subsystems (Bought by Imaiton)
nextio.com SSD and application acceleration solutions
NexusMN / Computex VAR (e.g. Formerly Nexus MN and Stratos – now Computex)
nfpa.org National Fire Protection Association
nicira.com Software defined networking and IOV (Bought by VMware)
nimblestorage.com Converged iSCSI SAN, backup and DR
nimbula.com Cloud and application orchestration management tools
Nimbusdata.com iSCSI storage
nirvanix.com Cloud storage provider (Ceased operations)
njvc.com Cloud and technology service provider
Nylte DCIM software tools
noggin.intel.com/rr/ Intel Recommended Reading List
noliosoft.com Cloud application management tools
Norlight.com Data communications and services
nortel.com Networking products
nortelnetworks.com Optical networking
northernsoft.com Medical software
novastor.com Backup and cloud data protection tools
Novell.com Server software vendor (Bought by Attachmate)
Novuscg.com Storage management and services (Bought by IBM)
Nomura Research Institute (NRI) AWS connect parter, Hosting/cloud/access services
nsic.org National Storage Industry Consortium
ntp.com Storage Management Software
NTT Communications Corporation AWS connect parter, Hosting/cloud/access services
numarasoftware.com IT Resource management, asset tracking tools
numonyx.com SSD memory solutions
nutanix.com Converged server and storage VM platform
nvelo.com Caching IO performance tools
nvidia.com Graphics and visualization tools
ocarinanetworks.com Data reduction and compression tools (Bought by Dell)
ocztechnology.com SSD drives and PCIe cards (Bought by Toshiba after bankruptcy)
offsitebackups.com Online and cloud backup solutions
olixir.com Removable storage for backup and data protection
omneon.com Archiving and fixed content storage solutions
onapp.com Cloud, CDN and storage tools
onaro.com Storage and Storage Network Management (Bought by NetApp)
onlinebackupsearch.com Various online backup reviews
oblinecourses.com Various online education courses for various topics
onpathtech.com Physical layer networking
onstor.com Clustered NAS storage including NAS gateways (Bought by LSI)
Ontrack.com Email and data protection software
OO Software Windows management and data protection tools
opalis.com Virtual data center management tools
OpenCompute OpenCompute consortium
opendatacenteralliance.org Open Data Center Alliance
open-e.com iSCSI and NAS storage
Open.IO Application aware object and bulk scale out software defined storage
openfabrics.org Open Fabric Alliance
OpenNebula Open source cloud software
opennetworking.org Open Networking Foundation
OpensourceSystems.com Data storage solutions
OpenStack Open source cloud compute and storage software
opnet.com Application and network performance management
Opsware.com Network and storage management (HP)
Optica Tech. Server and storage connectivity including FICON
opticatech.com Mainframe and ESCON encryption
oracle.com Data management software, hardware and services (Bought Sun/STK and others)
Ortera.com Performance monitoring and management
OSNEXUS Quantstor ZFS based Storage management solutions
Outsource Data Recovery Data Recovery services
overlandstorage.com Tape and backup, data retention, NAS solutions
owncloud.org Cloud infrastructure tools

Where To Learn More

View additional NAS, NVMe, SSD, NVM, SCM, Data Infrastructure and HDD related topics via the following links.

Can we get a side of context with them IOPS and other storage metrics?
WHEN AND WHERE TO USE NAND FLASH SSD FOR VIRTUAL SERVERS
Revisiting RAID storage remains relevant and resources
NVMe overview and primer – Part I
Part 1 of HDD for content servers series Trends and Content Application Servers
Part 2 of HDD for content servers series Content application server decisions and testing plans
Part 3 of HDD for content servers series Test hardware and software configuration
Part 4 of HDD for content servers series Large file I/O processing
Part 5 of HDD for content servers series Small file I/O processing
Part 6 of HDD for content servers series General I/O processing
Part 7 of HDD for content servers series How HDD continue to evolve over different generations and wrap up
As the platters spin, HDD’s for cloud, virtual and traditional storage environments
How many IOPS can a HDD, HHDD or SSD do?
Hard Disk Drives (HDD) for Virtual Environments
Server and Storage I/O performance and benchmarking tools
Server storage I/O performance benchmark workload scripts Part I and Part II
How to test your HDD, SSD or all flash array (AFA) storage fundamentals
What is the best server storage I/O workload benchmark? It depends
I/O, I/O how well do you know about good or bad server and storage I/Os?
Big Files Lots of Little File Processing Benchmarking with Vdbench
Part II – NVMe overview and primer (Different Configurations)
Part III – NVMe overview and primer (Need for Performance Speed)
Part IV – NVMe overview and primer (Where and How to use NVMe)
Part V – NVMe overview and primer (Where to learn more, what this all means)
PCIe Server I/O Fundamentals
If NVMe is the answer, what are the questions?
NVMe Wont Replace Flash By Itself
Via Computerweekly – NVMe discussion: PCIe card vs U.2 and M.2
Intel and Micron unveil new 3D XPoint Non Volatie Memory (NVM) for servers and storage
Part II – Intel and Micron new 3D XPoint server and storage NVM
Part III – 3D XPoint new server storage memory from Intel and Micron
Server storage I/O benchmark tools, workload scripts and examples (Part I) and (Part II)
Data Infrastructure Overview, Its Whats Inside of Data Centers
All You Need To Know about Remote Office/Branch Office Data Protection Backup (free webinar with registration)
Software Defined, Converged Infrastructure (CI), Hyper-Converged Infrastructure (HCI) resources
The SSD Place (SSD, NVM, PM, SCM, Flash, NVMe, 3D XPoint, MRAM and related topics)
The NVMe Place (NVMe related topics, trends, tools, technologies, tip resources)
Data Protection Diaries (Archive, Backup/Restore, BC, BR, DR, HA, RAID/EC/LRC, Replication, Security)
Software Defined Data Infrastructure Essentials (CRC Press 2017) including SDDC, Cloud, Container and more
Various Data Infrastructure related events, webinars and other activities
www.objectstoragecenter.com and Software Defined, Cloud, Bulk and Object Storage Fundamentals
Server Storage I/O Network PCIe Fundamentals

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

What This All Means

Visit the following additional data infrastructure and IT data center related links.

Links A-E
Links F-J
Links K-O
Links P-T
Links U-Z
Other Links

Ok, nuff said, for now.

February 21, 2017November 26, 2023

Six plus data center software defined management dashboards tools

Software defined data infrastructure management insight tools

server storage I/O trends

Updated 1/17/2018

Managing data infrastructures involves using software defined management dashboards tools. Recently I found in my inbox a link to a piece 6 Dashboards for Managing Every Modern Data Center that caught my attention. I was hoping to see who the six different datacenter technologies, dashboard solutions tools were instead of finding list of dashboard considerations for modern data centers and data infrastructures.

Turns out the piece was nothing more than a list of six items featured as part of the vendors (Sunbird) piece about what to look for in a dashboard (e.g. their product). Sure there were some of the usual key performance indicator (KPI) associated with or related to IT Service Management (ITSM), Data Center Infrastructure (Insight/Information) Management (DCIM), Configuration and Change management databases (CMDB), availability, capacity and Performance Management Databases (PMDB) among others.

Space
Inventory
Connectivity
Change
Environment
Power

Dashboard Discussions

Keep in mind however that there are many different types of dashboards (and consoles), some are active along with analytics including correlation, others are passive simply displaying. The focus area also various from physical data center facilities, to applications, to data infrastructures or components such as servers, storage, I/O networks, clouds, virtual, containers among others modern data centers.

Data Infrastructures (hardware, software, services, servers, storage, I/O and networks)

This is where some context comes into play as there are different types of dashboards for various audience, technology and focus areas (e.g. domains) across data infrastructure (and other entities). For example do a google search of “dashboard” and see what appears, or “IT dashboard”, “data center dashboard” vs. “datacenter dashboard” among others.

Additional KPIs include:

Performance, availability, Capacity and Economic (PACE) attributes
Service Level Objectives (SLO), Service Level Agreements (SLAs)
Recovery Time Objectives (RTO), Recovery Point Objectives (SLO)
IT Service Management (ITSM) and Data Center Infrastructure Management (DCIM)
Configuration and Change Management (e.g. things part of CMDB)
Performance, availability and capacity (e.g. things part of PMDB)
Various focus and layers, cross domain functionality views
Costs management including subscriptions, licenses and others

IT Data Center and Data Infrastructure Dashboard Options

For those of you who have made it this far, while not a comprehensive list, the following are some examples of vendors, services or solutions that either are, or have an association with data center, as well as data infrastructure management. Some dashboards or tools are homogenous in that they only work within a given area of focus such as particular cloud, service provider, vendor or solution set. Others are heterogeneous or federated working across different services, solutions, vendors and domain focus areas. Think of these as software defined management (SDM), or, software defined data infrastructure (SDDI) management, software defined data center (SDDC) management among other variations for the modern information factory.

There is a mix of tools that run on site (e.g. on premise) or via cloud services (e.g. manager your on site from the cloud). Likewise, some are for fee, others subscription and some are open source. In addition some of the tools are turnkey while others are do it yourself (DiY) or allow you to customize. Also keep in mind that depending on what your tradecraft (skills, experience, expertise) interest area is, these may or may not be applicable to you, while relevant to others. For example some such as Spiceworks tend to be more helpdesk focused while others on other data center or data infrastructure areas.

There are dashboards for or from AWS, Canonical (Ubuntu), Dell including EMC, Google, HPE, IBM, Microsoft System Center and Azure, NetApp, OpenStack, Oracle, Rackspace, Redhat, Rightscale, Servicenow, Softlayer, Suse and VMware among others.

Blue Medora (various data infrastructure monitoring)
Cloudkitty (open source cloud rating and chargeback)
Collectd (data infrastructure collection and monitoring)
cPanel and whm (web and hosting dashboards)
data infrastructure sddi cpanel

Dashbuilder (customize your dashboard)
Datadog (super easy to get access, download, install, configure and use)
Domo (various data infrastructure monitoring tools)
Extrahop (still waiting to be able to download and try their bits vs. watching a demo)
Firescope (data infrastructure insight and awareness)
Freezer (open source dashboard tools)
Komprise (interesting solution, would like try, however lots of gated material)
Nagios (data infrastructure monitoring)
Openit (data infrastructure tracking, report, monitoring)
Opvizor (data infrastructure monitoring and reporting)

storageio datadog dashboard

Panorama9 (various data infrastructure monitoring and reporting)
Quest (various tools)
Redhat Cloudforms (openstack and cloud management)
Rrdtools (data collection, logging and display)
Sisense (insight and awareness tools)
Solarwinds Server Application Monitor (SAM) among other tools
Teamquest (various monitoring, management, capacity planning tools)
Turbomomic (software defined data infrastructure insight tools)
Virtual Instruments (various monitoring and insight awareness along with analytics)

In addition to the above, there are tools such as Splunk among others that also provide insight and awareness to help avoid flying blind while managing your data center or data infrastructure.

Where to learn more

Learn more via the following links.

Data Infrastructure Primer and Overview (Its Whats Inside The Data Center)

E2E Awareness and insight for IT environments

Server and Storage I/O Benchmarking and Performance Resources

Data Center Infrastructure Management (DCIM) and IRM

The Value of Infrastructure Insight – Enabling Informed Decision Making

More storage and IO metrics that matter

Whats a data infrastructure?

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

What this all means

Without insight and awareness you are flying blind, how can you make informed decisions about your information factory, data infrastructures, data center along with applications. There are different focus areas for various audiences up and down the stack layers in data infrastructures and data centers. Key is having insight and awareness including knowing what are some different tool options.

Ok, nuff said, for now.

April 25, 2016November 26, 2023

Which Enterprise HDD for Content Server Platform

Which Enterprise HDD to use for a Content Server Platform

data infrastructure HDD server storage I/O trends

Updated 1/23/2018

Which enterprise HDD to use with a content server platform?

Insight for effective server storage I/O decision making
Server StorageIO Lab Review

This post is the first in a multi-part series based on a white paper hands-on lab report I did compliments of Equus Computer Systems and Seagate that you can read in PDF form here. The focus is looking at the Equus Computer Systems (www.equuscs.com) converged Content Solution platforms with Seagate Enterprise Hard Disk Drive (HDD’s). I was given the opportunity to do some hands-on testing running different application workloads with a 2U content solution platform along with various Seagate Enterprise 2.5” HDD’s handle different application workloads. This includes Seagate’s Enterprise Performance HDD’s with the enhanced caching feature.

Issues And Challenges

Even though Non-Volatile Memory (NVM) including NAND flash solid state devices (SSDs) have become popular storage for use internal as well as external to servers, there remains the need for HDD’s Like many of you who need to make informed server, storage, I/O hardware, software and configuration selection decisions, time is often in short supply.

A common industry trend is to use SSD and HDD based storage mediums together in hybrid configurations. Another industry trend is that HDD’s continue to be enhanced with larger space capacity in the same or smaller footprint, as well as with performance improvements. Thus, a common challenge is what type of HDD to use for various content and application workloads balancing performance, availability, capacity and economics.

Content Applications and Servers

Fast Content Needs Fast Solutions

An industry and customer trend are that information and data are getting larger, living longer, as well as there is more of it. This ties to the fundamental theme that applications and their underlying hardware platforms exist to process, move, protect, preserve and serve information.

Content solutions span from video (4K, HD, SD and legacy streaming video, pre-/post-production, and editing), audio, imaging (photo, seismic, energy, healthcare, etc.) to security surveillance (including Intelligent Video Surveillance [ISV] as well as Intelligence Surveillance and Reconnaissance [ISR]). In addition to big fast data, other content solution applications include content distribution network (CDN) and caching, network function virtualization (NFV) and software-defined network (SDN), to cloud and other rich unstructured big fast media data, analytics along with little data (e.g. SQL and NoSQL database, key-value stores, repositories and meta-data) among others.

Content Solutions And HDD Opportunities

A common theme with content solutions is that they get defined with some amount of hardware (compute, memory and storage, I/O networking connectivity) as well as some type of content software. Fast content applications need fast software, multi-core processors (compute), large memory (DRAM, NAND flash, SSD and HDD’s) along with fast server storage I/O network connectivity. Content-based applications benefit from having frequently accessed data as close as possible to the application (e.g. locality of reference).

Content solution and application servers need flexibility regarding compute options (number of sockets, cores, threads), main memory (DRAM DIMMs), PCIe expansion slots, storage slots and other connectivity. An industry trend is leveraging platforms with multi-socket processors, dozens of cores and threads (e.g. logical processors) to support parallel or high-concurrent content applications. These servers have large amounts of local storage space capacity (NAND flash SSD and HDD) and associated I/O performance (PCIe, NVMe, 40 GbE, 10 GbE, 12 Gbps SAS etc.) in addition to using external shared storage (local and cloud).

Where To Learn More

Part 1 of this series – Trends and Content Application Servers
Part 2 of this series – Content application server decisions and testing plans
Part 3 of this series – Test hardware and software configuration
Part 4 of this series – Large file I/O processing
Part 5 of this series – Small file I/O processing
Part 6 of this series – General I/O processing
Part 7 of this series – How HDD continue to evolve over different generations and wrap up
As the platters spin, HDD’s for cloud, virtual and traditional storage environments
How many IOPS can a HDD, HHDD or SSD do?
Hard Disk Drives (HDD) for Virtual Environments
Server and Storage I/O performance and benchmarking tools
Additional Server StorageIO White Papers and Lab Reports, Solutions Briefs and Profiles, Tips and Articles
PDF White Paper version of this post
www.thenvmeplace.com and www.thessdplace.com

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

What This All Means

Fast content applications need fast content and flexible content solution platforms such as those from Equus Computer Systems and HDD’s from Seagate. Key to a successful content application deployment is having the flexibility to hardware define and software defined the platform to meet your needs. Just as there are many different types of content applications along with diverse environments, content solution platforms need to be flexible, scalable and robust, not to mention cost effective.

Continue reading part two of this multi-part series here where we look at how and what to test as well as project planning.

Ok, nuff said, for now.

January 24, 2014March 7, 2022

IT and data center sustainability, the other convergence zone

Storage I/O trends

IT and data center sustainability convergence

Recently Hailey Lynne McKeefry (@HaileyMcK), Editor in Chief over at Data Center Acceleration (@DataAccelerate) reached out for a conversation about well, data center themes and topics. Given Hailey’s background in covering technology as well as business supply chain we somehow ended up talking about business, IT and data center sustainability. Hailey wrote a piece about Driving for Datacenter Sustainability and in addition I was honored to be an invited guest for a live on-line chat yesterday (you can view the conversation here).

Excerpt from Haileys piece:

Too often, sustainability efforts in the datacenter are written off as feel-good, public relations efforts. In reality, green is about economics — and done well, it can save the datacenter tons of cash.

"You mention green, and datacenter managers run or cringe and roll their eyes, because there’s been so much green washing done in the past few years," said Greg Schulz, founder of IT consultancy StorageIO. "It’s really about green economics, though, and getting more work done with the same budget."

Many different faces of IT and data center sustainability

Granted, when you here the term sustainability, IT and data centers you may think of different things depending on your view or area of focus.

For some it will be Green or environmental focused such as use of renewable and EH&S themes, recycling among others Related to the previous item some will see sustainability as being tied to energy, either tied to cost, availability/accessibility, standby or alternative and renewable Yet for others, it will mean business continuance (BC), disaster recovery (DR), business resiliency (BR), high availability or reliability availability service (RAS) among others Then the economics concerns of keeping the business running to discuss top and bottom line concerns.

Otoh, if your focus is on one of the above or a subset of one of them, you might not view the other areas as being tied to sustainability.

It data center sustainability

Likewise, you might even want to not be included in another other, let alone share your area with others. For example if your focus is on security you may not want to see or hear that data protection is part of sustainability, not to mention backup/restore, bc, dr and so forth.

Learning, education and knowledge sustainability

Part of sustainability is also continuing to learn about new things not only in your field or focus area, also in adjacent spaces.

Keep in mind that there is more of a data center or information factory than just a building or facility with power, cooling as there are the technologies, tools, people, process, delivery/distribution network, warehouse for storing raw and finished material, metrics and management that all go into delivering the product which is information services.

Hence there are many aspects to IT and data center sustainability and thus think more pragmatically about sustaining information factories, however lets also be realistic and not jump the shark by declaring everything as sustainable ;).

Check out the live talk chat that we had yesterday over at Data Center Acceleration by clicking here.

Ok, nuff said

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

May 19, 2013November 26, 2023

How many I/O iops can flash SSD or HDD do?

How many i/o iops can flash ssd or hdd do with vmware?

Updated 2/10/2018

A common question I run across is how many I/O iopsS can flash SSD or HDD storage device or system do or give.

The answer is or should be it depends.

This is the first of a two-part series looking at storage performance, and in context specifically around drive or device (e.g. mediums) characteristics across HDD, HHDD and SSD that can be found in cloud, virtual, and legacy environments. In this first part the focus is around putting some context around drive or device performance with the second part looking at some workload characteristics (e.g. benchmarks).

What about cloud, tape summit resources, storage systems or appliance?

Lets leave those for a different discussion at another time.

Getting started

Part of my interest in tools, metrics that matter, measurements, analyst, forecasting ties back to having been a server, storage and IO performance and capacity planning analyst when I worked in IT. Another aspect ties back to also having been a sys admin as well as business applications developer when on the IT customer side of things. This was followed by switching over to the vendor world involved with among other things competitive positioning, customer design configuration, validation, simulation and benchmarking HDD and SSD based solutions (e.g. life before becoming an analyst and advisory consultant).

Btw, if you happen to be interested in learn more about server, storage and IO performance and capacity planning, check out my first book Resilient Storage Networks (Elsevier) that has a bit of information on it. There is also coverage of metrics and planning in my two other books The Green and Virtual Data Center (CRC Press) and Cloud and Virtual Data Storage Networking (CRC Press). I have some copies of Resilient Storage Networks available at a special reader or viewer rate (essentially shipping and handling). If interested drop me a note and can fill you in on the details.

There are many rules of thumb (RUT) when it comes to metrics that matter such as IOPS, some that are older while others may be guess or measured in different ways. However the answer is that it depends on many things ranging from if a standalone hard disk drive (HDD), Hybrid HDD (HHDD), Solid State Device (SSD) or if attached to a storage system, appliance, or RAID adapter card among others.

Taking a step back, the big picture

Various HDD, HHDD and SSD’s

Server, storage and I/O performance and benchmark fundamentals

Even if just looking at a HDD, there are many variables ranging from the rotational speed or Revolutions Per Minute (RPM), interface including 1.5Gb, 3.0Gb, 6Gb or 12Gb SAS or SATA or 4Gb Fibre Channel. If simply using a RUT or number based on RPM can cause issues particular with 2.5 vs. 3.5 or enterprise and desktop. For example, some current generation 10K 2.5 HDD can deliver the same or better performance than an older generation 3.5 15K. Other drive factors (see this link for HDD fundamentals) including physical size such as 3.5 inch or 2.5 inch small form factor (SFF), enterprise or desktop or consumer, amount of drive level cache (DRAM). Space capacity of a drive can also have an impact such as if all or just a portion of a large or small capacity devices is used. Not to mention what the drive is attached to ranging from in internal SAS or SATA drive bay, USB port, or a HBA or RAID adapter card or in a storage system.

HDD fundamentals

How about benchmark and performance for marketing or comparison tricks including delayed, deferred or asynchronous writes vs. synchronous or actually committed data to devices? Lets not forget about short stroking (only using a portion of a drive for better IOP’s) or even long stroking (to get better bandwidth leveraging spiral transfers) among others.

Almost forgot, there are also thick, standard, thin and ultra thin drives in 2.5 and 3.5 inch form factors. What’s the difference? The number of platters and read write heads. Look at the following image showing various thickness 2.5 inch drives that have various numbers of platters to increase space capacity in a given density. Want to take a wild guess as to which one has the most space capacity in a given footprint? Also want to guess which type I use for removable disk based archives along with for onsite disk based backup targets (compliments my offsite cloud backups)?

Thick, thin and ultra thin devices

Beyond physical and configuration items, then there are logical configuration including the type of workload, large or small IOPS, random, sequential, reads, writes or mixed (various random, sequential, read, write, large and small IO). Other considerations include file system or raw device, number of workers or concurrent IO threads, size of the target storage space area to decide impact of any locality of reference or buffering. Some other items include how long the test or workload simulation ran for, was the device new or worn in before use among other items.

Tools and the performance toolbox

Then there are the various tools for generating IO’s or workloads along with recording metrics such as reads, writes, response time and other information. Some examples (mix of free or for fee) include Bonnie, Iometer, Iorate, IOzone, Vdbench, TPC, SPC, Microsoft ESRP, SPEC and netmist, Swifttest, Vmark, DVDstore and PCmark 7 among many others. Some are focused just on the storage system and IO path while others are application specific thus exercising servers, storage and IO paths.

Server, storage and IO performance toolbox

Having used Iometer since the late 90s, it has its place and is popular given its ease of use. Iometer is also long in the tooth and has its limits including not much if any new development, never the less, I have it in the toolbox. I also have Futremark PCmark 7 (full version) which turns out has some interesting abilities to do more than exercise an entire Windows PC. For example PCmark can use a secondary drive for doing IO to.

PCmark can be handy for spinning up with VMware (or other tools) lots of virtual Windows systems pointing to a NAS or other shared storage device doing real world type activity. Something that could be handy for testing or stressing virtual desktop infrastructures (VDI) along with other storage systems, servers and solutions. I also have Vdbench among others tools in the toolbox including Iorate which was used to drive the workloads shown below.

What I look for in a tool are how extensible are the scripting capabilities to define various workloads along with capabilities of the test engine. A nice GUI is handy which makes Iometer popular and yes there are script capabilities with Iometer. That is also where Iometer is long in the tooth compared to some of the newer generation of tools that have more emphasis on extensibility vs. ease of use interfaces. This also assumes knowing what workloads to generate vs. simply kicking off some IOPs using default settings to see what happens.

Another handy tool is for recording what’s going on with a running system including IO’s, reads, writes, bandwidth or transfers, random and sequential among other things. This is where when needed I turn to something like HiMon from HyperIO, if you have not tried it, get in touch with Tom West over at HyperIO and tell him StorageIO sent you to get a demo or trial. HiMon is what I used for doing start, stop and boot among other testing being able to see IO’s at the Windows file system level (or below) including very early in the boot or shutdown phase.

Here is a link to some other things I did awhile back with HiMon to profile some Windows and VDI activity test profiling.

What’s the best tool or benchmark or workload generator?

The one that meets your needs, usually your applications or something as close as possible to it.

Various 2.5 and 3.5 inch HDD, HHDD, SSD with different performance

Where To Learn More

View additional NAS, NVMe, SSD, NVM, SCM, Data Infrastructure and HDD related topics via the following links.

Can we get a side of context with them IOPS and other storage metrics?
WHEN AND WHERE TO USE NAND FLASH SSD FOR VIRTUAL SERVERS
Revisiting RAID storage remains relevant and resources
NVMe overview and primer – Part I
Part 1 of HDD for content servers series Trends and Content Application Servers
Part 2 of HDD for content servers series Content application server decisions and testing plans
Part 3 of HDD for content servers series Test hardware and software configuration
Part 4 of HDD for content servers series Large file I/O processing
Part 5 of HDD for content servers series Small file I/O processing
Part 6 of HDD for content servers series General I/O processing
Part 7 of HDD for content servers series How HDD continue to evolve over different generations and wrap up
As the platters spin, HDD’s for cloud, virtual and traditional storage environments
How many IOPS can a HDD, HHDD or SSD do?
Hard Disk Drives (HDD) for Virtual Environments
Server and Storage I/O performance and benchmarking tools
Server storage I/O performance benchmark workload scripts Part I and Part II
How to test your HDD, SSD or all flash array (AFA) storage fundamentals
What is the best server storage I/O workload benchmark? It depends
I/O, I/O how well do you know about good or bad server and storage I/Os?
Big Files Lots of Little File Processing Benchmarking with Vdbench
Part II – NVMe overview and primer (Different Configurations)
Part III – NVMe overview and primer (Need for Performance Speed)
Part IV – NVMe overview and primer (Where and How to use NVMe)
Part V – NVMe overview and primer (Where to learn more, what this all means)
PCIe Server I/O Fundamentals
If NVMe is the answer, what are the questions?
NVMe Wont Replace Flash By Itself
Via Computerweekly – NVMe discussion: PCIe card vs U.2 and M.2
Intel and Micron unveil new 3D XPoint Non Volatie Memory (NVM) for servers and storage
Part II – Intel and Micron new 3D XPoint server and storage NVM
Part III – 3D XPoint new server storage memory from Intel and Micron
Server storage I/O benchmark tools, workload scripts and examples (Part I) and (Part II)
Data Infrastructure Overview, Its Whats Inside of Data Centers
All You Need To Know about Remote Office/Branch Office Data Protection Backup (free webinar with registration)
Software Defined, Converged Infrastructure (CI), Hyper-Converged Infrastructure (HCI) resources
The SSD Place (SSD, NVM, PM, SCM, Flash, NVMe, 3D XPoint, MRAM and related topics)
The NVMe Place (NVMe related topics, trends, tools, technologies, tip resources)
Data Protection Diaries (Archive, Backup/Restore, BC, BR, DR, HA, RAID/EC/LRC, Replication, Security)
Software Defined Data Infrastructure Essentials (CRC Press 2017) including SDDC, Cloud, Container and more
Various Data Infrastructure related events, webinars and other activities
www.objectstoragecenter.com and Software Defined, Cloud, Bulk and Object Storage Fundamentals
Server Storage I/O Network PCIe Fundamentals

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

What This All Means

That depends, however continue reading part II of this series to see some results for various types of drives and workloads.

Ok, nuff said, for now.

AWS Cloud Application Data Protection Webinar

Where to learn more

What this all means and wrap-up

Share this:

Data Infrastructure Resource Links Server Storage I/O Network

Where to learn more

What This All Means

Share this:

AWS Announces New S3 Cloud Storage Security Encryption Features

Default Encryption

Permission Checks

Cross-Region Replication ACL Overwrite and KMS

Detailed Inventory Report

PrivateLink for AWS Services

Where To Learn More

What This All Means

Share this:

Volume 17, Issue VII (Pre VMworld 2017)

In This Issue

Data Infrastructure and IT Industry Activity Trends

Server StorageIO Commentary in the news

Server StorageIOblog Posts

Server StorageIO Data Infrastructure Tips and Articles

Events and Activities

Share this:

Hot Popular New Trending Data Infrastructure Vendors To Watch

Some Popular Trending Vendors To Watch

Some Popular Trending Technology Trends

Where To Learn More

What This All Means

Share this:

New SDDI Essentials Book by Greg Schulz of Server StorageIO

Cloud, Converged, Virtual Fundamental Server Storage I/O Tradecraft

Software Defined Data Infrastructure Essentials Book Announcement

Where To Purchase Your Copy

What People Are Saying About Software Defined Data Infrastructure Essentials Book

Book Features

Where To Learn More

What This All Means

Share this:

Volume 17, Issue V

In This Issue

Data Infrastructure and IT Industry Activity Trends

Server StorageIOblog Posts

Server StorageIO Commentary in the news

Events and Activities

Server StorageIO Industry Resources and Links

Share this:

Broadcom aka Avago aka LSI announces SAS SATA NVMe Adapters with RAID

Who Is Broadcom?

What Was Announced?

Wait, Doesnt NVMe use PCIe

Who Is This For?

Where to Learn More

What this all means

Share this:

Data Infrastructure IT Industry Related Resource Links K to O

IT Data Center and Data Infrastructure Industry Resources

Where To Learn More

What This All Means

Share this:

Software defined data infrastructure management insight tools

Dashboard Discussions

IT Data Center and Data Infrastructure Dashboard Options

Where to learn more

What this all means

Share this:

Which Enterprise HDD to use for a Content Server Platform

Issues And Challenges

Fast Content Needs Fast Solutions

Content Solutions And HDD Opportunities

Where To Learn More

What This All Means

Share this:

IT and data center sustainability convergence

Many different faces of IT and data center sustainability

Learning, education and knowledge sustainability

Share this:

How many i/o iops can flash ssd or hdd do with vmware?

Getting started