Category: Data and Information Management

Posted on

Which Enterprise HDD for Content Server Platform

Which Enterprise HDD to use for a Content Server Platform

data infrastructure HDD server storage I/O trends

Updated 1/23/2018

Which enterprise HDD to use with a content server platform?

Insight for effective server storage I/O decision making
Server StorageIO Lab Review

Which enterprise HDD to use for content servers

This post is the first in a multi-part series based on a white paper hands-on lab report I did compliments of Equus Computer Systems and Seagate that you can read in PDF form here. The focus is looking at the Equus Computer Systems (www.equuscs.com) converged Content Solution platforms with Seagate Enterprise Hard Disk Drive (HDD’s). I was given the opportunity to do some hands-on testing running different application workloads with a 2U content solution platform along with various Seagate Enterprise 2.5” HDD’s handle different application workloads. This includes Seagate’s Enterprise Performance HDD’s with the enhanced caching feature.

Issues And Challenges

Even though Non-Volatile Memory (NVM) including NAND flash solid state devices (SSDs) have become popular storage for use internal as well as external to servers, there remains the need for HDD’s Like many of you who need to make informed server, storage, I/O hardware, software and configuration selection decisions, time is often in short supply.

A common industry trend is to use SSD and HDD based storage mediums together in hybrid configurations. Another industry trend is that HDD’s continue to be enhanced with larger space capacity in the same or smaller footprint, as well as with performance improvements. Thus, a common challenge is what type of HDD to use for various content and application workloads balancing performance, availability, capacity and economics.

Content Applications and Servers

Fast Content Needs Fast Solutions

An industry and customer trend are that information and data are getting larger, living longer, as well as there is more of it. This ties to the fundamental theme that applications and their underlying hardware platforms exist to process, move, protect, preserve and serve information.

Content solutions span from video (4K, HD, SD and legacy streaming video, pre-/post-production, and editing), audio, imaging (photo, seismic, energy, healthcare, etc.) to security surveillance (including Intelligent Video Surveillance [ISV] as well as Intelligence Surveillance and Reconnaissance [ISR]). In addition to big fast data, other content solution applications include content distribution network (CDN) and caching, network function virtualization (NFV) and software-defined network (SDN), to cloud and other rich unstructured big fast media data, analytics along with little data (e.g. SQL and NoSQL database, key-value stores, repositories and meta-data) among others.

Content Solutions And HDD Opportunities

A common theme with content solutions is that they get defined with some amount of hardware (compute, memory and storage, I/O networking connectivity) as well as some type of content software. Fast content applications need fast software, multi-core processors (compute), large memory (DRAM, NAND flash, SSD and HDD’s) along with fast server storage I/O network connectivity. Content-based applications benefit from having frequently accessed data as close as possible to the application (e.g. locality of reference).

Content solution and application servers need flexibility regarding compute options (number of sockets, cores, threads), main memory (DRAM DIMMs), PCIe expansion slots, storage slots and other connectivity. An industry trend is leveraging platforms with multi-socket processors, dozens of cores and threads (e.g. logical processors) to support parallel or high-concurrent content applications. These servers have large amounts of local storage space capacity (NAND flash SSD and HDD) and associated I/O performance (PCIe, NVMe, 40 GbE, 10 GbE, 12 Gbps SAS etc.) in addition to using external shared storage (local and cloud).

Where To Learn More

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

Fast content applications need fast content and flexible content solution platforms such as those from Equus Computer Systems and HDD’s from Seagate. Key to a successful content application deployment is having the flexibility to hardware define and software defined the platform to meet your needs. Just as there are many different types of content applications along with diverse environments, content solution platforms need to be flexible, scalable and robust, not to mention cost effective.

Continue reading part two of this multi-part series here where we look at how and what to test as well as project planning.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

HDDs evolve for Content Application servers

HDDs evolve for Content Application servers

hdds evolve server storage I/O trends

Updated 1/23/2018

Enterprise HDDs evolve for content server platform

Insight for effective server storage I/O decision making
Server StorageIO Lab Review

Which enterprise HDD to use for content servers

This is the seventh and final post in this multi-part series (read part six here) based on a white paper hands-on lab report I did compliments of Servers Direct and Seagate that you can read in PDF form here. The focus is looking at the Servers Direct (www.serversdirect.com) converged Content Solution platforms with Seagate Enterprise Hard Disk Drive (HDD’s). The focus of this post is comparing how HDD continue to evolve over various generations boosting performance as well as capacity and reliability. This also looks at how there is more to HDD performance than the traditional focus on Revolutions Per Minute (RPM) as a speed indicator.

Comparing Different Enterprise 10K And 15K HDD Generations

There is more to HDD performance than RPM speed of the device. RPM plays an important role, however there are other things that impact HDD performance. A common myth is that HDD’s have not improved on performance over the past several years with each successive generation. Table-10 shows a sampling of various generations of enterprise 10K and 15K HDD’s (14) including different form factors and how their performance continues to improve.

different 10K and 15K HDDs
Figure-9 10K and 15K HDD performance improvements

Figure-9 shows how performance continues to improve with 10K and 15K HDD’s with each new generation including those with enhanced cache features. The result is that with improvements in cache software within the drives, along with enhanced persistent non-volatile memory (NVM) and incremental mechanical drive improvements, both read and write performance continues to be enhanced.

Figure-9 puts into perspective the continued performance enhancements of HDD’s comparing various enterprise 10K and 15K devices. The workload is the same TPC-C tests used earlier in a similar (14) (with no RAID). 100 simulated users are shown in figure-9 accessing a database on each of the different drives all running concurrently. The older 15K 3.5” Cheetah and 2.5” Savio used had a capacity of 146GB which used a database scale factor of 1500 or 134GB. All other drives used a scale factor 3000 or 276GB. Figure-9 also highlights the improvements in both TPS performance as well as lower response time with new HDD’s including those with performance enhanced cache feature.

The workloads run are same as the TPC-C ones shown earlier, however these drives were not configured with any RAID. The TPC-C activity used Benchmark Factory with similar setup and configuration to those used earlier including on a multi-socket, multi-core Windows 2012 R2 server supporting a Microsoft SQL Server 2012 database with a database for each drive type.

ENT 10K V3 2.5"

ENT (Cheetah) 15K 3.5"

Users

1

20

50

100

Users

1

20

50

100

TPS (TPC-C)

14.8

50.9

30.3

39.9

TPS (TPC-C)

14.6

51.3

27.1

39.3

Resp. Time (Sec.)

0.0

0.4

1.6

1.7

Resp. Time (Sec.)

0.0

0.3

1.8

2.1

ENT 10K 2.5" (with cache)

ENT (Savio) 15K 2.5"

Users

1

20

50

100

Users

1

20

50

100

TPS (TPC-C)

19.2

146.3

72.6

71.0

TPS (TPC-C)

15.8

59.1

40.2

53.6

Resp. Time (Sec.)

0.0

0.1

0.7

0.0

Resp. Time (Sec.)

0.0

0.3

1.2

1.2

ENT 15K V4 2.5"

Users

1

20

50

100

TPS (TPC-C)

19.7

119.8

75.3

69.2

Resp. Time (Sec.)

0.0

0.1

0.6

1.0

ENT 15K (enhanced cache) 2.5"

Users

1

20

50

100

TPS (TPC-C)

20.1

184.1

113.7

122.1

Resp. Time (Sec.)

0.0

0.1

0.4

0.2

Table-10 Continued Enterprise 10K and 15K HDD performance improvements

(Note 14) 10K and 15K generational comparisons were run on a separate comparable server to what was used for other test workloads. Workload configuration settings were the same as other database workloads including using Microsoft SQL Server 2012 on a Windows 2012 R2 system with Benchmark Factory driving the workload. Database memory sized was reduced however to only 8GB vs. 16GB used in other tests.

Where To Learn More

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

A little bit of flash in the right place with applicable algorithms goes a long way, an example being the Seagate Enterprise HDD’s with enhanced cache feature. Likewise, HDD’s are very much alive complementing SSD and vice versa. For high-performance content application workloads flash SSD solutions including NVMe, 12Gbps SAS and 6Gbps SATA devices are cost effective solutions. HDD’s continue to be cost-effective data storage devices for both capacity, as well as environments that do not need the performance of flash SSD.

For some environments using a combination of flash and HDD’s complementing each other along with cache software can be a cost-effective solution. The previous workload examples provide insight for making cost-effective informed storage decisions.

Evaluate today’s HDD’s on their effective performance running workloads as close as similar to your own, or, actually try them out with your applications. Today there is more to HDD performance than just RPM speed, particular with the Seagate Enterprise Performance 10K and 15K HDD’s with enhanced caching feature.

However the Enterprise Performance 10K with enhanced cache feature provides a good balance of capacity, performance while being cost-effective. If you are using older 3.5” 15K or even previous generation 2.5” 15K RPM and “non-performance enhanced” HDD’s, take a look at how the newer generation HDD’s perform, looking beyond the RPM of the device.

Fast content applications need fast content and flexible content solution platforms such as those from Servers Direct and HDD’s from Seagate. Key to a successful content application deployment is having the flexibility to hardware define and software defined the platform to meet your needs. Just as there are many different types of content applications along with diverse environments, content solution platforms need to be flexible, scalable and robust, not to mention cost effective.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Happy Earth Day 2016 Eliminating Digital and Data e-Waste

Happy Earth Day 2016 Eliminating Digital and Data e-Waste

server storage I/O trends

With Earth Day 2016 on April 22, here are some thoughts about electronic waste (e-waste).

For those involved in data management or data infrastructures, the following are five tips to help cut the overhead and resulting impact of digital e-waste and later physical e-waste. Most conversations involving e-waste focus on the physical aspects from disposing of electronics along with later impacts. While physical e-waste is an important topic, lets expand the conversation including other variations of e-waste including digital. By digital e-waste I’m referring to the use of physical items that end up contributing to traditional e-waste.

digital and data ewaste

Digital e-waste ranges from the overhead of keeping extra copies of data that result in an expanding data footprint that in turn requires extra physical resources and their impact. Addressing physical e-waste also means keeping digital (not the physical items) including data waste in perspective. Also note that digital or data waste may in fact not be waste per say if it exists as a by-product of making sure applications, data and resulting information are protected, preserved, secured and served for when needed. The warning is what can be done to make sure there are good useful effective and efficient copy data that has a relative low data footprint overhead impact, more on this later.

Here are six themes to consider to cut the impact without costing or compromising your organization when address e-waste (physical, digital, data).

1. Understand Digital e-waste

You might be familiar with the term e-waste (electronic waste), you know, those physical items that get discarded from supporting your digital lifestyle. The reason awareness around e-waste is important is because of the environmental impacts of discarding all those devices. The more known about the issue, impacts, causes and effects helps to drive awareness as well as insight into what can be done to mitigate those items.

ewaste

Devices range from smart and dumb cell phones, personal digital assistants (PDAs), tablets, notebook and workstation computers, MP3 devices, cameras, video display monitors along with larger servers, storage and networking technology, not to mention all the other Internet of Things (IoT) and Internet of Device (IoD) items. What’s important to know about physical e-waste is the impact of the various components. You can learn more about physical e-waste impact in general with a web search such as Google e-waste impact.

2. Reuse, Repurpose, Redeploy, Reconfigure, Re-Tool, Recycle

Reconfigure and retool where possible by re-driving installing newer, more energy-efficient high-capacity drives, or more performance effective devices. Besides replacing Hard Disk Drives (HDD), Solid State Devices (SSDs), magnetic tape among other mediums, look at the pro’s and con’s of replacing CPU processor sockets, upgrading memory and PCIe I/O cards for networking or storage among other enhancements. Pro’s include being able to use the chassis longer reducing amount of physical e-waste, however at some point it can be more cost-effective to do a total replacement. However the longer you can use the asset or device the more that has as a positive benefit to cut e-waste.

Repurpose, reuse and redeploy assets such as servers, storage and networking devices in a hand me down approach assuming there is a value or benefit in doing so.

Recycle when done, dispose of the technology properly including for storage secure erase of digital media and later physical handling.

3. Responsible Recycle and Disposition of technology (including secure digital destruction)

What are you doing with, how are you disposing of physical items ranging from laptops, workstations, tablets, phones, MP3 players, TVs and monitors, servers, network and storage devices among others when no longer needed? Are you securely erasing your digital data on HDDs as well as SSDs or even tape and optical devices before they are disposed of? If not, you should be. For example if you are not yet using or looking at Self Encrypting Drives (SED) including HDDs and SSDs for securing your data, start investigating them. Sure they have a security value proposition for when lost or stolen, however they can also cut the time to secure erase to a given standard from days or hours to minutes or seconds.

These will become e-waste

Smart shopping up front, what you want, what you need, how long can you leverage, spend more up front to get something that can last 3-5 years vs. discarding in 1.5-3 years.
Smart management with insight, know your cost and impacts, not just for PR purpose, for profit and practicality

4. Plan acquisitions with disposition in mind

Redesign and design for replacement, maximizing what you have or will acquire, using it for longer time to cut costs, improve productivity (and profitability) while reducing e-waste overhead contributing footprint.

For example, do you need or want to have the latest in new technology replacing that phone, tablet, watch or other IoT or IoD item as soon as something newer comes along? No worries if you are also doing something responsible with what was new and now old by such as donating or giving it to somebody else who might be able to get a few more years worth of use out of it before it becomes e-waste.

On the other hand, if you are acquiring technology with a 2-3 year useful life plan, what would it take to upgrade that item to a larger or more robust version using it for 3-5 years. Granted, you might not use it in its primary role for the longer duration, however can it be repurposed for some other uses? Also from a technology acquisition perspective, have a forecast and plan that can help you make smart, informed decisions up front knowing when upgrades or extra resources will be needed to prolong the usefulness of the item.

Of course you can also simply move everything to the cloud and out-source your e-waste footprint to the MSP or cloud provider.

5. Understand Changing Data Value

Keep in mind that data has either no value, some value or unknown value all of which can change over time. For example some data has value for seconds, minutes or hours and can then be discarded. Other data have some value which can be low or high which determines how as well as when, where and how to protect, preserve, secure and serve it when needed. Then there is data that has an unknown value. However, that can change over time.

Different and Changing Data Value

Over time your data may end up having no value meaning it can be discarded, or, it might have some value (low or high) meaning change how it should be protected, preserved, secured and served. Then there is data that may stay in limbo or unknown status indefinitely or until somebody, or some software or via other means decide if it has value or not.

The point is that to cut digital e-waste is to discard data with no value as soon as possible, protect, preserve, secure and serve data with value appropriately. Likewise, for all of that growing data with an unknown value, rethink how it is protected and stored, all of which has an impact on both physical as well as digital e-waste.

This means having insight and awareness into your environment, applications, data, settings, configuration and metadata, not only of the space being used, or when it was last updated. Also, look beyond when data was last modified or changed, look at when it was last read or accessed to decide how protected and secured including virus and other scans.

6. Data Footprint Reduction (DFR)

Implement data footprint reduction (DFR) to lower overhead impact not only at the target or downstream destination using compression, dedupe and other techniques. Also, move upstream to the source where the problem starts and address it there. Addressing at the source leverages various techniques from Archiving, Backup/Data Protection Modernization (rethinking what saved, when, how often, etc), Cleanup, Compression and Consolidation, Data management, Deletion and Dedupe along with storage tiering, RAID/Parity/Mirroring/Replication/Erasure Code and Advanced Parity/LRC/Forward Error Correction and other technologies.

For example if you have 10TB of data, how many copies do you have and why, how are those copies protected and what is their overhead. The issue and concern should not be primarily how many copies, rather, if those copies add or give value, then what can you do to keep them while reducing their overhead impact, besides simply trying to compress or dedupe everything. Hint, start exploring copy management as well as revisiting what you protect, when, where, why, how often along with options for implementing DFR as close to the data source as possible, as well as downstream.

Where To Learn More

What This All Means

Gain insight and awareness into what is occurring with physical and digital ewaste side stepping the greenwashing and other activity. Small steps implemented by many will have a big impact. Every bit, byte, block, blob, bucket file or object along with their copies have an impact, hopefully as well as a benefit, a question is how can you reduce the overhead while increasing your return on innovation reducing costs, complexity and overhead while enhancing organization capabilities. There are many techniques, technologies, tools and approaches to apply to various environments, after all, everything is not the same, yet there are similarities. Happy Earth Day 2016 and happy spring to those of you in the northern hemisphere (as well as elsewhere).

Ok, nuff said, for now

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved

Posted on

Big Files Lots of Little File Processing Benchmarking with Vdbench

Big Files Lots of Little File Processing Benchmarking with Vdbench


server storage data infrastructure i/o File Processing Benchmarking with Vdbench

Updated 2/10/2018

Need to test a server, storage I/O networking, hardware, software, services, cloud, virtual, physical or other environment that is either doing some form of file processing, or, that you simply want to have some extra workload running in the background for what ever reason? An option is File Processing Benchmarking with Vdbench.

I/O performance

Getting Started


Here’s a quick and relatively easy way to do it with Vdbench (Free from Oracle). Granted there are other tools, both for free and for fee that can similar things, however we will leave those for another day and post. Here’s the con to this approach, there is no Uui Gui like what you have available with some other tools Here’s the pro to this approach, its free, flexible and limited by your creative, amount of storage space, server memory and I/O capacity.

If you need a background on Vdbench and benchmarking, check out the series of related posts here (e.g. www.storageio.com/performance).

Get and Install the Vdbench Bits and Bytes


If you do not already have Vdbench installed, get a copy from the Oracle or Source Forge site (now points to Oracle here).

Vdbench is free, you simply sign-up and accept the free license, select the version down load (it is a single, common distribution for all OS) the bits as well as documentation.

Installation particular on Windows is really easy, basically follow the instructions in the documentation by copying the contents of the download folder to a specified directory, set up any environment variables, and make sure that you have Java installed.

Here is a hint and tip for Windows Servers, if you get an error message about counters, open a command prompt with Administrator rights, and type the command:

$ lodctr /r


The above command will reset your I/O counters. Note however that command will also overwrite counters if enabled so only use it if you have to.

Likewise *nix install is also easy, copy the files, make sure to copy the applicable *nix shell script (they are in the download folder), and verify Java is installed and working.

You can do a vdbench -t (windows) or ./vdbench -t (*nix) to verify that it is working.

Vdbench File Processing

There are many options with Vdbench as it has a very robust command and scripting language including ability to set up for loops among other things. We are only going to touch the surface here using its file processing capabilities. Likewise, Vdbench can run from a single server accessing multiple storage systems or file systems, as well as running from multiple servers to a single file system. For simplicity, we will stick with the basics in the following examples to exercise a local file system. The limits on the number of files and file size are limited by server memory and storage space.

You can specify number and depth of directories to put files into for processing. One of the parameters is the anchor point for the file processing, in the following examples =S:\SIOTEMP\FS1 is used as the anchor point. Other parameters include the I/O size, percent reads, number of threads, run time and sample interval as well as output folder name for the result files. Note that unlike some tools, Vdbench does not create a single file of results, rather a folder with several files including summary, totals, parameters, histograms, CSV among others.


Simple Vdbench File Processing Commands

For flexibility and ease of use I put the following three Vdbench commands into a simple text file that is then called with parameters on the command line.
fsd=fsd1,anchor=!fanchor,depth=!dirdep,width=!dirwid,files=!numfiles,size=!filesize

fwd=fwd1,fsd=fsd1,rdpct=!filrdpct,xfersize=!fxfersize,fileselect=random,fileio=random,threads=!thrds

rd=rd1,fwd=fwd1,fwdrate=max,format=yes,elapsed=!etime,interval=!itime

Simple Vdbench script

# SIO_vdbench_filesystest.txt

#

# Example Vdbench script for file processing

#

# fanchor = file system place  where directories and files will be created

# dirwid = how wide should the directories be (e.g. how many directories wide)

# numfiles = how many files per directory

# filesize = size in in k, m, g e.g. 16k = 16KBytes

# fxfersize = file I/O transfer size in kbytes

# thrds = how many threads or workers

# etime = how long to run in minutes (m) or hours (h)

# itime = interval sample time e.g. 30 seconds

# dirdep = how deep the directory tree

# filrdpct = percent of reads e.g. 90 = 90 percent reads

# -p processnumber = optional specify a process number, only needed if running multiple vdbenchs at same time, number should be unique

# -o output file that describes what being done and some config info

#

# Sample command line shown for Windows, for *nix add ./

#

# The real Vdbench script with command line parameters indicated by !=

#


fsd=fsd1,anchor=!fanchor,depth=!dirdep,width=!dirwid,files=!numfiles,size=!filesize


fwd=fwd1,fsd=fsd1,rdpct=!filrdpct,xfersize=!fxfersize,fileselect=random,fileio=random,threads=!thrds


rd=rd1,fwd=fwd1,fwdrate=max,format=yes,elapsed=!etime,interval=!itime

Big Files Processing Script


With the above script file defined, for Big Files I specify a command line such as the following.
$ vdbench -f SIO_vdbench_filesystest.txt fanchor=S:\SIOTemp\FS1 dirwid=1 numfiles=60 filesize=5G fxfersize=128k thrds=64 etime=10h itime=30 numdir=1 dirdep=1 filrdpct=90 -p 5576 -o SIOWS2012R220_NOFUZE_5Gx60_BigFiles_64TH_STX1200_020116

Big Files Processing Example Results


The following is one of the result files from the folder of results created via the above command for Big File processing showing totals.



Run totals


21:09:36.001 Starting RD=format_for_rd1


Feb 01, 2016 .Interval. .ReqstdOps.. ...cpu%...  read ....read.... ...write.... ..mb/sec... mb/sec .xfer.. ...mkdir... ...rmdir... ..create... ...open.... ...close... ..delete...

                          rate  resp total  sys   pct   rate  resp   rate  resp  read write  total    size  rate  resp  rate  resp  rate  resp  rate  resp  rate  resp  rate  resp

21:23:34.101   avg_2-28 2848.2  2.70   8.8 8.32   0.0    0.0  0.00 2848.2  2.70  0.00 356.0 356.02  131071   0.0  0.00   0.0  0.00   0.1 109176   0.1  0.55   0.1  2006   0.0  0.00


21:23:35.009 Starting RD=rd1; elapsed=36000; fwdrate=max. For loops: None


07:23:35.000 avg_2-1200 4939.5  1.62  18.5 17.3  90.0 4445.8  1.79  493.7  0.07 555.7 61.72 617.44  131071   0.0  0.00   0.0  0.00   0.0  0.00   0.1  0.03   0.1  2.95   0.0  0.00

Lots of Little Files Processing Script


For lots of little files, the following is used.



$ vdbench -f SIO_vdbench_filesystest.txt fanchor=S:\SIOTEMP\FS1 dirwid=64 numfiles=25600 filesize=16k fxfersize=1k thrds=64 etime=10h itime=30 dirdep=1 filrdpct=90 -p 5576 -o SIOWS2012R220_NOFUZE_SmallFiles_64TH_STX1200_020116

Lots of Little Files Processing Example Results


The following is one of the result files from the folder of results created via the above command for Big File processing showing totals.
Run totals


09:17:38.001 Starting RD=format_for_rd1


Feb 02, 2016 .Interval. .ReqstdOps.. ...cpu%...  read ....read.... ...write.... ..mb/sec... mb/sec .xfer.. ...mkdir... ...rmdir... ..create... ...open.... ...close... ..delete...

                          rate  resp total  sys   pct   rate  resp   rate  resp  read write  total    size  rate  resp  rate  resp  rate  resp  rate  resp  rate  resp  rate  resp

09:19:48.016    avg_2-5  10138  0.14  75.7 64.6   0.0    0.0  0.00  10138  0.14  0.00 158.4 158.42   16384   0.0  0.00   0.0  0.00 10138  0.65 10138  0.43 10138  0.05   0.0  0.00


09:19:49.000 Starting RD=rd1; elapsed=36000; fwdrate=max. For loops: None


19:19:49.001 avg_2-1200 113049  0.41  67.0 55.0  90.0 101747  0.19  11302  2.42 99.36 11.04 110.40    1023   0.0  0.00   0.0  0.00   0.0  0.00  7065  0.85  7065  1.60   0.0  0.00

Where To Learn More

View additional NAS, NVMe, SSD, NVM, SCM, Data Infrastructure and HDD related topics via the following links.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

The above examples can easily be modified to do different things particular if you read the Vdbench documentation on how to setup multi-host, multi-storage system, multiple job streams to do different types of processing. This means you can benchmark a storage systems, server or converged and hyper-converged platform, or simply put a workload on it as part of other testing. There are even options for handling data footprint reduction such as compression and dedupe.

Ok, nuff said, for now.

Gs

Greg Schulz - Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

RIP Windows SIS (Single Instance Storage), or at least in Server 2016

RIP Windows SIS, or at least in Server 2016

server storage I/O trends

I received as a Microsoft MVP a partner communication today from Microsoft of a heads up as well as pass on to others that Single Instance Storage (SIS) has been removed from Windows Server 2016 (Read the Microsoft Announcement here, or below). Windows SIS is part of Microsoft’s portfolio of tools and technology for implementing Data Footprint Reduction (DFR).

Granted Windows Server 2016 has not been released yet, however you can download and try out the latest release such as Technical Preview 4 (TP4), get the bits from Microsoft here. Learn more about some of the server and storage I/O enhancements in TP4 including storage spaces direct here.

Partner Communication from Microsoft

Partner Communication
Please relay or forward this notification to ISVs and hardware partners that have used Single Instance Storage (SIS) or implemented the SIS backup API.

Single Instance Storage (SIS) has been removed from Windows Server 2016
Summary:   Single Instance Storage (SIS), a file system filter driver used for NTFS file deduplication, has been removed from Windows Server. In Dec 2015, the SIS feature has been completely removed from Windows Server and Windows Storage Server editions.  SIS was officially deprecated in Windows Server 2012 R2 in this announcement and will be removed from future Windows Server Technical Preview releases.

Call to action:
Storage vendors that have any application dependencies on legacy SIS functions or SIS backup and restore APIs should verify that their applications behave as expected on Windows Server 2016 and Windows Storage Server 2016. Windows Server 2012 included Microsoft’s next generation of deduplication technology that uses variable-sized chunking and hashing and offers far superior deduplication rates. Users and backup vendors have already moved to support the latest Microsoft deduplication technology and should continue to do so.

Background:
SIS was developed and used in Windows Server since 2000, when it was part of Remote Installation Services. SIS became a general purpose file system filter driver in Windows Storage Server 2003 and the SIS groveler (the deduplication engine) was included in Windows Storage Server. In Windows Storage Server 2008, the SIS legacy read/write filter driver was upgraded to a mini-filter and it shipped in Windows Server 2008, Windows Server 2012 and Windows Server 2012 R2 editions. Creating SIS-controlled volumes could only occur on Windows Storage Server, however, all editions of Windows Server could read and write to volumes that were under SIS control and could restore and backup volumes that had SIS applied.

Volumes using SIS that are restored or plugged into Windows Server 2016 will only be able to read data that was not deduplicated. Prior to migrating or restoring a volume, users must remove SIS from the volume by copying it to another location or removing SIS using SISadmin commands.

The SIS components and features:

  • SIS Groveler. The SIS Groveler searched for files that were identical on the NTFS file system volume. It then reported those files to the SIS filter driver.
  • SIS Storage Filter. The SIS Storage Filter was a file system filter that managed duplicate copies of files on logical volumes. This filter copied one instance of the duplicate file into the Common Store. The duplicate copies were replaced with a link to the Common Store to improve disk space utilization.
  • SIS Link. SIS links were pointers within the file system, maintaining both application and user experience (including attributes such as file size and directory path) while I/O was transparently redirected to the actual duplicate file located within the SIS Common Store.
  • SIS Common Store. The SIS Common Store served as the repository for each file identified as having duplicates. Each SIS-maintained volume contained one SIS Common Store, which contained all of the merged duplicate files that exist on that volume.
  • SIS Administrative Interface. The SIS Administrative Interface gave network administrators easy access to all SIS controls to simplify management.
  • SIS Backup API. The SIS Backup API (Sisbkup.dll) helped OEMs create SIS-aware backup and restoration solutions.

References:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa362538(v=vs.85).aspx
https://msdn.microsoft.com/en-us/library/windows/desktop/aa362512(v=vs.85).aspx
https://msdn.microsoft.com/en-us/library/dexter.functioncatall.sis(v=vs.90).aspx
https://blogs.technet.com/b/filecab/archive/2012/05/21/introduction-to-data-deduplication-in-windows-server-2012.aspx
https://blogs.technet.com/b/filecab/archive/2006/02/03/single-instance-store-sis-in-windows-storage-server-r2.aspx

What this all means

Like it or not, SIS is being removed from Windows 2016 replaced by the new Microsoft deduplication or data footprint reduction (DFR) technology.

You have been advised…

RIP Windows SIS

Ok, nuff said (for now)

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved

Posted on

Server StorageIO October 2015 Update Newsletter


Server and StorageIO Update Newsletter

Volume 15, Issue X – Industry Trends, M&A, PTSA

Hello and welcome to this October 2015 Server StorageIO update newsletter. Fall has arrived here in the northern hemisphere which means its spring in the southern hemisphere, and getting colder here. While fall means cooler out-door temperature with winter just around the corner, in the IT/ITC industry, particular the data infrastructure sector (server, storage, I/O networking, hardware, software, cloud, physical, software defined virtual) things are very hot. Sure the various industry and vendor focused conferences, road shows and mini-events with associated new product, technology or services announcements (PTSA. There are also the various merger and acquisitions (M&A) that have occurred throughout the year including the recent Dell buying EMC, and Western Digital (WD) buying SANdisk among others.

This edition of the Server StorageIO update newsletter has a focus on industry trends perspectives including recent M&A and PTSA activity. In addition to industry fall industry M&A and PTSA activity, there also plenty of conference, seminars, workshops, webinars and other events some of which you can see here on the Server StorageIO events page.

On a slightly different note, for those interested and not aware of the European Union (EU) ruling earlier this month on data privacy (e.g. Safe Harbor), here and here are a couple of links to stories discussing the new ruling changes between the EU and US (among other countries). The EU data privacy rulings involve personal data being moved out of EU countries to US data centers such as cloud and application services firms.

Enjoy this edition of the Server StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and podcast’s along with in the news commentary appearing soon.

Cheers GS

In This Issue

  • Feature Topic
  • Industry Trends News
  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts
  • Videos and Podcasts
  • Events and Webinars
  • Recommended Reading List
  • Industry Activity Trends
  • Server StorageIO Lab reports
  • New and Old Vendor Update
  • Resources and Links

    • Feature Topic – TBD

    This months feature topic theme is industry trends perspectives including M&A activity.

    Some M&A, IPO and divestiture activity includes:

    Continue reading more about NVM, NVMe, NAND flash, SSD Server and storage I/O related topics at www.thessdplace.com as well as about I/O performance, monitoring and benchmarking tools at www.storageperformance.us.

     

    StorageIOblog Posts

    Recent and popular Server StorageIOblog posts include:

    View other recent as well as past blog posts here

    Server Storage I/O Industry Activity Trends (Cloud, Virtual, Physical)

    StorageIO news (image licensed for use from Shutterstock by StorageIO)

    Some new Products Technology Services Announcements (PTSA) include:

    • Amazon Web Service (AWS) Simple Storage Service (S3) Infrequent Access (IA) storage class for inactive data with immediate access vs. Glacier cold or frozen (dormant) data with slow or time delayed access. AWS also announced Snowball bulk data import/export 50TB appliance service in addition to their earlier offered capabilities.
    • EMC Rexray (part of EMCcode) and Mesosphere (for Mesos data center operating system) have joined to enable persistent Docker volumes for Mesos (e.g. data center operating system platform).
    • Microsoft Azure recent enhancements include file access of cloud storage (on-premises and within Azure cloud) leveraging SMB interfaces. Here is a primer on Azure cloud storage service offerings. View other recent Azure Cloud Storage, Compute, Database and Data Analytics service offerings here. In addition to Microsoft Azure cloud offerings or Windows 10 desktop operating system, you can also download WIndows Server 2016 Technical Preview 3 (TP3) and see what’s new here. Some of the features include Storage Spaces Direct (e.g. DAS storage) and replication among other features.

    View other recent news and industry trends here

    StorageIO Commentary in the news

    StorageIO news (image licensed for use from Shutterstock by StorageIO)
    Recent Server StorageIO commentary and industry trends perspectives about news, activities tips, and announcements.

    • NetworkComputing: Dell buying EMC: The Storage Ramifications
    • EnterpriseTech: VMware Targets Synergies in Dell EMC Deal 
    • HPCwire: Dell to Buy EMC for $67B
    • EnterpriseStorageForum: Data Storage: Do We Really Need to Store Everything?
    • EnterpriseStorageForum: Why Hard Drives Are Here to Stay (For Now)
    • EnterpriseStorageForum: Top Ten Ways to Use OpenStack for Storage
    • EnterpriseStorageForum: Are We Heading for Storage Armageddon?

    View more Server, Storage and I/O hardware as well as software trends comments here

    Vendors you may not have heard of

    Various vendors (and service providers) you may not know or heard about recently.

    • Hedvig – Converged server storage software management tools
    • Infinidat – Another Moshe Yanai Storage System Startup
    • Mesosphere – Mesos Data Center Operating System management tools
    • Plexxi – Networking startup with former EMC executive Rich Napolitano as CEO
    • ScaleMP – Scale-out server aggregation management tools

    Check out more vendors you may know, have heard of, or that are perhaps new on the Server StorageIO Industry Links page here (over 1,000 entries and growing).

    StorageIO Tips and Articles

    Recent Server StorageIO articles appearing in different venues include:

    • Virtual Blocks (VMware Blogs):  EVO:RAIL – What Is It And Why Does It Matter?
      This is the first of a multi-part series looking at Converged Infrastructures (CI), Hyper-Converged Infrastructures (HCI), Cluster in Box (CiB) and other unified solution bundles. There is a trend of industry adoption talking about CI, HCI, CiB and other bundled solutions, along with growing IT customer adoption and deployment. Different sized organizations are looking at various types of CI solutions to meet various application and workloads needs. Read more here.
    • WServerNews.com:  Cloud (Microsoft Azure) storage considerations
      Let’s say that you have been tasked with, or decided that it is time to use (or try) public cloud storage such as Microsoft Azure. Ok, now what do you do and what decisions need to be made? Keep in mind that Microsoft Azure like many other popular public clouds provides many difference services available for fee (subscription) along with free trials. These services include applications, compute, networking, storage along with development and management platform tools. Read more here.
    • NetworkComputing:  Selecting Storage: Buzzword Bingo
      The storage industry is rife with buzzwords. Here are some of the popular ones storage buyers need to navigate carefully to find storage products that truly meet their needs. Read more here.

    • InfoStor:  What’s The Best Storage Benchmark? It Depends…

    • EnterpriseStorageForum:  NAND, DRAM, SAS/SCSI & SATA/AHCI: Not Dead, Yet!

    Check out these resources and links technology, techniques, trends as well as tools. View more tips and articles here

    StorageIO Videos and Podcasts

    StorageIO podcasts are also available via and at StorageIO.tv

    StorageIO Webinars and Industry Events

    Deltaware Emerging Technology Summit November 10, 2015

    Dell Data Protection Summit Nov 4, 2015 7AM PT

    Microsoft MVP Summit Nov 2-5, 2015

    Server Storage I/O Dutch Workshop Seminar Series
    Nijkerk Netherlands October 13-16 2015

    October 13 – Symposium: Software Defined Storage Management
    October 14 – Server Storage I/O Fundamental Trends
    October 15 – Symposium – Data Center Infrastructure Management (DCIM)
    October 16 – “Converged Day” Server and Storage Decision making

    Learn more and register at the Brouwer Consultancy website here.

    September 23 – Webinar Redmond Magazine & Dell Data Protection
    The New World Order of Data Protection – Focus on Recovery
    Learn more about the 9Rs of data protection and recovery

    See more webinars and other activities on the Server StorageIO Events page here.

    From StorageIO Labs

    Research, Reviews and Reports

    Quick Look: SATA and NVMe Flash SSD Performance
    SATA and NVMe flash SSD performance

    Non-Volatile Memory (NVM) Express (NVMe) continues to evolve as a technology for enabling and improving server storage I/O for NVM including nand flash SSD storage. NVMe streamlines performance enabling more work to be done (e.g. IOPs), data to be moved (bandwidth) at a lower response time using less CPU. The above figure is a quick look comparing nand flash SSD being accessed via SATA III (6Gbps) on the left and NVMe (x4) on the right. As with any server storage I/O performance comparisons there are many variables and take them with a grain of salt. While IOPs and bandwidth are often discussed, keep in mind that with the new protocol, drivers and device controllers with NVMe that streamline I/O less CPU is needed. Learn more about NVM, NVMe, flash, SSD and related topics at www.thessdplace.com.

    View other StorageIO lab review reports here

    Server StorageIO Recommended Reading List

    The following are various recommended reading including books, blogs and videos. If you have not done so recently, also check out the Intel Recommended Reading List (here) where you will also find a couple of my books.

    Seven Databases in Seven Weeks guide to no SQL via Amazon.com

    The Human Face of Big Data book review. To say this is a big book would be an understatement, then again, big data is a big topic with a lot of diversity if you open your eyes and think in a pragmatic way, which once you open and see the pages you will see. This is physically a big book (11x 14 inches) with lots of pictures, texts, stories, factoids and thought stimulating information of the many facets and dimensions of big data across 224 pages. The Human Face of Big Data is more than a coffee table or picture book as it is full of with information, factoids and perspectives how information and data surround us every day. Open up a copy of The Human Face of Big Data and you will see examples of how data and information are all around us, and our dependence upon it. Read more here.

    Server StorageIO Industry Resources and Links

    Check out these useful links and pages:

    storageio.com/links
    objectstoragecenter.com
    storageioblog.com/data-protection-diaries-main/
    storageperformance.us
    thenvmeplace
    thessdplace.com
    storageio.com/raid
    storageio.com/ssd

    Ok, nuff said

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Dude, Dell is Getting (Buying) an EMC and VMware Deal

    Storage I/O trends

    Dude, Dell is Getting (Buying) an EMC and VMware Deal

    Some of you might remember the marketing campaign "Dude you’re getting a Dell" to show somebody buying a Dell computer.

    Today, Dell as in Michael Dell and his corporation Dell along with partner Silver Lake investment announced a $67B USD deal that they are acquiring EMC along with their stake in VMware which will stay an independently public traded company. Dell brings strength in small and medium-mid market strength and supplier to cloud and other managed service providers, Dell financing combines with EMC strength and enterprise portfolio. This deal also reunites the two parties who before had a strong storage joint venture with Dell OEMing EMC storage for about a decade before going their separate ways in the late 2000s.

    Dell buying EMC

    Key points

    • Privately held Dell is acquiring EMC and its various business units
    • VMware will stay independent public company with Dell as major owner
    • EMC based in Hopkinton Massachusetts will be headquarters for new Dell Systems Business Unit
    • Dell Systems Business Unit will also be headquarters for Dell servers
    • New Dell Systems Business Unit joint with EMC is expected to be a $30B USD plus sized entity
    • Dell see’s revenue synergies of about 3x over 1x cost of the combined entities
    • Dell see’s ability to generate cash to service debt coming from increased revenue growth
    • EMC global support, professional services, consulting to complement Dell capabilities
    • Ability for both partners to leverage their best of strengths from SMB to enterprise to cloud

    What this means big picture

    Basically EMC has gone private under the Dell umbrella while VMware remains an independent publicly traded company, granted with EMC and now Dell being the primary shareholder of that entity. Dell went private back in 2013 with its founder Michael Dell along with Silver Lake Partners as key investors. EMC has been under pressure from activist investors to sell off its investment in VMware to increase shareholder and was rumored to have been in acquisition discussions with other organizations such as HP. Now EMC (e.g. the non-VMware part) is effectively a private held company as the Dell Systems Business Unit to be initially headquartered in Hopkinton Massachusetts (EMC Headquarters) while Dell Corporation headquarters will remain in Austin Texas.

    The server business will be based in Hopkinton, which will be targeted at around a $30B USD business. Ironic that Massachusetts used to be a focus for server vendors from Dell (acquired by Compaq and then HP), Wang, DG (acquired by EMC) among others. This transaction puts Massachusetts back on the map as the Dell System Business Unit will also now be home to Dell servers. As of the announcement, there is an expectation that the Hopkinton headquarters will grow vs. shrink. Granted., some consolidation can be expected.

    Some questions that exist (among many others)

    What about Pivotal?

    One of the questions I have is that during the announcement discussions, not much if anything has been said about Pivotal and its future role or how it will be folded in, or set up as a tracking stock or similar activity. Also something to keep in mind as food for thought, or speculation, is that GE is an investor in Pivotal and GE has made noise about becoming more prominent player in software, just saying. In the meantime, let’s wait and see what happens with Pivotal.

    What about Lenovo relationship?

    After the last Dell breakup, EMC established a partnership and initiative with Lenovo to jointly produce servers that had been being sourced from Dell or others, as well as EMC moving its Iomega SMB storage business into the Lenovo initiative. Note that about a year ago Lenovo bought the former IBM x86 server business. What will become of that partnership for servers, as well as for Iomega moving forward?

    How will product rationalization occur?

    There is some product overlap in the storage business, as well as backup/data protection among some other areas. However looking at the bigger picture, there is not much if any overlap. Where there is overlap, one near-term approach that might (this is speculation) occur is to segment potential competing products into Enterprise and Systems business vs. SMB or entry-level. This could occur for storage products such as Dell Compellent, Exanet based Fluid NAS, EqualLogic and MD (OEM from NetApp) vs. those from EMC such as VMAX, VNX, Isilon, XtremIO, Datadomain among others. Likewise, there will need to be some rationalization for backup and data protection products such as EMC Networker, Avamar vs. Dell AppAssure, vRanger, NetVault as well as their OEM partners Commvault and Symantec among others.

    VCE gets leveraged as part of go to market?

    EMC took over ownership of VCE in 2014 with Cisco still involved, in fact if a product has Vblock in its name, it will be a Cisco server and network. However look for other VCE solutions to appear as well as the VxRACK announced earlier this year. I would expect new converge infrastructure (CI), hyper-converged infrastructure (HCI) and Cluster-in-Box (CiB) solutions from VCE that would include Dell servers in the future leveraging different software (VMware among others).

    How will Dell OEM business drive things?

    Dell has had a server OEM business that has supplied technology to others, including in the past EMC. This business moves in under the new System Business Unit as part of what is or was EMC. Beyond servers, it will be interesting to see how that business unit can also move other technologies into the OEM or high volume market including to cloud and managed service providers who buy in bulk.

    Will this cause Cisco an EMC partner to buy another storage vendor?

    Maybe, that depends on what Cisco wants to do moving forward in addition to remaining a partner with EMC. Of course, if Cisco were to go storage shopping, who would that be? Perhaps DDN, Nimble or NetApp?

    With Michael Dell now having done one of, if not the largest tech deals in history, how will Larry Ellison of Oracle react?

    It has been said that the difference between God and Larry Ellison is that God was not interested in becoming Larry Ellison, however, is Larry Ellison still interested in industry bragging rights meaning will he want to do a big block buster deal involving Oracle to get some headlines, or enjoy his semi-retirement, perhaps buying a bankrupt country or something?

    Where to read, watch and learn more

    Storage I/O trends

    What this all means and wrap up

    Certainly there are many more questions about server, storage, I/O networking, cloud, virtual, software, hardware, security and management tools along with service and support that will get addressed in follow-up discussions.

    Near term, the combined entity needs to get out front and sell to customers, partners and prospects that EMC is not going away, or that Dell is going to get in the way of existing business. The two need to run as is pursuing and closing each others respective business making sure that competitors do not create barriers to deals closing and disrupting revenue. In other words, neither Dell nor EMC can afford to foster a revenue prevention department now, nor can either afford to allow any other competitor to become a revenue prevention department as a service (e.g. costing either EMC or Dell revenue).

    Overall this deal has some interesting upside synergies and potential, granted, we will need to see how things unfold.

    Disclosure: Dell and EMC have been Server StorageIO clients, and StorageIO uses Dell as well as Lenovo servers among others technologies including VMware.

    Ok, nuff said, for now…

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved

    Posted on

    Server StorageIO September 2015 Update Newsletter


    Server and StorageIO Update Newsletter

    Volume 15, Issue IX

    Hello and welcome to this September 2015 Server StorageIO update newsletter. Summer has wrapped up here in the northern hemisphere which means the fall conference season has started. In addition to large conferences, there are also many smaller events including the sessions I will be doing in Nijkerk Holland week of October 13-16, along with others (in-person and on-line) throughout the fall.

    Enjoy this edition of the Server StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and podcast’s along with in the news commentary appearing soon.

    Cheers GS

    In This Issue

  • Industry Trends News
  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts
  • Videos and Podcasts
  • Events and Webinars
  • Recommended Reading List
  • Industry Activity Trends
  • New and Old Vendor Update
  • Resources and Links

    • StorageIOblog Posts

    Recent and popular Server StorageIOblog posts include:

    View other recent as well as past blog posts here

    Server Storage I/O Industry Activity Trends (Cloud, Virtual, Physical)

    StorageIO news (image licensed for use from Shutterstock by StorageIO)

    • AWS adds new S3 (Simple Storage Service) class (Infrequent Access)
    • EMC releases ScaleIO software appliance bundle
    • VMware VSAN V6.1, VSAN Witness appliance and more
    • All Flash Array (AFA) SSD startup Pure Storage files for IPO
    • Microsoft Windows Server 2016 Tech Preview 3 (TP3) features

    View other recent news and industry trends here

    StorageIO Commentary in the news

    StorageIO news (image licensed for use from Shutterstock by StorageIO)
    Recent Server StorageIO commentary and industry trends perspectives about news, activities tips, and announcements.

    • EnterpriseStorageForum: Top Ten Ways to Use OpenStack for Storage
    • EnterpriseStorageForum: Are We Heading for Storage Armageddon?

    View more Server, Storage and I/O hardware as well as software trends comments here

    Vendors you may not have heard of

    Various vendors (and service providers) you may not know or heard about recently.

    • Formation Data Systems – Software Defined Storage Management Solutions
    • Tesora – OpenStack Trove Database Management tools

    Check out more vendors you may know, have heard of, or that are perhaps new on the Server StorageIO Industry Links page here (over 1,000 entries and growing).

    StorageIO Tips and Articles

    Recent Server StorageIO articles appearing in different venues include:

    • NetworkComputing:  Selecting Storage: It’s All About The Applications
      Choosing the right storage for your applications depends on using the PACE model, evaluating Performance, Availability, Capacity and Economics (e.g. PACE). Often when I discuss mainstream applications with people, the perception is that bandwidth only applies to big data and analytics, video, and high-performance compute (HPC) or supercomputing applications such as those used in the seismic, geo, energy, video security surveillance, or entertainment industries. The reality is that those applications can be bandwidth or throughput intensive, but they can also need a large number of small I/Os that need many IOPs to handle metadata related processing. Even bulk storage repositories for archiving, solutions using scale-out NAS, and object storage have a mix of IOPs and bandwidth. Read more here.


    • EnterpriseStorageForum:  NAND, DRAM, SAS/SCSI and SATA/AHCI: Not Dead, Yet

      Manufacturers are coming out with new non-volatile memory (NVM) media like3D XPoint. Does that mean that DRAM and other NVM media such as NAND flash are now dead?

      Do new NVM storage access protocols such as NVM Express (NVMe) mean SCSI/SAS and AHCI/SATA are now dead?

      My simple answer is no, they all have bright futures. Read more here.

    Check out these resources and links technology, techniques, trends as well as tools. View more tips and articles here

    StorageIO Videos and Podcasts

    StorageIO podcasts are also available via and at StorageIO.tv

    StorageIO Webinars and Industry Events

    Deltaware Emerging Technology Summit November 10, 2015

    Microsoft MVP Summit Nov 2-5, 2015

    Server Storage I/O Dutch Workshop Seminar Series
    Nijkerk Netherlands October 13-16 2015

    October 13 – Symposium: Software Defined Storage Management
    October 14 – Server Storage I/O Fundamental Trends
    October 15 – Symposium – Data Center Infrastructure Management (DCIM)
    October 16 – “Converged Day” Server and Storage Decision making

    Learn more and register at the Brouwer Consultancy website here.

    September 23 – Webinar Redmond Magazine & Dell Data Protection
    The New World Order of Data Protection – Focus on Recovery
    Learn more about the 9Rs of data protection and recovery

    See more webinars and other activities on the Server StorageIO Events page here.

    Server StorageIO Recommended Reading List

    The following are various recommended reading including books, blogs and videos. If you have not done so recently, also check out the Intel Recommended Reading List (here) where you will also find a couple of my books.

    Seven Databases in Seven Weeks guide to no SQL via Amazon.com
    Seven Databases in Seven Weeks (A Guide to Modern Databases and the NoSQL Movement) is a book written Eric Redmond (@coderoshi) and Jim Wilson (@hexlib), that takes a look at several non SQL based database systems. Coverage includes PostgreSQL, Riak, Apache HBase, MongoDB, Apache CouchDB, Neo4J and Redis with plenty of code and architecture examples. Also covered include relational vs. key value, columnar and document based systems among others. Read more here.

    Server StorageIO Industry Resources and Links

    Check out these useful links and pages:

    storageio.com/links
    objectstoragecenter.com
    storageioblog.com/data-protection-diaries-main/
    storageperformance.us
    thenvmeplace
    thessdplace.com
    storageio.com/raid
    storageio.com/ssd

    Ok, nuff said

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    August Server StorageIO Update Newsletter – NVM and Flash SSD Focus

    Volume 15, Issue VIII

    Hello and welcome to this August 2015 Server StorageIO update newsletter. Summer is wrapping up here in the northern hemisphere which means the fall conference season has started, holidays in progress as well as getting ready for back to school time. I have been spending my summer working on various things involving servers, storage, I/O networking hardware, software, services from cloud to containers, virtual and physical. This includes OpenStack, VMware vCloud Air, AWS, Microsoft Azure, GCS among others, as well as new versions of Microsoft Windows and Servers, Non Volatile Memory (NVM) including flash SSD, NVM Express (NVMe), databases, data protection, software defined, cache, micro-tiering and benchmarking using various tools among other things (some are still under wraps).

    Enjoy this edition of the Server StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and podcast’s along with in the news commentary appearing soon.

    Cheers GS

    In This Issue

  • Feature Topic
  • Industry Trends News
  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts
  • Videos and Podcasts
  • Events and Webinars
  • Recommended Reading List
  • Industry Activity Trends
  • Server StorageIO Lab reports
  • New and Old Vendor Update
  • Resources and Links

    • Feature Topic – Non Volatile Memory including NAND flash SSD

    Via Intel History of Memory
    Via Intel: Click above image to view history of memory

    This months feature topic theme is Non Volatile Memory (NVM) which includes technologies such as NAND flash commonly used in Solid State Devices (SSDs) storage today, as well as in USB thumb drive, mobile and hand-held devices among many other uses. NVM spans servers, storage, I/O devices along with mobile and handheld among many other technologies. In addition to NAND flash, other forms of NVM include Non Volatile Random Access Memory (NVRAM), Read Only Memory (ROM) along with some emerging new technologies including the recently announced Intel and Micron 3D XPoint among others.

    • NVMe: The Golden Ticket for Faster Flash Storage? (Via EnterpriseStorageForum)
    • What should I consider when using SSD cloud? (Via SearchCloudStorage)
    • MSP CMG, Sept. 2014 Presentation (Flash back to reality – Myths and Realities
    • Flash and SSD Industry trends perspectives plus benchmarking tips) – PDF
    • Selecting Storage: Start With Requirements (Via NetworkComputing)
    • Spot The Newest & Best Server Trends (Via Processor)
    • Market ripe for embedded flash storage as prices drop (Via Powermore (Dell))

    Continue reading more about NVM, NVMe, NAND flash, SSD Server and storage I/O related topics at www.thessdplace.com as well as about I/O performance, monitoring and benchmarking tools at www.storageperformance.us.

     

    StorageIOblog Posts

    Recent and popular Server StorageIOblog posts include:

    View other recent as well as past blog posts here

    Server Storage I/O Industry Activity Trends (Cloud, Virtual, Physical)

    StorageIO news (image licensed for use from Shutterstock by StorageIO)

    • PMC Announces NVMe SSD Controllers (Via TomsITpro)
    • New SATA SSD powers elastic cloud agility for CSPs (Via Cbronline)
    • Toshiba Solid-State Drive Family Features PCIe Technology (Via Eweek)
    • SanDisk aims CloudSpeed Ultra SSD at cloud providers (Via ITwire)
    • Everspin & Aupera reveal MRAM Module M.2 Form Factor (Via BusinessWire)
    • PMC-Sierra Scales Storage with PCIe, NVMe (Via EEtimes)
    • Seagate Grows Its Nytro Enterprise Flash Storage Line (Via InfoStor)
    • New SAS Solid State Drive From Seagate Micron Alliance (Via Seagate)
    • Samsung ups the SSD ante with faster, higher capacity drives (Via ITworld)

    View other recent news and industry trends here

    StorageIO Commentary in the news

    StorageIO news (image licensed for use from Shutterstock by StorageIO)
    Recent Server StorageIO commentary and industry trends perspectives about news, activities tips, and announcements.

    • Processor: Comments on Spot The Newest & Best Server Trends
    • Processor: Comments on A Snapshot Strategy For Backups & Data Recovery
    • EnterpriseStorageForum: Comments on Defining the Future of DR Storage
    • EnterpriseStorageForum: Comments on Top Ten Tips for DR as a Service
    • EnterpriseStorageForum: Comments on NVMe: Golden Ticket for Faster Storage

    View more Server, Storage and I/O hardware as well as software trends comments here

    Vendors you may not have heard of

    Various vendors (and service providers) you may not know or heard about recently.

    • Scala – Scale out storage management software tools
    • Reduxio – Enterprise hybrid storage with data services
    • Jam TreeSize Pro – Data discovery and storage resource analysis and reporting

    Check out more vendors you may know, have heard of, or that are perhaps new on the Server StorageIO Industry Links page here (over 1,000 entries and growing).

    StorageIO Tips and Articles

    Recent Server StorageIO articles appearing in different venues include:

    • IronMountain:  Information Lifecycle Management: Which Data Types Have Value?
      It’s important to keep in mind that on a fundamental level, there are three types of data: information that has value, information that does not have value and information that has unknown value. Data value can be measured along performance, availability, capacity and economic attributes, which define how the data gets managed across different tiers of storage. In general data can have value, unknown value or no value. Read more here.
    • EnterpriseStorageForum:  Is Future Storage Converging Around Hyper-Converged?
      Depending on who you talk or listen to, hyper-converged storage is either the future of storage, or it is a hype niche market that is not for everybody, particular not larger environments. How converged is the hyper-converged market? There are many environments that can leverage CI along with HCI, CiB or other bundles solutions. Granted, not all of those environments will converge around the same CI, CiB and HCI or pod solution bundles as everything is not the same in most IT environments and data centers. Not all markets, environments or solutions are the same. Read more here.

    Check out these resources and links technology, techniques, trends as well as tools. View more tips and articles here

    StorageIO Videos and Podcasts

    StorageIO podcasts are also available via and at StorageIO.tv

    StorageIO Webinars and Industry Events

    Server Storage I/O Workshop Seminars
    Nijkerk Netherlands October 13-16 2015

    VMworld August 30-September 3 2015

    See additional webinars and other activities on the Server StorageIO Events page here.

    From StorageIO Labs

    Research, Reviews and Reports

    Enmotus FuzeDrive (Server based Micro-Tiering)
    Enmotus FuzeDrive
    • Micro-teiring of reads and writes
    • FuzeDrive for transparent tiering
    • Dynamic tiering with selectable options
    • Monitoring and diagnostics tools
    • Transparent to operating systems
    • Hardware transparent (HDD and SSD)
    • Server I/O interface agnostic
    • Optional RAM cache and file pinning
    • Maximize NVM flash SSD investment
    • Compliment other SDS solutions
    • Use for servers or workstations

    Enmotus FuzeDrive provides micro-tiering boosting performance (reads and writes) of storage attached to physical bare metal servers, virtual and cloud instances including Windows and Linux operating systems across various applications. In the simple example above five separate SQL Server databases (260GB each) were placed on a single 6TB HDD. A TPCC workload was run concurrently against all databases with various numbers of users. One workload used a single 6TB HDD (blue) while the other used a FuzeDrive (green) comprised of a 6TB HDD and a 400GB SSD showing basic micro-tiering improvements.

    View other StorageIO lab review reports here

    Server StorageIO Recommended Reading List

    The following are various recommended reading including books, blogs and videos. If you have not done so recently, also check out the Intel Recommended Reading List (here) where you will also find a couple of my books.

    Get Whats Yours via Amazon.com
    While not a technology book, you do not have to be at or near retirement age to be planning for retirement. Some of you may already be at or near retirement age, for others, its time to start planning or refining your plans. A friend recommended this book and I’m recommending it to others. Its pretty straight forward and you might be surprised how much money people may be leaving on the table! Check it out here at Amazon.com.

    Server StorageIO Industry Resources and Links

    Check out these useful links and pages:

    storageio.com/links
    objectstoragecenter.com
    storageioblog.com/data-protection-diaries-main/
    storageperformance.us
    thenvmeplace
    thessdplace.com
    storageio.com/raid
    storageio.com/ssd

    Ok, nuff said

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Some August 2015 Amazon Web Services (AWS) and Microsoft Azure Cloud Updates

    Storage I/O trends

    Some August 2015 Amazon Web Services (AWS) and Microsoft Azure Cloud Updates

    Cloud Services Providers continue to extend their feature, function and capabilities and the following are two examples. Being a customer of both Amazon Web Services (AWS) as well as Microsoft Azure (among others), I receive monthly news updates about service improvements along with new features. Here are a couple of examples involving recent updates from AWS and Azure.

    Azure enhancements

    Microsoft Azure customer update

    Azure Premium Storage generally available in Japan East

    Solid State Device (SSD) based Azure Premium Storage is now available in Japan East region. Add up to 32 TB and more than 64,000 IOPs (read operations) per virtual machine with  Azure Premium Storage. Learn more about Azure storage and pricing here.

    Azure Data Factory generally available

    Data Factory is a cloud based data integration service for automated management as well as movement and transformation of data, learn more and view pricing options here.

    AWS Partner Updates

    Recent Amazon Web Services (AWS) customer update included the following pertaining to partner storage solutions.

    AWS partner updates

    AWS Partner Network APN

    Learn more about AWS Partner Network (APN) here or click on the above image.

    AWS APN competency programs include:

    • Storage
    • Healthcare
    • Life Sciences
    • SAP Solutions
    • Microsoft Solutions
    • Oracle Solutions
    • Marketing and Commerce
    • Big Data
    • Security
    • Digital Media

    AWS Partner Network (APN) Solutions for Storage include:

    Archiving to AWS Glacier

  • Commvault
  • NetApp (AltaVault)

    • Backup to AWS using S3

  • CloudBerry Lab
  • Commvault
  • Ctera
  • Druva
  • NetApp (AltaVault)

    • Primary Cloud File and NAS storage complementing on-premises (e.g. your local) storage

  • Avere
  • Ctera
  • NetApp (Cloud OnTap)
  • Panzura
  • SoftNAS
  • Zadara

    • Secure File Transfer

  • Aspera
  • Signiant

    • Note that the above are those listed on the AWS Storage Partner Page as of this being published and subject to change. Likewise other solutions that are not part of the AWS partner program may not be listed.

    Where to read, watch and learn more

    Storage I/O trends

    What this all means and wrap up

    Cloud Service Providers (CSP) continue to enhance their capabilities, as well as their footprints as part of growth. In addition to technology, tools and number of regions, sites and data centers, the CSPs are also expanding their partner networks both about how many partners, also in the scope of those partnerships. Some of these partnerships are in the scope of the cloud as a destination, others are for enabling hybrid where public clouds become an extension complementing traditional IT. Everything is not the same in most environments and one type of cloud approach does not have to suit or fit all needs, hence the value of hybrid cloud deployment and usage.

    Ok, nuff said, for now…

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Supermicro CSE-M14TQC Use your media bay to add 12 Gbps SAS SSD drives to your server

    Storage I/O trends

    Supermicro CSE-M14TQC Use your media bay to add 12 Gbps SAS SSD drives to your server

    Do you have a computer server, workstation or mini-tower PC that needs to have more 2.5" form factor hard disk drive (HDD), solid state device (SSD) or hybrid flash drives added yet no expansion space?

    Do you also want or need the HDD or SSD drive expansion slots to be hot swappable, 6 Gbps SATA3 along with up to 12 Gbps SAS devices?

    Do you have an available 5.25" media bay slot (e.g. where you can add an optional CD or DVD drive) or can you remove your existing CD or DVD drive using USB for software loading?

    Do you need to carry out the above without swapping out your existing server or workstation on a reasonable budget, say around $100 USD plus tax, handling, shipping (your prices may vary)?

    If you need implement the above, then here is a possible solution, or in my case, an real solution.

    Via StorageIOblog Supermicro 4 x 2.5 12Gbps SAS enclosure CSE-M14TQC
    Supermicro CSE-M14TQC with hot swap canister before installing in one of my servers

    In the past I have used a solution from Startech that supports up to 4 x 2.5" 6 Gbps SAS and SATA drives in a 5.25" media bay form factor installing these in my various HP, Dell and Lenovo servers to increase internal storage bays (slots).

    Via Amazon.com StarTech SAS and SATA expansion
    Via Amazon.com StarTech 4 x 2.5" SAS and SATA internal enclosure

    I still use the StarTech device shown (read earlier reviews and experiences here, here and here) above in some of my servers which continue to be great for 6Gbps SAS and SATA 2.5" HDDs and SSDs. However for 12 Gbps SAS devices, I have used other approaches including external 12 Gbps SAS enclosures.

    Recently while talking with the folks over at Servers Direct, I mentioned how I was using StarTech 4 x 2.5" 6Gbps SAS/SATA media bay enclosure as a means of boosting the number of internal drives that could be put into some smaller servers. The Servers Direct folks told me about the Supermicro CSE-M14TQC which after doing some research, I decided to buy one to complement the StarTech 6Gbps enclosures, as well as external 12 Gbps SAS enclosures or other internal options.

    What is the Supermicro CSE-M14TQC?

    The CSE-M14TQC is a 5.25" form factor enclosure that enables four (4) 2.5" hot swappable (if your adapter and OS supports hot swap) 12 Gbps SAS or 6 Gbps SATA devices (HDD and SSD) to fit into the media bay slot normally used by CD/DVD devices in servers or workstations. There is a single Molex male power connector on the rear of the enclosure that can be used to attach to your servers available power using applicable connector adapters. In addition there are four seperate drive connectors (e.g. SATA type connectors) that support up to 12 Gbps SAS per drive which you can attach to your servers motherboard (note SAS devices need a SAS controller), HBA or RAID adapters internal ports.

    Cooling is provided via a rear mounted 12,500 RPM 16 cubic feet per minute fan, each of the four drives are hot swappable (requires operating system or hypervisor support) contained in a small canister (provided with the enclosure). Drives easily mount to the canister via screws that are also supplied as part of the enclosure kit. There is also a drive activity and failure notification LED for the devices. If you do not have any available SAS or SATA ports on your servers motherboard, you can use an available PCIe slot and add a HBA or RAID card for attaching the CSE-M14TQC to the drives. For example, a 12 Gbps SAS (6 Gbps SATA) Avago/LSI RAID card, or a 6 Gbps SAS/SATA RAID card.

    Via Supermicro CSE-M14TQC rear details (4 x SATA and 1 Molex power connector)

    Via StorageIOblog Supermicro 4 x 2.5 rear view CSE-M14TQC 12Gbps SAS enclosure
    CSE-M14TQCrear view before installation

    Via StorageIOblog Supermicro CSE-M14TQC 12Gbps SAS enclosure cabling
    CSE-M14TQC ready for installation with 4 x SATA (12 Gbps SAS) drive connectors and Molex power connector

    Tip: In the case of the Lenovo TS140 that I initially installed the CSE-M14TQC into, there is not a lot of space for installing the drive connectors or Molex power connector to the enclosure. Instead, attach the cables to the CSE-M14TQC as shown above before installing the enclosure into the media bay slot. Simply attach the connectors as shown and feed them through the media bay opening as you install the CSE-M14TQC enclosure. Then attach the drive connectors to your HBA, RAID card or server motherboard and the power connector to your power source inside the server.

    Note and disclaimer, pay attention to your server manufactures power loading and specification along with how much power will be used by the HDD or SSD’s to be installed to avoid electrical power or fire issues due to overloading!

    Via StorageIOblog Supermicro CSE-M14TQC enclosure Lenovo TS140
    CSE-M14TQC installed into Lenovo TS140 empty media bay

    Via StorageIOblog Supermicro CSE-M14TQC drive enclosure Lenovo TS140

    CSE-M14TQC installed with front face plated installed on Lenovo TS140

    Where to read, watch and learn more

    Storage I/O trends

    What this all means and wrap up

    If you have a server that simply needs some extra storage capacity by adding some 2.5" HDDs, or boosting performance with fast SSDs yet do not have any more internal drive slots or expansion bays, leverage your media bay. This applies to smaller environments where you might have one or two servers, as well as for environments where you want or need to create a scale out software defined storage or hyper-converged platform using your own hardware. Another option is that if you have a lab or test environment for VMware vSphere ESXi Windows, Linux, Openstack or other things, this can be a cost-effective approach to adding both storage space capacity as well as performance and leveraging newer 12Gbps SAS technologies.

    For example, create a VMware VSAN cluster using smaller servers such as Lenovo TS140 or equivalent where you can install a couple of 6TB or 8TB higher capacity 3.5" drive in the internal drive bays, then adding a couple of 12 Gbps SAS SSDs along with a couple of 2.5" 2TB (or larger) HDDs along with a RAID card, and high-speed networking card. If VMware VSAN is not your thing, how about setting up a Windows Server 2012 R2 failover cluster including Scale Out File Server (SOFS) with Hyper-V, or perhaps OpenStack or one of many other virtual storage appliances (VSA) or software defined storage, networking or other solutions. Perhaps you need to deploy more storage for a big data Hadoop based analytics system, or cloud or object storage solution? On the other hand, if you simply need to add some storage to your storage or media or gaming server or general purpose server, the CSE-M14TQC can be an option along with other external solutions.

    Ok, nuff said

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    VMware vCloud Air Server StorageIOlab Test Drive with videos

    Server Storage I/O trends

    VMware vCloud Air Server StorageIOlab Test Drive with videos

    Recently I was invited by VMware vCloud Air to do a free hands-on test drive of their actual production environment. Some of you may already being using VMware vSphere, vRealize and other software defined data center (SDDC) aka Virtual Server Infrastructure (VSI) or Virtual Desktop Infrastructure (VDI) tools among others. Likewise some of you may already be using one of the many cloud compute or Infrastructure as a Service (IaaS) such as Amazon Web Services (AWS) Elastic Cloud Compute (EC2), Centurylink, Google Cloud, IBM Softlayer, Microsoft Azure, Rackspace or Virtustream (being bought by EMC) among many others.

    VMware vCloud Air provides a platform similar to those just mentioned among others for your applications and their underlying resource needs (compute, memory, storage, networking) to be fulfilled. In addition, it should not be a surprise that VMware vCloud Air shares many common themes, philosophies and user experiences with the traditional on-premises based VMware solutions you may be familiar with.

    VMware vCloud Air overview

    You can give VMware vCloud Air a trial for free while the offer lasts by clicking here (service details here). Basically if you click on the link and register a new account for using VMware vCloud Air they will give you up to $500 USD in service credits to use in the real production environment while the offer lasts which iirc is through end of June 2015.

    Server StorageIO test drive VMware vCloud Air video I
    Click on above image to view video part I

    Server StorageIO test drive VMware vCloud Air part II
    Click on above image to view video part II

    What this means is that you can go and setup some servers with as many CPUs or cores, memory, Hard Disk Drive (HDD) or flash Solid State Devices (SSD) storage, external IP networks using various operating systems (Centos, Ubuntu, Windows 2008, 20012, 20012 R2) for free, or until you use up the service credits.

    Speaking of which, let me give you a bit of a tip or hint, even though you can get free time, if you provision a fast server with lots of fast SSD storage and leave it sit idle over night or over a weekend, you will chew up your free credits rather fast. So the tip which should be common sense is if you are going to do some proof of concepts and then leave things alone for a while, power the virtual cloud servers off to stretch your credits further. On the other hand, if you have something that you want to run on a fast server with fast storage over a weekend or longer, give that a try, just pay attention to your resource usage and possible charges should you exhaust your service credits.

    My Server StorageIO test drive mission objective

    For my test drive, I created a new account by using the above link to get the service credits. Note that you can use your regular VMware account with vCloud Air, however you wont get the free service credits. So while it is a few minutes of extra work, the benefit was worth it vs. simply using my existing VMware account and racking up more cloud services charges on my credit card. As part of this Server StorageIOlab test drive, I created two companion videos part I here and part II here that you can view to follow along and get a better idea of how vCloud works.

    VMware vCloud Air overview
    Phase one, create the virtual data center, database server, client servers and first setup

    My goal was to set up a simple Virtual Data Center (VDC) that would consist of five Windows 2012 R2 servers, one would be a MySQL database server with the other four being client application servers. You can download MySQL from here at Oracle as well as via other sources. For applications to simplify things I used Hammerdb as well as Benchmark Factory that is part of the Quest Toad tool set for database admins. You can download a free trial copy of Benchmark Factory here, and HammerDB here. Another tool that I used for monitoring the servers is Spotlight on Windows (SoW) which is also free here. Speaking of tools, here is a link to various server and storage I/O performance as well as monitoring tools.

    Links to tools that I used for this test-drive included:

    Setting up a virtual data center vdc
    Phase one steps and activity summary

    Summary of phase one of vdc
    Recap of what was done in phase one, watch the associated video here.

    After the initial setup (e.g. part I video here), the next step was to add some more virtual machines and take a closer look at the environment. Note that most of the work in setting up this environment was Windows, MySQL, Hammerdb, Benchmark Factory, Spotlight on Windows along with other common tools so their installation is not a focus in these videos or this post, perhaps a future post will dig into those in more depth.

    Summary of phase two of the vdc
    What was done during phase II (view the video here)

    VMware vCloud Air vdc trest drive

    There is much more to VMware vCloud Air and on their main site there are many useful links including overviews, how-too tutorials, product and service offering details and much more here. Besides paying attention to your resource usage and avoid being surprised by service charges, two other tips I can pass along that are also mentioned in the videos (here and here) is to pay attention what region you setup your virtual data centers in, second is have your network thought out ahead of time to streamline setting up the NAT and firewall as well as gateway configurations.

    Where to learn more

    Learn more about data protection and related topics, themes, trends, tools and technologies via the following links:

    Server Storage I/O trends

    What this all means and wrap-up

    Overall I like the VMware vCloud Air service which if you are VMware centric focused will be a familiar cloud option including integration with vCloud Director and other tools you may already have in your environment. Even if you are not familiar with VMware vSphere and associated vRealize tools, the vCloud service is intuitive enough that you can be productive fairly quickly. On one hand vCloud Air does not have the extensive menu of service offerings to choose from such as with AWS, Google, Azure or others, however that also means a simpler menu of options to choose from and simplify things.

    I had wanted to spend some time actually using vCloud and the offer to use some free service credits in the production environment made it worth making the time to actually setup some workloads and do some testing. Even if you are not a VMware focused environment, I would recommend giving VMware vCloud Air a test drive to see what it can do for you, as opposed to what you can do for it…

    Ok, nuff said for now

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Modernizing Data Protection = Using new and old things in new ways

    Server Storage I/O trends

    Modernizing Data Protection = Using new and old things in new ways

    This is part of an ongoing series of posts that part of www.storageioblog.com/data-protection-diaries-main/ on data protection including archiving, backup/restore, business continuance (BC), business resiliency (BC), data footprint reduction (DFR), disaster recovery (DR), High Availability (HA) along with related themes, tools, technologies, techniques, trends and strategies.

    data protection trends

    Keep in mind that a fundamental goal of an Information Technology (IT) organization is to protect, preserve and serve data and information in a cost-effective as well as productive way when needed. There is no such thing as an information recession with more data being generated and processed. In addition to more of it, data is also getting larger, having more dependencies on it being available as well as living longer (e.g. retention).

    Proof Points, No Data or Information Recession

    A quick easy proof point of more data and it getting larger is your cell phone and the pictures it take. Compare the size of those photos today to what you had in your previous generation of smart phone or even digital camera as the Mega Pixels (e.g. resolution and size of data) increased, along with the size of media (e.g. storage) to save those to also grew. Another proof point is look at your presentations, documents, web sites and other mediums with how the amount of rich or unstructured content (e.g. photos, videos) exists on those now vs. a few years ago. Yet another proof-point is to look at your structured little data databases and how there are more rows and columns, as well as how some of those columns have gotten larger or are point to external "blobs" or "objects" that have also gotten larger.

    Industry trend and challenges

    There has been industry buzz the past several years around data protection modernizing, modernizing data protection or simply modernizing backup along with modernizing your data and information infrastructure. Many of these conversations focus around swapping out an older technology in favor of whatever the new industry buzzword trend is (e.g. swap tape for disk, disk for cloud) or perhaps from one data protection, backup, archive or copy tool for another. Some of these conversations also focus around swapping legacy for virtual, cloud or some other variation of software defined marketing.

    Data protection strategy

    The Opportunity to do new things

    What is common with all the above is basically swapping out one technology, tool, medium or technique for another new one yet using it in old ways. For example tape gets swapped for disk, yet the same approach to when, where, why, how often and what gets copied or protected is left the same. Sure some new tools and technologies get introduced.  However when was the last time you put the tools down, took a step back and revisited the fundamental questions of how and why you are doing data protection the way it is being done? When was the last time you thought about data protection as an asset or business enabler as opposed to a cost center, overhead or after thought?

    Data protection tool box
    What’s in your data protection toolbox, do you know what to use when?

    What about modernizing beyond the tools

    One of the challenges with modernizing is that there is a cost involved including people time, staff skills as well as budgets not to mention keeping things running, so how do you go about paying for any improvements? Sure you can go get a data infrastructure or habitat for technology aka data home improvement loan, however there are costs associated to that.

    Big data garbage in = big data garbage out

    What about reducing data protection costs?

    So why not self-fund the improvements and modernization activities by finding and removing costs, eliminating complexity vs. moving and masking issues? Part of this can be accomplished by simply revisiting if you are treating all your applications and data the same from a data protection perspective. Are you providing a data protection service ability to your organization that is based on business wants or business needs? For example, does the business want recovery time objective (RTO) 0 and recovery point objective (RPO) 0 for all applications, while it needs RTO 4 hours and RPO 15 minutes for application-a while application-b requires RTO 12 hours and RPO of 2 hours and application must have RTO 24 hours with RPO of 12 hours?

    As a reminder RTO is how much time, or how quickly you need your applications and data to be restored and made ready for use. RPO is the point in time to where data needs to be protected as of, or the amount of data or time frame data could be lost or missing. Thus RTO = 0 means instant recovery no downtime and RPO = 0 means no loss of data. RTO one day and RPO of ten (10) minutes means applications and their data are ready for use within 24 hours and no more than 10 minutes of data can be lost (e.g. the granularity of protection coverage)., Also keep in mind that you can have various RTO and RPO combinations to meet your specific application along with business needs as part of a tiered data protection strategy implementation.

    With RTO and RPO in mind, when was the last time you sat down with the business and applications people to revisit what they want vs. what they must have? From these conversation you can easily Transition into how long to keep, how many copies in what place among other things which in turn allows you to review data protection as well as start using both old and new technologies, tools and techniques in new ways.

    Where to learn more

    Learn more about data protection and related topics, themes, trends, tools and technologies via the following links:

    Server Storage I/O trends

    What this all means and wrap-up

    Data protection is a broad topic that spans from logical and physical security to HA, BC, BR, DR, archiving (including life beyond compliance) along with various tools, technologies, techniques. Key is aligning those to the needs of the business or organization for today’s as well as tomorrows requirements. Instead of doing things what has been done in the past that may have been based on what was known or possible due to technology capabilities, why not start using new and old things in new ways. Let’s start using all the tools in the data protection toolbox regardless of if they are new or old, cloud, virtual, physical, software defined product or service in new ways while keeping the requirements of the business in focus.

    Keeping with the theme of protect preserve and serve, data protection to be modernized needs to become and be seen as a business asset or enabler vs. an after thought or cost over-head topic. Also, keep in mind that only you can prevent data loss, are your restores ready for when you need them? as well as one of the fundamental goals of IT is to protect, preserve and serve information including its applications as well as data when, where needed in a cost-effective way.

    What say you?

    Ok, nuff said for now

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    April 2015 Server StorageIO Update Newsletter

    Volume 15, Issue IV

    Hello and welcome to this April 2015 Server and StorageIO update newsletter.

    This months newsletter has a focus on cloud and object storage for bulk data, unstructured data, big data, archiving among other scenarios.

    Enjoy this edition of the Server and StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and Podcasts along with in the news commentary appearing soon.

    Storage I/O trends

    StorageIOblog posts

    April StorageIOblog posts include:

    View other recent as well as past blog posts here

    April Newsletter Feature Theme
    Cloud and Object Storage Fundamentals

    There are many facets to object storage including technology implementation, products, services, access and architectures for various applications and use scenarios. The following is a short synopsis of some basic terms and concepts associated with cloud and object storage.

    Common cloud and object storage terms

    • Account or project – Top of the hierarchy that represent owner or billing information for a service that where buckets are also attached.
    • Availability Zone (AZ) can be rack of servers and storage or data center where data is spread across for storage and durability.

      • AWS regions and availability zones (AZ)
      Example of some AWS Regions and AZ’s

    • Bucket or Container – Where objects or sub-folders containing objects are attached and accessed. Note in some environments such as AWS S3 you can have sub-folders in a bucket.
    • Connector or how your applications access the cloud or object storage such as via an API, S3, Swift, Rest, CDMI, Torrent, JSON, NAS file, block of other access gateway or software.
    • Durability – Data dispersed with copies in multiple locations to survive failure of storage or server hardware, software, zone or even region. Availability = Access + Durability.
    • End-point – Where or what your software, application or tool and utilities or gateways attach to for accessing buckets and objects.
    • Ephemeral – Temporary or non-persistent
    • Eventual consistency – Data is eventually made consistency, think in terms of asynchronous or deferred writes where there is a time lag vs. synchronous or real-time updates.
    • Immutable – Persistent, non-altered or write once read many copy of data. Objects generally are not updated, rather new objects created.

      • Object storage and cloud
      Via Cloud Virtual Data Storage (CRC)

    • Object – Byte (or bit) stream that can be as small as one byte to as large as several TBytes (some solutions and services support up to 5TByte sized objects). The object contains what ever data in any organization along with meta data. Different solutions and services support from a couple hundred KBytes of meta-data to MBytes worth of meta-data. In terms of what can be stored in an object, anything from files, videos, images, virtual disks (VMDK’s, VHDX), ZIP or tar files, backup and archive save sets, executable images or ISO’s, anything you want.
    • OPS – Objects per second or how many objects accessed similar to a IOP. Access includes gets, puts, list, head, deletes for a CRUD interface e.g. Created, Read, Update, Delete.
    • Region – Location where data is stored that can include one or more data centers also known as Availability Zones.
    • Sub-folder – While object storage can be accessed in a flat name space for commonality and organization some solutions and service support the notion of sub-folder that resemble traditional directory hierarchy.

    Learn more in Cloud Virtual Storage Networking (CRC) and www.objectstoragecenter.com

    Storage I/O trends

    OpenStack Manila (e.g. Folders and Files)

    AWS recently announced their new cloud based Elastic File Storage (EFS) to compliment their existing Elastic Block Storage (EBS) offerings. However are you aware of what is going on with cloud files within OpenStack?

    For those who are familiar with OpenStack or simply talk about it and Swift object storage, or perhaps Cinder block storage, are you aware that there is also a file (NAS or Network Attached Storage) component called Manila?

    In concept Manila should provide a similar capability to what AWS has recently announce with their Elastic File Service (EFS), or depending on your perspective, perhaps the other way around. If you are familiar and have done anything with Manila what are your initial thoughts and perspectives.

    What this all means

    People routinely tell me this is the most exciting and interesting times ever in servers, storage, I/O networking, hardware, software, backup or data protection, performance, cloud and virtual or take your pick too which I would not disagree.

    However, for the past several years (no, make that decade), there is new and more interesting things including in adjacent areas.

    I predict that at least for the next few years (no, make that decades), we will continue to see plenty of new and interesting things, questions include.

    However, what’s applicable to you and your environment vs. simply fun and interesting to watch?

    Ok, nuff said, for now

    Cheers gs

     

    In This Issue

  • Industry Trends Perspectives News
  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts
  • Events and Webinars
  • StorageIOblog posts
  • Server StorageIO Lab reports
  • Resources and Links

    • Industry News and Activity

    Recent Industry news and activity

    View other recent industry activity here

    StorageIO Commentary in the news

    StorageIO news (image licensed for use from Shutterstock by StorageIO)
    Recent Server StorageIO commentary and industry trends perspectives about news, activities and announcements.

    CyberTrend: Comments on Software Defined Data Center and Virtualization

    View more trends comments here

    StorageIO Tips and Articles

    Check out these resources and links on server storage I/O performance and benchmarking tools. View more tips and articles here

    Various Industry Events

    EMCworld – May 4-6 2015 (Las Vegas)

    Interop – April 29 2015 (Las Vegas)
    Presenting
    Smart Shopping for Your Enterprise Storage Strategy

    View other recent and upcoming events here

    Webinars


    BrightTalk Webinar – June 23 2015
    Server Storage I/O Innovation Update

    View other webinars here

    Videos and Podcasts

    Data Protection Gumbo Podcast
    Protect Preserve and Serve Data

    In this episode, Greg Schulz is a guest on Data Protection Gumbo hosted by Demetrius Malbrough(@dmalbrough). The conversation covers various aspects of data protection which has a focus of protect preserve and serve information, applications and data across different environments and customer segments.

    While we discuss enterprise and SMB data protection, we also talk about trends from Mobile to the cloud among many others tools, technologies and techniques. Check out the podcast here.

    Springtime in Kentucky
    With Kendrick Coleman of EMCcode
    Cloud Object Storage S3motion and more

    In this episode, @EMCcode (Part of EMC) developer advocate Kendrick Coleman (@KendrickColeman) joins me (e.g. Greg Schulz) for a conversation.

    Conversation covers what is EMCcode, EMC Federation, Cloud Foundry, clouds, object storage, buckets, containers, objects, node.js, Docker, OpenStack, AWS S3, micro services, and the S3motion tool Kendrick developed.

    S3motion is a good tool to have in your server storage I/O tool box for working with cloud and object storage along with others such as Cloudberry, S3fs, Cyberduck, S3 browser among many others. You can get S3motion for free from git hub here Check out the companion blog post for this podcast here.

    StorageIO podcast’s are also available via Server Storage I/O audio podcastServer Storage I/O video & at StorageIO.tv

    From StorageIO Labs

    Research, Reviews and Reports

    AWS S3 Cross-Region Replication

    AWS S3 Cross region replication
    Moving and Replicating Buckets/Containers, Sub folders and Objects (Click on Image to read about AWS Cross-Region Replication)

    View other StorageIO lab review reports here

    Resources and Links

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved