Tag: storage i/o

Posted on

PCIe Fundamentals Server Storage I/O Network Essentials

Updated 8/31/19

PCIe Fundamentals Server Storage I/O Network Essentials

PCIe fundamentals data infrastructure trends

This piece looks at PCIe Fundamentals topics for server, storage, I/O network data infrastructure environments. Peripheral Computer Interconnect (PCI) Express aka PCIe is a Server, Storage, I/O networking fundamentals component. This post is an excerpt from chapter 4 (Chapter 4: Servers: Physical, Virtual, Cloud, and Containers) of my new book Software Defined Data Infrastructure Essentials – Cloud, Converged and Virtual Fundamental Server Storage I/O Tradecraft (CRC Press 2017) Available via Amazon.com and other global venues. In this post, we look various PCIe fundamentals to learn and expand or refresh your server, storage, and I/O and networking tradecraft skills experience.

PCIe fundamentals Server Storage I/O Fundamentals

PCIe fundamental common server I/O component

Common to all servers is some form of a main system board, which can range from a few square meters in supercomputers, data center rack, tower, and micro towers converged or standalone, to small Intel NUC (Next Unit of Compute), MSI and Kepler-47 footprint, or Raspberry Pi-type desktop servers and laptops. Likewise, PCIe is commonly found in storage and networking systems, appliances among other devices.

For example, a blade server will have multiple server blades or modules, each with its motherboard, which shares a common back plane for connectivity. Another variation is a large server such as an IBM “Z” mainframe, Cray, or another supercomputer that consists of many specialized boards that function similar to a smaller-sized motherboard on a larger scale.

Some motherboards also have mezzanine or daughter boards for attachment of additional I/O networking or specialized devices. The following figure shows a generic example of a two-socket, with eight-memory-channel-type server architecture.

PCIe fundamentals SDDC, SDI, SDDI Server fundamentals
Generic computer server hardware architecture. Source: Software Defined Data Infrastructure Essentials (CRC Press 2017)

The above figure shows several PCIe, USB, SAS, SATA, 10 GbE LAN, and other I/O ports. Different servers will have various combinations of processor, and Dual Inline Memory Module (DIMM) Dynamic RAM (DRAM) sockets along with other features. What will also vary are the type and some I/O and storage expansion ports, power and cooling, along with management tools or included software.

PCIe, Including Mini-PCIe, NVMe, U.2, M.2, and GPU

At the heart of many servers I/O and connectivity solutions are the PCIe industry-standard interface (see PCIsig.com). PCIe is used to communicate with CPUs and the outside world of I/O networking devices. The importance of a faster and more efficient PCIe bus is to support more data moving in and out of servers while accessing fast external networks and storage.

For example, a server with a 40-GbE NIC or adapter would have to have a PCIe port capable of 5 GB per second. If multiple 40-GbE ports are attached to a server, you can see where the need for faster PCIe interfaces come into play.

As more VM are consolidated onto PM, as applications place more performance demand either regarding bandwidth or activity (IOPS, frames, or packets) per second, more 10-GbE adapters will be needed until the price of 40-GbE (also 25, 50 or 100 Gbe) becomes affordable. It is not if, but rather when you will grow into the performance needs on either a bandwidth/throughput basis or to support more activity and lower latency per interface.

PCIe is a serial interface specified for how servers communicate between CPUs, memory, and motherboard-mounted as well as AiC devices. This communication includes support attachment of onboard and host bus adapter (HBA) server storage I/O networking devices such as Ethernet, Fibre Channel, InfiniBand, RapidIO, NVMe (cards, drives, and fabrics), SAS, and SATA, among other interfaces.

In addition to supporting attachment of traditional LAN, SAN, MAN, and WAN devices, PCIe is also used for attaching GPU and video cards to servers. Traditionally, PCIe has been focused on being used inside of a given server chassis. Today, however, PCIe is being deployed on servers spanning nodes in dual, quad, or CiB, CI, and HCI or Software Defined Storage (SDS) deployments. Another variation of PCIe today is that multiple servers in the same rack or proximity can attach to shared devices such as storage via PCIe switches.

PCIe components (hardware and software) include:

  • Hardware chipsets, cabling, connectors, endpoints, and adapters
  • Root complex and switches, risers, extenders, retimers, and repeaters
  • Software drivers, BIOS, and management tools
  • HBAs, RAID, SSD, drives, GPU, and other AiC devices
  • Mezzanine, mini-PCIe, M.2, NVMe U.2 (8639 drive form factor)

There are many different implementations of PCIe, corresponding to generations representing speed improvements as well as physical packing options. PCIe can be deployed in various topologies, including a traditional model where an AiC such as GbE or Fibre Channel HBA connects the server to a network or storage device.

Another variation is for a server to connect to a PCIe switch, or in a shared PCIe configuration between two or more servers. In addition to different generations and topologies, there are also various PCIe form factors and physical connectors (see the following figure), ranging from AiC of various length and height, as well as M.2 small-form-factor devices and U.2 (8639) drive form-factor device for NVMe, among others.

Note that the presence of M.2 does not guarantee PCIe NVMe, as it also supports SATA.

Likewise, different NVMe devices run at various PCIe speeds based on the number of lanes. For example, in the following figure, the U.2 (8639) device (looks like a SAS device) shown is a PCIe x4.

SDDC, SDI, SDDI PCIe NVMe U.2 8639 drive fundamentals
PCIe devices NVMe U.2, M.2, and NVMe AiC. (Source: StorageIO Labs.)

PCIe leverages multiple serial unidirectional point-to-point links, known as lanes, compared to traditional PCI, which used a parallel bus design. PCIe interfaces can have one (x1), four (x4), eight (x8), sixteen (x16), or thirty-two (x32) lanes for data movement. Those PCIe lanes can be full-duplex, meaning data is sent and received at the same time, providing improved effective performance.

PCIe cards are upward-compatible, meaning that an x4 can work in an x8, an x8 in an x16, and so forth. Note, however, that the cards will not perform any faster than their specified speed; an x4 in an x8 slot will only run at x8. PCIe cards can also have single, dual, or multiple external ports and interfaces. Also, note that there are still some motherboards with legacy PCI slots that are not interoperable with PCIe cards and vice versa.

Note that PCIe cards and slots can be mechanically x1, x4, x8, x16, or x32, yet electrically (or signal) wired to a slower speed, based on the type and capabilities of the processor sockets and corresponding chipsets being used. For example, you can have a PCIe x16 slot (mechanical) that is wired for x8, which means it will only run at x8 speed.

In addition to the differences between electrical and mechanical slots, also pay attention to what generation the PCIe slots are, such as Gen 2 or Gen 3 or higher. Also, some motherboards or servers will advertise multiple PCIe slots, but those are only active with a second or additional processor socket occupied by a CPU. For example, a PCIe card that has dual x4 external PCIe ports requiring full PCIe bandwidth will need at least PCIe x8 attachment in the server slot. In other words, for full performance, the external ports on a PCIe card or device need to match the external electrical and mechanical card type and vice versa.

Recall big “B” as in Bytes vs. little “b” as in bits; for example, a PCIe Gen 3 x4 electrical could provide up to 4 GB/s bandwidth (your mileage and performance will vary), which translates to 8 × 4 GB or 32 Gbits/s. In the following table below, there is a mix of Big “B” Bytes per second and small “b” bits per second.

Each generation of PCIe has improved on the previous one by increasing the effective speed of the links. Some of the speed improvements have come from faster clock rates while implementing lower overhead encoding (e.g., from 8 b/10 b to 128 b/130 b).

For example, PCIe Gen 3 raw bit or line rate is 8 GT/s or 8 Gbps or about 2 GBps by using a 128 b/130 b encoding scheme that is very efficient compared to PCIe Gen 2 or Gen 1, which used an 8 b/10 b encoding scheme. With 8 b/10 b, there is a 20% overhead vs. a 1.5% overhead with 128 b/130 b (i.e., of 130 bits sent, 128 bits contain data, and 2 bits are for overhead).

PCIe Gen 1

PCIe Gen 2

PCIe Gen 3

PCIe Gen 4

PCIe Gen 5

Raw bit rate

2.5 GT/s

5 GT/s

8 GT/s

16 GT/s

32 GT/s

Encoding

8 b/10 b

8 b/10 b

128 b/130 b

128 b/130 b

128 b/130 b

x1 Lane bandwidth

2 Gb/s

4 Gb/s

8 Gb/s

16 Gb/s

32 Gb/s

x1 Single lane (one-way)

~250 MB/s

~500 MB/s

~1 GB/s

~2 GB/s

~4GB/s

x16 Full duplex (both ways)

~8 GB/s

~16 GB/s

~32 GB/s

~64 GB/s

~128 GB/s

Above Table: PCIe Generation and Sample Lane Comparison

Note that PCIe Gen 3 is the currently generally available shipping technology with PCIe Gen 4 appearing in the not so distant future, with PCIe Gen 5 in the wings appearing a few more years down the road.

By contrast, older generations of Fibre Channel and Ethernet also used 8 b/10 b, having switched over to 64 b/66 b encoding with 10 Gb and higher. PCIe, like other serial interfaces and protocols, can support full-duplex mode, meaning that data can be sent and received concurrently.

PCIe Bit Rate, Encoding, Giga Transfers, and Bandwidth

Let’s clarify something about data transfer or movement both internal and external to a server. At the core of a server, there is data movement within the sockets of the processors and its cores, as well as between memory and other devices (internal and external). For example, the QPI bus is used for moving data between some Intel processors whose performance is specified in giga transfers (GT).

PCIe is used for moving data between processors, memory, and other devices, including internal and external facing devices. Devices include host bus adapters (HBAs), host channel adapters (HCAs), converged network adapters (CNAs), network interface cards (NICs) or RAID cards, and others. PCIe performance is specified in multiple ways, given that it has a server processor focus which involves GT for raw bit rate as well as effective bandwidth per lane.

Note to keep in perspective PCIe mechanical as well as electrical lanes in that a card or slot may be advertised as say x8 mechanical (e.g., its physical slot form factor) yet only be x4 electrical (how many of those lanes are used or enabled). Also in the case of an adapter that has two or more ports, if the device is advertised as x8 does that mean it is x8 per port or x4 per port with an x8 connection to the PCIe bus.

Effective bandwidth per lane can be specified as half- or full-duplex (data moving in one or both directions for send and receive). Also, effective bandwidth can be specified as a single lane (x1), four lanes (x4), eight lanes (x8), sixteen lanes (x16), or 32 lanes (x32), as shown in the above table. The difference in speed or bits moved per second between the raw bit or line rate, and the effective bandwidth per lane in a single direction (i.e., half-duplex) is the encoding that is common to all serial data transmissions.

When data gets transmitted, the serializer/deserializer, or serdes, convert the bytes into a bit stream via encoding. There are different types of encoding, ranging from 8 b/10 b to 64 b/66 b and 128 b//130 b, shown in the following table.

Single 1542-byte frame

64 × 1542-byte frames

Encoding Scheme

Overhead

Data Bits

Encoding Bits

Bits Transmitted

Data Bits

Encoding Bits

Bits Transferred

8 b/10 b

20%

12,336

3,084

15,420

789,504

197,376

986,880

64 b/66 b

3%

12,336

386

12,738

789,504

24,672

814,176

128 b/130 b

1.5%

12,336

194

12,610

789,504

12,336

801,840

Above Table: Low-Level Serial Encoding Data Transmit Efficiency

In these encoding schemes, the smaller number represents the amount of data being sent, and the difference is the overhead. Note that this is different yet related to what occurs at a higher level with the various network protocols such as TCP/IP (IP). With IP, there is a data payload plus addressing and other integrity and management features in a given packet or frame.

The 8-b/10-b, 64-b/66-b or 128-b/130-b encoding is at the lower physical layer. Thus, a small change there has a big impact and benefit when optimized. Table 4.2 shows comparisons of various encoding schemes using the example of moving a single 1542-byte packet or frame, as well as sending (or receiving) 64 packets or frames that are 1542 bytes in size.

Why 1542? That is a standard IP packet including data and protocol framing without using jumbo frames (MTU or maximum transmission units).

What does this have to do with PCIe? GbE, 10-GbE, 40-GbE, and other physical interfaces that are used for moving TCP/IP packets and frames interface with servers via PCIe.

This encoding is important as part of server storage I/O tradecraft regarding understanding the impact of performance and network or resource usage. It also means understanding why there are fewer bits per second of effective bandwidth (independent of compression or deduplication) vs. line rate in either half- or full-duplex mode.

Another item to note is that looking at encoding such as the example given in the above table shows how a relatively small change at a large scale can have a big effective impact benefit. If the bits and bytes encoding efficiency and effectiveness scenario in Table 4.2 do not make sense, then try imagining 13 MINI Cooper automobiles each with eight people in it (yes, that would be a tight fit) end to end on the same road.

Now imagine a large bus that takes up much less length on the road than the 13 MINI Coopers. The bus holds 128 people, who would still be crowded but nowhere near as cramped as eight people in a MINI, plus 24 additional people can be carried on the bus. That is an example of applying basic 8-b/10-b encoding (the MINI) vs. applying 128-b/130-b encoding (the bus) and is also similar to PCIe G3 and G4, which use 128-b/130-b encoding for data movement.

PCIe Topologies

The basic PCIe topology configuration has one or more devices attached to the root complex shown in the following figure via an AiC or onboard device connector. Examples of AiC and motherboard-mounted devices that attach to PCIe root include LAN or SAN HBA, networking, RAID, GPU, NVM or SSD, among others. At system start-up, the server initializes the PCIe bus and enumerates the devices found with their addresses.

PCIe devices attach (shown in the following figure) to a bus that communicates with the root complex that connects with processor CPUs and memory. At the other end of a PCIe device is an end-point target, a PCIe switch that in turn has end-point targets attached. From a software standpoint, hypervisor or operating system device drivers communicate with the PCI devices that in turn send or receive data or perform other functions.

SDDC, SDI, SDDI PCIe fundamentals
Basic PCIe root complex with a PCIe switch or expander.

Note that in addition to PCIe AiC such as HBAs, GPU, and NVM SSD, among others that install into PCIe slots, servers also have converged storage or disk drive enclosures that support a mix of SAS, SATA, and PCIe. These enclosure backplanes have a connector that attaches to a SAS or SATA onboard port, or a RAID card, as well as to a PCIe riser card or motherboard connector. Depending on what type of drive is installed in the connector, either the SAS, SATA, or NVMe (AiC, U.2, and M2) using PCIe communication paths are used.

In addition to traditional and switched PCIe, using PCIe switches as well as nontransparent bridging (NTB), various other configurations can be deployed. These include server to server for clustering, failover, or device sharing as well as fabrics. Note that this also means that while traditionally found inside a server, PCIe can today use an extender, retimer, and repeaters extended across servers within a rack or cabinet.

A nontransparent bridge (NTB) is a point-to-point connection between two PCIe-based systems that provide electrical isolation yet functions as a transport bridge between two different address domains. Hosts on either side of the NTB see their respective memory or I/O address space. The NTB presents an endpoint exposed to the local system where writes are mirrored to memory on the remote system to allow the systems to communicate and share devices using associated device drivers. For example, in the following figure, two servers, each with a unique PCIe root complex, address, and memory map, are shown using NTB to any communication between the systems while maintaining data integrity.

SDDC, SDI, SDDI PCIe two server fundamentals
PCIe dual server example using NTB along with switches.

General PCIe considerations (slots and devices) include:

  • Power consumption (and heat dissipation)
  • Physical and software plug-and-play (good interoperability)
  • Drivers (in-the-box, built into the OS, or add-in)
  • BIOS, UEFI, and firmware being current versions
  • Power draw per card or adapters
  • Type of processor, socket, and support chip (if not an onboard processor)
  • Electrical signal (lanes) and mechanical form factor per slot
  • Nontransparent bridge and root port (RP)
  • PCI multi-root (MR), single-root (SR), and hot plug
  • PCIe expansion chassis (internal or external)
  • External PCIe shared storage

Various operating system and hypervisor commands are available for viewing and managing PCIe devices. For example, on Linux, the “lspci” and “lshw–c pci” commands displays PCIe devices and associated information. On a VMware ESXi host, the “esxcli hardware pci list” command will show various PCIe devices and information, while on Microsoft Windows systems, “device manager” (GUI) or “devcon” (command line) will show similar information.

Who Are Some PCIe Fundamentals Vendors and Service Providers

While not an exhaustive list, here is a sampling of some vendors and service providers involved in various ways with PCIe from solutions to components to services to trade groups include Amphenol (connectors and cables), AWS (cloud data infrastructure services), Broadcom (PCIe components), Cisco (servers), DataOn (servers), Dell EMC (servers, storage, software), E8 (storage software), Excelero (storage software), HPE (storage, servers), Huawei (storage, servers), IBM, Intel (storage, servers, adapters), Keysight (test equipment and tools).

Others include Lenovo (servers), Liqid (composable data infrastructure), Mellanox (server and storage adapters), Micron (storage devices), Microsemi (PCIe components), Microsoft (Cloud and Software including S2D), Molex (connectors, cables), NetApp, NVMexpress.org (NVM Express trade group organizations), Open Compute Project (server, storage, I/O network industry group), Oracle, PCISIG (PCIe industry trade group), Samsung (storage devices), ScaleMP (composable data infrastructure), Seagate (storage devices), SNIA (industry trade group), Supermicro (servers), Tidal (composable data infrastructure), Vantar (formerly known as HDS), VMware (Software including vSAN), and WD among others.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

PCIe fundamentals are resources for building legacy and software-defined data infrastructures (SDDI), software-defined infrastructures (SDI), data centers and other deployments from laptop to large scale, hyper-scale cloud service providers. Learn more about Servers: Physical, Virtual, Cloud, and Containers in chapter 4 of my new book Software Defined Data Infrastructure Essentials (CRC Press 2017) Available via Amazon.com and other global venues. Meanwhile, PCIe fundamentals continues to evolve as a Server, Storage, I/O networking fundamental component.

Ok, nuff said, for now.
Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

NVMe Wont Replace Flash By Itself They Complement Each Other

NVMe Wont Replace Flash By Itself They Complement Each Other

server storage I/O data infrastructure trends

Updated 2/2/2018

NVMe Wont Replace Flash By Itself They Complement Each Other

>various NVM flash and SSD devices
Various Solid State Devices (SSD) including NVMe, SAS, SATA, USB, M.2

There has been some recent industry marketing buzz generated by a startup to get some attention by claiming via a study sponsored by including the startup that Non-Volatile Memory (NVM) Express (NVMe) will replace flash storage. Granted, many IT customers as well as vendors are still confused by NVMe thinking it is a storage medium as opposed to an interface used for accessing fast storage devices such as nand flash among other solid state devices (SSDs). Part of that confusion can be tied to common SSD based devices rely on NVM that are persistent memory retaining data when powered off (unlike the memory in your computer).

NVMe is an access interface and protocol

Instead of saying NVMe will mean the demise of flash, what should or could be said however some might be scared to say it is that other interfaces and protocols such as SAS (Serial Attached SCSI), AHCI/SATA, mSATA, Fibre Channel SCSI Protocol aka FCP aka simply Fibre Channel (FC), iSCSI and others are what can be replaced by NVMe. NVMe is simply the path or roadway along with traffic rules for getting from point a (such as a server) to point b (some storage device or medium e.g. flash SSD). The storage medium is where data is stored such as magnetic for Hard Disk Drive (HDD) or tape, nand flash, 3D XPoint, Optane among others.

NVMe and NVM better together

NVMe and NVM including flash are better together

The simple quick get to the point is that NVMe (e.g. Non Volatile Memory aka NVM Express [NVMe]) is an interface protocol (like SAS/SATA/iSCSI among others) used for communicating with various nonvolatile memory (NVM) and solid state device (SSDs). NVMe is how data gets moved between a computer or other system and the NVM persistent memory such as nand flash, 3D XPoint, Spintorque or other storage class memories (SCM).

In other words, the only thing NVMe will, should, might or could kill off would be the use of some other interface such as SAS, SATA/AHCI, Fibre Channel, iSCSI along with propritary driver or protocols. On the other hand, given the extensibility of NVMe and how it can be used in different configurations including as part of fabrics, it is an enabler for various NVMs also known as persistent memories, SCMs, SSDs including those based on NAND flash as well as emerging 3D XPoint (or Intel version) among others.

Where To Learn More

View additional NVMe, SSD, NVM, SCM, Data Infrastructure and related topics via the following links.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

Context matters for example, NVM as the medium compared to NVMe as the interface and access protocols. With context in mind you can compare like or similar apples to apples such as nand flash, MRAM, NVRAM, 3D XPoint, Optane among other persistent memories also known as storage class memories, NVMs and SSDs. Likewise with context in mind NVMe can be compared to other interfaces and protocols such as SAS, SATA, PCIe, mSATA, Fibre Channel among others. The following puts all of this into context including various packaging options, interfaces and access protocols, functionality and media.

NVMe is the access for NVM flash
Putting IT all together

Will NVMe kill off flash? IMHO no not by itself, however NVMe combined with some other form of NVM, SCM, persistent memory as a storage medium may eventually combine as an alternative to NVMe and flash (or SAS/SATA and flash). However, for now at least for many applications, NVMe is in your future (along with flash among other storage mediums), the questions include when, where, why, how, with what among other questions (and answers). NVMe wont replace flash by itself (at least yet) as they complement each other.

Keep in mind, if NVMe is the answer, what are the questions.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Part 2 – Which HDD for Content Applications – HDD Testing

Part 2 – Which HDD for Content Applications – HDD Testing

HDD testing server storage I/O trends

Updated 1/23/2018

Which enterprise HDD to use with a content server, hdd testing, how and what to do

Insight for effective server storage I/O decision making
Server StorageIO Lab Review

Which enterprise HDD to use for content servers

This is the second in a multi-part series (read part one here) based on a white paper hands-on lab report I did compliments of Servers Direct and Seagate that you can read in PDF form here. The focus is looking at the Servers Direct (www.serversdirect.com) converged Content Solution platforms with Seagate Enterprise Hard Disk Drive (HDD’s). In this post we look at some decisions and configuration choices to make for testing content applications servers as well as project planning.

Content Solution Test Objectives

In short period, collect performance and another server, storage I/O decision-making information on various HDD’s running different content workloads.

Working with the Servers Direct staff a suitable content solution platform test configuration was created. In addition to providing two Intel-based content servers, Servers Direct worked with their partner Seagate to arrange for various enterprise-class HDD’s to be evaluated. For these series of content application tests, being short on time, I chose to do run some simple workloads including database, basic file (large and small) processing and general performance characterization.

Content Solution Decision Making

Knowing how Non-Volatile Memory (NVM) NAND flash SSD (1) devices (drives and PCIe cards) perform, what would be the best HDD based storage option for my given set of applications? Different applications have various performance, capacity and budget considerations. Different types of Seagate Enterprise class 2.5” Small Form Factor (SFF) HDD’s were tested.

While revolutions per minute (RPM) still plays a role in HDD performance, there are other factors including internal processing capabilities, software or firmware algorithm optimization, and caching. Most HDD’s today have some amount of DRAM for read caching and other operations. Seagate Enterprise Performance HDD’s with the enhanced caching feature (2) are examples of devices accelerate storage I/O speed vs. traditional 10K and 15K RPM drives.

Project Planning And Preparation

Workload to be tested included:

  • Database read/writes
  • Large file processing
  • Small file processing
  • General I/O profile

Project testing consisted of five phases, some of which overlapped with others:

Phase 1 – Plan
Identify candidate workloads that could be run in the given amount of time, determine time schedules and resource availability, create a project plan.

Phase 2 – Define
Hardware define and software define the test platform.

Phase 3 – Setup
The objective was to assess plug-play capability of the server, storage and I/O networking hardware with a Linux OS before moving on to the reported workloads in the next phase. Initial setup and configuration of hardware and software, installation of additional devices along with software configuration, troubleshooting, and learning as applicable. This phase consisted of using Ubuntu Linux 14.04 server as the operating system (OS) along with MySQL 5.6 as a database server during initial hands-on experience.

Phase 4 – Execute
This consisted of using Windows 2012 R2 server as the OS along with Microsoft SQL Server on the system under test (SUT) to support various workloads. Results of this phase are reported below.

Phase 5 – Analyze      
Results from the workloads run in phase 3 were analyzed and summarized into this document.

(Note 1) Refer to Seagate 1200 12 Gbps Enterprise SAS SSD StorageIO lab review

(Note 2) Refer to Enterprise SSHD and Flash SSD Part of an Enterprise Tiered Storage Strategy

Planning And Preparing The Tests

As with most any project there were constraints to contend with and work around.

Test constraints included:

  • Short-time window
  • Hardware availability
  • Amount of hardware
  • Software availability

Three most important constraints and considerations for this project were:

  • Time – This was a project with a very short time “runway”, something common in most customer environments who are looking to make a knowledgeable server, storage I/O decisions.
  • Amount of hardware – Limited amount of DRAM main memory, sixteen 2.5” internal hot-swap storage slots for HDD’s as well as SSDs. Note that for a production content solution platform; additional DRAM can easily be added, along with extra external storage enclosures to scale memory and storage capacity to fit your needs.
  • Software availability – Utilize common software and management tools publicly available so anybody could leverage those in their own environment and tests.

The following content application workloads were profiled:

  • Database reads/writes – Updates, inserts, read queries for a content environment
  • Large file processing – Streaming of large video, images or other content objects.
  • Small file processing – Processing of many small files found in some content applications
  • General I/O profile – IOP, bandwidth and response time relevant to content applications

Where To Learn More

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

There are many different types of content applications ranging from little data databases to big data analytics as well as very big fast data such as for video. Likewise there are various workloads and characteristics to test. The best test and metrics are those that apply to your environment and application needs.

Continue reading part three of this multi-part series here looking at how the systems and HDD’s were configured and tested.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Different NVMe Configurations

server storage I/O trends
Updated 1/12/2018

This is the second in a five-part mini-series providing a primer and overview of NVMe. View companion posts and more material at www.thenvmeplace.com.

The many different faces or facets of NVMe configurations

NVMe can be deployed and used in many ways, the following are some examples to show you its flexibility today as well as where it may be headed in the future. An initial deployment scenario is NVMe devices (e.g. PCIe cards, M2 or 8639 drives) installed as storage in servers or as back-end storage in storage systems. Figure 2 below shows a networked storage system or appliance that uses traditional server storage I/O interfaces and protocols for front-end access, however with back-end storage being all NVMe, or a hybrid of NVMe, SAS and SATA devices.
NVMe as back-end server storage I/O interface to NVM
Figure 2 NVMe as back-end server storage I/O interface to NVM storage

A variation of the above is using NVMe for shared direct attached storage (DAS) such as the EMC DSSD D5. In the following scenario (figure 3), multiple servers in a rack or cabinet configuration have an extended PCIe connection that attached to a shared storage all flash array using NVMe on the front-end. Read more about this approach and EMC DSSD D5 here or click on the image below.

EMC DSSD D5 NVMe
Figure 3 Shared DAS All Flash NVM Storage using NVMe (e.g. EMC DSSD D5)

Next up in figure 4 is a variation of the previous example, except NVMe is implemented over an RDMA (Remote Direct Memory Access) based fabric network using Converged 10GbE/40GbE or InfiniBand in what is known as RoCE (RDMA over Converged Ethernet pronounced Rocky).

NVMe over Fabric RoCE
Figure 4 NVMe as a “front-end” interface for servers or storage systems/appliances

Where To Learn More

View additional NVMe, SSD, NVM, SCM, Data Infrastructure and related topics via the following links.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

Watch for more topology and configuration options as NVMe along with associated hardware, software and I/O networking tools and technologies emerge over time.

Continue reading about NVMe with Part III (Need for Performance Speed) in this five-part series, or jump to Part I, Part IV or Part V.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

How to test your HDD SSD or all flash array (AFA) storage fundamentals

How to test your HDD SSD AFA Hybrid or cloud storage

server storage data infrastructure i/o hdd ssd all flash array afa fundamentals

Updated 2/14/2018

Over at BizTech Magazine I have a new article 4 Ways to Performance Test Your New HDD or SSD that provides a quick guide to verifying or learning what the speed characteristic of your new storage device are capable of.

An out-take from the article used by BizTech as a "tease" is:

These four steps will help you evaluate new storage drives. And … psst … we included the metrics that matter.

Building off the basics, server storage I/O benchmark fundamentals

The four basic steps in the article are:

  • Plan what and how you are going to test (what’s applicable for you)
  • Decide on a benchmarking tool (learn about various tools here)
  • Test the test (find bugs, errors before a long running test)
  • Focus on metrics that matter (what’s important for your environment)

Server Storage I/O performance

Where To Learn More

View additional NAS, NVMe, SSD, NVM, SCM, Data Infrastructure and HDD related topics via the following links.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

To some the above (read the full article here) may seem like common sense tips and things everybody should know otoh there are many people who are new to servers storage I/O networking hardware software cloud virtual along with various applications, not to mention different tools.

Thus the above is a refresher for some (e.g. Dejavu) while for others it might be new and revolutionary or simply helpful. Interested in HDD’s, SSD’s as well as other server storage I/O performance along with benchmarking tools, techniques and trends check out the collection of links here (Server and Storage I/O Benchmarking and Performance Resources).

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

I/O, I/O how well do you know good bad ugly server storage I/O iops?

How well do you know good bad ugly I/O iops?

server storage i/o iops activity data infrastructure trends

Updated 2/10/2018

There are many different types of server storage I/O iops associated with various environments, applications and workloads. Some I/Os activity are iops, others are transactions per second (TPS), files or messages per time (hour, minute, second), gets, puts or other operations. The best IO is one you do not have to do.

What about all the cloud, virtual, software defined and legacy based application that still need to do I/O?

If no IO operation is the best IO, then the second best IO is the one that can be done as close to the application and processor as possible with the best locality of reference.

Also keep in mind that aggregation (e.g. consolidation) can cause aggravation (server storage I/O performance bottlenecks).

aggregation causes aggravation
Example of aggregation (consolidation) causing aggravation (server storage i/o blender bottlenecks)

And the third best?

It’s the one that can be done in less time or at least cost or effect to the requesting application, which means moving further down the memory and storage stack.

solving server storage i/o blender and other bottlenecks
Leveraging flash SSD and cache technologies to find and fix server storage I/O bottlenecks

On the other hand, any IOP regardless of if for block, file or object storage that involves some context is better than those without, particular involving metrics that matter (here, here and here [webinar] )

Server Storage I/O optimization and effectiveness

The problem with IO’s is that they are a basic operations to get data into and out of a computer or processor, so there’s no way to avoid all of them, unless you have a very large budget. Even if you have a large budget that can afford an all flash SSD solution, you may still meet bottlenecks or other barriers.

IO’s require CPU or processor time and memory to set up and then process the results as well as IO and networking resources to move data too their destination or retrieve them from where they are stored. While IO’s cannot be eliminated, their impact can be greatly improved or optimized by, among other techniques, doing fewer of them via caching and by grouping reads or writes (pre-fetch, write-behind).

server storage I/O STI and SUT

Think of it this way: Instead of going on multiple errands, sometimes you can group multiple destinations together making for a shorter, more efficient trip. However, that optimization may also mean your drive will take longer. So, sometimes it makes sense to go on a couple of quick, short, low-latency trips instead of one larger one that takes half a day even as it accomplishes many tasks. Of course, how far you have to go on those trips (i.e., their locality) makes a difference about how many you can do in a given amount of time.

Locality of reference (or proximity)

What is locality of reference?

This refers to how close (i.e., its place) data exists to where it is needed (being referenced) for use. For example, the best locality of reference in a computer would be registers in the processor core, ready to be acted on immediately. This would be followed by levels 1, 2, and 3 (L1, L2, and L3) onboard caches, followed by main memory, or DRAM. After that comes solid-state memory typically NAND flash either on PCIe cards or accessible on a direct attached storage (DAS), SAN, or NAS device. 

server storage I/O locality of reference

Even though a PCIe NAND flash card is close to the processor, there still remains the overhead of traversing the PCIe bus and associated drivers. To help offset that impact, PCIe cards use DRAM as cache or buffers for data along with meta or control information to further optimize and improve locality of reference. In other words, this information is used to help with cache hits, cache use, and cache effectiveness vs. simply boosting cache use.

SSD to the rescue?

What can you do the cut the impact of IO’s?

There are many steps one can take, starting with establishing baseline performance and availability metrics.

The metrics that matter include IOP’s, latency, bandwidth, and availability. Then, leverage metrics to gain insight into your application’s performance.

Understand that IO’s are a fact of applications doing work (storing, retrieving, managing data) no matter whether systems are virtual, physical, or running up in the cloud. But it’s important to understand just what a bad IO is, along with its impact on performance. Try to identify those that are bad, and then find and fix the problem, either with software, application, or database changes. Perhaps you need to throw more software caching tools, hypervisors, or hardware at the problem. Hardware may include faster processors with more DRAM and faster internal busses.

Leveraging local PCIe flash SSD cards for caching or as targets is another option.

You may want to use storage systems or appliances that rely on intelligent caching and storage optimization capabilities to help with performance, availability, and capacity.

Where to gain insight into your server storage I/O environment

There are many tools that you can be used to gain insight into your server storage I/O environment across cloud, virtual, software defined and legacy as well as from different layers (e.g. applications, database, file systems, operating systems, hypervisors, server, storage, I/O networking). Many applications along with databases have either built-in or optional tools from their provider, third-party, or via other sources that can give information about work activity being done. Likewise there are tools to dig down deeper into the various data information infrastructure to see what is happening at the various layers as shown in the following figures.

application storage I/O performance
Gaining application and operating system level performance insight via different tools

windows and linux storage I/O performance
Insight and awareness via operating system tools on Windows and Linux

In the above example, Spotlight on Windows (SoW) which you can download for free from Dell here along with Ubuntu utilities are shown, You could also use other tools to look at server storage I/O performance including Windows Perfmon among others.

vmware server storage I/O
Hypervisor performance using VMware ESXi / vsphere built-in tools

vmware server storage I/O performance
Using Visual ESXtop to dig deeper into virtual server storage I/O performance

vmware server storage i/o cache
Gaining insight into virtual server storage I/O cache performance

Wrap up and summary

There are many approaches to address (e.g. find and fix) vs. simply move or mask data center and server storage I/O bottlenecks. Having insight and awareness into how your environment along with applications is important to know to focus resources. Also keep in mind that a bit of flash SSD or DRAM cache in the applicable place can go along way while a lot of cache will also cost you cash. Even if you cant eliminate I/Os, look for ways to decrease their impact on your applications and systems.

Where To Learn More

View additional NAS, NVMe, SSD, NVM, SCM, Data Infrastructure and HDD related topics via the following links.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

>Keep in mind: SSD including flash and DRAM among others are in your future, the question is where, when, with what, how much and whose technology or packaging.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

September October 2014 Server and StorageIO Update Newsletter

September and October 2014

Hello and welcome to this joint September and October Server and StorageIO update newsletter. Since the August newsletter, things have been busy with a mix of behind the scenes projects, as well as other activities including several webinars, on-line along with in-person events in the US as well as Europe.

Enjoy this edition of the Server and StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and podcasts along with in the news commentary appearing soon.

Cheers gs

Industry Trends and Perspectives

Storage trends

In September I was invited to do a key-note opening presentation at the MSP area CMG event. Theme for the September CMG event was "Flash – A Real Life Experience" with a focus of what people are doing, how testing and evaluating including use of hybrid solutions as opposed to vendor marketing sessions. My session was titled "Flash back to reality – Myths and Realities, Flash and SSD Industry trends perspectives plus benchmarking tips and can be found here. Thanks to Tom Becchetti an the MSP CMG (@mspcmg) folks for a great event.

There are many facets to hybrid storage including different types of media (SSD and HDD’s) along with unified or multi-protocol access. Then there are hybrid storage that spans local and public clouds. Here is a link to an on-line Internet Radio show via Information Week along with on-line chat about Hybrid Storage for Government.

Some things I’m working with or keeping an eye on include Cloud, Converged solutions, Data Protection, Business Resiliency, DCIM, Docker, InfiniBand, Microsoft (Hyper-V, SOFS, SMB 3.0), Object Storage, SSD, SDS, VMware and VVOL among others items.

Commentary In The News

StorageIO news

A lot has been going on in the IT industry since the last StorageIO Update newsletter. The following are some StorageIO industry trends perspectives comments that have appeared in various venues. Cloud conversations continue to be popular including concerns about privacy, security and availability. Here are some comments at SearchCloudComputing: about moving on from cloud deployment heartbreak.

Nand flash Solid State Devices (SSD) continue to increase in customer deployments, over at Processor, here are some here are some comments on Incorporating SSD’s Into Your Storage Plan. Also on SSD, here are some perspectives making the Argument For Flash-Based Storage. Some other comments over at Processer.com include looking At Disaster Recovery As A Service, tips to Avoid In Data Center Planning, making the most of Enterprise Virtualization, as well as New Tech, Advancements To Justify Servers. Part of controlling and managing storage costs is having timely insight, metrics that matter, here are some more perspectives and also here.

Over at SearchVirtualStorage I have some comments on how to configure and manage storage for a virtual desktop environment (VDI) while over at TechPageOne there are perspectives on top reasons to switch to Windows 8. 

Some other comments and perspectives are over at EnterpriseStorageForum including Top 10 Ways to Improve Data Center Energy Efficiency. At InfoStor there are comments and tips about Object Storage, while at SearchDataBackup I have some perspectives about Symantec being broken up.

View other industry trends comments at the here

Tips and Articles

Recent Server and StorageIO tips and articles appearing in various venues include over at SearchCloudStorage a series of discussion often asked question pieces:

Are you concerned with the security of the cloud?
Is the cost of cloud storage really cheaper?
What’s important to know about cloud privacy policy?
Are more than five nines of availability really possible?
What to look for enterprise file sync-and-share app?
How primary storage clouds and cloud backup differ?
What should I consider when using SSD cloud?
What is difference between a snapshot and a clone?

View other recent as well as past tips and articles here

StorageIOblog posts

Recent StorageIOblog posts include:

View other recent as well as past blog posts here

In This Issue

  • Industry Trends Perspectives
  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts

    • Events & Activities

    September 25, 2014
    MSP CMG – Flash and SSD performance

    October 8-10, 2014
    Nijkerk Netherlands Brouwer Seminar Series

    November 11-13, 2014
    AWS re:Invent Las Vegas

    View other recent and upcoming events here

    Webinars

    November 13 9AM PT
    BrightTalk – Software Defined Storage

    November 11 10AM PT
    Google+ Hangout Dell BackupU

    November 11 9AM PT
    BrightTak – Software Defined Data Centers

    October 16 9AM PT
    BrightTalk – Cloud Storage Decision Making

    October 15 1PM PT
    BrightTalk – Hybrid Cloud Trends

    October 7 11AM PT
    BackupU – Data Protection Management

    September 18 8AM CT
    Nexsan – Hybrid Storage

    September 18 9AM PT
    BrightTalk – Converged Storage

    September 17 1PM PT
    BrightTalk – DCIM

    September 16 1PM PT
    BrightTalk – Data Center Convergence

    September 16 Noon PT
    BrightTalk – BC, BR and DR

    September 16 1PM CT
    StarWind – SMB 3.0 & Microsoft SOFS

    September 16 9AM PT
    Google+ Hangout – BackupU – Replication

    September 2 11AM PT
    Dell BackupU – Replication

    Videos and Podcasts

    Docker for Smarties
    Video: Docker for Smarties

    StorageIO podcasts are also available via and at StorageIO.tv

    From StorageIO Labs

    Research, Reviews and Reports

    Enterprise 12Gbps SAS and SSD’s
    Better Together – Part of an Enterprise Tiered Storage Strategy
    In this StorageIO Industry Trends Perspective thought leadership white paper we look at how enterprise class SSD’s and 12Gbps SAS address current and next generation tiered storage for virtual, cloud, traditional Little and Big Data environments. This report includes proof points running various workloads including Database TPC-B, TPC-E, Microsoft Exchange in the StorageIO Labs along with cache software comparing SSD, SSHD and HDD’s. Read the  white paper  compliments of Seagate 1200 12Gbs SAS SSD’s.

    Seagate SSD White Paper

    Resources and Links

    Check out these useful links and pages:
    storageio.com/links
    objectstoragecenter.com
    storageioblog.com/data-protection-diaries-main/
    storageio.com/ssd
    storageio.com/ssd

    Ok, nuff said (for now)

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    VMware VVOLs storage I/O fundementals (Part 1)

    VMware VVOL’s storage I/O fundamentals (Part I)

    Note that this is a three part series with the first piece here (e.g. Are VMware VVOL’s in your virtual server and storage I/O future?), the second piece here (e.g.VMware VVOL’s and storage I/O fundamentals Part 1) and the third piece here (e.g. VMware VVOL’s and storage I/O fundamentals Part 2).

    Some of you may already be participating in the VMware beta of VVOL involving one of the initial storage vendors also in the beta program.

    Ok, now let’s go a bit deeper, however if you want some good music to listen to while reading this, check out @BruceRave GoDeepMusic.Net and shows here.

    Taking a step back, digging deeper into Storage I/O and VVOL’s fundamentals

    Instead of a VM host accessing its virtual disk (aka VMDK) which is stored in a VMFS formatted data store (part of ESXi hypervisor) built on top of a SCSI LUN (e.g. SAS, SATA, iSCSI, Fibre Channel aka FC, FCoE aka FC over Ethernet, IBA/SRP, etc) or an NFS file system presented by a storage system (or appliance), VVOL’s push more functionality and visibility down into the storage system. VVOL’s shift more intelligence and work from the hypervisor down into the storage system. Instead of a storage system simply presenting a SCSI LUN or NFS mount point and having limited (coarse) to no visibility into how the underlying storage bits, bytes as well as blocks are being used, storage systems gain more awareness.

    Keep in mind that even files and objects still get ultimately mapped to pages and blocks aka sectors even on nand flash-based SSD’s. However also keep an eye on some new technology such as the Seagate Kinetic drive that instead of responding to SCSI block based commands, leverage object API’s and associated software on servers. Read more about these emerging trends here and here at objectstoragecenter.com.

    With a normal SCSI LUN the underlying storage system has no knowledge of how the upper level operating system, hypervisor, file system or application such as a database (doing raw IO) is allocating the pages or blocks of memory aka storage. It is up to the upper level storage and data management tools to map from objects and files to the corresponding extents, pages and logical block address (LBA) understood by the storage system. In the case of a NAS solution, there is a layer of abstractions placed over the underlying block storage handling file management and the associated file to LBA mapping activity.

    Storage I/O basics
    Storage I/O and IOP basics and addressing: LBA’s and LBN’s

    Getting back to VVOL, instead of simply presenting a LUN which is essentially a linear range of LBA’s (think of a big table or array) that the hypervisor then manages data placement and access, the storage system now gains insight into what LBA’s correspond to various entities such as a VMDK or VMX, log, clone, swap or other VMware objects. With this more insight, storage systems can now do native and more granular functions such as clone, replication, snapshot among others as opposed to simply working on a coarse LUN basis. The similar concepts extend over to NAS NFS based access. Granted, there are more to VVOL’s including ability to get the underlying storage system more closely integrated with the virtual machine, hypervisor and associated management including supported service manage and classes or categories of service across performance, availability, capacity, economics.

    What about VVOL, VAAI and VASA?

    VVOL’s are building from earlier VMware initiatives including VAAI and VASA. With VAAI, VMware hypervisor’s can off-load common functions to storage systems that support features such as copy, clone, zero copy among others like how a computer can off-load graphics processing to a graphics card if present.

    VASA however provides a means for visibility, insight and awareness between the hypervisor and its associated management (e.g. vCenter etc) as well as the storage system. This includes storage systems being able to communicate and publish to VMware its capabilities for storage space capacity, availability, performance and configuration among other things.

    With VVOL’s VASA gets leveraged for unidirectional (e.g. two-way) communication where VMware hypervisor and management tools can tell the storage system of things, configuration, activities to do among others. Hence why VASA is important to have in your VMware CASA.

    What’s this object storage stuff?

    VVOL’s are a form of object storage access in that they differ from traditional block (LUN’s) and files (NAS volumes/mount points). However, keep in mind that not all object storage are the same as there are object storage access and architectures.

    object storage
    Object Storage basics, generalities and block file relationships

    Avoid making the mistake of when you hear object storage that means ANSI T10 (the folks that manage the SCSI command specifications) Object Storage Device (OSD) or something else. There are many different types of underlying object storage architectures some with block and file as well as object access front ends. Likewise there are many different types of object access that sit on top of object architectures as well as traditional storage system.

    Object storage I/O
    An example of how some object storage gets accessed (not VMware specific)

    Also keep in mind that there are many different types of object access mechanism including HTTP Rest based, S3 (e.g. a common industry defacto standard based on Amazon Simple Storage Service), SNIA CDMI, SOAP, Torrent, XAM, JSON, XML, DICOM, IL7 just to name a few, not to mention various programmatic bindings or application specific implementations and API’s. Read more about object storage architectures, access and related topics, themes and trends at www.objecstoragecenter.com

    Lets take a break here and when you are ready, click here to read the third piece in this series VMware VVOL’s and storage I/O fundamentals Part 2.

    Ok, nuff said (for now)

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Are VMware VVOLs in your virtual server and storage I/O future?

    Are VMware VVOL’s in your virtual server and storage I/O future?

    Note that this is a three part series with the first piece here (e.g. Are VMware VVOL’s in your virtual server and storage I/O future?), the second piece here (e.g. VMware VVOL’s and storage I/O fundamentals Part 1) and the third piece here (e.g. VMware VVOL’s and storage I/O fundamentals Part 2).

    With VMworld 2014 just around the corner, for some of you the question is not if Virtual Volumes (VVOL’s) are in your future, rather when, where, how and with what.

    What this means is that for some hands on beta testing is already occurring or will be soon, while for others that might be around the corner or down the road.

    Some of you may already be participating in the VMware beta of VVOL involving one of the first storage vendors also in the beta program.

    VMware vvol beta

    On the other hand, some of you may not be in VMware centric environments and thus VVOL’s may not yet be in your vocabulary.

    How do you know if VVOL are in your future if you don’t know what they are?

    First, to be clear, as of the time this was written VMware VVOL’s are not released and only in beta as well as having been covered in earlier VMworld’s. Consequently what you are going to read here is based on what VVOL material has already been made public in various venues including earlier VMworld’s and VMware blogs among other places.

    The quick synopsis of VMware VVOL’s overview:

  • Higher level of abstraction of storage vs. traditional SCSI LUN’s or NAS NFS mount points
  • Tighter level of integration and awareness between VMware hypervisors and storage systems
  • Simplified management for storage and virtualization administrators
  • Removing complexity to support increased scaling
  • Enable automation and service managed storage aka software defined storage management

    • VVOL considerations and your future

    As mentioned, as of this writing, VVOL’s are still a future item granted they exist in beta.

    For those of you in VMware environments, now is the time to add VVOL to your vocabulary which might mean simply taking the time to read a piece like this, or digging deeper into the theories of operations, configuration, usage, hints and tips, tutorials along with vendor specific implementations.

    Explore your options, and ask yourself, do you want VVOL or do you need it

    What support does your current vendor(s) have for VVOL or what is their statement of direction (SOD) which you might have to get from them under NDA.

    This means that there will be some first vendors with some of their products supporting VVOL’s with more vendors and products following (hence watch for many statements of direction announcements).

    Speaking of vendors, watch for a growing list of vendors to announce their current or plans for supporting VVOL’s, not to mention watch some of them jump up and down like Donkey in Shrek saying "oh oh pick me pick me".

    When you ask a vendor if they support VVOL’s, move beyond the simple yes or no, ask which of their specific products, it is a block (e.g. iSCSI) or NAS file (e.g. NFS) based and other caveats or configuration options.

    Watch for more information about VVOL’s in the weeks and months to come both from VMware along with from their storage provider partners.

    How will VVOL impact your organizations best practices, policies, workflow’s including who does what, along with associated responsibilities.

    Where to learn more

    Check out the companion piece to this that takes a closer look at storage I/O and VMware VVOL fundamentals here and here.

    Also check out this good VMware blog via Cormac Hogan (@CormacJHogan) that includes a video demo, granted its from 2012, however some of this stuff actually does take time and thus this is very timely. Speaking of VMware, Duncan Epping (aka @DuncanYB) at his Yellow-Bricks site has some good posts to check out as well with links to others including this here. Also check out the various VVOL related sessions at VMworld as well as the many existing, and soon to be many more blogs, articles and videos you can find via Google. And if you need a refresher, Why VASA is important to have in your VMware CASA.

    Of course keep an eye here or whichever venue you happen to read this for future follow-up and companion posts, and if you have not done so, sign up for the beta here as there are lots of good material including SDKs, configuration guides and more.

    VVOL Poll

    What are you VVOL plans, view results and cast your vote here

    Wrap up (for now)

    Hope you found this quick overview on VVOL’s of use, since VVOL’s at the time of this writing are not yet released, you will need to wait for more detailed info, or join the beta or poke around the web (for now).

    Keep an eye on and learn more about VVOL’s at VMworld 2014 as well as in various other venues.

    IMHO VVOL’s are or will be in your future, however the question will be is there going to be a back to the future moment for some of you with VVOL’s?

    Also what VVOL questions, comments and concerns are in your future and on your mind?

    And remember to check out the second part to this series here.

    Ok, nuff said (for now)

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    April and May 2014 Server and StorageIO Update newsletter


    Server and StorageIO Update newsletter – April and May 2014

    Welcome to the April and May 2014 edition of the StorageIO Update (newsletter) containing trends perspectives on cloud, virtualization and data infrastructure topics.

    The good news is that while spring is running late (as is this newsletter ;) here in the Stillwater MN area as well as other parts of the world, both are finally here. To say that a lot has been going on and things busy would be an understatement, however that is probably also the situation with you as well. So what has been going on during April and May 2014?

    Industry and Technology Updates

    Sony and Fujifilm (with their partner IBM) are trading marketing and proof of concept (POC) lab material in the efforts to show tape is still alive for data storage. Sony announced a month or so ago that it was moving the bar to 185TB per tape (without dedupe). Not to be out done, Fujifilm announced in late May that they in conjunction with IBM have a POC for a 154 TB LTO in the works.

    Greg Schulz Storage I/OGreg Schulz on break
    On the Hard Disk Drive (HDD) front, Seagate released a new 6TB device that they claim to be fast. I asked Seagate to send me one of the drives to see how fast it really is vs. their claims. While I have not completed all tests yet, what I can tell you is that the 6TB 3.5" 12Gbps SAS 7.2K RPM drive is like an american football linebacker or fullback. Its big, bulky, high-capacity, resilient with 10 to the 15 bit error rate (higher than normal high-capacity HDD’s) and fast.

    Sure the 6TB HDD is not in the speed race of a quick SSD or SSHD or 15K, however I was surprised at just how fast it is for its space capacity. Watch for a follow-up review in the not so distant future and if a WD 6TB drive were to show up on my door step can give some perspectives on that as well.

    As for SSD, they are following the trend paths of tape and HDD’s of increasing in space capacity, coming down in price and improving on resiliency. While I see HDD and even tape surviving for some time, granted in different roles, I’m also a firm believer that flash SSD in some form are in your future. The question is how much, when, where, with what and from whom. Needless to say there is plenty of SSD related hardware and software activity occurring in the StorageIO labs ;).

    Vendors and revenue earnings, is there storage slowdown?

    In other industry news and activity, vendor quarterly earnings are out and there is mixed information (see this recent post of if there is an information recession). IBM is one of those who have announced lowered storage related revenues as NetApp had mixed results (as did other vendors). In addition IBM is officially saying they are finally dropping the NetApp (FAS/ONTAP) based N series (was originally reported a week or so ago via Bloomberg). Note that IBM will continue to OEM NetApp E series (e.g. Engenio based). Some of you might remember (or do a Google search) that IBM indicated a few years back that it was De emphasizing the N series or moving away from it. Perhaps this time they really mean it while NetApp could move to embrace those VAR’s and IBM business partners to sell NetApp vs. IBM branded versions of the product. Here are some more perspectives appearing in SearchStorage. Watch for more about NetApp in a future follow-up post.

    In some other industry news, you might remember back in the February StorageIO update newsletter there was mention of Avago buying LSI. Now Avago is selling the flash business of LSI to Seagate for about $450M USD in the ongoing flash dance for cache and cash.

    Staying busy is a good thing

    What have I been doing during April and May 2014 to stay busy besides getting ready for spring and summer fun including in and around the water?

    • Attended NAB 2014 in Las Vegas where it is not just about archiving pertaining to data storage
    • Presented backup, restore, BC, DR and archiving including a keynote at the SNIA DSI conference
    • Was back in Las Vegas to attend EMCworld, I have some updates in the works from that event
    • Presented several BrightTalk Webinars (see events below) with more coming up in June
    • Release of new ITP white paper and StorageIO lab proof points with more in the works
    • More videos and pod casts, technology reviews including servers among other things
    • Participated including keynote at a vendor neutral archiving event in Europe
    • Providing industry commentary in different venues (see below) along with some writing
    • Not to mention various client consulting projects
    • Remember, work hard play hard, play hard and work hard!

    Whats in the works?

    Several projects and things are in the works that will show themselves in the coming weeks or months if not sooner. Some of which are more proof points coming out of the StorageIO labs involving software defined, converged, cloud, virtual, SSD, data protection and more.

    Speaking of Software Defined, join me for a free BrightTalk Webinar on June 12 on the many faces and facets of virtualization and software defined storage. Learn more about that event here as well as in the activities section down below.

    Watch for more StorageIO posts, commentary, perspectives, presentations, webinars, tips and events on information and data infrastructure topics, themes and trends. Data Infrastructure topics include among others cloud, virtual, legacy server, storage I/O networking, data protection, hardware and software.

    Enjoy this edition of the StorageIO Update newsletter and look forward to catching up with you live or online while out and about this spring.

    Ok, nuff said (for now)

    Cheers gs

    April and May 2014 Industry trend and perspectives

    Tips, commentary, articles and blog posts

    StorageIO Industry Trends and Perspectives

    The following is a synopsis of some StorageIOblog posts, articles and comments in different venues on various industry trends, perspectives and related themes about clouds, virtualization, data and storage infrastructure topics among related themes.

    StorageIO comments and perspectives in the news

    StorageIO in the news

    SearchStorage: Comments on IBM dropping N series, NetApp is still OEM to IBM
    InfoStor: Comments on Software Defined Storage: 10 Things You Need to Know
    SearchDataBackup: Comments about buying guides for enterprise Hard Disk Drives (HDD)
    SearchDataBackup: Conversation about data protection modernization
    InfoStor: Comments on cloud storage, 10 things you need to know
    InfoStor: Comments on Data Archiving: Life Beyond Compliance
    NetworkComputing: Comments on Sorting Through Storage Industry Hype
    StateTech: Comments on Secure Erasing HDDs and SSDs including planning in advance
    SNIA: Comments on CDMI Cloud Management Conformance Testing
    EnterpriseStorageForum: Comments on Hybrid Cloud Storage Tips
    NetworkComputing: Comments on Sorting Through Storage Industry Hype

    StorageIO tips and articles appearing in various venues

    StorageIO tips and articles

    Via InformationSecurityBuzz:  Dark Territories MH370 Do You Know Where Your Information Is? We still dont know 100% where the missing Malaysian airlines flight 370 is which amplifies the fact that there are still dar territories or gaps in coverage in this large world. Likewise there are gaps in coverage in many IT environments yet tools and technologies are available to gain better situational awareness and insight.

    Via The Virtualization Practice: This piece looks at the EMC ViPR V1.1 and SRM V3.0 (Software Defined Storage Management) announcements from earlier this year, along with links to earlier announcement and technology analysis. Note that EMC announced May 5, 2014 ViPR 2.0 along with their new Elastic Cloud Storage Appliance (ECS) among other enhancements at EMC World. Additional perspectives on ViPR 2.0, Elastic Cloud Storage Appliance and EMCworld announcement summary analysis can be found here in this video (with text) that I did (produced via TechTarget) while at EMCworld 2014. Watch for more coverage of ViPR 2.0 and other related new as well as updated items from EMCworld 2014 in upcoming posts, articles and commentary.

    Via InfoStor: Data Archiving: Life Beyond Compliance. Today many people think or assume based on what they hear that Archiving is only for regulatory archiving. Meanwhile some of you may remember a time before the regulatory compliance era of the early 2000s when Archiving was used as a general purpose tool, technology and solution to many IT data management storage challenges. This piece I did over at InfoStor looks at Data Archiving: Life Beyond Compliance and how Archiving is also a key technology that are part of Data Footprint Reduction (DFR) that also includes compression, dedupe, thin provisioning amount other techniques and tools. Here is a related Email Archiving piece (beyond compliance) from over at StateTech along with Practical tips in a piece over at VMware Communities.

    StorageIO video and audio pod casts

    StorageIOblog postStorageIOblog post
    Video conversation with Rob Emsley of EMC and me discussing data protection modernization moving beyond the product pitch!(Via TechTarget SearchDataBackup). In this conversation Rob and me talk about various aspects of data protection modernization including finding and fixing problems at the source, accidental architectures, using new (and old) things in new ways, rethinking data protection. However the conversation is a discussion about the topics, issues, trends, what can be done as opposed to a product pitch infomercial. Check out this video blog (vblog) of Rob and me via TechTarget SearchDataBackup, then weigh in with your comments.

    audioSNIA DSI David Dale
    Audio Podcast: Data Storage Innovation Conversation with SNIA Wayne Adams and David Dale
    In this episode, SNIA Chairman Emeritus Wayne Adams and current Chairman David Dale join me in a conversation from the Data Storage Innovation (DSI) 2014 conference event. DSI is a new event produced by SNIA targeted for IT professionals involved with data storage related topics, themes, technologies and tools spanning hardware, software, cloud, virtual and physical. In this conversation, we talk about the new DSI event, the diversity of new attendees who are attending their first SNIA event, along with other updates. Some of these updates include what is new with the SNIA Cloud Data Management Initiative (CDMI), Non Volatile Memory (think flash and SSD), SMIS, education and more. Listen in to our conversation in this podcast here as we cover cloud, convergence, software defined and more about data storage.

    audiocash coleman cleardb
    Audio Podcast: Catching up with Cash Coleman talking ClearDB, cloud database and Johnny Cash
    In this episode from the SNIA DSI 2014 event I am joined by Cashton Coleman (@Cash_Coleman). Cashton (Cash) is a Software architect, product mason, family bonder, life builder, idea founder along with Founder & CEO of SuccessBricks, Inc., makers of ClearDB. ClearDB is a provider of MySQL database software tools for cloud and physical environments. We talk about ClearDB, what they do and whom they do it with including deployments in cloud’s as well as onsite. For example if you are using some of the Microsoft Azure cloud services using MySQL, you may already be using this technology. However, there is more to the story and discussion including how Cash got his name, how to speed up databases for little and big data among other topics. Check out ClearDB and listen in to the conversation with Cash podcast here.

    audio
    Audio Podcast: Matt Vogt talks VMware vCOP in his first ever podcast
    In this episode from the Computex Rethink your Datacenter for 2017 planning and strategy event I am joined by Matt Vogt (@MattVogt). Matt is a Principal Architect with Computex Technology Solutions as well as certified VMware specialist and fellow vExpert. We talk about the role of automation for performance and capacity optimization along with how VMware vCop plays an important role. Listen in to learn more about how to gain insight and situational awareness to make informed decisions for your data infrastructure environment with Matt. Check out Matt’s blog here at blog.mattvogt.net and listen in to the podcast here.

    StorageIO audio podcasts are also available via
    and at StorageIO.tv

    StorageIOblog posts and perspectives

    StorageIOblog post

  • Is there an information or data recession, are you using less storage (with polls)
  • Lenovo TS140 Server and Storage IO Review Part I here and Part II here
  • Nand flash SSD server storage I/O conversations: See more SSD stories here
  • Data Protection Diaries: March 31 World Backup Day is Restore Data Test, read more here
  • March 2014 StorageIO Update Newsletter: Click here to read more

    • StorageIO White Papers, Solution Briefs and StorageIO Lab reports

    White Paper

    New White Paper: Solid State Hybrid Drives (SSHD)
    Enterprise SSHD and Flash SSD – Better Together – Part of an Enterprise Tiered Storage Strategy The question to ask yourself is not if flash Solid State Device (SSD) technologies are in your future. Instead the questions are when, where, using what, how to configure and related themes. SSD including traditional DRAM and NAND flash-based technologies are like real estate where location matters; however, there are different types of properties to meet various needs.

    This means leveraging different types of NAND flash SSD technologies in different locations in a complementary and cooperative aka hybrid way. In this StorageIO Industry Trends Perspective thought leadership white paper we look at how enterprise class Solid State Hybrid Drives (SSHD) and how they address current and next generation tiered storage for virtual, cloud, traditional Little and Big Data infrastructure environments. This includes providing proof points running various workloads including Database TPC-B, TPC-E and Microsoft Exchange in the StorageIO Labscomparing SSHD, SSD and different HDDs. Read more in this StorageIO Industry Trends and Perspective (ITP) white paper compliments of Seagate Enterprise Turbo SSHD. Read the companion blog post here that includes more proof points for large file transfer performance.

    Remember to check out our objectstoragecenter.com page where you will find a growing collection of information and links on cloud and object storage themes, technologies and trends from various sources.

    If you are interested in data protection including Backup/Restore, BC, DR, BR and Archiving along with associated technologies, tools, techniques and trends visit our storageioblog.com/data-protection-diaries-main/ page. For those who follow SSD and related technologies, we have organized a series of items at storageio.com/ssd.

    StorageIO events and activities

    Server and StorageIO seminars, conferences, web cats, events, activities

    The StorageIO calendar continues to evolve, here are some recent and upcoming activities including live in-person seminars, conferences, keynote and speaking activities as well as on-line webinars, twitter chats, Google+ hangouts among others.

    June 12, 2014The Many Facets of Virtual Storage and Software Defined Storage VirtualizationWebinar
    9AM PT
    June 11, 2014The Changing Face and Landscape of Enterprise StorageWebinar
    9AM PT
    May 14, 2014Brouwer Storage ConsultancyKeynote – Healthcare Vendor Neutral Archiving SymposiumNijkerk Netherlands
    May 5-7, 2014EMC WorldLas Vegas
    April 23, 2014SNIA DSI EventKeynote: Enabling Data Infrastructure Return On Innovation – The Other ROIbackup, restore, BC, DR and archiving
    April 22, 2014SNIA DSI EventThe Cloud Hybrid “Homerun” – Life Beyond The Hypebackup, restore, BC, DR and archiving
    April 16, 2014Open Source and Cloud Storage – Enabling business, or a technology enabler?Webinar
    9AM PT
    April 9, 2014Storage Decision Making for Fast, Big and Very Big Data EnvironmentsWebinar
    9AM PT

    Click here to view other upcoming along with earlier event activities. Watch for more 2014 events to be added soon to the StorageIO events calendar page. Topics include data protection modernization (backup/restore, HA, BC, DR, archive), data footprint reduction (archive, compression, dedupe), storage optimization, SSD, object storage, server and storage virtualization, big data, little data, cloud and object storage, performance and management trends among others.

    Vendors, VAR’s and event organizers, give us a call or send an email to discuss having us involved in your upcoming pod cast, web cast, virtual seminar, conference or other events.

    StorageIO Update Newsletter Archives

    Click here to view previous StorageIO Update newsletters (HTML and PDF versions) at www.storageio.com/newsletter. Subscribe to this newsletter (and pass it along) by clicking here (Via Secure Campaigner site). View archives of past StorageIO update news letters as well as download PDF versions at: www.storageio.com/newsletter

    Ok, nuff said (for now)

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Nand flash SSD NVM SCM server storage I/O memory conversations

    Updated 8/31/19
    Server Storage I/O storageioblog SDDC SDDI Data Infrastructure trends

    The SSD Place NVM, SCM, PMEM, Flash, Optane, 3D XPoint, MRAM, NVMe Server, Storage, I/O Topics

    Now and then somebody asks me if I’m familiar with flash or nand flash Solid State Devices (SSD) along with other non-volatile memory (NVM) technologies and trends including NVM Express (NVMe).

    Having been involved with various types of SSD technology, products and solutions since the late 80s initially as a customer in IT (including as a lunch customer for DEC’s ESE20 SSD’s), then later as a vendor selling SSD solutions, as well as an analyst and advisory consultant cover the technologies, I tell the person asking, well, yes, of course.

    That gave me the idea as well as to help me keep track of some of the content and make it easy to find by putting it here in this post (which will be updated now and then).

    Thus this is a collection of articles, tips, posts, presentations, blog posts and other content on SSD including nand flash drives, PCIe cards, DIMMs, NVM Express (NVMe), hybrid and other storage solutions along with related themes.

    Also if you can’t find it here, you can always do a Google search like this or this to find some more material (some of which is on this page).

    HDD, SSHD, HHDD and HDD

    Flash SSD Articles, posts and presentations

    The following are some of my tips, articles, blog posts, presentations and other content on SSD. Keep in mind that the question should not be if SSD are in your future, rather when, where, with what, from whom and how much. Also keep in mind that a bit of SSD as storage or cache in the right place can go a long way, while a lot of SSD will give you a benefit however also cost a lot of cash.

    • How to Prepare for the NVMe Server Storage I/O Wave (Via Micron.com)
    • Why NVMe Should Be in Your Data Center (Via Micron.com)
    • NVMe U2 (8639) vs. M2 interfaces (Via Gamersnexus)
    • Enmotus FuzeDrive MicroTiering (StorageIO Lab Report)
    • EMC DSSD D5 Rack Scale Direct Attached Shared SSD All Flash Array Part I (Via StorageIOBlog)
    • Part II – EMC DSSD D5 Direct Attached Shared AFA (Via StorageIOBlog)
    • NAND, DRAM, SAS/SCSI & SATA/AHCI: Not Dead, Yet! (Via EnterpriseStorageForum)
    • Non Volatile Memory (NVM), NVMe, Flash Memory Summit and SSD updates (Via StorageIOblog)
    • Microsoft and Intel showcase Storage Spaces Direct with NVM Express at IDF ’15 (Via TechNet)
    • MNVM Express solutions (Via SuperMicro)
    • Gaining Server Storage I/O Insight into Microsoft Windows Server 2016 (Via StorageIOblog)
    • PMC-Sierra Scales Storage with PCIe, NVMe (Via EEtimes)
    • RoCE updates among other items (Via InfiniBand Trade Association (IBTA) December Newsletter)
    • NVMe: The Golden Ticket for Faster Flash Storage? (Via EnterpriseStorageForum)
    • What should I consider when using SSD cloud? (Via SearchCloudStorage)
    • MSP CMG, Sept. 2014 Presentation (Flash back to reality – Myths and Realities – Flash and SSD Industry trends perspectives plus benchmarking tips)– PDF
    • Selecting Storage: Start With Requirements (Via NetworkComputing)
    • PMC Announces Flashtec NVMe SSD NVMe2106, NVMe2032 Controllers With LDPC (Via TomsITpro)
    • Exclusive: If Intel and Micron’s “Xpoint” is 3D Phase Change Memory, Boy Did They Patent It (Via Dailytech)
    • Intel & Micron 3D XPoint memory — is it just CBRAM hyped up? Curation of various posts (Via Computerworld)
    • How many IOPS can a HDD, HHDD or SSD do (Part I)?
    • How many IOPS can a HDD, HHDD or SSD do with VMware? (Part II)
    • I/O Performance Issues and Impacts on Time-Sensitive Applications (Via CMG)
    • Via EnterpriseStorageForum: 5 Hot Storage Technologies to Watch
    • Via EnterpriseStorageForum: 10-Year Review of Data Storage
    • Via CustomPCreview: Samsung SM961 PCIe NVMe SSD Shows Up for Pre-Order
    • StorageIO Industry Trends Perspective White Paper: Seagate 1200 Enterprise SSD (12Gbps SAS) with proof points (e.g. Lab test results)
    • Companion: Seagate 1200 12Gbs Enterprise SAS SSD StorgeIO lab review (blog post part I and Part II)
    • NewEggBusiness: Seagate 1200 12Gbs Enterprise SAS SSD StorgeIO lab review Are NVMe m.2 drives ready for the limelight?
    • Google (Research White Paper): Disks for Data Centers (vs. just SSD)
    • CMU (PDF White Paper): A Large-Scale Study of Flash Memory Failures in the Field
    • Via ZDnet: Google doubles Cloud Compute local SSD capacity: Now it’s 3TB per VM
    • EMC DSSD D5 Rack Scale Direct Attached Shared SSD All Flash Array Part I (Via StorageIOBlog)
    • Part II – EMC DSSD D5 Direct Attached Shared AFA (Via StorageIOBlog)
    • NAND, DRAM, SAS/SCSI & SATA/AHCI: Not Dead, Yet! (Via EnterpriseStorageForum)
    • Here’s why Western Digital is buying SanDisk (Via ComputerWorld)
    • HP, SanDisk partner to bring storage-class memory to market (Via ComputerWorld)
    • Non Volatile Memory (NVM), NVMe, Flash Memory Summit and SSD updates (Via StorageIOblog)
    • Microsoft and Intel showcase Storage Spaces Direct with NVM Express at IDF ’15 (Via TechNet)
    • PMC-Sierra Scales Storage with PCIe, NVMe (Via EEtimes)
    • Seagate Grows Its Nytro Enterprise Flash Storage Line (Via InfoStor)
    • New SAS Solid State Drive First Product From Seagate Micron Alliance (Via Seagate)
    • Wow, Samsung’s New 16 Terabyte SSD Is the World’s Largest Hard Drive (Via Gizmodo)
    • Samsung ups the SSD ante with faster, higher capacity drives (Via ITworld)
    • PMC Announces Flashtec NVMe SSD NVMe2106, NVMe2032 Controllers With LDPC (Via TomsITpro)
    • New SATA SSD powers elastic cloud agility for CSPs (Via Cbronline)
    • Toshiba Solid-State Drive Family Features PCIe Technology (Via Eweek)
    • SanDisk aims CloudSpeed Ultra SSD at cloud providers (Via ITwire)
    • Everspin & Aupera reveal all-MRAM Storage Module in M.2 Form Factor (Via BusinessWire)
    • Intel, Micron Launch “Bulk-Switching” ReRAM (Via EEtimes)
    • Exclusive: If Intel and Micron’s “Xpoint” is 3D Phase Change Memory, Boy Did They Patent It (Via Dailytech)
    • Intel & Micron 3D XPoint memory — is it just CBRAM hyped up? Curation of various posts (Via Computerworld)
    • NVMe: The Golden Ticket for Faster Flash Storage? (Via EnterpriseStorageForum)

    server I/O hirearchy

    • What should I consider when using SSD cloud? (Via SearchCloudStorage)
    • MSP CMG, September 2014 Presentation (Flash back to reality – Myths and Realities Flash and SSD Industry trends perspectives plus benchmarking tips) – PDF
    • Selecting Storage: Start With Requirements (Via NetworkComputing)
    • Spot The Newest & Best Server Trends (Via Processor)
    • Market ripe for embedded flash storage as prices drop (Via Powermore (Dell))
    • 2015 Tech Preview: SSD and SMBs (Via ChannelProNetworks )
    • How to test your HDD, SSD or all flash array (AFA) storage fundamentals (Via StorageIOBlog)
    • Processor: Comments on What Abandoned Data Is Costing Your Company
    • Processor: Comments on Match Application Needs & Infrastructure Capabilities
    • Processor: Comments on Explore The Argument For Flash-Based Storage
    • Processor: Comments on Understand The True Cost Of Acquiring More Storage
    • Processor: Comments on What Resilient & Highly Available Mean
    • Processor: Explore The Argument For Flash-Based Storage
    • SearchCloudStorage What should I consider when using SSD cloud?
    • StorageSearch.com: (not to be confused with TechTarget, good site with lots of SSD related content)
    • StorageSearch.com: What kind of SSD world… 2015?
    • StorageSearch.com: Various links about SSD
    • FlashStorage.com: (Various flash links curated by Tegile and analyst firm Actual Tech Media [Scott D. Lowe])
    • StorageSearch.com: How fast can your SSD run backwards?
    • Seagate has shipped over 10 Million storage HHDD’s (SSHDs), is that a lot?
    • Are large storage arrays dead at the hands of SSD?
    • Can we get a side of context with them IOPS and other storage metrics?
    • Cisco buys Whiptail continuing the SSD storage I/O flash cash cache dash
    • EMC VFCache respinning SSD and intelligent caching (Part I)
    • Flash Data Storage: Myth vs. Reality (Via InfoStor)
    • Have SSDs been unsuccessful with storage arrays (with poll)?
    • How many IOPS can a HDD, HHDD or SSD do (Part I)?
    • How many IOPS can a HDD, HHDD or SSD do with VMware? (Part II)
    • I/O Performance Issues and Impacts on Time-Sensitive Applications (Via CMG)

    server storage i/o memory hirearchy

    • Spiceworks SSD and related conversation here and here, profiling IOPs here, and SSD endurance here.
    • SSD is in your future, How, when, with what and where you will be using it (PDF Presentation)
    • SSD for Virtual (and Physical) Environments: Part I Spinning up to speed on SSD (Via TheVirtualizationPractice), Part II, The call to duty, SSD endurance, Part III What SSD is best for you?, and Part IV what’s best for your needs.
    • IT and storage economics 101, supply and demand
    • SSD, flash and DRAM, DejaVu or something new?
    • The Many Faces of Solid State Devices/Disks (SSD)
    • The Nand Flash Cache SSD Cash Dance (Via InfoStor)
    • The Right Storage Option Is Important for Big Data Success (Via FedTech)

    server storage i/o nand flash ssd options

    • Viking SATADIMM: Nand flash SATA SSD in DDR3 DIMM slot?
    • WD buys nand flash SSD storage I/O cache vendor Virident (Via VMware Communities)
    • What is the best kind of IO? The one you do not have to do
    • When and Where to Use NAND Flash SSD for Virtual Servers (Via TheVirtualizationPractice)
    • Why SSD based arrays and storage appliances can be a good idea (Part I)
    • Why SSD based arrays and storage appliances can be a good idea (Part II)
    • Q&A on Access data more efficiently with automated storage tiering and flash (Via SearchSolidStateStorage)
    • InfoStor: Flash Data Storage: Myth vs. Reality (Via InfoStor)
    • Enterprise Storage Forum: Not Just a Flash in the Pan (Via EnterpriseStorageForum)

    SSD Storage I/O and related technologies comments in the news

    The following are some of my commentary and industry trend perspectives that appear in various global venues.

    Storage I/O ssd news

    • Comments on using Flash Drives To Boost Performance (Via Processor)
    • Comments on selecting the Right Type, Amount & Location of Flash Storage (Via Toms It Pro)
    • Comments Google vs. AWS SSD: Which is the better deal? (Via SearchAWS)
    • Tech News World: SANdisk SSD comments and perspectives.
    • Tech News World: Samsung Jumbo SSD drives perspectives
    • Comments on Why Degaussing Isn’t Always Effective (Via StateTech Magazine)
    • Processor: SSD (FLASH and RAM)
    • SearchStorage: FLASH and SSD Storage
    • Internet News: Steve Wozniak joining SSD startup
    • Internet News: SANdisk sale to Toshiba
    • SearchSMBStorage: Comments on SanDisk and wireless storage product
    • StorageAcceleration: Comments on When VDI Hits a Storage Roadblock and SSD
    • Statetechmagazine: Boosting performance with SSD
    • Edtechmagazine: Driving toward SSDsStorage I/O trends
    • SearchStorage: Seagate SLC and MLC flash SSD
    • SearchWindowServer: Making the move to SSD in a SAN/NAS
    • SearchSolidStateStorage: Comments SSD marketplace
    • InfoStor: Comments on SSD approaches and opportunities
    • SearchSMBStorage: Solid State Devices (SSD) benefits
    • SearchSolidState: Comments on Fusion-IO flash SSD and API’s
    • SeaarchSolidStateStorage: Comments on SSD industry activity and OCZ bankruptcy
    • Processor: Comments on Plan Your Storage Future including SSD
    • Processor: Comments on Incorporate SSDs Into Your Storage PlanStorage I/O ssd news
    • Digistor: Comments on SSD and flash storage
    • ITbusinessEdge: Comments on flash SSD and hybrid storage environments
    • SearchStorage: Perspectives on Cisco buying SSD storage vendor Whiptail
    • StateTechMagazine: Comments on all flash SSD storage arrays
    • Processor: Comments on choosing SSDs for your data center needs
    • Searchsolidstatestorage: Comments on how to add solid state devices (SSD) to your storage system
    • Networkcomputing: Comments on SSD/Hard Disk Hybrids Bridge Storage Divide
    • Internet Evolution: Comments on IBM buying flash SSD vendor TMS
    • ITKE: Comments on IBM buying flash SSD vendor TMSStorage I/O trends
    • Searchsolidstatestorage: SSD, Green IT and economic benefits
    • IT World Canada: Cloud computing, dot be scared, look before you leap
    • SearchStorage: SSD in storage systems
    • SearchStorage: SAS SSD
    • SearchSolidStateStorage: Comments on Access data more efficiently with automated storage tiering and flash
    • InfoStor: Comments on EMC’s Light to Speed: Flash, VNX, and Software-Defined
    • EnterpriseStorageForum: Cloud Storage Mergers and Acquisitions: What’s Going On?

    Check out the Server StorageIO NVM Express (NVMe) focus page aka www.thenvmeplace.com for additional related content. nterested in data protection, check out the data protection diaries series of posts here, or cloud and object storage here, and server storage I/O performance benchmarking here. Also check out the StorageIO events and activities page here, as well as tips and articles here, news commentary here, along out newsletter here.

    Ok, nuff said (for now)

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Matt Vogt of Computex talks VMware vCOPs in his first ever podcast

    Storage I/O trends

    Matt Vogt of Computex talks VMware vCOPs in his first ever podcast

    audio

    In this episode from the Computex Rethink your Datacenter for 2017 planning and strategy event I am joined by Matt Vogt (@MattVogt).

    Introducing Matt Vogt

    Matt is a Principal Architect with Computex Technology Solutions as well as certified VMware specialist and fellow vExpert.

    audio

    Not only is this the first appearance by Matt on the StorageIO Podcasts, it is also his first time as a guest on any podcast so I’m honored to host his global debut podcast appearance here.

    We talk about the role of automation for performance and capacity optimization along with how VMware vCOPs plays an important role.

    Listen in to learn more about how to gain insight and situational awareness to make informed decisions for your data infrastructure environment with Matt.

    Check out Matt’s blog here at blog.mattvogt.net and listen in to the pod cast here.

    Also available via 

    Ok, nuff said.

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Some Windows Server Storage I/O related commands

    Storage I/O trends

    Some Windows Server Storage I/O related commands

    The following are some commands and tools for Microsoft Windows environments that are useful for storage I/O activities (among others).

    Microsoft Windows

    Finding a Windows physical disk, SSD or storage system device name

    So you may know and how to find out the more familiar Windows storage device (Solid State Device – SSD, Hard Disk Drives – HDD among others) names such as A:, B:, C:, D: as what you can view from the Windows Explorer, Computer or Admin tools.

    Windows storage devices

    However what if you need to find out a physical name for raw (not mounted) and mounted devices for configuration? For example, if you have a tool that wants the physical name for your C: drive that might be \\.\PhysicalDrive0\?

    No worries, use the command WMIC DISKDRIVE LIST BRIEF

    WIndows physical device name

    Need more detail about the devices beyond what is shown above?

    Then use WMIC DISKDRIVE LIST or as in the above example, direct the output to a file with the results shown below (scroll to the left or right to see more detail information).

            Availability  BytesPerSector  Capabilities  CapabilityDescriptions                 CompressionMethod  ConfigManagerErrorCode  ConfigManagerUserConfig  DefaultBlockSize  Description  DeviceID            ErrorCleared  ErrorDescription  ErrorMethodology  Index  InstallDate  InterfaceType  LastErrorCode  Manufacturer            MaxBlockSize  MaxMediaSize  MediaLoaded  MediaType              MinBlockSize  Model                                  Name                NeedsCleaning  NumberOfMediaSupported  Partitions  PNPDeviceID                                                  PowerManagementCapabilities  PowerManagementSupported  SCSIBus  SCSILogicalUnit  SCSIPort  SCSITargetId  SectorsPerTrack  Signature   Size           Status  StatusInfo  SystemName  TotalCylinders  TotalHeads  TotalSectors  TotalTracks  TracksPerCylinder  
              512             {3, 4}        {"Random Access", "Supports Writing"}                     0                       FALSE                                      Disk drive   \\.\PHYSICALDRIVE2                                                    2                   SCSI                          (Standard disk drives)                              TRUE         Fixed hard disk media                ATA ST3000DM001-1CH1 SCSI Disk Device  \\.\PHYSICALDRIVE2                                         0           SCSI\DISK&VEN_ATA&PROD_ST3000DM001-1CH1\5&3626375C&0&000600                                                         0        0                3         6             63               0           3000590369280  OK                  DBIOTEST    364801          255         5860528065    93024255     255                
              512             {3, 4}        {"Random Access", "Supports Writing"}                     0                       FALSE                                      Disk drive   \\.\PHYSICALDRIVE3                                                    3                   SCSI                          (Standard disk drives)                              TRUE         Fixed hard disk media                SEAGATE ST600MP0034 SCSI Disk Device   \\.\PHYSICALDRIVE3                                         0           SCSI\DISK&VEN_SEAGATE&PROD_ST600MP0034\5&3626375C&0&000A00                                                          0        0                3         10            63                           600124654080   OK                  DBIOTEST    72961           255         1172118465    18605055     255                
              512             {3, 4}        {"Random Access", "Supports Writing"}                     0                       FALSE                                      Disk drive   \\.\PHYSICALDRIVE4                                                    4                   SCSI                          (Standard disk drives)                              TRUE         Fixed hard disk media                SEAGATE ST600MX0004 SCSI Disk Device   \\.\PHYSICALDRIVE4                                         0           SCSI\DISK&VEN_SEAGATE&PROD_ST600MX0004\5&3626375C&0&000C00                                                          0        0                3         12            63                           600124654080   OK                  DBIOTEST    72961           255         1172118465    18605055     255                
              512             {3, 4}        {"Random Access", "Supports Writing"}                     0                       FALSE                                      Disk drive   \\.\PHYSICALDRIVE1                                                    1                   SCSI                          (Standard disk drives)                              TRUE         Fixed hard disk media                SEAGATE ST9300603SS SCSI Disk Device   \\.\PHYSICALDRIVE1                                         0           SCSI\DISK&VEN_SEAGATE&PROD_ST9300603SS\5&3626375C&0&000400                                                          0        0                3         4             63                           299992412160   OK                  DBIOTEST    36472           255         585922680     9300360      255                
              512             {3, 4}        {"Random Access", "Supports Writing"}                     0                       FALSE                                      Disk drive   \\.\PHYSICALDRIVE0                                                    0                   SCSI                          (Standard disk drives)                              TRUE         Fixed hard disk media                VMware Virtual disk SCSI Disk Device   \\.\PHYSICALDRIVE0                                         2           SCSI\DISK&VEN_VMWARE&PROD_VIRTUAL_DISK\5&1982005&1&000000                                                           0        0                2         0             63               -873641784  64420392960    OK                  DBIOTEST    7832            255         125821080     1997160      255

    Remembering (or learning) Xcopy

    Some of you might be familiar with Xcopy and if not, it is a handy tool for copying files, folders and directories to local as well as networked storage. Some handy Xcopy command switches include:

    /j = use un-buffered IO for large files
    /y = suppress prompting
    /c = continue if error
    /E = copy sub directories
    /H = copy hidden files
    /Q = quiet mode (don’t list files being copied)

    In the following example the content of the Videos folder and its sub-directories (83.5GB) are copied to another destination. Note the Time /T command that is also shown which is useful for timing how long the copy takes (e.g. subtract start-time from end-time and you have elapsed time). In this example 83.5GB are copied from one place to another on the same SSD device and using the results of the Time /T command the elapsed time was about 12 minutes.

    Windows SSD TRIM
    Xcopy command example

    Diskpart, don’t be scared, however be careful

    Ever have a Windows storage device or system that failed to boot, or a problem with a partition, volume or other issue?

    How about running into a situation where you are not able to format a device that you know and can confirm is ok to erase, yet you get a message that the volume is write protected or read only?

    Diskpart is handy, powerful and potentially dangerous tool if you are not careful as you could mistakenly drop a good volume or partition (e.g. the importance of having good backups). However Diskpart can be used to help repair storage devices that have boot problems, or for clearing read only attributes among other tasks. If you are prefer GUI interfaces, many of the Diskpart functions can also be done via Disk Management interface (e.g. Control Panel -> All Control Panel Items -> Administrative Tools -> Computer Management -> Storage -> Disk Management ). Note that Diskpart to do certain functions will need to be run as Administrator.

    windows diskpart

    In the above example the LIST DISK command shows what disks are present (on-line or off-line) which means that you may see devices here that do not show up elsewhere. Also shown is selecting a disk and then listing partitions, selecting a partition and showing attributes. The Attribute command can be used for clearing Read Only modes should a partition become write protected.

    Hint, ever have a device that was once had VMware installed on it, then you move it to Windows and try to reformat for use and get a Read Only error? If so, you will want to have a look at Diskpart and the Attribute commands. However BE CAREFULL and pay attention which disk, partition and volumes you are working with as you can easily cause a problem that would result in testing how good your backups are.

    Is SATA SSD TRIM Enabled?

    If you have a SATA SSD the TRIM command is a form of garbage collection that is supported with Windows 7 (SAS drives use the SCSI UNMAP). Not sure if your system has TRIM enabled? Try the following command as administrator. Note that if you see a result of "0" then TRIM is enabled while a value of "1" means that it is disabled for your system.

    Windows SSD TRIM

    Want to learn more about TRIM, check out this piece from Intel as well as this Microsoft Windows item.

    Having issues with collecting CPU and performance statistics?

    Have an issue or problem collecting your system statistics, or when running a benchmark, workload generation tool such as vdbench and getting an "Unable to obtain CPU statistics"?

    Try the Lodctr /R command (as administrator), however read this Microsoft Tip first to learn more.

    Windows Lodctr /R

    Sdelete and drive erase

    Like its name implies, if you do not have this tool, you can download it here from Microsoft to not only delete files, folders, as well as write "0" patterns across a disk to secure erase it. You can specify the number of times you want to run the write "0" patterns across a disk to meet your erasure requirements.

    There is also another use for Sdelete which is if you need or want to pre-condition a SSD or other device such as for testing, you can run a pre-conditioning pass using Sdelete.

    Some command options include -p #n where "n" is the number of times to run, -s recursive to process sub-directories, -z to write "0" or zero out the space on the device, -c for clean, -a to process read-only attributes. Learn more and get your copy of Sdelete from Microsoft here.

    Rufus, Seatools, Samsung Disk Magician and Cyberduck

    A handy tool available from Seagate (may only work with Seagate and their partner devices) is SeaTools that can give drive information, health and status as well as perform various tests including SMART.

    Seagate Seatools
    Seagate Seatools example

    Different HDD and SSD as well as storage system vendors give tools for configuration, monitoring, management and in some cases data movement with their solutions. Samsung SSD Magician is a tool I have installed for managing my SSDs (830 and 840 Pros) that has features for updating firmware, drive health as well as performance optimization. Other hand tools include the Samsung copy tool based on Clonix as Acronis among other clone or data migration utilities (more on those in a future post).

    Samsung SSD Magician
    Samsung SSD Magician

    While the Microsoft WIndows USB Tool is handy for dealing with Microsoft ISO, however for creating USB’s with ISO’s such as for installing VMware or Linux on bare metal systems, Rufus is a handy tool to have in the tool-box.

    Rufus ISO to USB tool

    Another useful tool that functions as an SSH and FTP utility is Cyberduck that also supports access to Amazon S3 among other cloud services.

    There are many other tools for server, storage I/O and other activities on WIndows, not to mention other platforms, however hopefully you find the above useful.

    How about it, what’s your favorite Windows server, storage I/O tools and commands?

    Ok, nuff said (for now)

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Goodbye 2013, hello 2014, predictions past, present and future

    Storage I/O trends

    Good by 2013 and hello 2014 along with predictions past, present and future

    First, for those who may have missed this, thanks to all who helped make 2013 a great year!

    2013 season greetings

    Looking back at 2013 I saw a continued trend of more vendors and their media public relations (PR) people reaching out to have their predictions placed in articles, posts, columns or trends perspectives pieces.

    Hmm, maybe a new trend is predictions selfies? ;)

    Not to worry, this is not a wrapper piece for a bunch of those pitched and placed predictions requests that I received in 2013 as those have been saved for a rainy or dull day when we need to have some fun ;) .

    What about 2013 server storage I/O networking, cloud, virtual and physical?

    2013 end up with some end of year spree’s including Avago acquiring storage I/O and networking vendor LSI for about $6.6B USD (e.g. SSD cards, RAID cards, cache cards, HBA’s (Host Bus Adapters), chips and other items) along with Seagate buying Xyratex for about $374M USD (a Seagate suppliers and a customer partner).

    Xyratex is known by some for making the storage enclosures that house hard disk drive (HDD’s) and Solid State Device (SSD) drives that are used by many well-known, and some not so well-known systems and solution vendors. Xyratex also has other pieces of their business such as appliances that combine their storage enclosures for HDD and SSD’s along with server boards, along with a software group focus on High Performance Compute (HPC) Lustre. There is another part of the Xyratex business that is not as well-known which is the test equipment used by disk drive manufacturers such as Seagate as part of their manufacturing process. Thus the Seagate acquisition moves them up market with more integrated solutions to offer to their (e.g. Seagate and Xyratex) joint customers, as well as streamline their own supply chain and costs (not to mention sell equipment to the other remaining drive manufactures WD and Toshiba).

    Storage I/O trends

    Other 2013 acquisitions included (Whiptail by Cisco, Virident by WD (who also bought several other companies), Softlayer by IBM) along with various mergers, company launches, company shutdowns (cloud storage Nirvanix and SSD maker OCZ bankruptcy filing), and IPO’s (some did well like Nimble while Violin not so well), while earlier high-flying industry darlings such as FusionIO are now the high-flung darling targets of the shareholder sock lawsuit attorneys.

    2013 also saw the end of SNW (Storage Network World), jointly produced by SNIA and Computerworld Storage in the US after more than a decade. Some perspectives from the last US SNW held October 2013 can be found in the Fall 2013 StorageIO Update Newsletter here, granted those were before the event was formal announced as being terminated.

    Speaking of events, check out the November 2013 StorageIO Update Newsletter here for perspectives from attending the Amazon Web Services (AWS) re:Invent conference which joins VMworld, EMCworld and a bunch of other vendor world events.

    Lets also not forget Dell buying itself in 2013.

    StorageIO in the news

    Click on the following links read (and here) more about various 2013 industry perspectives trends commentary of mine in various venues, along with tips, articles, newsletters, events, pod cast, videos and other items.

    What about 2014?

    Perhaps 2014 will build on the 2013 momentum of the annual rights of pages refereed to as making meaningless future year trends and predictions as being passe?

    Not that there is anything wrong with making predictions for the coming year, particular if they actually have some relevance, practicality not to mention track record.

    However that past few years seems to have resulted in press releases along with product (or services) plugs being masked as predictions, or simply making the same predictions for the coming year that did not come to be for the earlier year (or the one before that or before that and so forth).

    On the other hand, from an entertainment perspective, perhaps that’s where we will see annual predictions finally get classified and put into perspectives as being just that.

    Storage I/O trends

    Now for those who still cling to as well as look forward to annual predictions, ok, simple, we will continue in 2014 (and beyond) from where we left off in 2013 (and 2012 and earlier) meaning more (or continued):

    • Software defined "x" (replace "x" with your favorite topic) industry discussion adoption yet customer adoption or deployment question conversations.

    • Cloud conversations shifted from lets all go to the cloud as the new shiny technology to questioning the security, privacy, stability, vendor or service viability not to mention other common sense concerns that should have been discussed or looked into earlier. I have also heard from people who say Amazon (as well as Verizon, Microsoft, Blue host, Google, Nirvanix, Yahoo and the list goes on) outages are bad for the image of clouds as they shake people’s confidences. IMHO people confidence needs to be shaken to that of having some common sense around clouds including don’t be scared, be ready, do your homework and basic due diligence. This means cloud conversations over concerns set the stage for increased awareness into decision-making, usage, deployment and best practices (all of which are good things for continued cloud deployments). However if some vendors or pundits feel that people having basic cloud concerns that can be addressed is not good for their products or services, I would like to talk with them because they may be missing an opportunity to create long-term confidence with their customers or prospects.

    • VDI as a technology being deployed continues to grow (e.g. customer adoption) while the industry adoption (buzz or what’s being talked about) has slowed a bit which makes sense as vendors jump from one bandwagon to the new software defined bandwagon.

    • Continued awareness around modernizing data protection including backup/restore, business continuance (BC), disaster recovery (DR), high availability, archiving and security means more than simply swapping out old technology for new, yet using it in old ways. After all, in the data center and information factory not everything is the same. Speaking of data protection, check out the series of technology neutral webcast and video chats that started last fall as part of BackupU brought to you by Dell. Even though Dell is the sponsor of the series (that’s a disclosure btw ;) ) the focus of the sessions is on how to use different tools, technologies and techniques in new ways as well as having the right tools for different tasks. Check out the information as well as register to get a free Data Protection chapter download from my book Cloud and Virtual Data Storage Networking (CRC Press) at the BackupU site as well as attend upcoming events.

    • The nand flash solid state devices (SSD) cash-dash (and shakeout) continues with some acquisitions and IPO’s, as well as disappearances of some weaker vendors, while appearance of some new. SSD is showing that it is real in several ways (despite myths, fud and hype some of which gets clarified here) ranging from some past IPO vendors (e.g. FusiuonIO) seeing exit of their CEO and founders while their stock plummets and arrival of shareholder investor lawsuits, to Violins ho-hum IPO. What this means is that the market is real, it has a very bright future, however there is also a correction occurring showing that reality may be settling in for the long run (e.g. next couple of decades) vs. SSD being in the realm of unicorns.

      • Storage I/O trends

    • Internet of Things (IoT) and Internet of Devices (IoD) may give some relief for Big Data, BYOD, VDI, Software Defined and Cloud among others that need a rest after they busy usage that past few years. On the other hand, expect enhanced use of earlier buzzwords combined with IoT and IOD. Of course that also means plenty of questions around what is and is not IoD along with IoT and if actually relevant to what you are doing.

    • Also in 2014 some will discover storage and related data infrastructure topics or some new product / service thus having a revolutionary experience that storage is now exciting while others will have a DejaVu moment that it has been exciting for the past several years if not decades.

    • More big data buzz as well as realization by some that a pragmatic approach opens up a bigger broader market, not to mention customers more likely to realize they have more in common with big data than it simply being something new forcing them to move cautiously.

    • To say that OpenStack and related technologies will continue to gain both industry and customer adoption (and deployment) status building off of 2013 in 2014 would be an understatement, not to mention too easy to say, or leave out.

    • While SSD’s continue to gain in deployment, after the question is not if, rather when, where, with what and how much nand flash SSD is in your future, HDD’s continue to evolve for physical, virtual and cloud environments. This also includes Seagate announcing a new (Kinetic) Ethernet attached HDD (note that this is not a NAS or iSCSI device) that uses a new key value object storage API for storing content data (more on this in 2014).

    • This also means realizing that large amounts of little data can result in back logs of lots of big data, and that big data is growing into very fast big data, not to mention realization by some that HDFS is just another distributed file system that happens to work with Hadoop.

    • SOHO’s and lower end of SMB begin to get more respect (and not just during the week of Consumer Electronic Show – CES).

    • Realization that there is a difference between Industry Adoption and Customer Deployment, not to mention industry buzz and traction vs. customer adoption.

    server storage I/O trends

    What about beyond 2014?

    That’s easy, many of the predictions and prophecies that you hear about for the coming year have also been pitched in prior years, so it only makes sense that some of those will be part of the future.

    • If you have seen or experienced something you are more likely to have DejaVu.
    • Otoh if you have not seen or experienced something you are more likely to have a new and revolutionary moment!
    • Start using new (and old) things in new ways vs. simply using new things in old ways.
    • Barrier to technology convergence, not to mention new technology adoption is often people or their organizations.
    • Convergence is still around, cloud conversations around concerns get addressed leading to continued confidence for some.
    • Realization that data infrastructure span servers, storage I/O networking, cloud, virtual, physical, hardware, software and services.
    • That you can not have software defined without hardware and hardware defined requires software.
    • And it is time for me to get a new book project (or two) completed in addition to helping others with what they are working on, more on this in the months to come…

    Here’s my point

    The late Jim Morrison of the Doors said "There are things known and things unknown and in between are the doors.".

    The doors via Amazon.com
    Above image and link via Amazon.com

    Hence there is what we know about 2013 or will learn about the past in the future, then there is what will be in 2014 as well as beyond, hence lets step through some doors and see what will be. This means learn and leverage lessons from the past to avoid making the same or similar mistakes in the future, however doing so while looking forward without a death grip clinging to the past.

    Needless to say there will be more to review, preview and discuss throughout the coming year and beyond as we go from what is unknown through doors and learn about the known.

    Thanks to all who made 2013 a great year, best wishes to all, look forward to seeing and hearing from you in 2014!

    Ok, nuff said (for now)

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved