Non Volatile Memory (NVM), NVMe, Flash Memory Summit and SSD updates

I attended the Flash Memory Summit in Santa Clara CA last week and not surprisingly there were many announcements about Non-Volatile Memory (NVM) along with related enabling technologies. Some of these announcements were component based intended for original equipment manufactures (OEMs) ranging from startup to established, systems integrators (SI), value added resellers (VAR’s) while others were more customer solution focused. From a customer solution focus, some of the technologies were consumer oriented while others for business and some for cloud scale service providers.

Recent NVM, NVMe and Flash SSD news

A sampling of some recent NVM, NVMe and Flash related news includes among others:

PMC Announces Flashtec NVMe SSD NVMe2106, NVMe2032 Controllers (Via TomsITpro)
New SATA SSD powers elastic cloud agility for CSPs (Via Cbronline)
Toshiba Solid-State Drive Family Features PCIe Technology (Via Eweek)
SanDisk aims CloudSpeed Ultra SSD at cloud providers (Via ITwire)
Everspin & Aupera show all-MRAM Storage Module in M.2 Form Factor (Via BusinessWire)
Intel and Micron unveil new 3D XPoint Non Volatile Memory (NVM) for servers and storage (part I, part II and part III)
PMC-Sierra Scales Storage with PCIe, NVMe (Via EEtimes)
Seagate Grows Its Nytro Enterprise Flash Storage Line (Via InfoStor)
New SAS Solid State Drive First Product From Seagate Micron Alliance (Via Seagate)
Wow, Samsung’s New 16 Terabyte SSD Is the World’s Largest Hard Drive (Via Gizmodo)
Samsung ups the SSD ante with faster, higher capacity drives (Via ITworld)

NVMe primer

Via Intel: Click above image to view history of memory via Intel site

NVM includes technologies such as NAND flash commonly used in Solid State Devices (SSD’s) storage today, as well as in USB thumb drive, mobile and hand-held devices among many other uses. NVM spans servers, storage, I/O devices along with mobile and handheld among many other technologies. In addition to NAND flash, other forms of NVM include Non Volatile Random Access Memory (NVRAM), Read Only Memory (ROM) along with some emerging new technologies including the recently announced Intel and Micron 3D XPoint among others.

Server Storage I/O memory (and storage) hierarchy

Keep in mind that memory is storage and storage is persistent memory as well as that there are different classes, categories and tiers of memory and storage as shown above to meet various performance, availability, capacity and economic requirements. Besides NVM ranging from flash to NVRAM to emerging 3D XPoint among others, another popular topic that is gaining momentum is NVM Express (NVMe). NVMe (more material here at www.thenvmeplace.com) is a new server storage I/O access method and protocol for fast access to NVM based products. NVMe is an alternative to existing block based server storage I/O access protocols such as AHCI/SATA and SCSI/SAS devices commonly used for access Hard Disk Drives (HDD) along with SSD among other things.

Comparing AHCI/SATA, SCSI/SAS and NVMe all of which can coexist to address different needs.

Leveraging the common PCIe hardware interface, NVMe based devices (that have an NVMe controller) can be accessed via various operating systems (and hypervisors such as VMware ESXi) with both in the box drivers or optional third-party device drivers. Devices that support NVMe can be 2.5" drive format packaged that use a converged 8637/8639 connector (e.g. PCIe x4) coexisting with SAS and SATA devices as well as being add in card (AIC) PCIe cards supporting x4, x8 and other implementations. Initially NVMe is being positioned as a back-end to servers (or storage systems) interface for accessing fast flash and other NVM based devices.

NVMe as a "back-end" I/O interface in a server or storage system accessing NVM storage/media devices

NVMe as a “front-end” interface for servers (or storage systems/appliances) to use NVMe based storage systems

NVMe has also been shown to work over low latency, high-speed RDMA based network interfaces including RoCE (RDMA over Converged Ethernet) and InfiniBand (read more here, here and here involving Mangstor, Mellanox and PMC among others). What this means is that like SCSI based SAS which can be both a back-end drive (HDD, SSD, etc) access protocol and interface, NVMe can in addition to being used for back-end can also be used as a front-end of server to storage interface like how Fibre Channel SCSI_Protocol (aka FCP), SCSI based iSCSI, SCSI RDMA Protocol via InfiniBand (among others) are used.

NVMe and shared PCIe

NVMe features

Main features of NVMe include among others:

Lower latency due to improve drivers and increased queues (and queue sizes)
Lower CPU used to handler larger number of I/Os (more CPU available for useful work)
Higher I/O activity rates (IOPs) to boost productivity unlock value of fast flash and NVM
Bandwidth improvements leveraging various fast PCIe interface and available lanes
Dual-pathing of devices like what is available with dual-path SAS devices
Unlock the value of more cores per processor socket and software threads (productivity)
Various packaging options, deployment scenarios and configuration options
Appears as a standard storage device on most operating systems
Plug-play with in-box drivers on many popular operating systems and hypervisors

Watch for more about NVMe as it continues to gain in both industry adoption and deployment as well as customer adoption and deployment.

Where to read, watch and learn more

NVMe: The Golden Ticket for Faster Flash Storage? (Via EnterpriseStorageForum)
What should I consider when using SSD cloud? (Via SearchCloudStorage)
MSP CMG, September 2014 Presentation (Flash back to reality – Myths and Realities Flash and SSD Industry trends perspectives plus benchmarking tips) – PDF
Selecting Storage: Start With Requirements (Via NetworkComputing)
Spot The Newest & Best Server Trends (Via Processor)
Intel and Micron unveil new 3D XPoint Non Volatile Memory (NVM) for servers and storage (part I, part II and part III)
Market ripe for embedded flash storage as prices drop (Via Powermore (Dell))
Continue reading more about NVM, NVMe, NAND flash, SSD Server and storage I/O related topics at www.thessdplace.com as well as about I/O performance, monitoring and benchmarking tools at www.storageperformance.us.

What this all means and wrap up

The question is not if NVM is in your future, it is! Instead the questions are what type of NVM including NAND flash among other mediums will be deployed where, using what type of packaging or solutions (drives, cards, systems, appliances, cloud) for what role (as storage, primary memory, persistent cache) along with how much among others. For some environments the solution is already, or will be All NVM Arrays (ANA) or All Flash Arrays (AFA) or All SSD Arrays (ASA) while for others the home run will be hybrid based solutions that work for you, fitting in and adapting to your environment as it changes.

Also keep in mind that a little bit of fast memory including NVM based flash among others in the right place can have a big benefit. My experiences using NVMe to use flash enabled NVMe devices on Windows and Linux systems is that you can see lower response times at higher-IOP’s however also with lower CPU consumption particular when compared to 6Gbps SATA. Likewise bandwidth can easily be pushed to the limits of the NVMe device as well as PCIe interface being used such as x4 or x8 depending on implementation. That is also a warning and something to watch out for comparing apples to oranges in that while NVMe uses PCIe, understand when looking at different results if those are for x4 or x8 or faster PCIe as their mere presence of using PCIe does not mean you are running at full potential.

Keep an eye on NVMe as a new high-speed, low-latency server storage I/O access protocol for unlocking the full performance capabilities of fast NVM based storage as well as leveraging the multiple cores in today’s fast processors. Does this mean AHCI/SATA or SCSI/SAS are now dead? Some will claim that, however at least near-term for next few years (if not longer), those interfaces will continue to be used where they make sense, as well as where they can save dollars specifically for cost sensitive, high-capacity environments that do not need the full performance of NVMe just yet.

As for the Flash Memory Summit event in Santa Clara, that was a good day with time well spent in briefings, meetings, demo’s and add hoc discussions on the expo floor.

Ok, nuff said

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

greg