NVMe Place NVM Non Volatile Memory Express Resources

Updated 8/31/19

Welcome to NVMe place NVM Non Volatile Memory Express Resources. NVMe place is about Non Volatile Memory (NVM) Express (NVMe) with Industry Trends Perspectives, Tips, Tools, Techniques, Technologies, News and other information.

Disclaimer

Please note that this NVMe place resources site is independent of the industry trade and promoters group NVM Express, Inc. (e.g. www.nvmexpress.org). NVM Express, Inc. is the sole owner of the NVM Express specifications and trademarks.

Image used with permission of NVM Express, Inc.

Visit the NVM Express industry promoters site here to learn more about their members, news, events, product information, software driver downloads, and other useful NVMe resources content.

The NVMe Place resources and NVM including SCM, PMEM, Flash

NVMe place includes Non Volatile Memory (NVM) including nand flash, storage class memories (SCM), persistent memories (PM) are storage memory mediums while NVM Express (NVMe) is an interface for accessing NVM. This NVMe resources page is a companion to The SSD Place which has a broader Non Volatile Memory (NVM) focus including flash among other SSD topics. NVMe is a new server storage I/O access method and protocol for fast access to NVM based storage and memory technologies. NVMe is an alternative to existing block based server storage I/O access protocols such as AHCI/SATA and SCSI/SAS devices commonly used for access Hard Disk Drives (HDD) along with SSD among other things.

Comparing AHCI/SATA, SCSI/SAS and NVMe all of which can coexist to address different needs.

Leveraging the standard PCIe hardware interface, NVMe based devices (that have an NVMe controller) can be accessed via various operating systems (and hypervisors such as VMware ESXi) with both in the box drivers or optional third-party device drivers. Devices that support NVMe can be 2.5″ drive format packaged that use a converged 8637/8639 connector (e.g. PCIe x4) coexisting with SAS and SATA devices as well as being add-in card (AIC) PCIe cards supporting x4, x8 and other implementations. Initially, NVMe is being positioned as a back-end to servers (or storage systems) interface for accessing fast flash and other NVM based devices.

NVMe as a “back-end” I/O interface for NVM storage media

NVMe as a “front-end” interface for servers or storage systems/appliances

NVMe has also been shown to work over low latency, high-speed RDMA based network interfaces including RoCE (RDMA over Converged Ethernet) and InfiniBand (read more here, here and here involving Mangstor, Mellanox and PMC among others). What this means is that like SCSI based SAS which can be both a back-end drive (HDD, SSD, etc) access protocol and interface, NVMe can also being used for back-end can also be used as a front-end of server to storage interface like how Fibre Channel SCSI_Protocol (aka FCP), SCSI based iSCSI, SCSI RDMA Protocol via InfiniBand (among others) are used.

NVMe features

Main features of NVMe include among others:

• Lower latency due to improve drivers and increased queues (and queue sizes)
• Lower CPU used to handle larger number of I/Os (more CPU available for useful work)
• Higher I/O activity rates (IOPs) to boost productivity unlock value of fast flash and NVM
• Bandwidth improvements leveraging various fast PCIe interface and available lanes
• Dual-pathing of devices like what is available with dual-path SAS devices
• Unlock the value of more cores per processor socket and software threads (productivity)
• Various packaging options, deployment scenarios and configuration options
• Appears as a standard storage device on most operating systems
• Plug-play with in-box drivers on many popular operating systems and hypervisors

NVMe and shared PCIe (e.g. shared PCIe flash DAS)

NVMe related content and links

The following are some of my tips, articles, blog posts, presentations and other content, along with material from others pertaining to NVMe. Keep in mind that the question should not be if NVMe is in your future, rather when, where, with what, from whom and how much of it will be used as well as how it will be used.

• Use Intel Optane NVMe U.2 SFF 8639 SSD drive in PCIe slot
• NVMe overview and primer – Part I
• Part II – NVMe overview and primer (Different Configurations)
• Part III – NVMe overview and primer (Need for Performance Speed)
• Part IV – NVMe overview and primer (Where and How to use NVMe)
• Part V – NVMe overview and primer (Where to learn more, what this all means)

• PCIe Server I/O Fundamentals
• If NVMe is the answer, what are the questions?
• NVMe Wont Replace Flash By Itself
• Via Computerweekly – NVMe discussion: PCIe card vs U.2 and M.2
• Server storage I/O benchmark tools, workload scripts and examples (Part I) and (Part II)

• Via GizModo: Comments on Intel Optane 800P NVMe M.2 SSD
• Via InfoStor: 8 Big Enterprise SSD Trends to Expect in 2017
• Why NVMe Should Be in Your Data Center – Preparing for Tomorrow’s Data Center Today (StorageIO Guest Post Via Micron.com)
• Via SearchStoragae: Comments on Top 10 Tips on Solid State Storage Adoption Strategy
• Via J Metz’s Blog – Vendor neutral bibliography of material by subject matter for NVMe

• Via InfoStor – SSD Trends, Tips and Topics
• StorageIOblog: Get in the NVMe SSD game (if you are not already)
• Via VMware VirtualBlocks Blog – Base NVM Express – Part One
• Dell EMC PowerEdge MX 7000 Kinetic Based Data Infrastructure Architecture
• Via J Metz’s Blog – Vendor neutral learning NVMe A Program of Study
• Via StorageIOblog: VMware vSAN v6.6
• Via StorageIOblog: Cisco announces 32Gb FC and NVMe fabrics

• Data Infrastructure Primer and Overview (Its Whats Inside The Data Center)
• SSD, flash, Non-volatile memory (NVM), NVMe, storage Trends, Tips & Topics
• Cloud and Object storage are in your future, what are some questions?”>Cloud and Object Storage Questions
• Data Infrastructure Server Storage I/O related tradecraft overview
• Data Infrastructure Server Storage I/O Tradecraft Trends

• Via NextPlatform – Assessing tradeoffs NVMe storage Scale
• Via NextPlatform – PCIe flash revamped to compete with NVMe drives
• Via NextPlatform – Flash disruption comes to server main memory
• Via NextPlatform – Hybrid arrays fight back against all flash
• Via NextPlatform – Making remote NVMe flash look local fast

• Via NextPlatform – Hyperscaling Consumer Flash NVMe
• Via NextPlatform – Purified Implementation NVMe storage
• Via NextPlatform – Impatient fabrics Micron forges NVMe arrays
• Via NextPlatform – The truth about flash memory in the datacenter

• Via Pure Storage: Announces new NVMe storage
• Via Micron Blog (Guest Post by Greg Schulz): What’s next for NVMe and your Data Center – Preparing for Tomorrow Today
• Via ChannelProSMB: Comments on NVMe (and SSD) and server storage I/O
• EnterpriseStorageForum: Comments Top 10 Enterprise SSD Market Trends
• SearchSolidStateStorage: Comments on How to add solid-state storage to your enterprise data storage systems

• Microsoft TechNet: Understand the cache in Storage Spaces Direct
• Microsoft Technet: Don’t do it: consumer-grade solid-state drives (SSD) in Storage Spaces Direct
• Why Micron NVMe SSDs (Via Micron.com)
• New Path to Storage I/O Performance and Resiliency With NVMe (Via Micron.com)
• How NVMe Will Revolutionize Server and Storage I/O(Via Micron.com)

• How to Prepare for the NVMe Server Storage I/O Wave (Via Micron.com)
• Why NVMe Should Be in Your Data Center (Via Micron.com)
• NVMe U2 (8639) vs. M2 interfaces (Via Gamersnexus)
• Enmotus FuzeDrive MicroTiering (StorageIO Lab Report)
• EMC DSSD D5 Rack Scale Direct Attached Shared SSD All Flash Array Part I (Via StorageIOBlog)

• Part II – EMC DSSD D5 Direct Attached Shared AFA (Via StorageIOBlog)
• NAND, DRAM, SAS/SCSI & SATA/AHCI: Not Dead, Yet! (Via EnterpriseStorageForum)
• Non Volatile Memory (NVM), NVMe, Flash Memory Summit and SSD updates (Via StorageIOblog)
• Microsoft and Intel showcase Storage Spaces Direct with NVM Express at IDF ’15 (Via TechNet)
• MNVM Express solutions (Via SuperMicro)

• Gaining Server Storage I/O Insight into Microsoft Windows Server 2016 (Via StorageIOblog)
• PMC-Sierra Scales Storage with PCIe, NVMe (Via EEtimes)
• RoCE updates among other items (Via InfiniBand Trade Association (IBTA) December Newsletter)
• NVMe: The Golden Ticket for Faster Flash Storage? (Via EnterpriseStorageForum)
• What should I consider when using SSD cloud? (Via SearchCloudStorage)

• MSP CMG, Sept. 2014 Presentation (Flash back to reality – Myths and Realities – Flash and SSD Industry trends perspectives plus benchmarking tips)– PDF
• Selecting Storage: Start With Requirements (Via NetworkComputing)
• PMC Announces Flashtec NVMe SSD NVMe2106, NVMe2032 Controllers With LDPC (Via TomsITpro)
• Exclusive: If Intel and Micron’s “Xpoint” is 3D Phase Change Memory, Boy Did They Patent It (Via Dailytech)
• Intel & Micron 3D XPoint memory — is it just CBRAM hyped up? Curation of various posts (Via Computerworld)

• How many IOPS can a HDD, HHDD or SSD do (Part I)?
• How many IOPS can a HDD, HHDD or SSD do with VMware? (Part II)
• I/O Performance Issues and Impacts on Time-Sensitive Applications (Via CMG)
• Via EnterpriseStorageForum: 5 Hot Storage Technologies to Watch
• Via EnterpriseStorageForum: 10-Year Review of Data Storage

• Via EnterpriseStorageForum: Where All-Flash Storage Makes No Sense
• Via EnterpriseStorageForum: Top Tips for Enterprise SSD Form Factor Selection and Deployment
• Who Will Be Top Of Storage World?
• Intel announces new processors
• Server Storage I/O CI, HCI overview
• Data Infrastructure Tradecraft Overview
• SSD, flash and NVM Trends
• If NVMe is the answer, what are the questions?

Non-Volatile Memory (NVM) Express (NVMe) continues to evolve as a technology for enabling and improving server storage I/O for NVM including nand flash SSD storage. NVMe streamline performance enabling more work to be done (e.g. IOPs), data to be moved (bandwidth) at a lower response time using less CPU.

The above figure is a quick look comparing nand flash SSD being accessed via SATA III (6Gbps) on the left and NVMe (x4) on the right. As with any server storage I/O performance comparisons there are many variables and take them with a grain of salt. While IOPs and bandwidth are often discussed, keep in mind that with the new protocol, drivers and device controllers with NVMe that streamline I/O less CPU is needed.

Additional NVMe Resources

Also check out the Server StorageIO companion micro sites landing pages including thessdplace.com (SSD focus), data protection diaries (backup, BC/DR/HA and related topics), cloud and object storage, and server storage I/O performance and benchmarking here.

If you are in to the real bits and bytes details such as at device driver level content check out the Linux NVMe reflector forum. The linux-nvme forum is a good source if you are developer to stay up on what is happening in and around device driver and associated topics.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Disclaimer

Disclaimer: Please note that this site is independent of the industry trade and promoters group NVM Express, Inc. (e.g. www.nvmexpress.org). NVM Express, Inc. is the sole owner of the NVM Express specifications and trademarks. Check out the NVM Express industry promoters site here to learn more about their members, news, events, product information, software driver downloads, and other useful NVMe resources content.

Image used with permission of NVM Express, Inc.

Wrap Up

Watch for updates with more content, links and NVMe resources to be added here soon.

Ok, nuff said (for now)

Cheers
Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.