Tag: server

Posted on

Cloud Conversations AWS Azure Service Maps via Microsoft

Cloud Conversations AWS Azure Service Maps via Microsoft

server storage I/O data infrastructure trends

Updated 1/21/2018

Microsoft has created an Amazon Web Service AWS Azure Service Map. The AWS Azure Service Map is a list created by Microsoft looks at corresponding services of both cloud providers.

Azure AWS service map via Microsoft.com
Image via Azure.Microsoft.com

Note that this is an evolving work in progress from Microsoft and use it as a tool to help position the different services from Azure and AWS.

Also note that not all features or services may not be available in different regions, visit Azure and AWS sites to see current availability.

As with any comparison they are often dated the day they are posted hence this is a work in progress. If you are looking for another Microsoft created why Azure vs. AWS then check out this here. If you are looking for an AWS vs. Azure, do a simple Google (or Bing) search and watch all the various items appear, some sponsored, some not so sponsored among others.

Whats In the Service Map

The following AWS and Azure services are mapped:

  • Marketplace (e.g. where you select service offerings)
  • Compute (Virtual Machines instances, Containers, Virtual Private Servers, Serverless Microservices and Management)
  • Storage (Primary, Secondary, Archive, Premium SSD and HDD, Block, File, Object/Blobs, Tables, Queues, Import/Export, Bulk transfer, Backup, Data Protection, Disaster Recovery, Gateways)
  • Network & Content Delivery (Virtual networking, virtual private networks and virtual private cloud, domain name services (DNS), content delivery network (CDN), load balancing, direct connect, edge, alerts)
  • Database (Relational, SQL and NoSQL document and key value, caching, database migration)
  • Analytics and Big Data (data warehouse, data lake, data processing, real-time and batch, data orchestration, data platforms, analytics)
  • Intelligence and IoT (IoT hub and gateways, speech recognition, visualization, search, machine learning, AI)
  • Management and Monitoring (management, monitoring, advisor, DevOps)
  • Mobile Services (management, monitoring, administration)
  • Security, Identity and Access (Security, directory services, compliance, authorization, authentication, encryption, firewall
  • Developer Tools (workflow, messaging, email, API management, media trans coding, development tools, testing, DevOps)
  • Enterprise Integration (application integration, content management)

Down load a PDF version of the service map from Microsoft here.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

On one hand this can and will likely be used as a comparison however use caution as both Azure and AWS services are rapidly evolving, adding new features, extending others. Likewise the service regions and site of data centers also continue to evolve thus use the above as a general guide or tool to help map what service offerings are similar between AWS and Azure.

By the way, if you have not heard, its Blogtober, check out some of the other blogs and posts occurring during October here.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Dell EMC VMware September 2017 Software Defined Data Infrastructure Updates

Dell EMC VMware September 2017 Software Defined Data Infrastructure Updates

server storage I/O data infrastructure trends

Dell EMC VMware September 2017 Software Defined Data Infrastructure Updates

vmworld 2017

September was a busy month including VMworld in Las Vegas that featured many Dell EMC VMware (among other) software defined data infrastructure updates and announcements.

A summary of September VMware (and partner) related announcements include:

VMware on AWS via Amazon.com
VMware and AWS via Amazon Web Services

VMware and AWS

Some of you might recall VMware earlier attempt at public cloud with vCloud Air service (see Server StorageIO lab test drive here) which has since been depreciated (e.g. retired). This new approach by VMware leverages the large global presence of AWS enabling customers to set up public or hybrid vSphere, vSAN and NSX based clouds, as well as software defined data centers (SDDC) and software defined data infrastructures (SDDI).

VMware Cloud on AWS exists on a dedicated, single-tenant (unlike Elastic Cloud Compute (EC2) multi-tenant instances or VMs) that supports from 4 to 16 underlying host per cluster. Unlike EC2 virtual machine instances, VMware Cloud on AWS is delivered on elastic bare-metal (e.g. dedicated private servers aka DPS). Note AWS EC2 is more commonly known, AWS also has other options for server compute including Lambda micro services serverless containers, as well as Lightsail virtual private servers (VPS).

Besides servers with storage optimized I/O featuring low latency NVMe accessed SSDs, and applicable underlying server I/O networking, VMware Cloud on AWS leverages the VMware software stack directly on underlying host servers (e.g. there is no virtualization nesting taking place). This means more robust performance should be expected like in your on premise VMware environment. VM workloads can move between your onsite VMware systems and VMware Cloud on AWS using various tools. The VMware Cloud on AWS is delivered and managed by VMware, including pricing. Learn more about VMware Cloud on AWS here, and here (VMware PDF) and here (VMware Hands On Lab aka HOL).

Read more about AWS September news and related updates here in this StorageIOblog post.

VMware PKS
VMware and Pivotal PKS via VMware.com

Pivotal Container Service (PKS) and Google Kubernetes Partnership

During VMworld VMware, Pivotal and Google announced a partnership for enabling Kubernetes container management called PKS (Pivotal Container Service). Kubernetes is evolving as a popular open source container microservice serverless management orchestration platform that has roots within Google. What this means is that what is good for Google and others for managing containers, is now good for VMware and Pivotal. In related news, VMware has become a platinum sponsor of the Cloud Native Compute Foundation (CNCF). If you are not familiar with CNCF, add it to your vocabulary and learn more here at www.cncf.io.

Other VMworld and September VMware related announcements

Hyper converged data infrastructure provider Maxta has announced a VMware vSphere Escape Pod (parachute not included ;) ) to facilitate migration from ESXi based to Red Hat Linux hypervisor environments. IBM and VMware for cloud partnership, along with Dell EMC, IBM and VMware joint cloud solutions. White listing of VMware vSphere VMs for enhanced security combine with earlier announced capabilities.

Note that both VMware with vSphere ESXi and Microsoft with Hyper-V (Windows and Azure based) are supporting various approaches for securing Virtual Machines (VMs) and the hosts they run on. These enhancements are moving beyond simply encrypting the VMDK or VHDX virtual disks the VMs reside in or use, as well as more than password, ssh and other security measures. For example Microsoft is adding support for host guarded fabrics (and machine hosts) as well as shielded VMs. Keep an eye on how both VMware and Microsoft extend the data protection and security capabilities for software defined data infrastructures for their solutions and services.

Dell EMC Announcements

At VMworld in September Dell EMC announcements included:

  • Hyper Converged Infrastructure (HCI) and Hybrid Cloud enhancements
  • Data Protection, Goverence and Management suite updates
  • XtremIO X2 all flash array (AFA) availability optimized for vSphere and VDI

HCI and Hybrid Cloud enhancements include VxRail Appliance, VxRack SDDC (vSphere 6.5, vSAN 6.6, NSX 6.3) along with hybrid cloud platforms (Enterprise Hybrid Cloud and Native Hybrid Cloud) along with vSAN Ready Nodes (vSAN 6.6 and encryption) and VMware Ready System. Note that Dell EMC in addition to supporting VMware hybrid clouds also previously announced solutions for Microsoft Azure Stack back in May.

Software Defined Data Infrastructure Essentials at VMworld Bookstore

xxxx

Software Defined Data Infrastructure Essentials (CRC Press) at VMworld bookstore

My new book Software Defined Data Infrastructure Essentials (CRC Press) made its public debut in the VMware book store where I did a book signing event. You can get your copy of Software Defined Data Infrastructure Essentials which includes Software Defined Data Centers (SDDC) along with hybrid, multi-cloud, serverless, converged and related topics at Amazon among other venues. Learn more here.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

What This All Means

A year ago at VMworld the initial conversations were started around what would become the VMware Cloud on AWS solution. Also a year ago besides VMware Integrated Containers (VIC) and some other pieces, the overall container and in particular related management story was a bit cloudy (pun intended). However, now the fog and cloud seem to be clearing with the PKS solution, along with details of VMware Cloud on AWS. Likewise vSphere, vSAN and NSX along with associated vRealize tools continue to evolve as well as customer deployment growing. All in all, VMware continues to evolve, let’s see how things progress now over the year until the next VMworld.

By the way, if you have not heard, its Blogtober, check out some of the other blogs and posts occurring during October here.

Ok, nuff said, for now.
Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (and vSAN). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

Microsoft Azure September 2017 Software Defined Data Infrastructure Updates

Microsoft Azure September 2017 Software Defined Data Infrastructure Updates

server storage I/O data infrastructure trends

Microsoft and Azure September 2017 Software Defined Data infrastructure Updates

September was a busy month for data infrastructure topics as well as Microsoft in terms of new and enhanced technologies. Wrapping up September was Microsoft Ignite where Azure, Azure Stack, Windows, O365, AI, IoT, development tools announcements occurred, along with others from earlier in the month. As part of the September announcements, Microsoft released a new version of Windows server (e.g. 1709) that has a focus for enhanced container support. Note that if you have deployed Storage Spaces Direct (S2D) and are looking to upgrade to 1709, do your homework as there are some caveats that will cause you to wait for the next release. Note that there had been new storage related enhancements slated for the September update, however those were announced at Ignite to being pushed to the next semi-annual release. Learn more here and also here.

Azure Files and NFS

Microsoft made several Azure file storage related announcements and public previews during September including Native NFS based file sharing as companion to existing Azure Files, along with public preview of new Azure File Sync Service. Native NFS based file sharing (public preview announced, service is slated to be available in 2018) is a software defined storage deployment of NetApp OnTAP running on top of Azure data infrastructure including virtual machines and leverage Azure underlying storage.

Note that the new native NFS is in addition to the earlier native Azure Files accessed via HTTP REST and SMB3 enabling sharing of files inside Azure public cloud, as well as accessible externally from Windows based and Linux platforms including on premises. Learn more about Azure Storage and Azure Files here.

Azure File Sync (AFS)

Azure File Sync AFS

Azure File Sync (AFS) has now entered public preview. While users of Windows-based systems have been able to access and share Azure Files in the past, AFS is something different. I have used AFS for some time now during several private preview iterations having seen how it has evolved, along with how Microsoft listens incorporating feedback into the solution.

Lets take a look at what is AFS, what it does, how it works, where and when to use it among other considerations. With AFS, different and independent systems can now synchronize file shares through Azure. Currently in the AFS preview Windows Server 2012 and 2016 are supported including bare metal, virtual, and cloud based. For example I have had bare metal, virtual (VMware), cloud (Azure and AWS) as part of participating in a file sync activities using AFS.

Not to be confused with some other storage related AFS including Andrew File System among others, the new Microsoft Azure File Sync service enables files to be synchronized across different servers via Azure. This is different then the previous available Azure File Share service that enables files stored in Azure cloud storage to be accessed via Windows and Linux systems within Azure, as well as natively by Windows platforms outside of Azure. Likewise this is different from the recently announced Microsoft Azure native NFS file sharing serving service in partnership with NetApp (e.g. powered by OnTAP cloud).

AFS can be used to synchronize across different on premise as well as cloud servers that can also function as cache. What this means is that for Windows work folders served via different on premise servers, those files can be synchronized across Azure to other locations. Besides providing a cache, cloud tiering and enterprise file sync share (EFSS) capabilities, AFS also has robust optimization for data movement to and from the cloud and across sites, along with management tools. Management tools including diagnostics, performance and activity monitoring among others.

Check out the AFS preview including planning for an Azure File Sync (preview) deployment (Docs Microsoft), and for those who have Yammer accounts, here is the AFS preview group link.

Microsoft Azure Blob Events via Microsoft

Azure Blob Storage Tiering and Event Triggers

Two other Azure storage features that are in public preview include blob tiering (for cold archiving) and event triggers for events. As their names imply, blob tiering enables automatic migration from active to cold inactive storage of dormant date. Event triggers are policies rules (code) that get executed when a blob is stored to do various functions or tasks. Here is an overview of blob events and a quick start from Microsoft here.

Keep in mind that not all blob and object storage are the same, a good example is Microsoft Azure that has page, block and append blobs. Append blobs are similar to what you might be familiar with other services objects. Here is a Microsoft overview of various Azure blobs including what to use when.

Project Honolulu and Windows Server Enhancements

Microsoft has evolved from command prompt (e.g. early MSDOS) to GUI with Windows to command line extending into PowerShell that left some thinking there is no longer need for GUI. Even though Microsoft has extended its CLI with PowerShell spanning WIndows platforms and Azure, along with adding Linux command shell, there are those who still want or need a GUI. Project Honolulu is the effort to bring GUI based management back to Windows in a simplified way for what had been headless, and desktop less deployments (e.g. Nano, Server Core). Microsoft had Server Management Tools (SMT) accessible via the Azure Portal which has been discontinued.


Project Honolulu Image via Microsoft.com

This is where project Honolulu comes into play for managing Windows Server platforms. What this means is that for those who dont want to rely on or have a PowerShell dependency have an alternative option. Learn more about Project Honolulu here and here, including download the public preview here.

Storage Spaces Direct (S2D) Kepler Appliance

Data Infrastructure provider DataOn has announced a new turnkey Windows Server 2016 Storage Spaces Direct (S2D) powered Hyper-Converged Infrastructure (e.g. productization of project Kepler-47) solution with two node small form factor servers (partner with MSI). How small? Think suitcase or airplane roller board carry on luggage size.

What this means is that you can get into the converged, hyper-converged software defined storage game with Windows-based servers supporting Hyper-V virtual machines (Windows and Linux) including hardware for around $10,000 USD (varies by configuration and other options).

Azure and Microsoft Networking News

Speaking of Microsoft Azure public cloud, ever wonder what the network that enables the service looks like and some of the software defined networking (SDN) along with network virtualization function (NFV) objectives are, have a look at this piece from over at Data Center Knowledge.

In related Windows, Azure and other focus areas, Microsoft, Facebook and Telxius have completed the installation of a high-capacity subsea cable (network) to cross the atlantic ocean. Whats so interesting from a data infrastructure, cloud or legacy server storage I/O and data center focus perspective? The new network was built by the combined companies vs. in the past by a Telco provider consortium with the subsequent bandwidth sold or leased to others.

This new network is also 4,000 miles long including in depths of 11,000 feet, supports with current optics 160 terabits (e.g. 20 TeraBytes) per second capable of supporting 71 million HD videos streamed simultaneous. To put things into perspective, some residential Fiber Optic services can operate best case up to 1 gigabit per second (line speed) and in an asymmetrical fashion (faster download than uploads). Granted there are some 10 Gbit based services out there more common with commercial than residential. Simply put, there is a large amount of bandwidth increased across the atlantic for Microsoft and Facebook to support growing demands.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

What This All Means

Microsoft announced a new release of Windows Server at Ignite as part of its new semi-annual release cycle. This latest version of Windows server is optimized for containers. In addition to Windows server enhancements, Microsoft continues to extend Azure and related technologies for public, private and hybrid cloud as well as software defined data infrastructures.

By the way, if you have not heard, its Blogtober, check out some of the other blogs and posts occurring during October here.

Ok, nuff said, for now.
Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (and vSAN). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

September 2017 Server StorageIO Data Infrastructure Update Newsletter



Server StorageIO September 2017 Data Infrastructure Update Newsletter

Volume 17, Issue IX (September 2017)

Hello and welcome to the September 2017 issue of the Server StorageIO update newsletter.

With September being generally known as back to school month, the two September event bookends were VMware VMworld and Microsoft Ignite with many other things in between. Needless to say, a lot has happened in and around data infrastructure topic areas since the August newsletter (here if you missed it). Here is a post covering some of the things that I participated with during September including presentations at events in Las Vegas (VMworld), New York City (Wipro SDx Summit), SNIA SDC in Santa Clara, Fujifilm Executive Summitt in Seattle, Minneapolis/St. Paul CMG along with other activities.

Software-Defined Data Infrastructure Essentials SDDI SDDC

One of the activities I participated in with while at VMworld in Las Vegas was a book signing event at the VMware bookstore of my new book Software Defined Data Infrastructure Essentials (CRC Press) available at Amazon.com and other global venues.

September has been a busy month pertaining data infrastructure including server storage I/O related trends, activities, news, perspectives and related topics, so let’s have a look at them.

In This Issue

Enjoy this edition of the Server StorageIO data infrastructure update newsletter.

Cheers GS

Data Infrastructure and IT Industry Activity Trends

Some recent Industry Activities, Trends, News and Announcements include:

The month started out with VMworld in Las Vegas (e.g. one of the event bookends for the month). Rather than a long list of announcements in this newsletter, check out this StorageIOblog post covering VMworld, VMware and Dell EMC and related news. As part of VMworld, VMware and Amazon Web Services (AWS) announced news about their partnership. AWS also had several other enhancements and new product announcements during september that can be found in this StorageIOblog post here.

AWS, Dell EMC and VMware were not the only ones making news or announcements during September. Startup NVMe based storage startup Apeiron has announced a Splunk appliance to boost log and analytics processing performance. Gigamon has extended its public cloud monitoring, insight awareness and analytics capabilities including support for Microsoft Azure.

For those looking for the latest new emerging data infrastructure vendors to watch, add Vexta to your list of NVMe based storage systems. Vexta talks a lot about NVMe particular for their backend (e.g. where data stored on NVM based devices accessed via NVMe), access of their storage system is via traditional Fibre Channel (FC) or emerging NVMe over fabric.

Long time data infrastructure server and storage vendor HDS (Hitachi Data Systems) is no more (at least in name) having re branded themselves as Vantara focusing on IoT and Cloud analytics besides their traditional data center focus. Vantara combines what was HDS, Hitachi Insight Group and Pentaho into a single unit effectively based in what was HDS as a new, repackaged, refocused business unit.

Another longtime data infrastructure solution and service provider IBM announced a new Linux only zSeries (ZED) mainframe solution. Some might think the Mainframe is dead, others that it can only run Linux as a virtual guest in a virtual machine. On the other hand some might recall that there are native Linux implementations on the ZED including Ubuntu among others.

Also note that while IBM zOS mainframe operating systems use FICON for storage access, native ZED Linux systems can use open systems based Fibre Channel (FC) e.g. SCSI command set protocols. Is the ZED based Linux for everybody or every environment? Probably not, however for those who have large-scale Linux needs, it might be worth a look to do a total cost of ownership analysis. If nothing else, do your homework, play your cards right and you might have some leverage with the x86 based server crowd when it comes to negotiating leverage.

Cloud storage gateway vendor Nasuni has landed another $38 Million USD in funding, hopefully that will enable them to start landing some new and larger customer revenues growing their business. Meanwhile storage startup Qumulo has announced extending their global file fabric name space to include spanning AWS.

Attala Systems has announced next generation software defined storage for data infrastructures for Telco environments. Percona has added an experimental release of their MySQL engine enhancing performance for high volume, write intensive workloads along with improved cost effectiveness.

Software defined storage vendor Datacore announced enhancements to support fast databases for online transaction processing (OLTP) along with analytics. Meanwhile Linux provider SUSE continues to expand its software defined storage story based around Ceph. Panasas has enhanced its scale out high performance cluster file system global name space for HPC environments with 20 PByte support. Another longtime storage vendor X-IO (formerly known as Xiotech) announced their 4th generation of their Intelligent Storage Element (ISE).

September wrapped up with Microsoft Ignite conference along with many updated, enhancements and new features for Azure, Azure Stack, Windows among others. Read more about those and other Microsoft September announcements here in this StorageIOblog post.

Check out other industry news, comments, trends perspectives here.

Server StorageIO Commentary in the news

Recent Server StorageIO industry trends perspectives commentary in the news.

Via CDW: Comments on Is Your Network About To Fail?
Via EnterpriseStorageForum: Comments on Data Storage and Big Data Analytics
Via InfoGoto: Comments on Cloud FOMO (Fear of missing out)
Via InfoGoto: Comments on Building a Modern Data Strategy
Via InfoGoto: Comments on the future of Multi-Cloud Computing
Via InfoGoto: Comments on AI, Machine Learning and Data management
Via InfoGoto: Comments on Your riskiest data might be in plain sight
Via InfoGoto: Comments on Data Management Too Much To Handle
Via InfoGoto: Comments on Google Cloud Platform Gaining Data Storage Momentum
Via InfoGoto: Comments on Singapore High Rise Data Centers
Via InfoGoto: Comments on New Tape Storage Capacity
Via EnterpriseStorageForum: Comments on 8 ways to save on cloud storage
Via EnterpriseStorageForum: Comments on Google Cloud Platform and Storage

View more Server, Storage and I/O trends and perspectives comments here

Server StorageIOblog Posts

Recent and popular Server StorageIOblog posts include:

In Case You Missed It #ICYMI

View other recent as well as past StorageIOblog posts here

Server StorageIO Data Infrastructure Tips and Articles

Recent Server StorageIO industry trends perspectives commentary in the news.

Via EnterpriseStorageForum: Comments on Who Will Rule the Storage World?
Via InfoGoto: Comments on Google Cloud Platform Gaining Data Storage Momentum
Via InfoGoto: Comments on Singapore High Rise Data Centers
Via InfoGoto: Comments on New Tape Storage Capacity
Via EnterpriseStorageForum: Comments on 8 ways to save on cloud storage
Via EnterpriseStorageForum: Comments on Google Cloud Platform and Storage

View more Server, Storage and I/O trends and perspectives comments here

Server StorageIO Recommended Reading (Watching and Listening) List

In addition to my own books including Software Defined Data Infrastructure Essentials (CRC Press 2017), the following are Server StorageIO recommended reading, watching and listening list items. The list includes various IT, Data Infrastructure and related topics.

Intel Recommended Reading List (IRRL) for developers is a good resource to check out.

Its October which means that it is also Blogtober, check out some of the blogs and posts occurring during October here.

Preston De Guise aka @backupbear is Author of several books has an interesting new site Foolsrushin.info that looks at topics including Ethics in IT among others. Check out his new book Data Protection: Ensuring Data Availability (CRC Press 2017).

Brendan Gregg has a great site for Linux performance related topics here.

Greg Knieriemen has a must read weekly blog, post, column collection of whats going on in and around the IT and data infrastructure related industries, Check it out here.

Interested in file systems, CIFS, SMB, SAMBA and related topics then check out Chris Hertels book on implementing CIFS here at Amazon.com

For those involved with VMware, check out Frank Denneman VMware vSphere 6.5 host resource guide-book here at Amazon.com.

I often mention in presentations a must have for anybody involved with software defined anything, or programming for that matter which is the Niklaus Wirth classic Algorithms + Data Structures = Programs that you can get on Amazon.com here.

Another great book to have is Seven Databases in Seven Weeks which not only provides an overview of popular NoSQL databases such as Cassandra, Mongo, HBASE among others, lots of good examples and hands on guides. Get your copy here at Amazon.com.

Watch for more more items to be added to the book shelf soon.

Events and Activities

Recent and upcoming event activities.

Nov. 2, 2017 – Webinar – Modern Data Protection for Hyper-Convergence
Sep. 21, 2017 – MSP CMG – Minneapolis MN
Sep. 20, 2017 – Webinar – BC, DR and Business Resiliency (BR) tips
Sep. 14, 2017 – Fujifilm IT Executive Summit – Seattle WA
Sep. 12, 2017 – SNIA Software Developers Conference (SDC) – Santa Clara CA
Sep. 7, 2017 – Wipro SDX – Enabling, Planning Your Software Defined Journey
August 28-30, 2017 – VMworld – Las Vegas

See more webinars and activities on the Server StorageIO Events page here.

Server StorageIO Industry Resources and Links

Useful links and pages:
Microsoft TechNet – Various Microsoft related from Azure to Docker to Windows
storageio.com/links – Various industry links (over 1,000 with more to be added soon)
objectstoragecenter.com – Cloud and object storage topics, tips and news items
OpenStack.org – Various OpenStack related items
storageio.com/downloads – Various presentations and other download material
storageio.com/protect – Various data protection items and topics
thenvmeplace.com – Focus on NVMe trends and technologies
thessdplace.com – NVM and Solid State Disk topics, tips and techniques
storageio.com/converge – Various CI, HCI and related SDS topics
storageio.com/performance – Various server, storage and I/O benchmark and tools
VMware Technical Network – Various VMware related items

Ok, nuff said, for now.

Cheers
Gs

Greg Schulz – Multi-year Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

Getting Caught Up What Happened In September 2017

server storage I/O data infrastructure trends

Getting Caught Up, What Happened In September?

Seems like just yesterday it was the end of August with the start of VMworld in Las Vegas, now its the end of September and Microsoft Ignite in Orlando is wrapping up. Microsoft has made several announcements this week at Ignite including Azure cloud related, AI, IoT, Windows platforms, O365 among others. More about Microsoft Azure, Azure Stack, Windows Server, Hyper-V and related data infrastructure topics in future posts.

Like many of you, September is a busy time of the year, so here is a recap of some of what I have been doing for the past month (among other things).

vmworld 2017

VMworld Las Vegas

During VMworld US VMware announced enhanced workspace, security and endpoint solutions, Pivotal Container Service (PKS) with Google for Kubernetes serverless container management, DXC partnership for hybrid cloud management, security enablement via its AppDefense solutions, data infrastructure platform enhancements including integrated OpenStack, vRealize management tools, vSAN among others. VMware also made announcements including expanded multi-cloud and hybrid cloud support along with VMware on AWS as well as Dell EMC data protection for VMware and AWS environments.

xxxx

Software Defined Data Infrastructure Essentials (CRC Press) at VMworld bookstore

In other VMworld activity, my new book Software Defined Data Infrastructure Essentials (CRC Press) made its public debut in the VMware book store where I did a book signing event. You can get your copy of Software Defined Data Infrastructure Essentials which includes Software Defined Data Centers (SDDC) along with hybrid, multi-cloud, serverless, converged and related topics at Amazon among other venues. Learn more here.

Software Defined Everything (x)

In early September I was invited to present at the Wipro Software Defined Everything (x) event in New York City. This event follows Wipro invited me to present at in London England this past January at the inaugural SDx Summit event. At the New York City event my presentation was Planning and Enabling Your Journey to SDx which bridged the higher level big picture industry trends to the applied feet on the ground topics. Attendees of the event included customers, prospects, partners, various analyst firms along with Wipro personal.

At the Wipro event during a panel discussion a question was asked about definition of software defined. After the usual vendor and industry responses, mine was a simple, put the emphasis on Define as opposed to software, with a focus on what is the resulting outcome. In other words how and what are you defining (e.g. x) which could be storage, server, data center, data infrastructure, network among others to make a particular result, outcome, service or capability. While the emphasis is around defined, that also can mean curate, compose, craft, program or whatever you prefer to create an outcome.

Image via snia.org

Role of Storage in a Software Defined Data Infrastructure

At the Storage Network Industry Association (SNIA) Storage Developers Conference (SDC) in Santa Clara I did a talk about the role of Storage in Software Defined Data Infrastructures. The theme was that not only is there a role, storage is fundamental and essential for any software defined data infrastructure (as well as legacy) from cloud to container, serverless to virtual servers, converged and hybrid among others. Other themes included the changing role of storage along with how hardware needs software, software needs hardware, and serverless has hardware and software somewhere in the stack. Tradecraft along with other related data infrastructure topics were also discussed.

Data Infrastructures Protect Preserve Secure and Serve Information
Various IT and Cloud Infrastructure Layers including Data Infrastructures

While promoted as an event for storage developers by storage developers, based on a lot of the content presented, SNIA could easily increase attendance to a broader audience with some slight tweaks as well as messaging. If SNIA is looking to focus the event only for vendor storage developers, surprise surprise, there were developers there, however I also talked with IT customers who were there among other non developers. SDC IMHO is not a replacement for SNW, however with some simple adjustments in messaging from who shouldn’t attend to who should or could attend, more attendees and sponsors might just happen appear.

Check out the SNIA SDC presentations here, along with my presentation from the 2017 event here (among others).

tape and cloud storage

Tape in a Software Defined and Hybrid Cloud World

I was invited by Fujifilm to present at their recent 9th annual executive summit in Seattle. The Fujifilm event was attended by various partners, customers and industry folks covering a diverse set of topics. Focus areas spanned from legacy IT to hyper-scale to public cloud and High-Performance Compute (HPC) among others. Magnetic Tape (e.g. tape) may be going away from your data center, however, chances are if you are doing or storing things in the cloud, your data may end up on tape. In other words, not only does tape continue to evolve, its place and how used (as well as accessed) is also changing. Check out the Fujifilm site here where you can scroll down and check out mine and other presentations from the event.

Focus on Data Protection (and recovery)

September also saw hurricanes, tropical storms, flooding, earthquakes, and acts of natural events, to man-made accidental as well as intentional including software-defined threats such as ransomware, malware, virus, Equifax data information breaches, leaks, loss among other security concerns. A reminder that there are the headline-making news events, as well as those that may be more common yet not widely talked about. What this means is that big or small, full or partial damage, destruction, loss or loss of access, data protection should be proactive to enable recovery instead of an afterthought.

Think of data protection as an investment instead of cost overhead, however that also means finding ways to spread costs out while gaining more benefit. Also remember that if something can occur, fail or happen, it probably will. In other words, the question should not be if, rather when, with what impact. This also means evolving from backup/restore, disaster recovery to business resiliency that enables your applications and data to stay available as well as accessible. In other words, how well are you prepared?

Additional data protection related topics and content include:

  • Free Webinar (registration required) with tips for disaster recovery (DR) and business resiliency (BR)
  • Preventing Unexpected Disasters article tip via Iron Mountain
  • Server StorageIO data infrastructure data protection diaries (various tips and content)
  • Free webinar (registration required) planning for GDPR
  • Time to recover, do you know where backup data is (article from Computerweekly)
  • Ensuring your data infrastructure remains available (article from Networkworld)
  • Tips on preparing for Hurricane and storm season (via IronMountain)

Expanding Your Data Infrastructure Tradecraft

At the September Minneapolis St. Paul (MSP) Computer Measurement Group (CMG) event, I gave a presentation discussing industry trends perspectives, buzzword bingo updates including software defined, NVM (the media) vs. NVMe (the interface) benchmarking, tools, cloud, serverless and tradecraft. Tradecraft as a refresher are those skills and fundamental experiences you acquire over time including what tools, techniques to use for different scenarios.

As part of the CMG presentation, the discussion looked at expanding your data infrastructure tradecraft into adjacent areas around your current focus. Also discussed were the importance of context as different words have two or more meanings. For example SAS can mean Scandinavian Air System, Statistics Analysis Software the original unstructured and big data tool, as well as for storage Serial Attached SCSI. However there is another meaning for SAS which spans server, storage, networking, cloud, security and other focus areas which is Shared Access Signature.

Downloads the CMG and other presentations from the Server StorageIO website here.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

What This All Means

The above are some of the things I was involved with during September with themes of data infrastructure, data protection, software defined cloud, virtual, serverless containers, servers, storage, I/O networking, SSD including NVMe, performance and capacity planning, metrics that matter, management among other topics. It was great meeting many new people at the various venues this past month, likewise seeing old acquaintances and friends. Also thanks to all who have ordered copies of my new book Software Defined Data Infrastructure Essentials along with your comments. Check out the Server StorageIO data infrastructure update newsletter for other related activity, industry trends among other topics. Now lets see how fast October and the rest of 2017 goes.

Ok, nuff said, for now.
Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (and vSAN). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

Hot Popular New Trending Data Infrastructure Vendors To Watch

Hot Popular New Trending Data Infrastructure Vendors To Watch

server storage I/O data infrastructure trends

Updated 1/21/2018

A common question I get asked is who are the hot popular new trending data infrastructure vendors to watch. This post looks at some data infrastructure vendors to watch and keep an eye on.

Keep in mind that there is a difference between industry adoption and customer deployment, the former being what the industry (e.g. Vendors, resellers, integrators, investors, consultants, analyst, press, media, analysts, bloggers or other influences) like, want and need to talk about. Then there is customer adoption and deployment which is what is being bought, installed and used.

Some Popular Trending Vendors To Watch

The following is far from an exhaustive list however here are some that come to mind that I’m watching.

Apcera – Enterprise class containers and management tools
AWS – Rolls our new services like a startup with size momentum of a legacy player
Blue Medora – Data Infrastructure insight, software defined management
Broadcom – Avago/LSI, legacy Broadcom, Emulex, Brocade acquisition interesting portfolio
Chelsio – Server, storage and data Infrastructure I/O technologies
Commvault – Data protection and backup solutions
Compuverde – Software defined storage
Data Direct Networks (DDN) – Scale out and high performance storage
Datadog – Software defined management, data infrastructure insight, analytics, reporting
Datrium – Converged software defined data infrastructure solutions
Dell EMC Code – Rexray container persistent storage management
Docker – Container and management tools
E8 Storage – NVMe based storage solutions
Elastifile – Scale out software defined storage and file system
Enmotus – MicroTiering that works with Windows, Linux and various cloud platforms
Everspin – storage class memories and NVDIMM
Excelero – NVMe based storage
Hedvig – Scale out software defined storage
Huawei – While not common in the US, in Europe and elsewhere they are gaining momentum
Intel – Watch what they do with Optane and storage class memories
Kubernetes – Container software defined management
Liqid – Stealth Colorado startup focusing on PCIe fabrics and composable infrastructure
Maxta – Hyper converged infrastructure (HCI) and software defined data infrastructure vendor
Mellanox – While not a startup, keep an eye on what they are doing with their adapters
Micron – Watch what they do with 3D XPoint storage class memory and SSD
Microsoft – Not a startup, however keep an eye on Azure, Azure Stack, Window Server with S2D, ReFS, tiering, CI/HCI as well as Linux services on Windows.
Minio – Software defined storage solutions
NetApp – While FAS/Ontap and Solidfire get the headlines, E series generates revenue, keep an eye on StorageGrid and AltaVault
Neuvector – Container management and security
Noobaa – Software defined storage and more
NVIDA – No longer just another graphics process unit based company
Pivot3 – An original HCI software defined players, granted, some of their competitors might not think so
Pluribus Networks – Software Defined Networks for Software Defined Data Infrastructures
Portwork – Container management and persistent storage
Rozo Systems – Scale out software defined storage and file system
Rubrik – Data Protection software, reminds me of a startup called Commvault 20 years ago.
ScaleMP – Composable scale out software defined servers
Storpool – Scale out software defined storage
Stratoscale – Software defined data infrastructure and hybrid solutions
SUSE – Linux distribution looking to expand their offerings, gain more insight
Tidalscale – Composable software defined data infrastructures
Turbonomic – Software Defined Management, insight, analytics and automation
Ubuntu – Known for their Linux distribution, check out their Metal as a Service (MaaS) technology
Veeam – Data protection and backup solutions
technology
Virtuozzo – Software defined storage and data infrastructure technologies
VMware – AWS, vSAN, NSX, Integrated Containers and much more
WekaIO – Scale out software defined storage and file system

Some Popular Trending Technology Trends

  • ARM, ASIC, FPGA, GPU servers among others
  • Converged Infrastructure (CI), Hyper Converged Infrastructure (HCI), Composable Infrastructure
  • Analytics, reporting, insight, machine learning (ML), artificial intelligence (AI), automation
  • Software Defined including Cloud, Virtual, Containers, Docker, kubernetes, mesos, serverless, micro services
  • Data protection, backup/restore, archive, security, business resiliency (BR), business continuance (BC), disaster recovery (DR)
  • Non-volatile memory (NMV), NVM Express (NVMe), storage class memories (SCM), persistent memory, nand flash, SSD

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

Data Infrastructures Protect Preserve Secure and Serve Information
Various IT and Cloud Infrastructure Layers including Data Infrastructures

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

There are always more hot popular new or trending data infrastructure vendors to watch, which ones are you keeping an eye on?

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Travel Fun Crossword Puzzle For VMworld 2017 Las Vegas

Travel Fun Crossword Puzzle For VMworld 2017 Las Vegas

server storage I/O data infrastructure trends

Some of you may be traveling to VMworld 2017 in Las Vegas next week to sharpen, expand, refresh or share your VMware and data infrastructure tradecraft (skills, experiences, expertise, knowledge). Here is something fun to sharpen your VMware skills while traveling. Most of these should be pretty easy meaning that you do not have to be a Unicorn, full of vCertifications, vCredentials or a 9, 8, 7, 6, 5, 4, 3, 2 or 1st time vExpert or top 100 vBlogger. However if you need the answers they are below.

VMworld 2017 crossword puzzle SDDI, SDDC

Note that you can also click here to get a PDF version that is larger (or click on the image) that also has the answers.

Software Defined Data Infrastructure Essentials SDDI SDDC

For those of you who will be in Las Vegas at VMworld next week, stop by the VMworld Book Store at 1PM on Tuesday (the 29th) where I will be doing a book signing event for my new book Software Defined Data Infrastructure Essentials (CRC Press), stop by and say hello. Note there are also Kindle and other electronic versions of my new SDDI Essentials Book on Amazon.com and other venues if you need something to read during your upcoming travels.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

Data Infrastructures Protect Preserve Secure and Serve Information
Various IT and Cloud Infrastructure Layers including Data Infrastructures

What This All Means

Have a safe and fun trip on your way to Las Vegas for next weeks VMworld, enjoy the crossword puzzle, and if you need the answers, they are located here (PDF), see you at VMworld 2017 in Last Vegas.

Ok, nuff said, for now.
Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (and vSAN). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

Announcing Software Defined Data Infrastructure Essentials Book by Greg Schulz

New SDDI Essentials Book by Greg Schulz of Server StorageIO

Cloud, Converged, Virtual Fundamental Server Storage I/O Tradecraft

server storage I/O data infrastructure trends

Update 1/21/2018
Over the past several months I have posted, commenting, presenting and discussing more about Software Defined Data Infrastructure Essentials aka SDDI or SDDC and SDI. Now it is time to announce my new book (my 4th solo project), Software Defined Data Infrastructure Essentials Book (CRC Press). Software Defined Data Infrastructure Essentials is now generally available at various global venues in hardcopy, hardback print as well as various electronic versions including via Amazon and CRC Press among others. For those attending VMworld 2017 in Las Vegas, I will be doing a book signing, meet and greet at 1PM Tuesday August 29 in the VMworld book store, as well as presenting at various other fall industry events.

Software Defined Data Infrastructure Essentials Book Announcement

(Via Businesswire) Stillwater, Minnesota – August 23, 2017  – Server StorageIO, a leading independent IT industry advisory and consultancy firm, in conjunction with publisher CRC Press, a Taylor and Francis imprint, announced the release and general availability of “Software-Defined Data Infrastructure Essentials,” a new book by Greg Schulz, noted author and Server StorageIO founder.

Software Defined Data Infrastructure Essentials

The Software Defined Data Infrastructure Essentials book covers physical, cloud, converged (and hyper-converged), container, and virtual server storage I/O networking technologies, revealing trends, tools, techniques, and tradecraft skills.

Data Infrastructures Protect Preserve Secure and Serve Information
Various IT and Cloud Infrastructure Layers including Data Infrastructures

From cloud web scale to enterprise and small environments, IoT to database, software-defined data center (SDDC) to converged and container servers, flash solid state devices (SSD) to storage and I/O networking,, the book helps develop or refine hardware, software, services and management experiences, providing real-world examples for those involved with or looking to expand their data infrastructure education knowledge and tradecraft skills.

Software Defined Data Infrastructure Essentials book topics include:

    • Cloud, Converged, Container, and Virtual Server Storage I/O networking
    • Data protection (archive, availability, backup, BC/DR, snapshot, security)
    • Block, file, object, structured, unstructured and data value
    • Analytics, monitoring, reporting, and management metrics
    • Industry trends, tools, techniques, decision making
    • Local, remote server, storage and network I/O troubleshooting
    • Performance, availability, capacity and  economics (PACE)

Where To Purchase Your Copy

Order via Amazon.com and CRC Press along with Google Books among other global venues.

What People Are Saying About Software Defined Data Infrastructure Essentials Book

“From CIOs to operations, sales to engineering, this book is a comprehensive reference, a must-read for IT infrastructure professionals, beginners to seasoned experts,” said Tom Becchetti, advisory systems engineer.

“We had a front row seat watching Greg present live in our education workshop seminar sessions for ITC professionals in the Netherlands material that is in this book. We recommend this amazing book to expand your converged and data infrastructure knowledge from beginners to industry veterans.”

Gert and Frank Brouwer – Brouwer Storage Consultancy

“Software-Defined Data Infrastructures provides the foundational building blocks to improve your craft in several areas including applications, clouds, legacy, and more.  IT professionals, as well as sales professionals and support personal, stand to gain a great deal by reading this book.”

Mark McSherry- Oracle Regional Sales Manager

“Greg Schulz has provided a complete ‘toolkit’ for storage management along with the background and framework for the storage or data infrastructure professional (or those aspiring to become one).”
Greg Brunton – Experienced Storage and Data Management Professional

“Software-defined data infrastructures are where hardware, software, server, storage, I/O networking and related services converge inside data centers or clouds to protect, preserve, secure and serve applications and data,” said Schulz.  “Both readers who are new to data infrastructures and seasoned pros will find this indispensable for gaining and expanding their knowledge.”

SDDI and SDDC components

More About Software Defined Data Infrastructure Essentials
Software Defined Data Infrastructures (SDDI) Essentials provides fundamental coverage of physical, cloud, converged, and virtual server storage I/O networking technologies, trends, tools, techniques, and tradecraft skills. From webscale, software-defined, containers, database, key-value store, cloud, and enterprise to small or medium-size business, the book is filled with techniques, and tips to help develop or refine your server storage I/O hardware, software, Software Defined Data Centers (SDDC), Software Data Infrastructures (SDI) or Software Defined Anything (SDx) and services skills. Whether you are new to data infrastructures or a seasoned pro, you will find this comprehensive reference indispensable for gaining as well as expanding experience with technologies, tools, techniques, and trends.

Software Defined Data Infrastructure Essentials SDDI SDDC content

This book is the definitive source providing comprehensive coverage about IT and cloud Data Infrastructures for experienced industry experts to beginners. Coverage of topics spans from higher level applications down to components (hardware, software, networks, and services) that get defined to create data infrastructures that support business, web, and other information services. This includes Servers, Storage, I/O Networks, Hardware, Software, Management Tools, Physical, Software Defined Virtual, Cloud, Docker, Containers (Docker and others) as well as Bulk, Block, File, Object, Cloud, Virtual and software defined storage.

Additional topics include Data protection (Availability, Archiving, Resiliency, HA, BC, BR, DR, Backup), Performance and Capacity Planning, Converged Infrastructure (CI), Hyper-Converged, NVM and NVMe Flash SSD, Storage Class Memory (SCM), NVMe over Fabrics, Benchmarking (including metrics matter along with tools), Performance Capacity Planning and much more including whos doing what, how things work, what to use when, where, why along with current and emerging trends.

Book Features

ISBN-13: 978-1498738156
ISBN-10: 149873815X
Hardcover: 672 pages
(Available in Kindle and other electronic formats)
Over 200 illustrations and 70 plus tables
Frequently asked Questions (and answers) along with many tips
Various learning exercises, extensive glossary and appendices
Publisher: Auerbach/CRC Press Publications; 1 edition (June 19, 2017)
Language: English

SDDI and SDDC toolbox

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

Data Infrastructures Protect Preserve Secure and Serve Information
Various IT and Cloud Infrastructure Layers including Data Infrastructures

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

Data Infrastructures exist to protect, preserve, secure and serve information along with the applications and data they depend on. With more data being created at a faster rate, along with the size of data becoming larger, increased application functionality to transform data into information means more demands on data infrastructures and their underlying resources.

Software-Defined Data Infrastructure Essentials: Cloud, Converged, and Virtual Fundamental Server Storage I/O Tradecraft is for people who are currently involved with or looking to expand their knowledge and tradecraft skills (experience) of data infrastructures. Software-defined data centers (SDDC), software data infrastructures (SDI), software-defined data infrastructure (SDDI) and traditional data infrastructures are made up of software, hardware, services, and best practices and tools spanning servers, I/O networking, and storage from physical to software-defined virtual, container, and clouds. The role of data infrastructures is to enable and support information technology (IT) and organizational information applications.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

Everything is not the same in business, organizations, IT, and in particular servers, storage, and I/O. This means that there are different audiences who will benefit from reading this book. Because everything and everybody is not the same when it comes to server and storage I/O along with associated IT environments and applications, different readers may want to focus on various sections or chapters of this book.

If you are looking to expand your knowledge into an adjacent area or to understand whats under the hood, from converged, hyper-converged to traditional data infrastructures topics, this book is for you. For experienced storage, server, and networking professionals, this book connects the dots as well as provides coverage of virtualization, cloud, and other convergence themes and topics.

This book is also for those who are new or need to learn more about data infrastructure, server, storage, I/O networking, hardware, software, and services. Another audience for this book is experienced IT professionals who are now responsible for or working with data infrastructure components, technologies, tools, and techniques.

Learn more here about Software Defined Data Infrastructure (SDDI) Essentials book along with cloud, converged, and virtual fundamental server storage I/O tradecraft topics, order your copy from Amazon.com or CRC Press here, and thank you in advance for learning more about SDDI and related topics.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Like IT Data Centers Do You Take Trade Show Exhibit Infrastructure For Granted?

Do You Take Trade Show Exhibit Infrastructure For Granted?

server storage I/O data infrastructure trends

Think about this for a moment; do you assume that Information Technology (IT) and Cloud based data centers along with their associated Data Infrastructure supporting various applications will be accessible when needed. Likewise, when you go to a trade show, conference, symposium, user group or another conclave is it assumed that the trade show, exposition (expo), exhibits, booths, stands or demo areas will be ready, waiting and accessible?

IT industry cloud software defined events

Fire Disrupts Flash Memory Summit Conference Exhibits

This past week at the Flash Memory Summit (FMS) conference trade show event in Santa Clara California, what normally would be taken for granted (e.g. expo hall and exhibits) were disrupted. The disruption (more here and here) was caused by an early morning fire in one of the exhibitor’s booths (stand) in the expo hall (view some photos here via Toms Hardware.com).

Fortunately, nobody was hurt, at least physically, and damage (physically) appears to have been isolated.

However while the key notes, panels, and other presentations did take place as part of the show must go on, the popular exhibit expo hall did not. Granted for some people who only attend conferences or seminar events for the presentation content, lack of the exhibition hall simply meant no free giveaways.

On the other hand, for those who attend events like FMS mainly for the exhibition hall experience, the show did not go on, perhaps resulting in a trip in vain (e.g. how you might be able to recoup some travel costs in some scenarios) for some people. For example, those who were attending to meet with a particular vendor, see a product technology, conduct some business or other meetings, do an interview, video, podcast, take some photos, or simply get some free stuff were disrupted.

Likewise those behind the scenes, from conference organizers, event staff not to mention the vendor’s sponsors who put resources (time, money, people, and equipment) into an exhibit were disrupted. Vendors were still able to issue their press releases and conduct their presentations, keynotes, panel discussions, however what about the lack of the expo.

Do We Take Data and Event Infrastructures For Granted

This begs the question of if trade show exhibits still have value, or can an event function without one?

I am not sure as some events can and do stand on their merit with presentation content being the primary focus, others the expo is the draw, many are hybrid with a mix of both.

A question and point of this piece is that how many people take conferences in general, and exhibits along with their associated Infrastructure for granted?

How many know or understand the amount of time, money, people resources and various tradecraft skills across different disciplines go into event planning, staging, coordination, the execution, so they occur?

This also ties into the theme of how many people only think and assume that IT data centers and clouds along with their data Infrastructure resources, services are available supporting applications along with data access to give information?

The same holds true for your telephone (plain old telephone system [POTS] and cellular or mobile) service, gas, electric, sewer, water, waste (garbage), traditional or network based television, internet provider, highways, railroads, airports, the list goes on.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

What This All Means

The good news is that nobody physically was injured this past week.

Granted some may have incurred emotional, monetary or public and marketing related injuries, however, those can be dealt with over time.

My point is, do we assume too much (perhaps rightfully so) that events, exhibits and other trade show conference related items are always on, always available, accessible open on time? With IT data center and clouds, you have different expectation levels of access, availability, durability, survivability for a given cost to meet service expectations.

Data center, cloud and data infrastructure SDC, SDDI, SDI, SDx

Next time you attend a webinar, seminar, conference, symposium, trade show, presentation, exhibit or expo, take a moment and look around at what you see, as well as what you do not see. Having been in involved in and around conferences, conventions, seminars, expos across different industries, both behind the scenes as well as on the public side, I do not take these events for granted.

Knowing what goes into the planning, coordination, scheduling, promotion, logistics, all the things behind the scenes, next time you go to an event, look around. What you can see that perhaps are not meant to be seen as part of their Infrastructure. In event venue exhibit halls as well as data centers, there are those things you see such as data infrastructure resources including racks of servers, storage, I/O networking, monitors, displays, work areas, heating ventilation air conditioning (HVAC) along with those you might not see.

What you might not see and take for granted are the smoke and fire detection along with suppression systems which at the Santa Clara convention center appeared to have done their job. There are also the electrical power and distribution systems; perhaps battery backed uninterruptible power systems (UPS) along with standby alternate generator power.

How about a big round of applause, thank you, Atta boy and Atta girl, acknowledgment and other signs of appreciation for all those involved behind the scenes who do the planning, preparation, coordination, setup, tear down and in person what you see at events.

Thank you to all who have, and continue to enable trade shows, conferences, seminars, exhibits, road shows among other events to take place, after all, the show must go on. In other words, like IT and cloud Data Centers, do you take trade show exhibit infrastructures for granted?

Ok, nuff said, for now.

Greg Schulz – Multi-year Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

NVMe Wont Replace Flash By Itself They Complement Each Other

NVMe Wont Replace Flash By Itself They Complement Each Other

server storage I/O data infrastructure trends

Updated 2/2/2018

NVMe Wont Replace Flash By Itself They Complement Each Other

>various NVM flash and SSD devices
Various Solid State Devices (SSD) including NVMe, SAS, SATA, USB, M.2

There has been some recent industry marketing buzz generated by a startup to get some attention by claiming via a study sponsored by including the startup that Non-Volatile Memory (NVM) Express (NVMe) will replace flash storage. Granted, many IT customers as well as vendors are still confused by NVMe thinking it is a storage medium as opposed to an interface used for accessing fast storage devices such as nand flash among other solid state devices (SSDs). Part of that confusion can be tied to common SSD based devices rely on NVM that are persistent memory retaining data when powered off (unlike the memory in your computer).

NVMe is an access interface and protocol

Instead of saying NVMe will mean the demise of flash, what should or could be said however some might be scared to say it is that other interfaces and protocols such as SAS (Serial Attached SCSI), AHCI/SATA, mSATA, Fibre Channel SCSI Protocol aka FCP aka simply Fibre Channel (FC), iSCSI and others are what can be replaced by NVMe. NVMe is simply the path or roadway along with traffic rules for getting from point a (such as a server) to point b (some storage device or medium e.g. flash SSD). The storage medium is where data is stored such as magnetic for Hard Disk Drive (HDD) or tape, nand flash, 3D XPoint, Optane among others.

NVMe and NVM better together

NVMe and NVM including flash are better together

The simple quick get to the point is that NVMe (e.g. Non Volatile Memory aka NVM Express [NVMe]) is an interface protocol (like SAS/SATA/iSCSI among others) used for communicating with various nonvolatile memory (NVM) and solid state device (SSDs). NVMe is how data gets moved between a computer or other system and the NVM persistent memory such as nand flash, 3D XPoint, Spintorque or other storage class memories (SCM).

In other words, the only thing NVMe will, should, might or could kill off would be the use of some other interface such as SAS, SATA/AHCI, Fibre Channel, iSCSI along with propritary driver or protocols. On the other hand, given the extensibility of NVMe and how it can be used in different configurations including as part of fabrics, it is an enabler for various NVMs also known as persistent memories, SCMs, SSDs including those based on NAND flash as well as emerging 3D XPoint (or Intel version) among others.

Where To Learn More

View additional NVMe, SSD, NVM, SCM, Data Infrastructure and related topics via the following links.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

Context matters for example, NVM as the medium compared to NVMe as the interface and access protocols. With context in mind you can compare like or similar apples to apples such as nand flash, MRAM, NVRAM, 3D XPoint, Optane among other persistent memories also known as storage class memories, NVMs and SSDs. Likewise with context in mind NVMe can be compared to other interfaces and protocols such as SAS, SATA, PCIe, mSATA, Fibre Channel among others. The following puts all of this into context including various packaging options, interfaces and access protocols, functionality and media.

NVMe is the access for NVM flash
Putting IT all together

Will NVMe kill off flash? IMHO no not by itself, however NVMe combined with some other form of NVM, SCM, persistent memory as a storage medium may eventually combine as an alternative to NVMe and flash (or SAS/SATA and flash). However, for now at least for many applications, NVMe is in your future (along with flash among other storage mediums), the questions include when, where, why, how, with what among other questions (and answers). NVMe wont replace flash by itself (at least yet) as they complement each other.

Keep in mind, if NVMe is the answer, what are the questions.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

New family of Intel Xeon Scalable Processors enable software defined data infrastructures (SDDI) and SDDC

Intel Xeon Scalable Processors SDDI and SDDC

server storage I/O data infrastructure trends

Today Intel announced a new family of Xeon Scalable Processors (aka Purely) that for some workloads Intel claims to be on average of 1.65x faster than their predecessors. Note your real improvement will vary based on workload, configuration, benchmark testing, type of processor, memory, and many other server storage I/O performance considerations.

Intel Scalable Xeon Processors
Image via Intel.com

In general the new Intel Xeon Scalable Processors enable legacy and software defined data infrastructures (SDDI), along with software defined data centers (SDDC), cloud and other environments to support expanding workloads more efficiently as well as effectively (e.g. boosting productivity).

Data Infrastructures and workloads

Some target application and environment workloads Intel is positioning these new processors for includes among others:

  • Machine Learning (ML), Artificial Intelligence (AI), advanced analytics, deep learning and big data
  • Networking including software defined network (SDN) and network function virtualization (NFV)
  • Cloud and Virtualization including Azure Stack, Docker and Kubernetes containers, Hyper-V, KVM, OpenStack VMware vSphere, KVM among others
  • High Performance Compute (HPC) and High Productivity Compute (e.g. the other HPC)
  • Storage including legacy and emerging software defined storage software deployed as appliances, systems or server less deployment modes.

Features of the new Intel Xeon Scalable Processors include:

  • New core micro architecture with interconnects and on die memory controllers
  • Sockets (processors) scalable up to 28 cores
  • Improved networking performance using Quick Assist and Data Plane Development Kit (DPDK)
  • Leverages Intel Quick Assist Technology for CPU offload of compute intensive functions including I/O networking, security, AI, ML, big data, analytics and storage functions. Functions that benefit from Quick Assist include cryptography, encryption, authentication, cipher operations, digital signatures, key exchange, loss less data compression and data footprint reduction along with data at rest encryption (DARE).
  • Optane Non-Volatile Dual Inline Memory Module (NVDIMM) for storage class memory (SCM) also referred to by some as Persistent Memory (PM), not to be confused with Physical Machine (PM).
  • Supports Advanced Vector Extensions 512  (AVX-512) for HPC and other workloads
  • Optional Omni-Path Fabrics in addition to 1/10Gb Ethernet among other I/O options
  • Six memory channels supporting up to 6TB of RDIMM with multi socket systems
  • From two to eight  sockets per node (system)
  • Systems support PCIe 3.x (some supporting x4 based M.2 interconnects)

Note that exact speeds, feeds, slots and watts will vary by specific server model and vendor options. Also note that some server system solutions have two or more nodes (e.g. two or more real servers) in a single package not to be confused with two or more sockets per node (system or motherboard). Refer to the where to learn more section below for links to Intel benchmarks and other resources.

Software Defined Data Infrastructures, SDDC, SDX and SDDI

What About Speeds and Feeds

Watch for and check out the various Intel partners who have or will be announcing their new server compute platforms based on Intel Xeon Scalable Processors. Each of the different vendors will have various speeds and feeds options that build on the fundamental Intel Xeon Scalable Processor capabilities.

For example Dell EMC announced their 14G server platforms at the May 2017 Dell EMC World event with details to follow (e.g. after the Intel announcements).

Some things to keep in mind include the amount of DDR4 DRAM (or Optane NVDIMM) will vary by vendors server platform configuration, motherboards, several sockets and DIMM slots. Also keep in mind the differences between registered (e.g. buffered RDIMM) that give good capacity and great performance, and load reduced DIMM (LRDIMM) that have great capacity and ok performance.

Various nvme options

What about NVMe

It’s there as these systems like previous Intel models support NVMe devices via PCIe 3.x slots, and some vendor solutions also supporting M.2 x4 physical interconnects as well.

server storageIO flash and SSD
Image via Software Defined Data Infrastructure Essentials (CRC)

Note that Broadcom formerly known as Avago and LSI recently announced PCIe based RAID and adapter cards that support NVMe attached devices in addition to SAS and SATA.

server storage data infrastructure sddi

What About Intel and Storage

In case you have not connected the dots yet, the Intel Xeon Scalable Processor based server (aka compute) systems are also a fundamental platform for storage systems, services, solutions, appliances along with tin-wrapped software.

What this means is that the Intel Xeon Scalable Processors based systems can be used for deploying legacy as well as new and emerging software-defined storage software solutions. This also means that the Intel platforms can be used to support SDDC, SDDI, SDX, SDI as well as other forms of legacy and software-defined data infrastructures along with cloud, virtual, container, server less among other modes of deployment.

Image Via Intel.com

Moving beyond server and compute platforms, there is another tie to storage as part of this recent as well as other Intel announcements. Just a few weeks ago Intel announced 64 layer triple level cell (TLC) 3D NAND solutions positioned for the client market (laptop, workstations, tablets, thin clients). Intel with that announcement increased the traditional aerial density (e.g. bits per square inch or cm) as well as boosting the number of layers (stacking more bits as well).

The net result is not only more bits per square inch, also more per cubic inch or cm. This is all part of a continued evolution of NAND flash including from 2D to 3D, MCL to TLC, 32 to 64 layer.  In other words, NAND flash-based Solid State Devices (SSDs) are very much still a relevant and continue to be enhanced technology even with the emerging 3D XPoint and Optane (also available via Amazon in M.2) in the wings.

server memory evolution
Via Intel and Micron (3D XPoint launch)

Keep in mind that NAND flash-based technologies were announced almost 20 years ago (1999), and are still evolving. 3D XPoint announced two years ago, along with other emerging storage class memories (SCM), non-volatile memory (NVM) and persistent memory (PM) devices are part of the future as is 3D NAND (among others). Speaking of 3D XPoint and Optane, Intel had announcements about that in the past as well.

Where To Learn More

Learn more about Intel Xeon Scalable Processors along with related technology, trends, tools, techniques and tips with the following links.

What This All Means

Some say the PC is dead and IMHO that depends on what you mean or define a PC as. For example if you refer to a PC generically to also include servers besides workstations or other devices, then they are alive. If however your view is that PCs are only workstations and client devices, then they are on the decline.

However if your view is that a PC is defined by the underlying processor such as Intel general purpose 64 bit x86 derivative (or descendent) then they are very much alive. Just as older generations of PCs leveraging general purpose Intel based x86 (and its predecessors) processors were deployed for many uses, so to are today’s line of Xeon (among others) processors.

Even with the increase of ARM, GPU and other specialized processors, as well as ASIC and FPGAs for offloads, the role of general purpose processors continues to increase, as does the technology evolution around. Even with so called server less architectures, they still need underlying compute server platforms for running software, which also includes software defined storage, software defined networks, SDDC, SDDI, SDX, IoT among others.

Overall this is a good set of announcements by Intel and what we can also expect to be a flood of enhancements from their partners who will use the new family of Intel Xeon Scalable Processors in their products to enable software defined data infrastructures (SDDI) and SDDC.

Ok, nuff said (for now…).

Cheers
Gs

Greg Schulz – Multi-year Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Watch for the spring 2017 release of his new book "Software-Defined Data Infrastructure Essentials" (CRC Press).

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

June 2017 Server StorageIO Data Infrastructures Update Newsletter

Volume 17, Issue VI

Hello and welcome to the June 2017 issue of the Server StorageIO update newsletter.

For those of you in the northern hemisphere it is time for summer holidays, while in the southern hemisphere its winter time. That means there is a lot going on outside of work, however June has also seen a lot of activity in and around IT data infrastructure along with data centers. Check out some of the industry trends news and updates below.

Software-Defined Data Infrastructure Essentials SDDI SDDC

A quick update following up from the May newsletter is that my new book is now available via Amazon.com, CRC Press and other venues in hardcopy hardcover as well as electronic versions. Think of this as the soft launch with a formal launch and more information being rolled out soon. For now, you can visit the landing page for Software Defined Data Infrastructure Essentials – Cloud, Converged, and Virtual Fundamental Server Storage I/O Tradecraft (CRC PRess/Taylor Francis/Auerbach) at storageio.com/book4 to learn more including view table of contents, preface, how organized among other items.

In This Issue

Enjoy this edition of the Server StorageIO update newsletter.

Cheers GS

Data Infrastructure and IT Industry Activity Trends

Some recent Industry Activities, Trends, News and Announcements include:

Cavium announced 10, 25, 50 and 50Gbps Ethernet server storage I/O NIC solutions (e.g. FastLine 41000 series).

The NVMe Express trade group (e.g. nvmexpress.org) announced the completion of NVMe 1.3 specification. New optional features include support for mobile platforms and book, along with scaling for enterprise as well as cloud environments. Learn more about specifications at the NVMexpress.org site as well as more NVMe material at thenvmeplace.com.

Keep in mind that if the answer is NVMe, what are the questions along with various options from front end to back-end, NVMe and PCIe, NVMeoF, U.2/8639, M2/NGFF among others.

The Fibre Channel Industry Association announced FC-NVMe interoperability plugfest and Gen 6 32GFC activity to support next generation data infrastructures and data centers.

Storage vendor Tegile announced they are joining the growing ranks of vendors adding NVMe support with their InteliFlash OS 3.7 along with other enhancements.

For those of you who are involved with Windows Servers environments along with server, storage and I/O networks, check out Darryl VanderPeijl multi-part series on RDMA, DCB, PFC, ETS and related topics.

HPE and Hedvig announced solutions combing forces to address hybrid cloud storage needs.

IBM and Cisco announced enhancements around their converged (Cisco powered servers) solution for VDI and Hybrid cloud workloads.

Big Data and Analytics vendor Mapr announced enhancements to their converged data management platform for cloud scale data fabrics.

Panzura has enhanced its Freedom software defined storage management solution with version 7 to support expanded unstructured data growth while easing management functions, along with performance updates.

Red Hat announced Ceph Storage 2.3 including Ceph 10.2 (Jewel) combing an NFS gateway.

Scality announced enhancements to its Ring software defined storage cloud and object solution including enhanced security along with data protection capabilities.

Check out other industry news, comments, trends perspectives here.

 

Server StorageIOblog Posts

Recent and popular Server StorageIOblog posts include:

View other recent as well as past StorageIOblog posts here

Server StorageIO Commentary in the news

Recent Server StorageIO industry trends perspectives commentary in the news.

Via EnterpriseStorageForum: 5 Hot Storage Technologies to Watch
Storage can be held back by slow I/O performance, which caused expensive compute resources and memory to be consumed. NVMe reduces wait time while increasing the amount of effective work, enabling higher-profitability compute. The storage I/O capabilities of flash can be fed across PCIe faster to enable multi-core processors to complete more useful work in less time.

Via EnterpriseStorageForum: 10-Year Review of Data Storage
The adoption of hybrid cloud and hybrid converged server storage has appeared more rapidly than many expected. And despite firm pronouncements of their demise, FC, tape and HDD are still very much with us.

Via CDW: Your IT Department Can Help Your Companys Bottom Line Heres How
Not only are the servers more robust performance wise, but they’ve got more compute capability, can handle more workloads, have more memory and also have better resiliency.

Via EnterpriseStorageForum: Top 10 Tips for Software-Defined Storage Deployment
Dell 14g PowerEdge Servers give you greater compute and IO capability, as well as the density you need, NVMe and 25 Gig Ethernet on board,

Via CDW: Meeting IoTs Demands for Networking

View more Server, Storage and I/O trends and perspectives comments here

Events and Activities

Recent and upcoming event activities.

Sep. 13-15, 2017 – Fujifilm IT Executive Summit – Seattle WA

August 28-30, 2017 – VMworld – Las Vegas

June 22, 2017 – Webinar – GDPR and Microsoft Environments

May 11, 2017 – Webinar – Email Archiving, Compliance and Ransomware

See more webinars and activities on the Server StorageIO Events page here.

Server StorageIO Industry Resources and Links

Useful links and pages:
Microsoft TechNet – Various Microsoft related from Azure to Docker to Windows
storageio.com/links – Various industry links (over 1,000 with more to be added soon)
objectstoragecenter.com – Cloud and object storage topics, tips and news items
OpenStack.org – Various OpenStack related items
storageio.com/protect – Various data protection items and topics
thenvmeplace.com – Focus on NVMe trends and technologies
thessdplace.com – NVM and Solid State Disk topics, tips and techniques
storageio.com/converge – Various CI, HCI and related SDS topics
storageio.com/performance – Various server, storage and I/O benchmark and tools
VMware Technical Network – Various VMware related items

Cheers
Gs

Greg Schulz – Multi-year Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

Part II Revisting AWS S3 Storage Gateway (Test Drive Deployment)

server storage I/O trends

Part II Revisiting AWS S3 Storage Gateway (Test Drive Deployment)

This Amazon Web Service (AWS) Storage Gateway Revisited posts is a follow-up to the AWS Storage Gateway test drive and review I did a few years ago (thus why it’s called revisited). As part of a two-part series, the first post looks at what AWS Storage Gateway is, how it has improved since my last review of AWS Storage Gateway along with deployment options. The second post in the series looks at a sample test drive deployment and use.

What About Storage Gateway Costs?

Costs vary by region, type of storage being used (files stored in S3, Volume Storage, EBS Snapshots, Virtual Tape storage, Virtual Tape storage archive), as well as type of gateway host, along with how access and used. Request pricing varies including data written to AWS storage by gateway (up to maximum of $125.00 per month), snapshot/volume delete, virtual tape delete, (prorate fee for deletes within 90 days of being archived), virtual tape archival, virtual tape retrieval. Note that there are also various data transfer fees that also vary by region and gateway host. Learn more about pricing here.

What Are Some Storage Gateway Alternatives

AWS and S3 storage gateway access alternatives include those from various third-party (including that are in the AWS marketplace), as well as via data protection tools (e.g. backup/restore, archive, snapshot, replication) and more commonly storage systems. Some tools include Cloudberry, S3FS, S3 motion, S3 Browser among many others.

Tip is when a vendor says they support S3, ask them if that is for their back-end (e.g. they can access and store data in S3), or front-end (e.g. they can be accessed by applications that speak S3 API). Also explore what format the application, tool or storage system stores data in AWS storage, for example, are files mapped one to one to S3 objects along with corresponding directory hierarchy, or are they stored in a save set or other entity.

AWS Storage Gateway Deployment and Management Tips

Once you have created your AWS account (if you did not already have one) and logging into the AWS console (note the link defaults to US East 1 Region), go to the AWS Services Dashboard and select Storage Gateway (or click here which goes to US East 1). You will be presented with three options (File, Volume or VTL) modes.

What Does Storage Gateway and Install Look Like

The following is what installing a AWS Storage Gateway for file and then volume looks like. First, access the AWS Storage Gateway main landing page (it might change by time you read this) to get started. Scroll down and click on the Get Started with AWS Storage Gateway button or click here.

AWS Storage Gateway Landing Page

Select type of gateway to create, in the following example File is chosen.

Select type of AWS storage gateway

Next select the type of file gateway host (EC2 cloud hosted, or on-premises VMware). If you choose VMware, an OVA will be downloaded (follow the onscreen instructions) that you deploy on your ESXi system or with vCenter. Note that there is a different VMware VM gateway OAV for File Gateway and another for Volume Gateway. In the following example VMware ESXi OVA is selected and downloaded, then accessed via VMware tools such as vSphere Web Client for deployment.

AWS Storage Gateway select download

Once your VMware OVA file is downloaded from AWS, install using your preferred VMware tool, in this case I used the vSphere Web Client.

AWS Storage Gateway VM deploy

Once you have deployed the VMware VM for File Storage Gateway, it is time to connect to the gateway using the IP address assigned (static or DHCP) for the VM. Note that you may need to allocate some extra VMware storage to the VM if prompted (this mainly applies to Volume Gateway). Also follow directions about setting NTP time, using paravirtual adapters, thick vs. thin provisioning along with IP settings. Also double-check to make sure your VM and host are set for high-performance power setting. Note that the default username is sguser and password is sgpassword for the gateway.

AWS Storage Gateway Connect

Once you successfully connect to the gateway, next step will be to configure file share settings.

AWS Storage Gateway Configure File Share

Configure file share by selecting which gateway to use (in case you have more than one), name of an S3 bucket name to create, type of storage (S3 Standard or IA), along with Access Management security controls.

AWS Storage Gateway Create Share

Next step is to complete file share creation, not the commands provided for Linux and Windows for accessing the file share.

AWS Storage Gateway Review Share Settings

Review file share settings

AWS Storage Gateway access from Windows

Now lets use the file share by accessing and mounting to a Windows system, then copy some files to the file share.

AWS Storage Gateway verify Bucket Items

Now let’s go to the AWS console (or in our example use S3 Browser or your favorite tool) and look at the S3 bucket for the file share and see what is there. Note that each file is an object, and the objects simply appear as a file. If there were sub-directory those would also exist. Note that there are other buckets that I have masked out as we are only interested in the one named awsgwydemo that is configured using S3 Standard storage.

AWS Storage Gateway Volume

Now lets look at using the S3 Storage Gateway for Volumes. Similar to deploying for File Gateway, start out at the AWS Storage Gateway page and select Volume Gateway, then select what type of host (EC2 cloud, VMware or Hyper-V (2008 R2 or 2012) for on-premises deployment). Lets use the VMware Gateway, however as mentioned above, this is a different OVA/OVF than the File Gateway.

AWS Storage Gateway Configure Volume

Download the VMware OVA/OVF from AWS, and then install using your preferred VMware tools making sure to configure the gateway per instructions. Note that the Volume Gateway needs a couple of storage devices allocated to it. This means you will need to make sure that a SCSI adapter exists (or add one) to the VM, along with the disks (HDD or SSD) for local storage. Refer to AWS documentation about how to size, for my deployment I added a couple of small 80GB drives (you can choose to put on HDD or SSD including NVMe). Note that when connecting to the gateway if you get an error similar to below, make sure that you are in fact using the Volume Gateway and not mistakenly using the File Gateway OVA (VM). Note that the default username is sguser and password is sgpassword for the gateway.

AWS Storage Gateway Connect To Volume

Now connect to the local Volume Storage Gateway and notice the two local disks allocated to it.

AWS Storage Gateway Cached Volume Deploy

Next its time to create the Gateway which are deploying a Volume Cached below.

AWS Storage Gateway Volume Create

Next up is creating a volume, along with its security and access information.

AWS Storage Gateway Volume Settings

Volume configuration continued.

AWS Storage Gateway Volume CHAP

And now some additional configuration of the volume including iSCSI CHAP security.

AWS Storage Gateway Windows Access

Which leads us up to some Windows related volume access and configuration.

AWS Storage Gateway Using iSCSI Volume

Now lets use the new iSCSI based AWS Storage Gateway Volume. On the left you can see various WIndows command line activity, along with corresponding configuration information on the right.

AWS Storage Gateway Being Used by Windows

And there you have it, a quick tour of AWS Storage Gateway, granted there are more options that you can try yourself.

AWS

Where To Learn More

What This All Means

Overall I like the improvements that AWS has made to the Storage Gateway along with the different options it provides. Something to keep in mind is that if you are planning to use the AWS Storage Gateway File serving sharing mode that there are caveats to multiple concurrent writers to the same bucket. I would not be surprised if some other gateway or software based tool vendors tried to throw some fud towards the Storage Gateway, however ask them then how they coordinate multiple concurrent updates to a bucket while preserving data integrity.

Which Storage Gateway variant from AWS to use (e.g. File, Volume, VTL) depends on what your needs are, same with where the gateway is placed (Cloud hosted or on-premises with VMware or Hyper-V). Keep an eye on your costs, and more than just the storage space capacity. This means pay attention to your access and requests fees, as well as different service levels, along with data transfer fees.

You might wonder what about EFS and why you would want to use AWS Storage Gateway? Good question, at the time of this post EFS has evolved from being internal (e.g. within AWS and across regions) to having an external facing end-point however there is a catch. That catch which might have changed by time you read this is that the end-point can only be accessed from AWS Direct Connect locations.

This means that if your servers are not in a AWS Direct Connect location, without some creative configuration, EFS is not an option. Thus Storage Gateway File mode might be an option in place of EFS as well as using AWS storage access tools from others. For example I have some of my S3 buckets mounted on Linux systems using S3FS for doing rsync or other operations from local to cloud. In addition to S3FS, I also have various backup tools that place data into S3 buckets for backup, BC and DR as well as archiving.

Check out AWS Storage Gateway yourself and see what it can do or if it is a fit for your environment.

Ok, nuff said (for now…).

Cheers
Gs

Greg Schulz – Multi-year Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Watch for the spring 2017 release of his new book "Software-Defined Data Infrastructure Essentials" (CRC Press).

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

Top vBlog 2017 Voting Now Open

server storage I/O trends

Top vBlog 2017 Voting Now Open

It is that time of the year again when Eric Siebert (@ericsiebert) over at vSphere-land holds his annual Top vBlog (e.g. VMware and Virtualization related) voting (vote here until June 30, 2017). The annual Top vBlog event enables fans to vote for their favorite blogs (to get them into the top 10, 25, 50 and 100) as well as rank them for different categories which appear on Eric’s vLaunchPad site.

This years Top vBlog voting is sponsored by TurboNomic (e.g. formerly known as VMturbo) who if you are not aware of, have some interesting technology for cross-platform (cloud, container, virtualization, hardware, software, services) data infrastructure management software tools.

Software Defined Data Infrastructure Management

The blogs and sites listed on Eric’s site have common theme linkage to Virtualization and in particular tend to be more VMware focused, however some are also hybrid agnostic spanning other technologies, vendors, services and tools. Some examples of the different focus areas include hypervisors, VDI, cloud, containers, management tools, scripting, networking, servers, storage, data protection including backup/restore, replication, BC, DR among others).

In addition to the main list of blogs (that are active), there are also sub lists for different categories including:

  • Top 100 (Also top 10, 25, 50) vBlogs
  • Archive of retired (e.g. not active or seldom post)
  • News and Information sites
  • Podcasts
  • Scripting Blogs
  • Storage related
  • Various Virtualization Blogs
  • VMware Corporate Blogs

What To Do

Get out and vote for your favorite (or blogs that you frequent) in appreciation to those who create virtualization, VMware and data infrastructure related content. Click here or on the image above to reach the voting survey site where you will find more information and rules. In summary, select 12 of your favorite or preferred blogs, then rank them from 1 (most favorite) to 12. Then select your favorites for other categories such as Female Blog, Independent, New Blog, News websites, Podcast, Scripting and Storage among others.

Note: You will find my StorageIOblog in the main category (e.g. where you select 12 and then rank), as well as in the Storage, Independent, as well as Podcast categories, and thank you in advance for your continued support.

Which Blogs Do I Recommend (Among Others)

Two of my favorite blogs (and authors) are not included as Duncan Epping (Yellow Bricks) former #1 and Frank Denneman former #4 chose not to take part this year opening the door for some others to move up into the top 10 (or 25, 50 and 100). Of those listed some of my blogs I find valuable include Cormac Hogan of VMware, Demitasse (Alastair Cooke), ESX Virtualization (Vladan Seget), Kendrick Coleman, NTPro.nl (Eric Sloof), Planet VM (Tom Howarth), Virtually Ghetto (William Lam), VM Blog (David Marshall), vsphere-land.com (Eric Siebert) and Wahl Networks (Chris Wahl) among others.

Where to learn more

What this all means

It’s that time of the year again to take a few moments and show some appreciation for your favorite or preferred blogs along with their authors who spend time to create content for those sites. Also check out Turbonomic as they are an interesting technology that I have kept an eye on for some time now and so should you. Thank you all in advance regardless of if you take part in the voting as I also appreciate your continued support by viewing these posts either at StorageIOblog.com site or one of the many downstream sites where you can also read the content.

Ok, nuff said (for now…).

Cheers
Gs

Greg Schulz – Multi-year Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Watch for the spring 2017 release of his new book "Software-Defined Data Infrastructure Essentials" (CRC Press).

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.