Category: IT Industry Activity

Posted on

EMCworld 2015 How Do You Want Your Storage Wrapped?

Server Storage I/O trends

EMCworld 2015 How Do You Want Your Storage Wrapped?

Back in early May I was invited by EMC to attend EMCworld 2015 which included both the public sessions, as well as several NDA based discussions. Keep in mind that there is the known, there is the unknown (or assumed or speculated) and in between there are NDA’s, nuff said on that. EMC covered my hotel and registration costs to attend the event in Las Vegas (thanks EMC, that’s a disclosure btw ;) and here is a synopsis of various EMCworld 2015 announcements.

What EMC announced

  • VMAX3 enhancements to the EMC enterprise flagship storage platform to keep it relevant for traditional legacy workloads as well as for in a converged, scale-out, cloud, virtual and software defined environment.
  • VNX 3200 entry-level All Flash Array (AFA) flash SSD system starting at $25,000 USD for a 3TB unified platform with full data services found in other VNX products.
  • vVNX aka Virtual VNX aka "project liberty" which is a community (e.g. free) software version of the VNX. vVNX is a Virtual Storage Appliance (VSA) that you download and run on a VMware platform. Learn more and download here. Note the install will do a CPU type check so forget about trying to run it on a Intel Nuc or similar, I tried just because I could, the install will protect you from doing such things.
  • Various data protection related items including new Datadomain platforms as well as software updates and integration with other EMC platforms (storage systems).
  • All Flash Array (AFA) XtremIO 4.0 enhancements including larger clusters, larger nodes to boost performance, capacity and availability, along with copy service updates among others improvements.
  • Preview of DSSD shared (inside a rack) external flash Solid State Device (SSD) including more details. While much of DSSD is still under NDA, EMC did provide more public details at EMCworld. Between what was displayed and announced publicly at EMCworld as well as what can be found via Google (or other searches) you can piece together more of the DSSD story. What is known publicly today is that DSSD leverages the new Non-Volatile Memory express (NVMe) access protocol built upon underlying PCIe technology. More on DSSD in future discussions,if you have not done so, get an NDA deep dive briefing on it from EMC.
  • ScaleIO is now available via a free download here including both Windows and Linux clients as well as instructions for those operating systems as well as VMware.
  • ViPR can also be downloaded here for free (has been previously available) from here as well as it has been placed into open source by EMC.

What EMC announced since EMCworld 2015

  • Acquisition of cloud services (and software tools) vendor Virtustream for $1.2B adding to the federation cloud services portfolio (companion to VMware vCloud Air).
  • Release of ECS 2.0 including a free download here. This new version of ECS (Elastic Cloud Storage) can be used independent of the ViPR controller, or in conjunction with ViPR. In addition ECS now has about 80% of the functionality of the Centera object storage platform. The remaining 20% functionality (mainly regulatory compliance governance) of Centera will be added to ECS in the future providing a migration path for Centera customers. In case you are wondering what does EMC do with Centera, Atmos, ViPR and now ECS, answer is that ECS can work with or without ViPR, second is that the functionality of Centera, Atmos are being rolled into ECS. ECS as a refresher is software that transforms general purpose industry standard servers with direct storage into a scale-out HDFS and object storage solution.
  • Check out EMCcode including S3motion that I use and have reviewed here. Also check out EMCcode Rex-Ray which if you are into docker containers, it should be of interest, I know I’m interested in it.

Server Storage I/O trends

What this all means and wrap-up

There were no single major explosive announcements however the sum of all the announcements together should not be over shadowed by the big tent made for TV (or web) big tent productions and entertainment. What EMC announced was effectively how would you like, how do you want and need your storage and associated data services along with management wrapped.

tin wrapped software

By being wrapped, do you want your software defined storage management and storage wrapped in a legacy turnkey solution such as VMAX3, VNX or Isilon, do you want or need it to be hybrid or all flash, converged and unified, block, file or object.

software wrapped storage

Or do you need or want the software defined storage management and storage to be "shrink wrapped" as a download so you can deploy on your own hardware "tin wrapped" or as a VSA "virtual wrapped" or cloud wrapped? Do you need or want the software defined storage management and storage to leverage anybody’s hardware while being open source?

server storage software wrapping

How do you need or want your storage to be wrapped to fit your specific needs, that IMHO was the essence of what EMC announced at EMCworld 2015, granted the motorcycles and other production entertainment was engaging as well as educational.

Ok, nuff said for now

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

Posted on

VMware vCloud Air Server StorageIOlab Test Drive with videos

Server Storage I/O trends

VMware vCloud Air Server StorageIOlab Test Drive with videos

Recently I was invited by VMware vCloud Air to do a free hands-on test drive of their actual production environment. Some of you may already being using VMware vSphere, vRealize and other software defined data center (SDDC) aka Virtual Server Infrastructure (VSI) or Virtual Desktop Infrastructure (VDI) tools among others. Likewise some of you may already be using one of the many cloud compute or Infrastructure as a Service (IaaS) such as Amazon Web Services (AWS) Elastic Cloud Compute (EC2), Centurylink, Google Cloud, IBM Softlayer, Microsoft Azure, Rackspace or Virtustream (being bought by EMC) among many others.

VMware vCloud Air provides a platform similar to those just mentioned among others for your applications and their underlying resource needs (compute, memory, storage, networking) to be fulfilled. In addition, it should not be a surprise that VMware vCloud Air shares many common themes, philosophies and user experiences with the traditional on-premises based VMware solutions you may be familiar with.

VMware vCloud Air overview

You can give VMware vCloud Air a trial for free while the offer lasts by clicking here (service details here). Basically if you click on the link and register a new account for using VMware vCloud Air they will give you up to $500 USD in service credits to use in the real production environment while the offer lasts which iirc is through end of June 2015.

Server StorageIO test drive VMware vCloud Air video I
Click on above image to view video part I

Server StorageIO test drive VMware vCloud Air part II
Click on above image to view video part II

What this means is that you can go and setup some servers with as many CPUs or cores, memory, Hard Disk Drive (HDD) or flash Solid State Devices (SSD) storage, external IP networks using various operating systems (Centos, Ubuntu, Windows 2008, 20012, 20012 R2) for free, or until you use up the service credits.

Speaking of which, let me give you a bit of a tip or hint, even though you can get free time, if you provision a fast server with lots of fast SSD storage and leave it sit idle over night or over a weekend, you will chew up your free credits rather fast. So the tip which should be common sense is if you are going to do some proof of concepts and then leave things alone for a while, power the virtual cloud servers off to stretch your credits further. On the other hand, if you have something that you want to run on a fast server with fast storage over a weekend or longer, give that a try, just pay attention to your resource usage and possible charges should you exhaust your service credits.

My Server StorageIO test drive mission objective

For my test drive, I created a new account by using the above link to get the service credits. Note that you can use your regular VMware account with vCloud Air, however you wont get the free service credits. So while it is a few minutes of extra work, the benefit was worth it vs. simply using my existing VMware account and racking up more cloud services charges on my credit card. As part of this Server StorageIOlab test drive, I created two companion videos part I here and part II here that you can view to follow along and get a better idea of how vCloud works.

VMware vCloud Air overview
Phase one, create the virtual data center, database server, client servers and first setup

My goal was to set up a simple Virtual Data Center (VDC) that would consist of five Windows 2012 R2 servers, one would be a MySQL database server with the other four being client application servers. You can download MySQL from here at Oracle as well as via other sources. For applications to simplify things I used Hammerdb as well as Benchmark Factory that is part of the Quest Toad tool set for database admins. You can download a free trial copy of Benchmark Factory here, and HammerDB here. Another tool that I used for monitoring the servers is Spotlight on Windows (SoW) which is also free here. Speaking of tools, here is a link to various server and storage I/O performance as well as monitoring tools.

Links to tools that I used for this test-drive included:

Setting up a virtual data center vdc
Phase one steps and activity summary

Summary of phase one of vdc
Recap of what was done in phase one, watch the associated video here.

After the initial setup (e.g. part I video here), the next step was to add some more virtual machines and take a closer look at the environment. Note that most of the work in setting up this environment was Windows, MySQL, Hammerdb, Benchmark Factory, Spotlight on Windows along with other common tools so their installation is not a focus in these videos or this post, perhaps a future post will dig into those in more depth.

Summary of phase two of the vdc
What was done during phase II (view the video here)

VMware vCloud Air vdc trest drive

There is much more to VMware vCloud Air and on their main site there are many useful links including overviews, how-too tutorials, product and service offering details and much more here. Besides paying attention to your resource usage and avoid being surprised by service charges, two other tips I can pass along that are also mentioned in the videos (here and here) is to pay attention what region you setup your virtual data centers in, second is have your network thought out ahead of time to streamline setting up the NAT and firewall as well as gateway configurations.

Where to learn more

Learn more about data protection and related topics, themes, trends, tools and technologies via the following links:

Server Storage I/O trends

What this all means and wrap-up

Overall I like the VMware vCloud Air service which if you are VMware centric focused will be a familiar cloud option including integration with vCloud Director and other tools you may already have in your environment. Even if you are not familiar with VMware vSphere and associated vRealize tools, the vCloud service is intuitive enough that you can be productive fairly quickly. On one hand vCloud Air does not have the extensive menu of service offerings to choose from such as with AWS, Google, Azure or others, however that also means a simpler menu of options to choose from and simplify things.

I had wanted to spend some time actually using vCloud and the offer to use some free service credits in the production environment made it worth making the time to actually setup some workloads and do some testing. Even if you are not a VMware focused environment, I would recommend giving VMware vCloud Air a test drive to see what it can do for you, as opposed to what you can do for it…

Ok, nuff said for now

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

Posted on

Modernizing Data Protection = Using new and old things in new ways

Server Storage I/O trends

Modernizing Data Protection = Using new and old things in new ways

This is part of an ongoing series of posts that part of www.storageioblog.com/data-protection-diaries-main/ on data protection including archiving, backup/restore, business continuance (BC), business resiliency (BC), data footprint reduction (DFR), disaster recovery (DR), High Availability (HA) along with related themes, tools, technologies, techniques, trends and strategies.

data protection trends

Keep in mind that a fundamental goal of an Information Technology (IT) organization is to protect, preserve and serve data and information in a cost-effective as well as productive way when needed. There is no such thing as an information recession with more data being generated and processed. In addition to more of it, data is also getting larger, having more dependencies on it being available as well as living longer (e.g. retention).

Proof Points, No Data or Information Recession

A quick easy proof point of more data and it getting larger is your cell phone and the pictures it take. Compare the size of those photos today to what you had in your previous generation of smart phone or even digital camera as the Mega Pixels (e.g. resolution and size of data) increased, along with the size of media (e.g. storage) to save those to also grew. Another proof point is look at your presentations, documents, web sites and other mediums with how the amount of rich or unstructured content (e.g. photos, videos) exists on those now vs. a few years ago. Yet another proof-point is to look at your structured little data databases and how there are more rows and columns, as well as how some of those columns have gotten larger or are point to external "blobs" or "objects" that have also gotten larger.

Industry trend and challenges

There has been industry buzz the past several years around data protection modernizing, modernizing data protection or simply modernizing backup along with modernizing your data and information infrastructure. Many of these conversations focus around swapping out an older technology in favor of whatever the new industry buzzword trend is (e.g. swap tape for disk, disk for cloud) or perhaps from one data protection, backup, archive or copy tool for another. Some of these conversations also focus around swapping legacy for virtual, cloud or some other variation of software defined marketing.

Data protection strategy

The Opportunity to do new things

What is common with all the above is basically swapping out one technology, tool, medium or technique for another new one yet using it in old ways. For example tape gets swapped for disk, yet the same approach to when, where, why, how often and what gets copied or protected is left the same. Sure some new tools and technologies get introduced.  However when was the last time you put the tools down, took a step back and revisited the fundamental questions of how and why you are doing data protection the way it is being done? When was the last time you thought about data protection as an asset or business enabler as opposed to a cost center, overhead or after thought?

Data protection tool box
What’s in your data protection toolbox, do you know what to use when?

What about modernizing beyond the tools

One of the challenges with modernizing is that there is a cost involved including people time, staff skills as well as budgets not to mention keeping things running, so how do you go about paying for any improvements? Sure you can go get a data infrastructure or habitat for technology aka data home improvement loan, however there are costs associated to that.

Big data garbage in = big data garbage out

What about reducing data protection costs?

So why not self-fund the improvements and modernization activities by finding and removing costs, eliminating complexity vs. moving and masking issues? Part of this can be accomplished by simply revisiting if you are treating all your applications and data the same from a data protection perspective. Are you providing a data protection service ability to your organization that is based on business wants or business needs? For example, does the business want recovery time objective (RTO) 0 and recovery point objective (RPO) 0 for all applications, while it needs RTO 4 hours and RPO 15 minutes for application-a while application-b requires RTO 12 hours and RPO of 2 hours and application must have RTO 24 hours with RPO of 12 hours?

As a reminder RTO is how much time, or how quickly you need your applications and data to be restored and made ready for use. RPO is the point in time to where data needs to be protected as of, or the amount of data or time frame data could be lost or missing. Thus RTO = 0 means instant recovery no downtime and RPO = 0 means no loss of data. RTO one day and RPO of ten (10) minutes means applications and their data are ready for use within 24 hours and no more than 10 minutes of data can be lost (e.g. the granularity of protection coverage)., Also keep in mind that you can have various RTO and RPO combinations to meet your specific application along with business needs as part of a tiered data protection strategy implementation.

With RTO and RPO in mind, when was the last time you sat down with the business and applications people to revisit what they want vs. what they must have? From these conversation you can easily Transition into how long to keep, how many copies in what place among other things which in turn allows you to review data protection as well as start using both old and new technologies, tools and techniques in new ways.

Where to learn more

Learn more about data protection and related topics, themes, trends, tools and technologies via the following links:

Server Storage I/O trends

What this all means and wrap-up

Data protection is a broad topic that spans from logical and physical security to HA, BC, BR, DR, archiving (including life beyond compliance) along with various tools, technologies, techniques. Key is aligning those to the needs of the business or organization for today’s as well as tomorrows requirements. Instead of doing things what has been done in the past that may have been based on what was known or possible due to technology capabilities, why not start using new and old things in new ways. Let’s start using all the tools in the data protection toolbox regardless of if they are new or old, cloud, virtual, physical, software defined product or service in new ways while keeping the requirements of the business in focus.

Keeping with the theme of protect preserve and serve, data protection to be modernized needs to become and be seen as a business asset or enabler vs. an after thought or cost over-head topic. Also, keep in mind that only you can prevent data loss, are your restores ready for when you need them? as well as one of the fundamental goals of IT is to protect, preserve and serve information including its applications as well as data when, where needed in a cost-effective way.

What say you?

Ok, nuff said for now

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

Posted on

April 2015 Server StorageIO Update Newsletter

Volume 15, Issue IV

Hello and welcome to this April 2015 Server and StorageIO update newsletter.

This months newsletter has a focus on cloud and object storage for bulk data, unstructured data, big data, archiving among other scenarios.

Enjoy this edition of the Server and StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and Podcasts along with in the news commentary appearing soon.

Storage I/O trends

StorageIOblog posts

April StorageIOblog posts include:

View other recent as well as past blog posts here

April Newsletter Feature Theme
Cloud and Object Storage Fundamentals

There are many facets to object storage including technology implementation, products, services, access and architectures for various applications and use scenarios. The following is a short synopsis of some basic terms and concepts associated with cloud and object storage.

Common cloud and object storage terms

  • Account or project – Top of the hierarchy that represent owner or billing information for a service that where buckets are also attached.
  • Availability Zone (AZ) can be rack of servers and storage or data center where data is spread across for storage and durability.

    • AWS regions and availability zones (AZ)
    Example of some AWS Regions and AZ’s

  • Bucket or Container – Where objects or sub-folders containing objects are attached and accessed. Note in some environments such as AWS S3 you can have sub-folders in a bucket.
  • Connector or how your applications access the cloud or object storage such as via an API, S3, Swift, Rest, CDMI, Torrent, JSON, NAS file, block of other access gateway or software.
  • Durability – Data dispersed with copies in multiple locations to survive failure of storage or server hardware, software, zone or even region. Availability = Access + Durability.
  • End-point – Where or what your software, application or tool and utilities or gateways attach to for accessing buckets and objects.
  • Ephemeral – Temporary or non-persistent
  • Eventual consistency – Data is eventually made consistency, think in terms of asynchronous or deferred writes where there is a time lag vs. synchronous or real-time updates.
  • Immutable – Persistent, non-altered or write once read many copy of data. Objects generally are not updated, rather new objects created.

    • Object storage and cloud
    Via Cloud Virtual Data Storage (CRC)

  • Object – Byte (or bit) stream that can be as small as one byte to as large as several TBytes (some solutions and services support up to 5TByte sized objects). The object contains what ever data in any organization along with meta data. Different solutions and services support from a couple hundred KBytes of meta-data to MBytes worth of meta-data. In terms of what can be stored in an object, anything from files, videos, images, virtual disks (VMDK’s, VHDX), ZIP or tar files, backup and archive save sets, executable images or ISO’s, anything you want.
  • OPS – Objects per second or how many objects accessed similar to a IOP. Access includes gets, puts, list, head, deletes for a CRUD interface e.g. Created, Read, Update, Delete.
  • Region – Location where data is stored that can include one or more data centers also known as Availability Zones.
  • Sub-folder – While object storage can be accessed in a flat name space for commonality and organization some solutions and service support the notion of sub-folder that resemble traditional directory hierarchy.

Learn more in Cloud Virtual Storage Networking (CRC) and www.objectstoragecenter.com

Storage I/O trends

OpenStack Manila (e.g. Folders and Files)

AWS recently announced their new cloud based Elastic File Storage (EFS) to compliment their existing Elastic Block Storage (EBS) offerings. However are you aware of what is going on with cloud files within OpenStack?

For those who are familiar with OpenStack or simply talk about it and Swift object storage, or perhaps Cinder block storage, are you aware that there is also a file (NAS or Network Attached Storage) component called Manila?

In concept Manila should provide a similar capability to what AWS has recently announce with their Elastic File Service (EFS), or depending on your perspective, perhaps the other way around. If you are familiar and have done anything with Manila what are your initial thoughts and perspectives.

What this all means

People routinely tell me this is the most exciting and interesting times ever in servers, storage, I/O networking, hardware, software, backup or data protection, performance, cloud and virtual or take your pick too which I would not disagree.

However, for the past several years (no, make that decade), there is new and more interesting things including in adjacent areas.

I predict that at least for the next few years (no, make that decades), we will continue to see plenty of new and interesting things, questions include.

However, what’s applicable to you and your environment vs. simply fun and interesting to watch?

Ok, nuff said, for now

Cheers gs

 

In This Issue

  • Industry Trends Perspectives News
  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts
  • Events and Webinars
  • StorageIOblog posts
  • Server StorageIO Lab reports
  • Resources and Links

    • Industry News and Activity

    Recent Industry news and activity

    View other recent industry activity here

    StorageIO Commentary in the news

    StorageIO news (image licensed for use from Shutterstock by StorageIO)
    Recent Server StorageIO commentary and industry trends perspectives about news, activities and announcements.

    CyberTrend: Comments on Software Defined Data Center and Virtualization

    View more trends comments here

    StorageIO Tips and Articles

    Check out these resources and links on server storage I/O performance and benchmarking tools. View more tips and articles here

    Various Industry Events

    EMCworld – May 4-6 2015 (Las Vegas)

    Interop – April 29 2015 (Las Vegas)
    Presenting
    Smart Shopping for Your Enterprise Storage Strategy

    View other recent and upcoming events here

    Webinars


    BrightTalk Webinar – June 23 2015
    Server Storage I/O Innovation Update

    View other webinars here

    Videos and Podcasts

    Data Protection Gumbo Podcast
    Protect Preserve and Serve Data

    In this episode, Greg Schulz is a guest on Data Protection Gumbo hosted by Demetrius Malbrough(@dmalbrough). The conversation covers various aspects of data protection which has a focus of protect preserve and serve information, applications and data across different environments and customer segments.

    While we discuss enterprise and SMB data protection, we also talk about trends from Mobile to the cloud among many others tools, technologies and techniques. Check out the podcast here.

    Springtime in Kentucky
    With Kendrick Coleman of EMCcode
    Cloud Object Storage S3motion and more

    In this episode, @EMCcode (Part of EMC) developer advocate Kendrick Coleman (@KendrickColeman) joins me (e.g. Greg Schulz) for a conversation.

    Conversation covers what is EMCcode, EMC Federation, Cloud Foundry, clouds, object storage, buckets, containers, objects, node.js, Docker, OpenStack, AWS S3, micro services, and the S3motion tool Kendrick developed.

    S3motion is a good tool to have in your server storage I/O tool box for working with cloud and object storage along with others such as Cloudberry, S3fs, Cyberduck, S3 browser among many others. You can get S3motion for free from git hub here Check out the companion blog post for this podcast here.

    StorageIO podcast’s are also available via Server Storage I/O audio podcastServer Storage I/O video & at StorageIO.tv

    From StorageIO Labs

    Research, Reviews and Reports

    AWS S3 Cross-Region Replication

    AWS S3 Cross region replication
    Moving and Replicating Buckets/Containers, Sub folders and Objects (Click on Image to read about AWS Cross-Region Replication)

    View other StorageIO lab review reports here

    Resources and Links

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Data Protection Gumbo = Protect Preserve and Serve Information

    Storage I/O trends

    Data Protection Gumbo = Protect Preserve and Serve Information

    Recently I was invited to be a guest on the podcast Data Protection Gumbo hosted by Demetrius Malbrough (@dmalbrough).

    Data Protection Gumbo Podcast Description
    Data Protection Gumbo is set up with the aim of expanding the awareness of anyone responsible for protecting mission critical data, by providing them with a mix of the latest news, data protection technologies, and interesting facts on topics in the Data Backup and Recovery industry.

    Data Protection Gumbo Also available on

    Protect Preserve and Serve Applications, Information and Data

    Keep in mind that a fundamental role of Information Technology (IT) is to protect, preserve and serve business or organizations information assets including applications, configuration settings and data for use when or where needed.

    Our conversation covers various aspects of data protection which has a focus of protect preserve and serve information, applications and data across different environments and customer segments. While we discuss enterprise and small medium business (SMB) data protection, we also talk about trends from Mobile to the cloud among many others tools, technologies and techniques.

    Where to learn more

    Learn more about data protection and related trends, tools and technologies via the following links:

    Data Protection Gumbo Also available on

    What this all means and wrap-up

    Data protection is a broad topic that spans from logical and physical security to high availability (HA), disaster recovery (DR), business continuance (BC), business resiliency (BR), archiving (including life beyond compliance) along with various tools, technologies, techniques. Keeping with the theme of protect preserve and serve, data protection to be modernized needs to become and be seen as a business asset or enabler vs. an after thought or cost over-head topic. Also, keep in mind that only you can prevent data loss, are your restores ready for when you need them?

    Check out Demetrius Data Protection Gumbo podcast, also check out his Linkedin Backup & Recovery Professionals group. Speaking of data protection, check out the www.storageioblog.com/data-protection-diaries-main/ page for more coverage of backup/restore, HA, BC, DR, archiving and restated themes.

    Ok, nuff said, for now..

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    S3motion Buckets Containers Objects AWS S3 Cloud and EMCcode

    Storage I/O trends

    S3motion Buckets Containers Objects AWS S3 Cloud and EMCcode

    It’s springtime in Kentucky and recently I had the opportunity to have a conversation with Kendrick Coleman to talk about S3motion, Buckets, Containers, Objects, AWS S3, Cloud and Object Storage, node.js, EMCcode and open source among other related topics which are available in a podcast here, or video here and available at StorageIO.tv.

    In this Server StorageIO industry trends perspective podcast episode, @EMCcode (Part of EMC) developer advocate Kendrick Coleman (@KendrickColeman) joins me for a conversation. Our conversation spans spring-time in Kentucky (where Kendrick lives) which means Bourbon and horse racing as well as his blog (www.kendrickcoleman.com).

    Btw, in the podcast I refer to Captain Obvious and Kendrick’s beard, for those not familiar with who or what @Captainobvious is that is made reference to, click here to learn more.


    @Kendrickcoleman     & @Captainobvious

    What about Clouds Object Storage Programming and other technical stuff?

    Of course we also talk some tech including what is EMCcode, EMC Federation, Cloud Foundry, clouds, object storage, buckets, containers, objects, node.js, Docker, Openstack, AWS S3, micro services, and the S3motion tool that Kendrick developed.

    Cloud and Object Storage Access
    Click to view video

    Kendrick explains the motivation behind S3motion along with trends in and around objects (including GET, PUT vs. traditional Read, Write) as well as programming among related topic themes and how context matters.

    S3motion for AWS S3 Google and object storage
    Click to listen to podcast

    I have used S3motion for moving buckets, containers and objects around including between AWS S3, Google Cloud Storage (GCS) and Microsoft Azure as well as to/from local. S3motion is a good tool to have in your server storage I/O tool box for working with cloud and object storage along with others such as Cloudberry, S3fs, Cyberduck, S3 browser among many others.

    You can get S3motion free from git hub here.

    Amazon Web Services AWS

    Where to learn more

    Here are some links to learn more about AWS S3, Cloud and Object Storage along with related topics

    Also available on

    What this all means and wrap-up

    Context matters when it comes to many things particular about objects as they can mean different things. Tools such as S3motion make it easy for moving your buckets or containers along with objects from one cloud storage system, solution or service to another. Also check out EMCcode to see what they are doing on different fronts from supporting new and greenfield development with Cloud Foundry and PaaS to Openstack to bridging current environments to the next generation of platforms. Also check out Kendricks blog site as he has a lot of good technical content as well as some other fun stuff to learn about. Look forward to having Kendrick on as a guest again soon to continue our conversations. In the meantime, check out S3motion to see how it can fit into your server storage I/O tool box.

    Ok, nuff said, for now..

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Cloud Conversations: AWS EFS Elastic File System (Cloud NAS) First Preview Look

    Storage I/O trends

    Cloud Conversations: AWS EFS Elastic File System (Cloud NAS) First Preview Look

    Amazon Web Services (AWS) recently announced (preview) new Elastic File System (EFS) providing Network File System (NFS) NAS (Network Attached Storage) capabilities for AWS Elastic Cloud Compute (EC2) instances. EFS AWS compliments other AWS storage offerings including Simple Storage Service (S3) along with Elastic Block Storage (EBS), Glacier and Relational Data Services (RDS) among others.

    Ok, that’s a lot of buzzwords and acronyms so lets break this down a bit.

    Amazon Web Services AWS

    AWS EFS and Cloud Storage, Beyond Buzzword Bingo

    • EC2 – Instances exist in various Availability Zones (AZ’s) in different AWS Regions. Compute instance with various operating systems including Windows and Ubuntu among others that also can be pre-configured with applications such as SQL Server or web services among others. EC2 instances vary from low-cost to high-performance compute, memory, GPU, storage or general purposed optimized. For example, some EC2 instances rely solely on EBS, S3, RDS or other AWS storage offerings while others include on-board Solid State Disk (SSD) like DAS SSD found on traditional servers. EC2 instances on EBS volumes can be snapshot to S3 storage which in turn can be replicated to another region.
    • EBS – Scalable block accessible storage for EC2 instances that can be configured for performance or bulk storage, as well as for persistent images for EC2 instances (if you choose to configure your instance to be persistent)
    • EFS – New file (aka NAS) accessible storage service accessible from EC2 instances in various AZ’s in a given AWS region
    • Glacier – Cloud based near-line (or by some comparisons off-line) cold-storage archives.
    • RDS – Relational Database Services for SQL and other data repositories
    • S3 – Provides durable, scalable low-cost bulk (aka object) storage accessible from inside AWS as well as via externally. S3 can be used by EC2 instances for bulk durable storage as well as being used as a target for EBS snapshots.
    • Learn more about EC2, EBS, S3, Glacier, Regions, AZ’s and other AWS topics in this primer here

    aws regions architecture

    What is EFS

    Implements NFS V4 (SNIA NFS V4 primer) providing network attached storage (NAS) meaning data sharing. AWS is indicating initial pricing for EFS at $0.30 per GByte per month. EFS is designed for storage and data sharing from multiple EC2 instances in different AZ’s in the same AWS region with scalability into the PBs.

    What EFS is not

    Currently it seems that EFS has an end-point inside AWS accessible via an EC2 instance like EBS. This appears to be like EBS where the storage service is accessible only to AWS EC2 instances unlike S3 which can be accessible from the out-side world as well as via EC2 instances.

    Note however, that depending on how you configure your EC2 instance with different software, as well as configure a Virtual Private Cloud (VPC) and other settings, it is possible to have an application, software tool or operating system running on EC2 accessible from the outside world. For example, NAS software such as those from SoftNAS and NetApp among many others can be installed on an EC2 instance and with proper configuration, as well as being accessible to other EC2 instances, they can also be accessed from outside of AWS (with proper settings and security).

    AWS EFS at this time is NFS version 4 based however does not support Windows SMB/CIFS, HDFS or other NAS access protocols. In addition AWS EFS is accessible from multiple AZ’s within a region. To share NAS data across regions some other software would be required.

    EFS is not yet as of this writing released and AWS is currently accepting requests to join the EFS preview here.

    Amazon Web Services AWS

    Where to learn more

    Here are some links to learn more about AWS S3 and related topics

    What this all means and wrap-up

    AWS continues to extend its cloud platform include both compute and storage offerings. EFS compliments EBS along with S3, Glacier and RDS. For many environments NFS support will be welcome while for others CIFS/SMB would be appreciated and others are starting to find that value in HDFS accessible NAS.

    Overall I like this announcement and look forward to moving beyond the preview.

    Ok, nuff said, for now..

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    March 2015 Server StorageIO Update Newsletter

     

     

    Volume 15, Issue III

    Hello and welcome to this March 2015 Server and StorageIO update newsletter. Here in the northern hemisphere at least by the calendar spring is here, weather wise winter continues to linger in some areas. March also means in the US college university sports tournaments with many focused on their NCAA men’s basketball championship brackets.

    Besides various college championships, March also has a connection to back up and data protection. Thus this months newsletter has a focus on data protection, after all March 31 is World Backup Day which means it should also be World Restore test day!

    Focus on Data Protection

    Data protection including backup/restore, business continuance (BC), disaster recovery (DR), business resiliency (BR) and archiving across physical, virtual and cloud environments.

    Data Protection Fundamentals

    A reminder on the importance of data protection including backup, BC, DR and related technologies is to make sure they are occuring as planned. Also test your copies and remember the 4 3 2 1 rule or guide.

    4 – Versions (different time intervals)
    3 – Copies of critical data (including versions)
    2 – Different media, devices or systems
    1 – Off-site (cloud or elsewhere)

    The above means having at least four (4) different versions from various points in time of your data. Having three (3) copies including various versions protects against one or more copies being corrupt or damaged. Placing those versions and copies on at least two (2) different storage systems, devices or media if something happens.

    While it might be common sense, a bad April Fools recovery joke would be finding out all of your copies were on the same device which is damaged. That might seem obvious however sometimes the obvious needs to be stated. Also make sure that at least one (1) of your copies is off-site either on off-line media (tape, disk, ssd, optical) or cloud.

    Take a few moments and to verify that your data protection strategy is being implemented and practiced as intended. Also test what is being copied including not only restore the data from cloud, disk, ssd or tape, also make sure you can actually read or use the data being protected. This means make sure that your security credentials including access certificates and decryption occur as expected.

    Watch for more news, updates industry trends perspectives commentary, tips, articles and other information at Storageio.com, StorageIOblog.com, various partner venues as well as in future newsletters.

    StorageIOblog posts

    Data Protection Diaries
    Are restores ready for World Backup Day?
    In case you forgot or did not know, World Backup Day is March 31 2015 (@worldbackupday) so now is a good time to be ready. The only challenge that I have with the World Backup Day (view their site here) that has gone on for a few years know is that it is a good way to call out the importance of backing up or protecting data.
    world backup day test your restore

    However it’s also time to put more emphasis and focus on being able to make sure those backups or protection copies actually work.

    By this I mean doing more than making sure that your data can be read from tape, disk, SSD or cloud service actually going a step further and verifying that restored data can actually be used (read, written, etc).

    The problem, issue and challenges are simple, are your applications, systems and data protected as well as can you use those protection copies (e.g. backups, snapshots, replicas or archives) when as well as were needed? Read more here about World Backup Day and what I’m doing as well as various tips to be ready for successful recovery and avoid being an April 1st fool ;).

    Cloud Conversations
    AWS S3 Cross Region Replication
    Amazon Web Services (AWS) announced several enhancements including a new Simple Storage Service (S3) cross-region replication of objects from a bucket (e.g. container) in one region to a bucket in another region.

    AWS also recently enhanced Elastic Block Storage (EBS) increasing maximum performance and size of Provisioned IOPS (SSD) and General Purpose (SSD) volumes. EBS enhancements included ability to store up to 16 TBytes of data in a single volume and do 20,000 input/output operations per second (IOPS). Read more about EBS and other AWS server, storage I/O  enhancements here.
    AWS regions and availability zones (AZ)
    Example of some AWS Regions and AZs

    AWS S3 buckets and objects are stored in a specific region designated by the customer or user (AWS S3, EBS, EC2, Glacier, Regions and Availability Zone primer can be found here). The challenge being addressed by AWS with S3 replication is being able to move data (e.g. objects) stored in AWS buckets in one region to another in a safe, secure, timely, automated, cost-effective way.

    Continue reading more here about AWS S3 bucket and object replication feature along with related material.

    Additional March StorageIOblog posts include:

    Server Storage I/O performance (Image licensed from Shutterstock by StorageIO)

     

     

    View other recent as well as past blog posts here

    In This Issue

    • Industry Trends Perspectives News
    • Commentary in the news
    • Tips and Articles
    • StorageIOblog posts
    • Events and Webinars
    • Recommended Reading List
    • StorageIOblog posts
    • Server StorageIO Lab reports
    • Resources and Links

     

    Industry News and Activity

    Recent Industry news and activity

    EMC sets up cloudfoundry Dojo
    AWS S3, EBS IOPs and other updates
    New backup/data protection vendor Rubrik
    Google adds nearline Cloud Storage
    AWS and Microsoft Cloud Price battle

    View other recent and upcoming events here

    StorageIO Commentary in the news

    StorageIO news (image licensed for use from Shutterstock by StorageIO)
    Recent Server StorageIO commentary and industry trends perspectives about news, activities and announcements.

    Processor: Enterprise Backup Solution Tips
    Processor: Failed & Old Drives
    EnterpriseStorageForum: Disk Buying Guide
    ChannelProNetwork: 2015 Tech and SSD
    Processor: Detect & Avoid Drive Failures

    View more trends comments here

    StorageIO Tips and Articles

    So you have a new storage device or system. How will you test or find its performance? Check out this quick-read tip on storage benchmark and testing fundamentals over at BizTech.

    Keeping with this months theme of data protection including backup/restore, BC, DR, BR and archiving, here are some more tips. These tips span server storage I/O networking hardware, software, cloud, virtual, performance, data protection applications and related themes including:

    • Test your data restores, can you read and actually use the data? Is you data decrypted, proper security certificates applied?
    • Remember to back up or protect your security encryption keys, certificates and application settings!
    • Revisit what format your data is being saved in including how will you be able to use data saved to the cloud. Will you be able to do a restore to a cloud server or do you need to make sure a copy of your backup tools are on your cloud server instances?

    Check out these resources and links on server storage I/O performance and benchmarking tools. View more tips and articles here

    Various Industry Events

    EMCworld – May 4-6 2015

    Interop – April 29 2015 (Las Vegas)

    Presenting Smart Shopping for Your Storage Strategy

    NAB – April 14-15 2015

    SNIA DSI Event – April 7-9

    View other recent and upcoming events here

    Webinars

    December 11, 2014 – BrightTalk
    Server & Storage I/O Performance

    December 10, 2014 – BrightTalk
    Server & Storage I/O Decision Making

    December 9, 2014 – BrightTalk
    Virtual Server and Storage Decision Making

    December 3, 2014 – BrightTalk
    Data Protection Modernization

    Videos and Podcasts

    StorageIO podcasts are also available via and at StorageIO.tv

    From StorageIO Labs

    Research, Reviews and Reports

    Datadynamics StorageX
    Datadynamics StorageX

    More than a data mover migration tool, StorageX is a tool for adding management and automation around unstructured local and distributed NAS (NFS, CIFS, DFS) file data. Read more here.

    View other StorageIO lab review reports here

    Recommended Reading List

    This is a new section being introduced in this edition of the Server StorageIO update mentioning various books, websites, blogs, articles, tips, tools, videos, podcasts along with other things I have found interesting and want to share with you.

      • Introducing s3motion (via EMCcode e.g. opensource) a tool for copying buckets and objects between public, private and hybrid clouds (e.g. AWS S3, GCS, Microsoft Azure and others) as well as object storage systems. This is a great tool which I have added to my server storage I/O cloud, virtual and physical toolbox. If you are not familiar with EMCcode check it out to learn more…
    • Running Hadoop on Ubuntu Linux (Series of tutorials) for those who want to get their hands dirty vs. using one of the All In One (AIO) appliances.
      • Yellow-bricks (Good blog focused on virtualization, VMware and other related themes) by Duncan Epping @duncanyb

    Resources and Links

    Check out these useful links and pages:
    storageio.com/links
    objectstoragecenter.com
    storageioblog.com/data-protection-diaries-main/
    storageperformance.us
    thessdplace.com
    storageio.com/raid
    storageio.com/ssd

    Enjoy this edition of the Server and StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and podcasts along with in the news commentary appearing soon.

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Cloud Conversations: AWS S3 Cross Region Replication storage enhancements

    Storage I/O trends

    Cloud Conversations: AWS S3 Cross Region Replication storage enhancements

    Amazon Web Services (AWS) recently among other enhancements announced new Simple Storage Service (S3) cross-region replication of objects from a bucket (e.g. container) in one region to a bucket in another region. AWS also recently enhanced Elastic Block Storage (EBS) increasing maximum performance and size of Provisioned IOPS (SSD) and General Purpose (SSD) volumes. EBS enhancements included ability to store up to 16 TBytes of data in a single volume and do 20,000 input/output operations per second (IOPS). Read more about EBS and other recent AWS server, storage I/O and application enhancements here.

    Amazon Web Services AWS

    The Problem, Issue, Challenge, Opportunity and Need

    The challenge is being able to move data (e.g. objects) stored in AWS buckets in one region to another in a safe, secure, timely, automated, cost-effective way.

    Even though AWS has a global name-space, buckets and their objects (e.g. files, data, videos, images, bit and byte streams) are stored in a specific region designated by the customer or user (AWS S3, EBS, EC2, Glacier, Regions and Availability Zone primer can be found here).

    aws regions architecture

    Understanding the challenge and designing a strategy

    The following diagram shows the challenge and how to copy or replicate objects in an S3 bucket in one region to a destination bucket in a different region. While objects can be copied or replicated without S3 cross-region replication, that involves essentially reading your objects pulling that data out via the internet and then writing to another place. The catch is that this can add extra costs, take time, consume network bandwidth and need extra tools (Cloudberry, Cyberduck, S3fuse, S3motion, S3browser, S3 tools (not AWS) and a long list of others).
    aws cross region replication

    What is AWS S3 Cross-region replication

    Highlights of AWS S3 Cross-region replication include:

    • AWS S3 Cross region replication is as its name implies, replication of S3 objects from a bucket in one region to a destination bucket in another region.
    • S3 replication of new objects added to an existing or new bucket (note new objects get replicated)
    • Policy based replication tied into S3 versioning and life-cycle rules
    • Quick and easy to set up for use in a matter of minutes via S3 dashboard or other interfaces
    • Keeps region to region data replication and movement within AWS networks (potential cost advantage)

    To activate, you simply enable versioning on a bucket, enable cross-region replication, indicate source bucket (or prefix of objects in bucket), specify destination region and target bucket name (or create one), then create or select an IAM (Identify Access Management) role and objects should be replicated.

    • Some AWS S3 cross-region replication things to keep in mind (e.g. considerations):
    • As with other forms of mirroring and replication if you add something on one side it gets replicated to other side
    • As with other forms of mirroring and replication if you deleted something from the other side it can be deleted on both (be careful and do some testing)
    • Keep costs in perspective as you still need to pay for your S3 storage at both locations as well as applicable internal data transfer and GET fees
    • Click here to see current AWS S3 fees for various regions

    S3 Cross-region replication and alternative approaches

    There are several regions around the world and up until today AWS customers could copy, sync or replicate S3 bucket contents between AWS regions manually (or via automation) using various tools such as Cloudberry, Cyberduck, S3browser and S3motion to name just a few as well as via various gateways and other technologies. Some of those tools and technologies are open-source or free, some are freemium and some are premium for a few that also vary by interface (some with GUI, others with CLI or APIs) including ability to mount an S3 bucket as a local network drive and use tools to sync or copy.

    However a catch with the above mentioned tools (among others) and approaches is that to replicate your data (e.g. objects in a bucket) can involve other AWS S3 fees. For example reading data (e.g. a GET which has a fee) from one AWS region and then copying out to the internet has fees. Likewise when copying data into another AWS S3 region (e.g. a PUT which are free) there is also the cost of storage at the destination.

    Storage I/O trends

    AWS S3 cross-region hands on experience (first look)

    For my first hands on (first look) experience with AWS cross-region replication today I enabled a bucket in the US Standard region (e.g. Northern Virginia) and created a new target destination bucket in the EU Ireland. Setup and configuration was very quick, literally just a few minutes with most of the time spent reading the text on the new AWS S3 dashboard properties configuration displays.

    I selected an existing test bucket to replicate and noticed that nothing had replicated over to the other bucket until I realized that new objects would be replicated. Once some new objects were added to the source bucket within a matter of moments (e.g. few minutes) they appeared across the pond in my EU Ireland bucket. When I deleted those replicated objects from my EU Ireland bucket and switched back to my view of the source bucket in the US, those new objects were already deleted from the source. Yes, just like regular mirroring or replication, pay attention to how you have things configured (e.g. synchronized vs. contribute vs. echo of changes etc.).

    While I was not able to do a solid quantifiable performance test, simply based on some quick copies and my network speed moving via S3 cross-region replication was faster than using something like s3motion with my server in the middle.

    It also appears from some initial testing today that a benefit of AWS S3 cross-region replication (besides being bundled and part of AWS) is that some fees to pull data out of AWS and transfer out via the internet can be avoided.

    Amazon Web Services AWS

    Where to learn more

    Here are some links to learn more about AWS S3 and related topics

    What this all means and wrap-up

    For those who are looking for a way to streamline replicating data (e.g. objects) from an AWS bucket in one region with a bucket in a different region you now have a new option. There are potential cost savings if that is your goal along with performance benefits in addition to using what ever might be working in your environment. Replicating objects provides a way of expanding your business continuance (BC), business resiliency (BR) and disaster recovery (DR) involving S3 across regions as well as a means for content cache or distribution among other possible uses.

    Overall, I like this ability for moving S3 objects within AWS, however I will continue to use other tools such as S3motion and s3sfs for moving data in and out of AWS as well as among other public cloud serves and local resources.

    Ok, nuff said, for now..

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Data Protection Diaries: Are your restores ready for World Backup Day 2015?

    Data Protection Diaries: Are your restores ready for World Backup Day 2015?

    This is part of an ongoing data protection diaries series of post about, well, cloud and data protection and what I’m doing pertaining to World Backup Day 2015 along with related topics.

    In case you forgot or did not know, World Backup Day is March 31 2015 (@worldbackupday) so now is a good time to be ready. The only challenge that I have with the World Backup Day (view their site here) that has gone on for a few years know is that it is a good way to call out the importance of backing up or protecting data. However its time to also put more emphasis and focus on being able to make sure those backups or protection copies actually work.

    By this I mean doing more than making sure that your data can be read from tape, disk, SSD or cloud service actually going a step further and verifying that restored data can actually be used (read, written, etc).

    The Problem, Issue, Challenge, Opportunity and Need

    The problem, issue and challenges are simple, are your applications, systems and data protected as well as can you use those protection copies (e.g. backups, snapshots, replicas or archives) when as well as were needed?

    storage I/O data protection

    The opportunity is simple, avoiding downtime or impact to your business or organization by being proactive.

    Understanding the challenge and designing a strategy

    The following is my preparation checklist for World Backup Data 2015 (e.g. March 31 2015) which includes what I need or want to protect, as well as some other things to be done including testing, verification, address (remediate or fix) known issues while identifying other areas for future enhancements. Thus perhaps like yours, data protection for my environment which includes physical, virtual along with cloud spanning servers to mobile devices is constantly evolving.

    collect TPM metrics from SQL Server with hammerdb
    My data protection preparation, checklist and to do list

    Finding a solution

    While I already have a strategy, plan and solution that encompasses different tools, technologies and techniques, they are also evolving. Part of the evolving is to improve while also exploring options to use new and old things in new ways as well as eat my down dog food or walk the talk vs. talk the talk. The following figure provides a representation of my environment that spans physical, virtual and clouds (more than one) and how different applications along with systems are protected against various threats or risks. Key is that not all applications and data are the same thus enabling them to be protected in different ways as well as over various intervals. Needless to say there is more to how, when, where and with what different applications and systems are protected in my environment than show, perhaps more on that in the future.

    server storageio and unlimitedio data protection
    Some of what my data protection involves for Server StorageIO

    Taking action

    What I’m doing is going through my checklist to verify and confirm the various items on the checklist as well as find areas for improvement which is actually an ongoing process.

    Do I find things that need to be corrected?

    Yup, in fact found something that while it was not a problem, identified a way to improve on a process that will once fully implemented enabler more flexibility both if a restoration is needed, as well as for general everyday use not to mention remove some complexity and cost.

    Speaking of lessons learned, check this out that ties into why you want 4 3 2 1 based data protection strategies.

    Storage I/O trends

    Where to learn more

    Here are some extra links to have a look at:

    Data Protection Diaries
    Cloud conversations: If focused on cost you might miss other cloud storage benefits
    5 Tips for Factoring Software into Disaster Recovery Plans
    Remote office backup, archiving and disaster recovery for networking pros
    Cloud conversations: Gaining cloud confidence from insights into AWS outages (Part II)
    Given outages, are you concerned with the security of the cloud?
    Data Archiving: Life Beyond Compliance
    My copies were corrupted: The 3-2-1 rule
    Take a 4-3-2-1 approach to backing up data
    Cloud and Virtual Data Storage Networks – Chapter 8 (CRC/Taylor and Francis)

    What this all means and wrap-up

    Be prepared, be proactive when it comes to data protection and business resiliency vs. simply relying reacting and recovering hoping that all will be ok (or works).

    Take a few minutes (or longer) and test your data protection including backup to make sure that you can:

    a) Verify that in fact they are working protecting applications and data in the way expected

    b) Restore data to an alternate place (verify functionality as well as prevent a problem)

    c) Actually use the data meaning it is decrypted, inflated (un-compressed, un-de duped) and security certificates along with ownership properties properly applied

    d) Look at different versions or generations of protection copies if you need to go back further in time

    e) Identify area of improvement or find and isolate problem issues in advance vs. finding out after the fact

    Time to get back to work checking and verifying things as well as attending to some other items.

    Ok, nuff said, for now…

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Cloud conversations: If focused on cost you might miss other cloud storage benefits

    Storage I/O trends

    Cloud conversations: If focused on cost you might miss other cloud storage benefits

    Drew Robb (@robbdrew) has a good piece (e.g. article) over at InfoStor titled Eight Ways to Avoid Cloud Storage Pricing Surprises that you can read here.

    Drew start’s his piece out with this nice analogy or story:

    Let’s begin with a cautionary tale about pricing: a friend hired a moving company as they quoted a very attractive price for a complex move. They lured her in with a low-ball price then added more and more “extras” to the point where their price ended up higher than many of the other bids she passed up. And to make matters worse, they are already two weeks late with delivery of the furniture and are saying it might take another two weeks.

    Drew extends his example in his piece to compare how some cloud providers may start with pricing as low as some amount only for the customer to be surprised when they did not do their homework to learn about the various fees.

    Note that most reputable cloud providers do not hide their fees even though there are myths that all cloud vendors have hidden fees, instead they list what those costs are on their sites. However that means the smart shopper or person procuring cloud services needs to go look for those fee’s and what they mean to avoid surprises. On the other hand if you can not find what extra fee’s would be along with what is or is not included in a cloud service price, to quote Jenny’s line in the movie Forest Gump, "…Run, Forest! Run!…".

    In Drew’s piece he mentions five general areas to keep an eye on pertaining cloud storage costs including:

    • Be Duly Diligent
    • Trace Out Application Interaction
    • Avoid Fixed Usage Rates
    • Beware Lowballing
    • Demand Enterprise Visibility

    Beware Lowballing

    In Drew’s piece, he includes a comment from myself shown below.

    Just as in the moving business, lowballing is alive and well in cloud pricing. Greg Schulz, an analyst with StorageIO Group, warned users to pay attention to services that have very low-cost per GByte/TByte yet have extra fees and charges for use, activity or place service caps. Compare those with other services that have higher base fees and attempt to price it based on your real storage and usage patterns.

    “Watch out for usage and activity fees with lower cost services where you may get charged for looking at or visiting your data, not to mention for when you actually need to use it,” said Schulz. “Also be aware of limits or caps on performance that may apply to a particular class of service.”

    As a follow-up to Drew’s good article, I put together the following thoughts that appeared earlier this year over at InfoStor titled Cloud storage: Is It All About Cost? that you can read here. In that article I start out with the basic question of:

    So what is your take on cloud storage, and in what context?

    Is cloud storage all about removing cost, cost cutting, free storage?

    Or perhaps even getting something else in addition to free storage?

    I routinely talk with different people from various backgrounds, environments from around the world, and the one consistency I hear when it comes to cloud services including storage is that there is no consistency.

    What I mean by this is that there are the cloud crowd cheerleaders who view or cheer for anything cloud related, some of them actually use the cloud vs. simply cheering.

    What does this have to do with cloud costs

    Simple, how do you know if cloud is cheaper or more expensive if you do not know your own costs?

    How do you know if cloud storage is available, reliable, durable if you do not have a handle on your environment?

    Are you making apples to oranges comparisons or simple trading or leveraging hype and fud for or against?

    Similar to regular storage, how you choose to use and configure on-site traditional storage for high-availability, performance, security among other best practices should be applied to cloud solutions. After all, only you can prevent cloud (or on premise) data loss, granted it is a shared responsibility. Shared responsibility means your service provider or system vendor needs to deliver quality robust solution that you can then take responsibility for configure to use with resiliency.

    For some of you perhaps cloud might be about lowering, reducing or cutting storage costs, perhaps even getting some other service(s) in addition to free storage.

    On the other hand, some of you might be

    Yet another class of cloud storage (e.g. AWS EBS) are those intended or optimized to be accessed from within a cloud via cloud servers or compute instances (e.g. AWS EC2 among others) vs. those that are optimized for both inside the cloud as well as outside the cloud access (e.g. AWS S3 or Glacier with costs shown here). I am using AWS examples; however, you could use Microsoft Azure (pricing shown here), Google (including their new Nearline service with costs shown here), Rackspace, (calculator here or other cloud files pricing here), HP Cloud (costs shown here), IBM Softlayer (object storage costs here) and many others.

    Not all types of cloud storage are the same, which is similar to traditional storage you may be using or have used in your environment in the past. For example, there is high-capacity low-cost storage, including magnetic tape for data protection, archiving of in-active data along with near-line hard disk drives (HDD). There are different types of HDDs, as well as fast solid-state devices (SSD) along with hybrid or SSHD storage used for different purposes. This is where some would say the topic of cloud storage is highly complex.

    Where to learn more

    Data Protection Diaries
    Cloud Conversations: AWS overview and primer)
    Only you can prevent cloud data loss
    Is Computer Data Storage Complex? It Depends
    Eight Ways to Avoid Cloud Storage Pricing Surprises
    Cloud and Object Storage Center
    Cloud Storage: Is It All About Cost?
    Cloud conversations: Gaining cloud confidence from insights into AWS outages (Part II)
    Given outages, are you concerned with the security of the cloud?
    Is the cost of cloud storage really cheaper than traditional storage?
    Are more than five nines of availability really possible?
    What should I look for in an enterprise file sync-and-share app?
    How do primary storage clouds and cloud for backup differ?
    What should I consider when using SSD cloud?
    What’s most important to know about my cloud privacy policy?
    Data Archiving: Life Beyond Compliance
    My copies were corrupted: The 3-2-1 rule
    Take a 4-3-2-1 approach to backing up data

    What this means

    In my opinion there are cheap clouds (products, services, solutions) and there are low-cost options as well as there are value and premium offerings. Avoid confusing value with cheap or low-cost as something might have a higher cost, however including more capabilities or fees included that if useful can be more value. Look beyond the up-front cost aspects of clouds also considering ongoing recurring fees for actually using a server or solution.

    If you can find low-cost storage at or below a penny per GByte per month that could be a good value if it also includes many free access, retrieval GETS head and lists for management or reporting. On the other hand, if you find a service that is at or below a penny per GByte per month however charges for any access including retrieval, as well as network bandwidth fees along with reporting, that might not be as good of a value.

    Look beyond the basic price and watch out for statements like "…as low as…" to understand what is required to get that "..as low as.." price. Also understand what the extra fee’s are which most of the reputable providers list these on their sites, granted you have to look for them. If you are already using cloud services, pay attention to your monthly invoices and track what you are paying for to avoid surprises.

    From my InfoStor piece:

    For cloud storage, instead of simply focusing on lowest cost of storage per capacity, look for value, along with ability to configure or use with as much resiliency as you need. Value will mean different things depending on your needs and cloud storage servers, yet the solution should be cost-effective with availability including durability, secure and applicable performance.

    Shopping for cloud servers and storage is similar to acquiring regular servers and storage in that you need to understand what you are acquiring along with up-front and recurring fee’s to understand the total cost of ownership and cost of operations not to mention making apples to apples vs. apples to oranges comparisons.

    Btw, instead of simply using lower cost cloud services to cut cost, why not also use those capabilities to create or park another copy of your important data somewhere else just to be safe…

    What say you about cloud costs?

    Ok, nuff said, for now…

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Top vblog voting V2.015 (Its IT award season, cast your votes)

    Top vblog voting V2.015 (Its IT award season, cast your votes)

    Storage I/O trends

    It’s that time of the year again for award season:

    • The motion picture association Academy awards (e.g. the Oscars)
    • The Grammys and other entertainment awards
    • As well as Eric Siebert (aka @ericsiebert) vsphere-land.com top vblog

    Vsphere-land.com top vblog

    Eric has run for several years now an annual top VMware, Virtualization, Storage and related blogs voting now taking place until March 16th 2015 (click on the image below). You will find a nice mix of new school, old school and a few current or future school theme blogs represented with some being more VMware specific. However there are also many blogs at the vpad site that have a cloud, virtual, server, storage, networking, software defined, development and other related themes.

    top vblog voting
    Click on the above image to cast your vote for favorite:

    • Ten blogs (e.g. select up to ten and then rank 1 through 10)
    • Storage blog
    • Scripting blog
    • VDI blog
    • New Blogger
    • Independent Blogger (e.g. non-vendor)
    • News/Information Web site
    • Podcast

    Call to action, take a moment to cast your vote

    My StorageIOblog.com has been on the vLaunchPad site for several years now as well as having syndicated content that also appears via some of the other venues listed there.

    Six time VMware vExpert

    In addition to my StorageIOblog and podcast, you will also find many of my fellow VMware vExperts among others at the vLaunchpad site so check them out as well.

    What this means

    This is a people’s choice process (yes it is a popularity process of sorts as well) however also a way of rewarding or thanking those who take time to create and share content with you and others. If you take time to read various blogs, listen to podcasts as well as consume other content, please take a few moments and cast your vote here (thank you in advance) which I hope includes StorageIOblog.com as part of the top ten, as well as being nominated in the Storage, Podcast and Independent blogger categories.

    Ok, nuff said, for now…

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    February 2015 Server StorageIO Update Newsletter

    Volume 15, Issue II

    Hello and welcome to this February 2015 Server and StorageIO update newsletter. The new year is off and running with many events already underway including the recent USENIX FAST conference and others on the docket over the next few months.

    Speaking of FAST (File and Storage Technologies) event which I attended last week, here is a link to where you can download the conference proceedings.

    In other events, VMware announced version 6 of their vSphere ESXi hypervisor and associated management tools including VSAN, VVOL among other items.

    This months newsletter has a focus on server storage I/O performance topics with various articles, tips, commentary and blog posts.

    Watch for more news, updates and industry trends perspectives coming soon.

    Commentary In The News

    StorageIO news

    Following are some StorageIO industry trends perspectives comments that have appeared in various print and on-line venues. Over at Processor there are comments on resilient & highly available, underutilized or unused servers, what abandoned data Is costing your company, align application needs with your infrastructure (server, storage, networking) resources.

    Also at processor explore flash based (SSD) storage, enterprise backup buying tips, re-evaluate server security, new tech advancements for server upgrades, and understand cost of acquiring storage.

    Meanwhile over at CyberTrend there are some perspectives on enterprise backup and better servers mean better business.

    View more trends comments here

    Tips and Articles

    So you have a new storage device or system.

    How will you test or find its performance?

    Check out this quick-read tip on storage benchmark and testing fundamentals over at BizTech. Also check out these resources and links on server storage I/O performance and benchmarking tools.

    View recent as well as past tips and articles here

    StorageIOblog posts

    Recent StorageIOblog posts include:

    View other recent as well as past blog posts here

    In This Issue

  • Industry Trends Perspectives
  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts

    • Events & Activities

    EMCworld – May 4-6 2015

    Interop – April 29 2015

    NAB – April 14-15 2015

    Deltaware Event – March 3 2015

    Feb. 18 – FAST 2015 – Santa Clara CA

    View other recent and upcoming events here

    Webinars

    December 11, 2014 – BrightTalk
    Server & Storage I/O Performance

    December 10, 2014 – BrightTalk
    Server & Storage I/O Decision Making

    December 9, 2014 – BrightTalk
    Virtual Server and Storage Decision Making

    December 3, 2014 – BrightTalk
    Data Protection Modernization

    November 13 9AM PT – BrightTalk
    Software Defined Storage

    Videos and Podcasts

    StorageIO podcasts are also available via and at StorageIO.tv

    From StorageIO Labs

    Research, Reviews and Reports

    StarWind Virtual SAN
    starwind virtual san

    Using less hardware with software defined storage management. This looks at the needs of Microsoft Hyper-V ROBO and SMB environments with software defined storage less hardware. Read more here.

    View other StorageIO lab review reports here.

    Resources and Links

    Check out these useful links and pages:
    storageio.com/links
    objectstoragecenter.com
    storageioblog.com/data-protection-diaries-main/
    storageperformance.us
    thessdplace.com
    storageio.com/raid
    storageio.com/ssd

    Enjoy this edition of the Server and StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and podcasts along with in the news commentary appearing soon.

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    VMware announces vSphere V6 and associated virtualization technologies

    VMware announces vSphere V6 and associated virtualization technologies

    server storage I/O trends

    VMware has announced version 6 (V6) of its software defined data center (SDDC) server virtualization hypervisor called vSphere aka ESXi. In addition to a new version of its software defined server hypervisor along with companion software defined management and convergence tools.

    VMware

    VMware vSphere Refresh

    As a refresh for those whose world does not revolve around VMware, vSphere and software defined data centers (believe it or not there are some who exist ;), ESXi is the hypervisor that virtualizes underlying physical machines (PM’s) known as hosts.

    software defined data center convergence
    The path to software defined data center convergence

    Guest operating systems (or other hypervisors using nesting) run as virtual machines (VM’s) on top of the vSphere hypervisor host (e.g. ESXi software). Various VMware management tools (or third-party) are used for managing the virtualized data center from initial configuration, configuration, conversion from physical to virtual (P2V) or virtual to virtual (V2V) along with data protection, performance, capacity planning across servers, storage and networks.

    virtual machines

    VMware vSphere is flexible and can adapt to different sized environments from small office home office (SOHO) or small SMB, to large SMB, SME, enterprise or cloud service provider. There are a free version of ESXi along with paid versions that include support and added management tool features. Besides the ESXi vSphere hypervisor, other commonly deployed modules include the vCenter administration along with Infrastructure Controller services platform among others. In addition, there are optional solution bundles to add support for virtual networking, cloud (public and private), data protection (backup/restore, replication, HA, BC, DR), big data among other capabilities.

    What is new with vSphere V6

    VMware has streamlined the installation, configuration and deployment of vSphere along with associated tools which for smaller environments makes things simply easier. For the larger environments, having to do less means being able to do more in the same amount of time which results in cost savings. In addition to easier to use, deploy and configure, VMware has extended the scaling capabilities of vSphere in terms of scaling-out (larger clusters), scaling-up (more and larger servers), as well as scaling-down (smaller environments and ease of use).

    cloud virtual software defined servers

    • Compute: Expanded support for new hardware, guest operating systems and general scalability in terms of physical, and virtual resources. For example increasing the number of virtual CPU (vCPUs), number of cluster nodes among other speeds and feeds enhancements.

    server storage I/O vsan

    • Storage: This is an area where several enhancements were made including updates for Storage I/O controls (Storage QoS and performance optimizations) with per VM reservations, NFS v4.1 with Kerberos client, Virtual SAN (VSAN) improvements (new back-end underlying file system) as well as new Virtual Volumes (vVOLs) for Storage Policy Based Management.
    • Availability: Improvements for vMotion (ability to live move virtual machines between physical servers (vmware hosts) including long distance fault-tolerance. Other improvements include faster replication, vMotion across vCenter servers, and long distance vMotion (up to 100ms round trip time latency).
    • Network: Network I/O Control (NIOC) provides per VM and dat (VM and data repository) bandwidth reservations for quality of service (QoS) performance optimization.
    • Management: Improvements for multi-site, virtual data centers, content-library (storage and versioning of files and objects including ISOs and OVFs (Open Virtualization Format files) that can be on a VMFS (VMware File System) dat or NFS volume, policy-based management and web-client performance enhancements.

    What is vVOL?

    The quick synopsis of VMware vVOL’s overview:

    • Higher level of abstraction of storage vs. traditional SCSI LUN’s or NAS NFS mount points
    • Tighter level of integration and awareness between VMware hypervisors and storage systems
    • Simplified management for storage and virtualization administrators
    • Removing complexity to support increased scaling
    • Enable automation and service managed storage aka software defined storage management

    server storage I/O volumes
    How data storage access and managed via VMware today (read more here)

    vVOL’s are not LUN’s like regular block (e.g. DAS or SAN) storage that use SAS, iSCSI, FC, FCoE, IBA/SRP, nor are they NAS volumes like NFS mount points. Likewise vVOL’s are not accessed using any of the various object storage access methods mentioned above (e.g. AWS S3, Rest, CDMI, etc) instead they are an application specific implementation. For some of you this approach of an applications specific or unique storage access method may be new, perhaps revolutionary, otoh, some of you might be having a DejaVu moment right about now.

    vVOL is not a LUN in the context of what you may know and like (or hate, even if you have never worked with them), likewise it is not a NAS volume like you know (or have heard of), neither are they objects in the context of what you might have seen or heard such as S3 among others.

    Keep in mind that what makes up a VMware virtual machine are the VMK, VMDK and some other files (shown in the figure below), and if enough information is known about where those blocks of data are or can be found, they can be worked upon. Also keep in mind that at least near-term, block is the lowest common denominator that all file systems and object repositories get built-up.

    server storage I/O vVOL basics
    How VMware data storage accessed and managed with vVOLs (read more here)

    Here is the thing, while vVOL’s will be accessible via a block interface such as iSCSI, FC or FCoE or for that matter, over Ethernet based IP using NFS. Think of these storage interfaces and access mechanisms as the general transport for how vSphere ESXi will communicate with the storage system (e.g. their data path) under vCenter management.

    What is happening inside the storage system that will be presented back to ESXi will be different than a normal SCSI LUN contents and only understood by VMware hypervisor. ESXi will still tell the storage system what it wants to do including moving blocks of data. The storage system however will have more insight and awareness into the context of what those blocks of data mean. This is how the storage systems will be able to more closely integrate snapshots, replication, cloning and other functions by having awareness into which data to move, as opposed to moving or working with an entire LUN where a VMDK may live.

    Keep in mind that the storage system will still function as it normally would, just think of vVOL as another or new personality and access mechanism used for VMware to communicate and manage storage. Watch for vVOL storage provider support from the who’s who of existing and startup storage system providers including Cisco, Dell, EMC, Fujitsu, HDS, HP, IBM, NetApp, Nimble and many others. Read more about Storage I/O fundamentals here and vVOLs here and here.

    What this announcement means

    Depending on your experiences, you might use revolutionary to describe some of the VMware vSphere V6 features and functionalities. Otoh, if you have some Dejavu moments looking pragmatically at what VMware is delivering with V6 of vSphere executing on their vision, evolutionary might be more applicable. I will leave it up to you do decide if you are having a Dejavu moment and what that might pertain to, or if this is all new and revolutionary, or something more along the lines of technolutionary.

    VMware continues to execute delivering on the Virtual Data Center aka Software Defined Data Center paradigm by increasing functionality, as well as enhancing existing capabilities with performance along with resiliency improvements. These abilities enable the aggregation of compute, storage, networking, management and policies for enabling a global virtual data center while supporting existing along with new emerging applications.

    Where to learn more

    If you were not part of the beta to gain early hands-on experience with VMware vSphere V6 and associated technologies, download a copy to check it out as part of making your upgrade or migration plans.

    Check out the various VMware resources including communities links here
    VMware vSphere Hypervisor getting started and general vSphere information (including download)
    VMware vSphere data sheet, compatibility guide along with speeds and feeds (size and other limits)
    VMware vExpert
    VMware Blogs and VMware vExpert page

    Various fellow VMware vExpert blogs including among many others vsphere-land, scott lowe, virtuallyghetto and yellow-bricks among many others found at the vpad here.

    StorageIO Out and About Update – VMworld 2014 (with Video)
    VMware vVOL’s and storage I/O fundamentals (Storage I/O overview and vVOL, details Part I and Part II)
    How many IOPs can a HDD or SSD do in a VMware environment (Part I and Part II)
    VMware VSAN overview and primer, DIY converged software defined storage on a budget

    Wrap up and summary

    Overall VMware vSphere V6 has a great set of features that support both ease of management for small environments as well as the scaling needs of larger organizations.

    Ok, nuff said, for now…

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved