OpenStack Archives

May 7, 2013March 7, 2022

EMC ViPR software defined object storage part III

Storage I/O trends

This is part III in a series of posts pertaining to EMC ViPR software defined storage and object storage. You can read part I here and part II here.

EMCworld

More on the object opportunity

Other object access includes OpenStack storage part Swift, AWS S3 HTTP and REST API access. This also includes ViPR supporting EMC Atmos, VNX and Isilon arrays as southbound persistent storage in addition.

object storage
Object (and cloud) storage access example

EMC is claiming that over 250 VNX systems can be abstracted to support scaling with stability (performance, availability, capacity, economics) using ViPR. Third party storage will be supported along with software such as OpenStack Swift, Ceph and others running on commodity hardware. Note that EMC has some history with object storage and access including Centera and Atmos. Visit the micro site I have setup called www.objectstoragecenter.com and watch for more content to be updated and added there.

More on the ViPR control plane and controller

ViPR differs from some others in that it does not sit in the data path all the time (e.g. between application servers and storage systems or cloud services) to cut potential for bottlenecks.

ViPR architecture

Organizations that can use ViPR include enterprise, SMB, CSP or MSP and hosting sites. ViPR can be used in a control mode to leverage underlying storage systems, appliances and services intelligence and functionality. This means ViPR can be used to complement as oppose to treat southbound or target storage systems and services as dumb disks or JBOD.

On the other hand, ViPR will also have a suite of data services such as snapshot, replication, data migration, movement, tiering to add value for when those do not exist. Customers will be free to choose how they want to use and deploy ViPR. For example leveraging underlying storage functionality (e.g. lightweight model), or in a more familiar storage virtualization model heavy lifting model. In the heavy lifting model more work is done by the virtualization or abstraction software to create an added value, however can be a concern for bottlenecks depending how deployed.

Software defined, storage hypervisor, virtual storage or storage virtualization?

Most storage virtualization, storage hypervisors and virtual storage solutions that are hardware or software based (e.g. software defined) implemented what is referred to as in band. With in band the storage virtualization software or hardware sits between the applications (northbound) and storage systems or services (southbound).

While this approach can be easier to carry out along with add value add services, it can also introduce scaling bottlenecks depending on implementations. Examples of in band storage virtualization includes Actifio, DataCore, EMC VMAX with third-party storage, HDS with third-party storage, IBM SVC (and their V7000 Storwize storage system based on it) and NetApp Vseries among others. An advantage of in band approaches is that there should not need to be any host or server-side software requirements and SAN transparency.

There is another approach called out-of-band that has been tried. However pure out-of-band requires a management system along with agents, drivers, shims, plugins or other software resident on host application servers.

Example of generic fast path control path model

ViPR takes a different approach, one that was seen a few years ago with EMC Invista called fast path, control path that for the most part stays out of the data path. While this is like out-of-band, there should not be a need for any host server-side (e.g. northbound) software. By being a fast path control path, the virtualization or abstraction and management functions stay out of the way for data being moved or work being done.

Hmm, kind of like how management should be, there to help when needed, out-of-the-way not causing overhead other times ;).

Is EMC the first (even with Invista) to leverage fast path control path?

Actually up until about a year or so ago, or shortly after HP acquired 3PAR they had a solution called Storage Virtualization Services Platform (SVPS) that was OEMd from LSI (e.g. StorAge). Unfortunately, HP decided to retire that as opposed to extend its capabilities for file and object access (northbound) as well as different southbound targets or destination services.

Whats this northbound and southbound stuff?

Simply put, think in terms of a vertical stack with host servers (PMs or VMs) on the top with applications (and hypervisors or other tools such as databases) on top of them (e.g. north).

software defined storage
Northbound servers, southbound storage systems and cloud services

Think of storage systems, appliances, cloud services or other target destinations on the bottom (or south). ViPR sits in between providing storage services and management to the northbound servers leveraging the southbound storage.

What host servers can VIPR support for serving storage?

VIPR is being designed to be server agnostic (e.g. virtual or physical), along with operating system agnostic. In addition VIPR is being positioned as capable of serving northbound (e.g. up to application servers) block, file or object as well as accessing southbound (e.g. targets) block, file and object storage systems, file systems or services.

Note that a difference between earlier similar solutions from EMC have been either block based (e.g. Invista, VPLEX, VMAX with third-party storage), or file based. Also note that this means VIPR is not just for VMware or virtual server environments and that it can exist in legacy, virtual or cloud environments.

ViPR image

Likewise VIPR is intended to be application agnostic supporting little data, big data, very big data ( VBD) along with Hadoop or other specialized processing. Note that while VIPR will support HDFS in addition to NFS and CIFS file based access, Hadoop will not be running on or in the VIPR controllers as that would live or run elsewhere.

How will VIPR be deployed and licensed?

EMC has indicated that the VIPR controller will be delivered as software that installs into a virtual appliance (e.g. VMware) running as a virtual machine (VM) guest. It is not clear when support will exist for other hypervisors (e.g. Microsoft Hyper-V, Citrix/XEN, KVM or if VMware vSphere with vCenter or simply on ESXi free version). As of the announcement pre briefing, EMC had not yet finalized pricing and licensing details. General availability is expected in the second half of calendar 2013.

Keep in mind that the VIPR controller (software) runs as a VM that can be hosted on a clustered hypervisor for HA. In addition, multiple VIPR controllers can exist in a cluster to further enhance HA.

Some questions to be addressed among others include:

How and where are IOs intercepted?
Who can have access to the APIs, what is the process, is there a developers program, SDK along with resources?
What network topologies are supported local and remote?
What happens when JBOD is used and no advanced data services exist?
What are the characteristics of the object access functionality?
What if any specific switches or data path devices and tools are needed?
How does a host server know to talk with its target and ViPR controller know when to intercept for handling?
Will SNIA CDMI be added and when as part of the object access and data services capabilities?
Are programmatic bindings available for the object access along with support for other APIs including IOS?
What are the performance characteristics including latency under load as well as during a failure or fault scenario?
How will EMC place Vplex and its caching model on a local and wide area basis vs. ViPR or will we see those two create some work together, if so, what will that be?

Bottom line (for now):

Good move for EMC, now let us see how they execute including driving adoption of their open APIs, something they have had success in the past with Centera and other solutions. Likewise, let us see what other storage vendors become supported or add support along with how pricing and licensing are rolled out. EMC will also have to articulate when and where to use ViPR vs. VPLEX along with other storage systems or management tools.

Ok, nuff said (for now)

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

May 6, 2013January 1, 2022

EMC ViPR virtual physical object and software defined storage (SDS)

Storage I/O trends

Introducing EMC ViPR

This is the first in a three part series, read part II here, and part III here.

During the recent EMCworld event in Las Vegas among other things, EMC announced ViPR (read announcement here) . Note that this ViPR is not the same EMC Viper project from a few years ago that was focused on data footprint reduction (DFR) including dedupe. ViPR has been in the works for a couple of years taking a step back rethinking how storage is can be used going forward.

EMCworld

ViPR is not a technology creation developed in a vacuum instead includes customer feedback, wants and needs. Its core themes are extensible, open and scalable.

EMCworld

On the other hand, ViPR addresses plenty of buzzword bingo themes including:

Agility, flexibility, multi-tenancy, orchestration
Virtual appliance and control plane
Data services and storage management
IT as a Service (ITaaS) and Infrastructure as a Service (IaaS)
Scaling with stability without compromise
Software defined storage
Public, private, hybrid cloud
Big data and little data
Block, file and object storage
Control plane and data plane
Storage hypervisor, virtualization and virtual storage
Heterogeneous (third-party) storage support
Open API and automation
Self-service portals, service catalogs

Buzzword bingo

Note that this is essentially announcing the ViPR product and program initiative with general availability slated for second half of 2013.

What is ViPR addressing?

IT and data infrastructure (server, storage, IO and networking hardware, software) challenges for traditional, virtual and cloud environments.

Data growth, after all, there is no such thing as an information recession with more data being generated, moved, processed, stored and retained for longer periods of time. Then again, people and data are both getting larger and living longer, for both little data and big data along with very big data.

Overhead and complexities associated with managing and using an expanding, homogenous (same vendor, perhaps different products) or heterogeneous (different vendors and products) data infrastructure across cloud, virtual and physical, legacy and emerging. This includes add, changes or moves, updates and upgrades, retirement and replacement along with disposition, not to mention protecting data in an expanding footprint.

road to cloud

Operations and service management, fault and alarm notification, resolution and remediation, rapid provisioning, removing complexity and cost of doing things vs. simply cutting cost and compromising service.

EMC ViPR

What is this software defined storage stuff?

There is the buzzword aspect, and then there is the solution and business opportunity.

First the buzzword aspect and bandwagon:

Software defined marketing (SDM) Leveraging software defined buzzwords.

Software defined data centers (SDDC) Leveraging software to derive more value from hardware while enabling agility, flexibility, and scalability and removing complexity. Think the Cloud and Virtual Data Center models including those from VMware among others.

Software defined networking (SDN) Rather than explain, simply look at Nicira that VMware bought in 2012.

Software defined storage (SDS) Storage software that is independent of any specific hardware, which might be a bit broad, however it is also narrower than saying anything involving software.

Software defined BS (SDBS) Something that usually happens as a result when marketers and others jump on a bandwagon, in this case software defined marketing.

Note that not everything involved with software defined is BS, only some of the marketing spins and overuse. The downside to the software defined marketing and SDBS is the usual reaction of skepticism, cynicism and dismissal, so let us leave the software defined discussion here for now.

software defined storage

An example of software defined storage can be storage virtualization, virtual storage and storage hypervisors that are hardware independent. Note that when I say hardware independent, that also means being able to support different vendors systems. Now if you want to have some fun with the software defined storage diehards or purist, tell them that all hardware needs software and all software needs hardware, even if virtual. Further hardware is defined by its software, however lets leave sleeping dogs lay where they rest (at least for now ;)).

Storage hypervisors were a 2012 popular buzzword bingo topic with plenty of industry adoption and some customer deployment. While 2012 saw plenty of SDM buzz including SDC, SDN 2013 is already seeing an increase including software defined servers, and software defined storage.

Regardless of what you view of software defined storage, storage hypervisor, storage virtualization and virtual storage is, the primary focus and goal should be addressing business and application needs. Unfortunately, some of the discussions or debates about what is or is not software defined and related themes lose focus of what should be the core goal of enabling business and applications.

Continue reading in part II of this series here including how ViPR works, who it is for and more analysis.

Ok, nuff said (for now)

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

November 3, 2012March 7, 2022

Ceph Day in Amsterdam and Sage Weil on Object Storage

Now also available via

This is a new episode in the continuing StorageIO industry trends and perspectives pod cast series (you can view more episodes or shows along with other audio and video content here) as well as listening via iTunes or via your preferred means using this RSS feed (https://storageio.com/StorageIO_Podcast.xml)

In this episode, I am at the Ceph day in Amsterdam Holland event at the Tobacco Theatre. My guest for this episode is Ceph (Cephalanthera) creator Sage Weil who is also the founder of inktank.com that provides services and support for the open source based Ceph project.

For those not familiar with Ceph, it is an open source distributed object scale out software platform that can be used for deploying cloud and managed services, general purpose storage for research, commercial, scientific, high performance computing (HPC) or high productivity computing (commercial) along with backup or data protection and archiving destinations.

During our conversation Sage presents an overview of what Ceph is (e.g. Ceph for non Dummies), where and how it can be used, some history of the project and how it fits in with or provides an alternative to other solutions. Sage also talks about the business or commercial considerations for open source based projects, importance of community and having good business mentors and partners as well as staying busy with his young family.

If you are a Ceph fan, gain more insight into Sage along with Ceph day sponsors Inktank and 42on. On the other hand, if you new to object storage, open source storage software or cloud storage, listen in to gain perspectives of where technology such as Ceph fits for public, private, hybrid or traditional environments.

Click here (right-click to download MP3 file) or on the microphone image to listen to the conversation with Sage and myself.

Also available via

Watch (and listen) for more StorageIO industry trends and perspectives audio blog posts pod casts and other upcoming events. Also be sure to heck out other related pod casts, videos, posts, tips and industry commentary at StorageIO.com and StorageIOblog.com.

Enjoy this episode Ceph Day in Amsterdam with Sage Weil.

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

October 6, 2012May 24, 2020

Does Dell have a cloudy cloud strategy story (Part II)?

StorageIO industry trends cloud, virtualization and big data

This is the second of two posts (here is the first post) that are part of ongoing industry trends and perspectives cloud conversations series that looks at Dell and their cloud strategy story.

So what does the first post have to do with Dell having a cloudy cloud strategy story?

Simple, there have been some rather low-key, almost quiet or muddled announcements (also here, here and here) about Dell and Nirvanix collaborating around public cloud storage. Keep in mind that Nirvanix and IBM not too long ago also announced a partnership that some jumped to the conclusion that big blue was about to buy the startup vendor, even though IBM already has other cloud and storage as a service, or backup as a service and DR as a service offerings, what the heck, the more the merrier for big blue?

What about Dell and their partnership with Nirvanix, (more on that in the first post) did somebody jump the gun, or jump the shark?

Is Dell trying to walk the tightrope between being a supplier to major cloud providers while carefully moving into the cloud services market themselves, or are they simply addressing point customer situation or opportunities, at least for the time being?

Alternatively, is this nothing more than Dell establishing another partnership with a technology partner who also happens to be in the services business, similar to what Dell is doing with OpenStack and others?

IMHO Dell has some of the pieces and partnerships and could be a strong contender in the SMB and SME private cloud space, along with VDI and related areas with their Citrix, Microsoft and VMware partnerships. This is also also leveraging their servers and, storage, software, networking and other solutions to supply service providers.

The rest comes down to what markets or areas of focus does Dell want to target, that would in turn dictate how to extend what they already have or what they need to go out and get or partner around.

What say you, what’s your take on Dells cloud strategy story and portfolio?

Ok, nuff said (for now).

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

October 6, 2012December 24, 2020

Does Dell have a cloudy cloud strategy story (Part I)?

StorageIO industry trends cloud, virtualization and big data

This is first of a two-part post (click here for second post) that is part of ongoing industry trends and perspective cloud conversations series that looks at Dell and their cloud strategy story. For background, some previous Dell posts are found here, here, here and here. Here is a link that has video of the live Dell Storage Customer Advisory (CAP) panel that Dell asked me to moderate back in June that touches on some related themes and topics. Btw, fwiw and for disclosure Dell AppAssure is a site advertiser on storageioblog.com ;).

Depending on your view of what is or is not a cloud service, product or solution, naturally you will then have various opinions of where Dell is at with their cloud strategy and story.

If you consider object based storage to be part of or a component of private clouds or at least for medical, healthcare and related focus, then Dell is already there with their DX object storage solutions (Caringo based).

From a scale out, clustered or grid file system, Dell bought Exanet in a post holiday shopping sale a few years back and has invested in its development having renamed it Fluid File System and initially available as the FS7000 series (EqualLogic) and more recently expanded systems such as the FS8600 (Compellent based), EqualLogic and NX3500 (MD3000 based).

If you view clouds as being part of services provided including via hosting or similar, Dell is already there via their Perot systems acquisitions.

If you view cloud as being part of VDI, or VDI being part of cloud, Dell is there with their tools including various acquisitions and solution bundles.

On the other hand if you view clouds as reference architectures across VMware vSphere, Microsoft Hyper-V and Citrix Xen among others, guess what, Dell is also there with their VIS.

Or, if you view private clouds as being a bundled solution (server, storage, hardware, software) such as EMC vBlock or NetApp FlexPod, then Dell vStart (not to be confused as being a service) is on the list with other infrastructure stack solutions.

How about being a technology supplier to what you may consider as being true cloud providers or enables including those who use OpenStack or other APIs and cloud tools, guess what, Dell is also there including at Rackspace (via public web info).

So the above all comes back to that Dell like many vendors who offer services, solutions and related items for data and information infrastructures have diverse offerings including servers, storage, networking, hardware, software and support. Dell like others similar to them has to find a balance between providing services that compete with their customers, as well as supplier such as to Rackspace. In this case Dell is no different from EMC who happened to move their Mozy backup service off to their VMware subsidiary and has managed to help define where VCE (and here) and ATMOS fit as products while being services capable. IBM has figured this out having a mix of old school services such as SmartCloud Services (or here), IBM Global Services and BCRS (business continuity recovery services), not to mention newer backup and storage cloud services, products and solutions they have acquired, or OEM or have reseller agreements with.

StorageIO industry trends cloud, virtualization and big data

HP has expanded their traditional focused EDS as well as other HP services along with products being joined by their Amazon like Cloud Services including compute, storage and content distribution network (CDN) capabilities. NetApp is taking the partnering route along with Cisco staying focused for at least now on being a partner supplier. Oracle, well Oracle is Oracle and they have a mix of products and services. In fact some might say Oracle is late to the cloud game however they have been in the game since the late 90s when they came out with Oracle online, granted the cloud purist will call that application service provider (e.g. ASP) vs. today’s applications as a service (AaaS) models.

Continue with the second post here, ok, nuff said (for now).

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

More on the object opportunity

More on the ViPR control plane and controller

Software defined, storage hypervisor, virtual storage or storage virtualization?

Is EMC the first (even with Invista) to leverage fast path control path?

Whats this northbound and southbound stuff?

What host servers can VIPR support for serving storage?

How will VIPR be deployed and licensed?

Bottom line (for now):

Share this:

Introducing EMC ViPR

What is ViPR addressing?

What is this software defined storage stuff?

Share this:

Share this:

Share this:

Share this: