Tag: swift

Posted on

Software Defined Cloud Bulk Object Storage News Trends

Cloud, Software Defined, Scale-Out, Object Storage News Trends

Welcome to the Cloud, Big Data, Software Defined, Bulk and Object Storage News Trends page.

This page contains various links pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage News Trends and resources. Other resources pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage include:

Software defined, Bulk, Cloud and Object Storage exist to support expanding and diverse application data demands. There are various types of cloud, bulk and object storage including public services such as Amazon Web Services (AWS) Simple Storage Service (S3), Google, Microsoft Azure, IBM Softlayer, Rackspace among many others. There are also solutions for hybrid and private deployment from Cisco, DDN, Dell EMC, Fujitsu, HDS, HPE, IBM, NetApp, Noobaa, OpenStack, Quantum, Rackspace, Scality, Seagate, Spectra, Storpool, Suse, Swift and WD among others.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

Cloud products and services among others, along with associated data infrastructures including object storage, file systems, repositories and access methods are at the center of bulk, big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.

Via InfoStor: Object Storage Is In Your Future
Via FujiFilm IT Summit: Software Defined Data Infrastructures (SDDI) and Hybrid Clouds
Via ChannelPartnersOnline: Selling Software-Defined Storage: Not All File Systems Are the Same
Via ITProPortal: IBM kills off its first cloud storage platform
Via MultiChannel: After ditching cloud business, Verizon inks Virtual Network Services deal with Amazon
Via ITBusinessEdge: Time to Rein in Cloud Storage
Via BusinessInsider: Kodiak Data Brings Enterprise Storage and Database Services to the Cloud Era with StorageCloud
Via SerchCloudStorge: Ctera Networks’ file-sharing services gain intelligent cache
Via StorageIOblog: Who Will Be At Top Of Storage World Next Decade?

cloud object storage

The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
Via CloudComputingAdmin: Cloud Storage Decision Making
Via Wservernews: Cloud storage considerations
Via PRNewsWire: Scality Accelerates Multi-Cloud Adoption with New Open-Source Release
Via EnterpriseStorageForum: Top 10 Tips for Software-Defined Storage Deployment
Via MultiChannel: Verizon Digital Media Services now offers integrated Microsoft Azure Storage
Via BroadCstingCable: Quantum’s New StorNext 6 Delivers Unparalleled Combination of Best-in-Class 4K Performance and Advanced Data Management
Via ShootOnLine: Morro Data Accelerates Media Collaboration With Cache and Sync Appliance
Via TechCrunch: Backblaze drops download pricing for its B2 storage platform by 60%
Via PRNewsWire: FileCloud Online Launches an All-In-One File Share, Sync and Backup SaaS Offering in a Challenge to Dropbox and Box

cloud object storage

Via HITInfrastructure: Benefits of Object Storage to Health IT Infrastructure
Via ITBusinessedge: Growing Diversity of Cloud Storage Solutions
Via Telecompaper: Interoute launches storage zones in Switzerland
Via TradeCalls:
Object Storage Market Size, Status and Forecast 2022
Via OpenStack: Creating object storage container causes user to be logged out, OpenStack Dashboard (Horizon)
Via RethinkResearch: Nokia claims a 40% cut in cloud DVR Capex, using new Skim Storage
Via Cloudwards: A Beginner’s Guide on How Cloud Backup With Microsoft Azure Works
Via ITProPortal: Lack of understanding of data inhibits move to the cloud

cloud object storage

Via InfoStor:Cloud Storage Concerns, Considerations and Trends
Via StreetInsider: IBM Announces New Pricing Model for Cloud Storage and NetApp Object Storage Partnership

Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
Via ComputerWeekly: Storage 101: Object storage vs block vs file
Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
Via Yahoo Finance: SwiftStack Joins Google Cloud Technology Partner Program
Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look

Via StorageIOblog: HPE Continues Buying Into Server Storage I/O Data Infrastructures
Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
Via TheStack:Google to launch nine new data centres over three new cloud regions
Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
Via DigitalJournal:Global Business Cloud Storage market 2017 share, Segmentation and Industrial Overview forecasts to 2022
Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
Via NextPlatform:How Microsoft Fell Hard for FPGAs
Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
Via NextPlatform:Google Courts Enterprise For Cloud Platform

cloud object storage

Via IBM RedBooks: Cloud Object Storage as a
Service – IBM Cloud Object Storage from Theory to Practice
Via VentureBeat: Microsoft confirms Azure storage issues around the world
Via Microsoft Azure: Azure status history
Via NextPlatform: Windows Server Comes To ARM Chips, But Only For Azure
Via NextPlatform: ARM And AMD X86 Server Chips Get Mainstream Lift From Microsoft
Via ITBusinessEdge: Google Expands Enterprise Cloud Service Footprint
Via ChannelWorld: Google announces three new cloud regions, contract discounts
Via AWS: AWS Service Health Status
Via AWS (S3 Feb 28 2017 Port mortem): Summary of the Amazon S3 Service Disruption
Via SearchCloudStorage: Open source object storage startup OpenIO adds hardware

cloud object storage

Via InfoStor: Comments on Cloud Storage Pros and Cons
Via CRN: StorageCraft Acquires Exablox
Via TVtechglobal: Signiant CEO: The media industry isnt just one monolithic thing
Via Computerweekly: Using the cloud as a storage tier
Via SearchCloudStorage: OpenIO joins object storage cloud scrum
Via CRN: Dell Partners With Scality On High-Density Object Storage
Via BroadwayWorld: The World’s Smallest Object Storage Solution, ‘Ceph on Olive’, is Released
Via SmartDataCollective: Why You Should Be Securing Big Data In The Cloud?
Via Geekwire: Feds certify Amazon and Microsoft clouds to handle sensitive government data
Via NewsFactor: SoftNAS Introduces File Services for NetApp StorageGRID
Via Redmondmag: AWS Cloud Storage Service Turns 10
Via DropBox Blog: Scaling to exabytes and beyond
Via Computerworld: Dropbox quits Amazon cloud, takes back 500 PB of data
Via HPCwire: Seagate Sets Sights on Broader HPC Market

cloud object storage

Via Channelnomics.com: Why Samsung’s Joyent buyout could frame its future
Via GeekyGadgets: Google Drive Cloud Storage Is Now A Little Easier To Organize Thanks To New Tools
Via ZDnet: Google doubles Cloud Compute local SSD capacity: Now it’s 3TB per VM
Via AndroidPolice: Barracuda Is Shutting Down The Copy Cloud Storage Service On May 1st
Via StorageSoup: EMC expands Elastic Cloud (object) Storage
Via Cloud Review: Access Multiple Cloud Apps from a Unified Interface
Via CyberSecurity: Cybersecurity and Accessible Storage Trends
Via eWeek: Cloud Storage Spending Poised for Increase in 2016
Via InfoStor: Object Storage Is In Your Future
Via BioScienceTechnology: Six Reasons to Add Object Storage to Your Genomics Lexicon
Via TechCrunch: Backblaze’s Low-Cost Cloud Storage Service Comes Out Of Beta
Via CloudComputingAdmin: Cloud Storage Decision Making
COSbench for cloud and object storage (Various): White Paper (CMU), PDF overview presentation, Get the bits (software) from Github
Via AWS Blogs: AWS Storage Update – New Lower Cost S3 Storage Option & Glacier Price Reduction
Via Techcrunch: IBM Acquires Cleversafe To Accelerate Hybrid Cloud Storage Strategy

cloud object storage

Via Techcrunch: Developers Can Now Import Data To Google Cloud Storage By Mailing In Their Hard Drives, Tapes And USB Flash Drives
Via vCloud: VMware vCloud Air Object Storage
Via SearchStorage: SwiftStack adds new features to object storage software solution
Via TheVarGuy: Vendor consortium create Ceph advisory board
Via OpenStack: Whats new in OpenStack Liberty
Via Microsoft Azure: File access cloud storage in addition to blobs and object storage
Via AWS: AWS Storage Update – New Lower Cost S3 Storage Option & Glacier Price Reduction
Via VMware Blogs: Showcasing VMware vCloud Air Object Storage powered by EMC at EMC World!
Via Venturebeat: Backblaze launches B2 object storage service at 1/4 of the cost of Amazon S3
Via Techcrunch: Wuala, The Secure P2P Cloud Storage Service Acquired By LaCie, To Shut Down
Via ChannelworldIN: 13 firms join to open up Ethernet, object-storage to everyone
Via ITportal: The elephant in the room: The trust deficit issue with public clouds
Via VCpost: Dropbox latest update: Cloud storage service provider now lets you drag and drop URLs with files
Via NetworkAsia: HP’s new converged appliance offers a path to hybrid cloud
Via Techtimes: Google Files For Patent To Search Your Desktop Files, Apps And Cloud Storage

cloud object storage

Via CloudTweaks: Questions To Ask Every Cloud Storage Provider
Via MarketWatch: Linux Foundation Brings Together Industry Leaders to Advance Cloud Object Storage Technologies
Via CBR: New Kinetic platform targets open source object storage
Via Techcrunch: Seagate, Toshiba, Scality And Others Launch Kinetic Open Storage Platform For Smart, Low-Overhead Cloud Object Storage
Via StorageIOblog: July 2015 Newsletter Focus on Content Solution Platforms
Via ITnewsline: Hitachi Data Systems Announces New Enhancements to the Hitachi Content Platform Portfolio
Via ITbusinessnet: Caringo Provides Only Software-Based Web-Scale Object Storage for Symantec Enterprise Vault
Via ITbusinessnet: Caringo Provides Only Software-Based Web-Scale Object Storage for Symantec Enterprise Vault
Via TopTechNews: Dropbox Adds USB Keys To Boost Security
Via ZDnet: SUSE brings its management software to the cloud
Via Forbes: Jumptuit Roadmap Video: A New Search Bar For Your Data At Facebook, Google and Dropbox
Via Forbes: Panzura Global File System — Disrupting the Storage Industry
Via TalkinCloud: Barracuda Launches Cloud Storage and Share Service CudaDrive
Via ITnews: Researchers debut rival to AWS cloud storage
Via NetworkAsia.net: HP launches new scale-out and scale-up Compute portfolios

cloud object storage

Via Server StorageIO: May / June 2015 Newsletter Focus on server storage packaging
Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
Via StudioDaily.com: At NAB, Vendors and Users Both Got Serious About 4K Workflow
Via OpenSource.com: The unofficial guide to OpenStack Summit Vancouver
Via InsideHPC: Deploying Hadoop on Lustre Storage: Lessons Learned and Best Practices
Via Yahoo Engineering Blog: Yahoo Cloud Object Store – Object Storage at Exabyte Scale
Via the Platform: Inside The Ceph Exascale Storage At Yahoo
Via Swift Summit: Taking the Mystery out of Erasure Codes: A Swift Implementation
Via GovTech: Storage Costs Cloud Police Cam Issue
Via BostonHerald: Booting Up: Storage costs cloud police cam issue
Via StorageIOblog: S3motion Buckets Containers Objects AWS S3 Cloud and EMCcode (with podcast)
Via StorageIOblog: If focused on cost you might miss other cloud storage benefits
Via ComputerWorld: Amazon offers network file storage in the cloud
Via ComputerWeekly: HGST marries helium HDDs and Himalaya in object storage Active Archive
Via GoogleCloudPlatform Blog: GCS Nearline Online storage at Offline price

cloud object storage

Via MarketWatch: Global Data Center Provider CyrusOne Announces Direct Connectivity to Google Cloud Platform
Via PRNewsWire: Quantum Announces New Archive Solutions Designed To Reduce Unstructured Data Storage Costs
Via Enterprise Storage Forum: Lustre buying guide
Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
Via StreetInsider: Western Digital (WDC) to Acquire Object Storage Software Amplidata
Via Enterprise Storage Forum: Dell Invests in Object Storage Startup Exablox
Via Enterprise Storage Forum: Introducing s3motion (S3 and object access docker based appliance)
Via Computerworld: Quantum enhances their cloud and object storage management with new StorNext software version
Via HealthIndustry: 2014 IDC MarketScape perspectives
Via MarketWired: ScaleOut Software Releases Version 5.2 of Its In-Memory Computing Platform
Via CRN: HP Inks Global Reseller Agreement With Object Storage Startup Scality
Via NetApp: NetApp Introduces Software-Defined Object Storage for the Hybrid Cloud
Via ZDnet: NetApp intros software-defined object storage for hybrid cloud adoption
Via SearchCloudStorage: Are more than five nines of availability really possible?

cloud object storage

Via SearchCloudStorage: How do primary storage clouds and cloud for backup differ?
Via SearchCloudStorage: What should I consider when using SSD cloud?
Via SearchCloudStorage: Given outages, are you concerned with the security of the cloud?
Via SearchCloudStorage: Is the cost of cloud storage really cheaper than traditional storage?
Via HostReview: Panzura Collaborates With Microsoft Azure To Transform Cloud Storage Into A Globally Distributed NAS
Via ITBusinessNet: Caringo Awarded Patent for Erasure Coding, Replication in Storage Cluster
Via PRWeb: Western Digital Capital Makes Strategic Investment in Amplidata
Via ITwire: Dovecot in for the long haul
Via InfoStor: Top Tips for Object Storage
Via Enterprise Storage ForumObject Storage Buying Guide
Via Enterprise Networking Planet: Cloud Gateways 101
Via eWeek: HP Unveils Smaller, Less Costly Helion Cloud Platform
Via PCadvisor: CA Technologies, AWS and Riverbed bring mainframe Cloud to big four banks
Via CMS wire: The Enterprise Cloud is Finally Moving Beyond Storage
Via Edtech: Microsoft 365’s Massive Cloud Storage Is ‘Icing on Cake’ to University’s Upgrade

cloud object storage

Via CRN: Quantum Acquires Symform, Plans To Bring Cloud-Based Backups To Enterprise
Via BusinessWire: OpenStack Object Storage (Swift) Adds Storage Policies for Enterprise Users and Cloud Operators
Via SearchStorage: EMC ViPR and Data Services (including Object storage)
Via Enterprise Storage Forum: Cloud Appliance Buying Guide
Via Datacenter Dynamics: Verizon, Amplidata and Cloud Object Storage
Via ZDnet: IBM drops SoftLayer cloud storage costs closer to its rivals
Via YellowBricks: How long will VSAN rebuilding take with large drives? via Duncan Epping (@DuncanYB) and his Yellow-Bricks site. Note that VMware VSAN uses one of many different forms of object storage (e.g. its not S3 or Swift or CDMI) rather a means for storing VMware vSphere entities and items (e.g. various types of objects)
Via Computerweekly: Object storage: An architecture for the petabyte era? Via Simon Robinson (@simonrob451) appearing at
Via MarketWatch: Western Digital/HGST announce alternative to Seagate Kinetic HDD for object storage
Via EMC: EMC Announces Elastic Cloud Storage Appliance via EMC
Via TechCrunch: RedHat acquires InkTank provider of Ceph services
Via CompueterWeekly: DataDirect Networks (DDN) WOS enhancements

cloud object storage

Hybrid Clouds: Bridging the Gap Between Public and Private Environments (Recorded for Playback: BrightTalk Webinar)
Book review: Rethinking Enterprise Storage and Hybrid Clouds by Microsoft/Storsimple and Marc Farley aka (@Gofarley (or what ever name he is currently using ;))
Goodbye 2013, hello 2014 predictions present and future
Enterprises should just ignore OpenStack via Virtualizedgeek aka Keith Townsend (@virtualizedgeek)
OpenStack Swift Object Storage for the Cloud Grows via The Var Guy (@thevarguy) aka Christopher Tozzi
For Seagate Kinetic to fly…it’s gonna take an ecosystem (Via @Seagate The Storage Effect Blog)
Object-defined storage (it’s not about object storage) (Via Enrico Signoretti @Juku)
Seagate Kinetic Cloud and Object Storage I/O platform
Why You Should Consider Object Storage (Via 21CIT)
Object Storage Is in Your Future, Even if You Use Files (Via 21CIT)
The Right Storage Option Is Important for Big Data Success (Via FedTech)
Hybrid cloud storage with an object storage back end(Via Marc Farley aka @MicroFarley at Microsoft Technet Blogs)
cloud object storage

Via SearchStorage: EMC ViPR aimed at service providers, enterprise storage (Video)
EMC ViPR virtual physical object and software defined storage
EMC ViPR software defined object storage part II
EMC ViPR software defined object storage part III
The rise of cloud-integrated storage and EMC’s ViPR (Via Marc Farley of Microsoft aka @MicroFarley)
ViPR: Under the covers (Via Chad Sakacc of EMC aka @sakacc)
Introducing EMC ViPR (Via Chuck Hollis aka @Chuckhollis of EMC)
EMC Tells their SDS story (e.g. ViPR) (Via the Solutions Architect aka Michael Letschin aka @mletschin)
Object Paucity (Via Martin Glassborow aka @storagebod)
Object Storage: You can call it NAS 2.0! (Via Enrico Signoretti aka @esignoretti)
Object storage is the platform, gateways are the solution (Via Enrico Signoretti aka @esignorettii)
Files Vs. Objects: the difference is in the value (Via Enrico Signoretti aka @esignorettii)
Conversation with Justin Stottlemyer of Shutterfly and object storage discussion (Pod cast)

cloud object storage

Cloud conversations: AWS EBS, Glacier and S3 overview
Ceph Day Amsterdam 2012 (Object and cloud storage)
Ceph Day in Amsterdam and Sage Weil on Object Storage (Podcast)
Mr. Backup (Curtis Preston) goes back to Ceph School (Video and Podcast)
AWS (Amazon) storage gateway, first, second and third impressions

cloud object storage

Where to learn more

The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

Click here to view (and hear) more content including cloud and object storage fundamentals.
Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
Via InfoStor: Cloud Storage Concerns, Considerations and Trends
Via InfoStor: Object Storage Is In Your Future
Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
Cloud conversations: AWS EBS, Glacier and S3 overview
AWS (Amazon) storage gateway, first, second and third impressions

Videos and podcasts at storageio.tv also available via Applie iTunes.

Human Face of Big Data
Human Face of Big Data (Book review)

Seven Databases in Seven weeks
Seven Databases in Seven Weeks (Book review)

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

Wrap up and summary

Object and cloud storage are in your future, the questions are when, where, with what and how among others. Watch for more content and links to be added here as well as on the main object storage center page including posts, presentations, pod casts, perspectives along with services and product solutions profiles.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Software Defined, Bulk, Cloud, Scale Out, Object Storage Fundamentals

Cloud, Bulk, Scale-Out, Object Storage Fundamentals

Welcome to the Cloud, Big Data, Software Defined, scale-out, Bulk and Object Storage Fundamentals page.

This page contains various resources, tips, essential topics pertaining to Software Defined, scale-out, Cloud, Bulk and Object and blob Storage Fundamentals. Other resources pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage include:

There are various types of cloud, bulk and object storage including public services such as Amazon Web Services (AWS) Simple Storage Service (S3), Google, Microsoft Microsoft Azure, IBM Softlayer, Rackspace among many others. There are also solutions for hybrid and private deployment from Cisco, Cloudian, Fujifilm, DDN, Dell EMC, Fujitsu, HDS, HPE, IBM, NetApp, Noobaa, OpenStack, Quantum, Rackspace, Scality, Seagate, Spectra, Storpool, Suse, Swift and WD among others.

Cloud products and services among others, along with associated data infrastructures including object storage, file systems, repositories and access methods are at the center of bulk, big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.

Cloud Object Storage Fundamentals Access and Architectures

There are many facets to object storage including technology implementation, products, services, access and architectures for various applications and use scenarios.

    • Project or Account – Top of the hierarchy that can represent the owner or billing information for a service that where buckets are also attached.
    • Region – Location where data is stored that can include one or more data centers also known as Availability Zones.

AWS S3 Cross region replication
Moving and Replicating Buckets/Containers, Subfolders and Objects

    • Availability Zone (AZ) or data center or server that implement durability and accessibility for availability within a region.

AWS Regions and Availability Zones AZs
Example of Regions and Availability Zones (AZs)

    • Bucket or Container – Where objects or sub-folders containing objects are attached and accessed.

Object storage fundamentals sddc and cloud software defined

    • Sub-folder – While object storage can be located in a flat namespace for commonality and organization some solutions and service support the notion of sub-folder that resemble traditional directory hierarchy.
    • Object – Byte (or bit) stream that can be as small as one byte to as large as several Tbytes (some solutions and services support up to 5TByte sized objects). The object contains whatever data in any organization along with metadata. Different solutions and services support from a couple hundred KBytes of meta-data to Mbytes worth of meta-data. Regarding what can be stored in an object, anything from files, videos, images, virtual disks (VMDKs, VHDX), ZIP or tar files, backup and archive save sets, executable images or ISO’s, anything you want.
    • End-point – Where or what your software, application or tool and utilities along with gateways attach to for accessing buckets and objects.

 

object storage fundamentals, sddc and cloud storage example

A common theme for object storage is flexibility, along with scaling (performance, availability, capacity, economics) along with extensibility without compromise or complexity. From those basics, there are many themes and variations from how data is protected (RAID or no RAID, hardware or software), deployed as a service or as tin wrapped software (an appliance), optimized for archiving or video serving or other applications.

Many facets of cloud and object storage access

One aspect of object and cloud storage is accessing or using object methods including application programming interfaces (API’s) vs. traditional block (LUN) or NAS (file) based approaches. Keep in mind that many object storage systems, software, and services support NAS file-based access including NFS, CIFS, HDFS  among others for compatibility and ease of use.

Likewise various API’s can be found across different object solutions, software or services including Amazon Web Services (AWS) Simple Storage Service (S3) HTTP REST based, among others. Other API’s will vary by specific vendor or product however can include IOS (e.g. Apple iPhone and iPad), WebDav, FTP, JSON, XML, XAM, CDMI, SOAP, and DICOM among others. Another aspect of object and cloud storage are expanded  and dynamic metadata.

While traditional file systems and NAS have simple or fixed metadata, object and cloud storage systems, services and solutions along with some scale-out file systems have ability to support user defined metadata. Specific systems, solutions, software, and services will vary on the amount of metadata that could range on the low-end from 100s of KBytes  to tens or more Mbytes.

cloud object storage

Where to learn more

The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

Click here to view software defined, bulk, cloud and object storage trend news.


StorageIO Founder Greg Schulz: File Services on Object Storage with HyperFile

Via InfoStor: Object Storage Is In Your Future
Via FujiFilm IT Summit: Software Defined Data Infrastructures (SDDI) and Hybrid Clouds
Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
Via InfoStor: Cloud Storage Concerns, Considerations and Trends
Via InfoStor: Object Storage Is In Your Future
Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
Cloud conversations: AWS EBS, Glacier and S3 overview
AWS (Amazon) storage gateway, first, second and third impressions
Cloud and Virtual Data Storage Networking (CRC Book)
Via ChannelPartnersOnline: Selling Software-Defined Storage: Not All File Systems Are the Same
Via ITProPortal: IBM kills off its first cloud storage platform
Via ITBusinessEdge: Time to Rein in Cloud Storage
Via SerchCloudStorge: Ctera Networks’ file-sharing services gain intelligent cache
Via StorageIOblog: Who Will Be At Top Of Storage World Next Decade?

Videos and podcasts at storageio.tv also available via Applie iTunes.

Human Face of Big Data
Human Face of Big Data (Book review)

Seven Databases in Seven weeks
Seven Databases in Seven Weeks (Book review)

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

Wrap up and summary

Object and cloud storage are in your future, the questions are when, where, with what and how among others.

Watch for more content and links to be added here soon to this object storage center page including posts, presentations, pod casts, polls, perspectives along with services and product solutions profiles.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Cloud and Object storage are in your future, what are some questions?

Cloud and Object storage are in your future, what are some questions?

server storage I/O trends

IMHO there is no doubt that cloud and object storage are in your future, what are some questions?

Granted, what type of cloud and object storage or service along with for work or entertainment are some questions.

Likewise, what are your cloud and object storage concerns (assuming you already have heard the benefits)?

Some other questions include when, where for different applications workload needs, as well as how and with what among others.

Keep in mind that there are many aspects to cloud storage and they are not all object, likewise, there are many facets to object storage.

Recently I did a piece over at InfoStor titled Cloud Storage Concerns, Considerations and Trends that looks at the above among other items including:

  • Is cloud storage cheaper than traditional storage?
  • How do you access cloud object storage from legacy block and file applications?
  • How do you implement on-site cloud storage?
  • Is enterprise file sync and share (EFSS) safe and secure?
  • Does cloud storage need to be backed up and protected?
  • What geographic location requirements or regulations apply to you?

When it comes to cloud computing and, in particular, cloud storage, context matters. Conversations are necessary to discuss concerns, as well as discuss various considerations, options and alternatives. People often ask me questions about the best cloud storage to use, concerns about privacy, security, performance and cost.

Some of the most common cloud conversations topics involve context :

  • Public, private or hybrid cloud; turnkey subscription service or do it yourself (DIY)?
  • Storage, compute server, networking, applications or development tools?
  • Storage application such as file sync and share like Dropbox?
  • Storage resources such as table, queues, objects, file or block?
  • Storage for applications in the cloud, on-site or hybrid?

Continue reading Cloud Storage Concerns, Considerations and Trends over at InfoStor.

Where To Learn More

Additional related content can be found at:

What This All Means

As I mentioned above, cloud and object storage are in your future, granted your future may not rely on just cloud or object storage. Take a few minutes to check out some of the conversation topics, tips and trends in my piece over at InfoStor Cloud Storage Concerns, Considerations and Trends along with more material at www.objectstoragecenter.com.

Btw, what are your questions, comments, concerns, claims or caveats as part of cloud and object storage conversations?

Ok, nuff said, for now…

Cheers
Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, vSAN and VMware vExpert. Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved

Posted on

April 2015 Server StorageIO Update Newsletter

Volume 15, Issue IV

Hello and welcome to this April 2015 Server and StorageIO update newsletter.

This months newsletter has a focus on cloud and object storage for bulk data, unstructured data, big data, archiving among other scenarios.

Enjoy this edition of the Server and StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and Podcasts along with in the news commentary appearing soon.

Storage I/O trends

StorageIOblog posts

April StorageIOblog posts include:

View other recent as well as past blog posts here

April Newsletter Feature Theme
Cloud and Object Storage Fundamentals

There are many facets to object storage including technology implementation, products, services, access and architectures for various applications and use scenarios. The following is a short synopsis of some basic terms and concepts associated with cloud and object storage.

Common cloud and object storage terms

  • Account or project – Top of the hierarchy that represent owner or billing information for a service that where buckets are also attached.
  • Availability Zone (AZ) can be rack of servers and storage or data center where data is spread across for storage and durability.

    • AWS regions and availability zones (AZ)
    Example of some AWS Regions and AZ’s

  • Bucket or Container – Where objects or sub-folders containing objects are attached and accessed. Note in some environments such as AWS S3 you can have sub-folders in a bucket.
  • Connector or how your applications access the cloud or object storage such as via an API, S3, Swift, Rest, CDMI, Torrent, JSON, NAS file, block of other access gateway or software.
  • Durability – Data dispersed with copies in multiple locations to survive failure of storage or server hardware, software, zone or even region. Availability = Access + Durability.
  • End-point – Where or what your software, application or tool and utilities or gateways attach to for accessing buckets and objects.
  • Ephemeral – Temporary or non-persistent
  • Eventual consistency – Data is eventually made consistency, think in terms of asynchronous or deferred writes where there is a time lag vs. synchronous or real-time updates.
  • Immutable – Persistent, non-altered or write once read many copy of data. Objects generally are not updated, rather new objects created.

    • Object storage and cloud
    Via Cloud Virtual Data Storage (CRC)

  • Object – Byte (or bit) stream that can be as small as one byte to as large as several TBytes (some solutions and services support up to 5TByte sized objects). The object contains what ever data in any organization along with meta data. Different solutions and services support from a couple hundred KBytes of meta-data to MBytes worth of meta-data. In terms of what can be stored in an object, anything from files, videos, images, virtual disks (VMDK’s, VHDX), ZIP or tar files, backup and archive save sets, executable images or ISO’s, anything you want.
  • OPS – Objects per second or how many objects accessed similar to a IOP. Access includes gets, puts, list, head, deletes for a CRUD interface e.g. Created, Read, Update, Delete.
  • Region – Location where data is stored that can include one or more data centers also known as Availability Zones.
  • Sub-folder – While object storage can be accessed in a flat name space for commonality and organization some solutions and service support the notion of sub-folder that resemble traditional directory hierarchy.

Learn more in Cloud Virtual Storage Networking (CRC) and www.objectstoragecenter.com

Storage I/O trends

OpenStack Manila (e.g. Folders and Files)

AWS recently announced their new cloud based Elastic File Storage (EFS) to compliment their existing Elastic Block Storage (EBS) offerings. However are you aware of what is going on with cloud files within OpenStack?

For those who are familiar with OpenStack or simply talk about it and Swift object storage, or perhaps Cinder block storage, are you aware that there is also a file (NAS or Network Attached Storage) component called Manila?

In concept Manila should provide a similar capability to what AWS has recently announce with their Elastic File Service (EFS), or depending on your perspective, perhaps the other way around. If you are familiar and have done anything with Manila what are your initial thoughts and perspectives.

What this all means

People routinely tell me this is the most exciting and interesting times ever in servers, storage, I/O networking, hardware, software, backup or data protection, performance, cloud and virtual or take your pick too which I would not disagree.

However, for the past several years (no, make that decade), there is new and more interesting things including in adjacent areas.

I predict that at least for the next few years (no, make that decades), we will continue to see plenty of new and interesting things, questions include.

However, what’s applicable to you and your environment vs. simply fun and interesting to watch?

Ok, nuff said, for now

Cheers gs

 

In This Issue

  • Industry Trends Perspectives News
  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts
  • Events and Webinars
  • StorageIOblog posts
  • Server StorageIO Lab reports
  • Resources and Links

    • Industry News and Activity

    Recent Industry news and activity

    View other recent industry activity here

    StorageIO Commentary in the news

    StorageIO news (image licensed for use from Shutterstock by StorageIO)
    Recent Server StorageIO commentary and industry trends perspectives about news, activities and announcements.

    CyberTrend: Comments on Software Defined Data Center and Virtualization

    View more trends comments here

    StorageIO Tips and Articles

    Check out these resources and links on server storage I/O performance and benchmarking tools. View more tips and articles here

    Various Industry Events

    EMCworld – May 4-6 2015 (Las Vegas)

    Interop – April 29 2015 (Las Vegas)
    Presenting
    Smart Shopping for Your Enterprise Storage Strategy

    View other recent and upcoming events here

    Webinars


    BrightTalk Webinar – June 23 2015
    Server Storage I/O Innovation Update

    View other webinars here

    Videos and Podcasts

    Data Protection Gumbo Podcast
    Protect Preserve and Serve Data

    In this episode, Greg Schulz is a guest on Data Protection Gumbo hosted by Demetrius Malbrough(@dmalbrough). The conversation covers various aspects of data protection which has a focus of protect preserve and serve information, applications and data across different environments and customer segments.

    While we discuss enterprise and SMB data protection, we also talk about trends from Mobile to the cloud among many others tools, technologies and techniques. Check out the podcast here.

    Springtime in Kentucky
    With Kendrick Coleman of EMCcode
    Cloud Object Storage S3motion and more

    In this episode, @EMCcode (Part of EMC) developer advocate Kendrick Coleman (@KendrickColeman) joins me (e.g. Greg Schulz) for a conversation.

    Conversation covers what is EMCcode, EMC Federation, Cloud Foundry, clouds, object storage, buckets, containers, objects, node.js, Docker, OpenStack, AWS S3, micro services, and the S3motion tool Kendrick developed.

    S3motion is a good tool to have in your server storage I/O tool box for working with cloud and object storage along with others such as Cloudberry, S3fs, Cyberduck, S3 browser among many others. You can get S3motion for free from git hub here Check out the companion blog post for this podcast here.

    StorageIO podcast’s are also available via Server Storage I/O audio podcastServer Storage I/O video & at StorageIO.tv

    From StorageIO Labs

    Research, Reviews and Reports

    AWS S3 Cross-Region Replication

    AWS S3 Cross region replication
    Moving and Replicating Buckets/Containers, Sub folders and Objects (Click on Image to read about AWS Cross-Region Replication)

    View other StorageIO lab review reports here

    Resources and Links

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    S3motion Buckets Containers Objects AWS S3 Cloud and EMCcode

    Storage I/O trends

    S3motion Buckets Containers Objects AWS S3 Cloud and EMCcode

    It’s springtime in Kentucky and recently I had the opportunity to have a conversation with Kendrick Coleman to talk about S3motion, Buckets, Containers, Objects, AWS S3, Cloud and Object Storage, node.js, EMCcode and open source among other related topics which are available in a podcast here, or video here and available at StorageIO.tv.

    In this Server StorageIO industry trends perspective podcast episode, @EMCcode (Part of EMC) developer advocate Kendrick Coleman (@KendrickColeman) joins me for a conversation. Our conversation spans spring-time in Kentucky (where Kendrick lives) which means Bourbon and horse racing as well as his blog (www.kendrickcoleman.com).

    Btw, in the podcast I refer to Captain Obvious and Kendrick’s beard, for those not familiar with who or what @Captainobvious is that is made reference to, click here to learn more.


    @Kendrickcoleman     & @Captainobvious

    What about Clouds Object Storage Programming and other technical stuff?

    Of course we also talk some tech including what is EMCcode, EMC Federation, Cloud Foundry, clouds, object storage, buckets, containers, objects, node.js, Docker, Openstack, AWS S3, micro services, and the S3motion tool that Kendrick developed.

    Cloud and Object Storage Access
    Click to view video

    Kendrick explains the motivation behind S3motion along with trends in and around objects (including GET, PUT vs. traditional Read, Write) as well as programming among related topic themes and how context matters.

    S3motion for AWS S3 Google and object storage
    Click to listen to podcast

    I have used S3motion for moving buckets, containers and objects around including between AWS S3, Google Cloud Storage (GCS) and Microsoft Azure as well as to/from local. S3motion is a good tool to have in your server storage I/O tool box for working with cloud and object storage along with others such as Cloudberry, S3fs, Cyberduck, S3 browser among many others.

    You can get S3motion free from git hub here.

    Amazon Web Services AWS

    Where to learn more

    Here are some links to learn more about AWS S3, Cloud and Object Storage along with related topics

    Also available on

    What this all means and wrap-up

    Context matters when it comes to many things particular about objects as they can mean different things. Tools such as S3motion make it easy for moving your buckets or containers along with objects from one cloud storage system, solution or service to another. Also check out EMCcode to see what they are doing on different fronts from supporting new and greenfield development with Cloud Foundry and PaaS to Openstack to bridging current environments to the next generation of platforms. Also check out Kendricks blog site as he has a lot of good technical content as well as some other fun stuff to learn about. Look forward to having Kendrick on as a guest again soon to continue our conversations. In the meantime, check out S3motion to see how it can fit into your server storage I/O tool box.

    Ok, nuff said, for now..

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    SNIA announces Cloud Data Management Initiative (CDMI) V1.1

    SNIA announces Cloud Data Management Initiative (CDMI) V1.1

    In case you missed it, the Storage Networking Industry Association (SNIA) recently released their version 1.1 of its Cloud Data Management Interface (CDMI) specification.

    Highlights of CDMI version 1.1 include:

  • New functionality to ease CDMI implementation with other cloud API’s (e.g. AWS S3, OpenStack Swift, etc.)
  • Expanded cloud data services along with backwards compatible to earlier versions among other enhancements.

    • Check out the full specification here.

    Speaking of SNIA and CDMI, check out this pod cast post of CDMI in a conversation with Wayne Adams and David Dale of SNIA.

    Ok, nuff said

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Seagate Kinetic Cloud and Object Storage I/O platform (and Ethernet HDD)

    Storage I/O trends

    Seagate Kinetic Cloud and Object Storage I/O platform

    Seagate announced today their Kinetic platform and drive designed for use by object API accessed storage including for cloud deployments. The Kinetic platform includes Hard Disk Drives (HDD) that feature 1Gb Ethernet (1 GbE) attached devices that speak object access API or what Seagate refers to as a key / value.

    Seagate Kinetic architecture

    What is being announced with Seagate Kinetic Cloud and Object (Ethernet HDD) Storage?

    • Kinetic Open Storage Platform – Ethernet drives, key / value (object access) API, partner software
    • Software developer’s kits (SDK) – Developer tools, documentation, drive simulator, code libraries, code samples including for SwiftStack and Riak.
    • Partner ecosystem

    What is Kinetic?

    While it has 1 GbE ports, do not expect to be able to use those for iSCSI or NAS including NFS, CIFS or other standard access methods. Being Ethernet based, the Kinetic drive only supports the key value object access API. What this means is that applications, cloud or object stacks, key value and NoSQL data repositories, or other software that adopt the API can communicate directly using object access.

    Seagate Kinetic storage

    Internal, the HDD functions as a normal drive would store and accessing data, the object access function and translation layer shifts from being in an Object Storage Device (OSD) server node to inside the HDD. The Kinetic drive takes on the key value API personality over 1 GbE ports instead of traditional Logical Block Addressing (LBA) and Logical Block Number (LBN) access using 3g, 6g or emerging 12g SAS or SATA interfaces. Instead Kinetic drives respond to object access (aka what Seagate calls key / value) API commands such as Get, Put among others. Learn more about object storage, access and clouds at www.objectstoragecenter.com.

    Storage I/O trends

    Some questions and comments

    Is this the same as what was attempted almost a decade ago now with the T10 OSD drives?

    Seagate claims no.

    What is different this time around with Seagate doing a drive that to some may vaguely resemble the predecessor failed T10 OSD approach?

    Industry support for object access and API development have progressed from an era of build it and they will come thinking, to now where the drives are adapted to support current cloud, object and key value software deployment.

    Wont 1GbE ports be too slow vs. 12g or 6g or even 3g SAS and SATA ports?

    Keep in mind those would be apples to oranges comparisons based on the protocols and types of activity being handled. Kinetic types of devices initially will be used for large data intensive applications where emphasis is on storing or retrieving large amounts of information, vs. low latency transactional. Also, keep in mind that one of the design premises is to keep cost low, spread the work over many nodes, devices to meet those goals while relying on server-side caching tools.

    Storage I/O trends

    Does this mean that the HDD is actually software defined?

    Seagate or other HDD manufactures have not yet noticed the software defined marketing (SDM) bandwagon. They could join the software defined fun (SDF) and talk about a software defined disk (SDD) or software defined HDD (SDHDD) however let us leave that alone for now.

    The reality is that there is far more software that exists in a typical HDD than what is realized. Sure some of that is packaged inside ASICs (Application Specific Integrated Circuits) or running as firmware that can be updated. However, there is a lot of software running in a HDD hence the need for power yet energy-efficient processors found in those devices. On a drive per drive basis, you may see a Kinetic device consume more energy vs. other equivalence HDDs due to the increase in processing (compute) needed to run the extra software. However that also represents an off-load of some work from servers enabling them to be smaller or do more work.

    Are these drives for everybody?

    It depends on if your application, environment, platform and technology can leverage them or not. This means if you view the world only through what is new or emerging then these drives may be for all of those environments, while other environments will continue to leverage different drive options.

    Object storage access

    Does this mean that block storage access is now dead?

    Not quite, after all there is still some block activity involved, it is just that they have been further abstracted. On the other hand, many applications, systems or environments still rely on block as well as file based access.

    What about OpenStack, Ceph, Cassandra, Mongo, Hbase and other support?

    Seagate has indicated those and others are targeted to be included in the ecosystem.

    Seagate needs to be careful balancing their story and message with Kinetic to play to and support those focused on the new and emerging, while also addressing their bread and butter legacy markets. The balancing act is communicating options, flexibility to choose and adopt the right technology for the task without being scared of the future, or clinging to the past, not to mention throwing the baby out with the bath water in exchange for something new.

    For those looking to do object storage systems, or cloud and other scale based solutions, Kinetic represents a new tool to do your due diligence and learn more about.

    Ok, nuff said (for now)

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    EMC ViPR software defined object storage part III

    Storage I/O trends

    This is part III in a series of posts pertaining to EMC ViPR software defined storage and object storage. You can read part I here and part II here.

    EMCworld

    More on the object opportunity

    Other object access includes OpenStack storage part Swift, AWS S3 HTTP and REST API access. This also includes ViPR supporting EMC Atmos, VNX and Isilon arrays as southbound persistent storage in addition.

    object storage
    Object (and cloud) storage access example

    EMC is claiming that over 250 VNX systems can be abstracted to support scaling with stability (performance, availability, capacity, economics) using ViPR. Third party storage will be supported along with software such as OpenStack Swift, Ceph and others running on commodity hardware. Note that EMC has some history with object storage and access including Centera and Atmos. Visit the micro site I have setup called www.objectstoragecenter.com and watch for more content to be updated and added there.

    More on the ViPR control plane and controller

    ViPR differs from some others in that it does not sit in the data path all the time (e.g. between application servers and storage systems or cloud services) to cut potential for bottlenecks.

    ViPR architecture

    Organizations that can use ViPR include enterprise, SMB, CSP or MSP and hosting sites. ViPR can be used in a control mode to leverage underlying storage systems, appliances and services intelligence and functionality. This means ViPR can be used to complement as oppose to treat southbound or target storage systems and services as dumb disks or JBOD.

    On the other hand, ViPR will also have a suite of data services such as snapshot, replication, data migration, movement, tiering to add value for when those do not exist. Customers will be free to choose how they want to use and deploy ViPR. For example leveraging underlying storage functionality (e.g. lightweight model), or in a more familiar storage virtualization model heavy lifting model. In the heavy lifting model more work is done by the virtualization or abstraction software to create an added value, however can be a concern for bottlenecks depending how deployed.

    Service categories

    Software defined, storage hypervisor, virtual storage or storage virtualization?

    Most storage virtualization, storage hypervisors and virtual storage solutions that are hardware or software based (e.g. software defined) implemented what is referred to as in band. With in band the storage virtualization software or hardware sits between the applications (northbound) and storage systems or services (southbound).

    While this approach can be easier to carry out along with add value add services, it can also introduce scaling bottlenecks depending on implementations. Examples of in band storage virtualization includes Actifio, DataCore, EMC VMAX with third-party storage, HDS with third-party storage, IBM SVC (and their V7000 Storwize storage system based on it) and NetApp Vseries among others. An advantage of in band approaches is that there should not need to be any host or server-side software requirements and SAN transparency.

    There is another approach called out-of-band that has been tried. However pure out-of-band requires a management system along with agents, drivers, shims, plugins or other software resident on host application servers.

    fast path control path
    Example of generic fast path control path model

    ViPR takes a different approach, one that was seen a few years ago with EMC Invista called fast path, control path that for the most part stays out of the data path. While this is like out-of-band, there should not be a need for any host server-side (e.g. northbound) software. By being a fast path control path, the virtualization or abstraction and management functions stay out of the way for data being moved or work being done.

    Hmm, kind of like how management should be, there to help when needed, out-of-the-way not causing overhead other times ;).

    Is EMC the first (even with Invista) to leverage fast path control path?

    Actually up until about a year or so ago, or shortly after HP acquired 3PAR they had a solution called Storage Virtualization Services Platform (SVPS) that was OEMd from LSI (e.g. StorAge). Unfortunately, HP decided to retire that as opposed to extend its capabilities for file and object access (northbound) as well as different southbound targets or destination services.

    Whats this northbound and southbound stuff?

    Simply put, think in terms of a vertical stack with host servers (PMs or VMs) on the top with applications (and hypervisors or other tools such as databases) on top of them (e.g. north).

    software defined storage
    Northbound servers, southbound storage systems and cloud services

    Think of storage systems, appliances, cloud services or other target destinations on the bottom (or south). ViPR sits in between providing storage services and management to the northbound servers leveraging the southbound storage.

    What host servers can VIPR support for serving storage?

    VIPR is being designed to be server agnostic (e.g. virtual or physical), along with operating system agnostic. In addition VIPR is being positioned as capable of serving northbound (e.g. up to application servers) block, file or object as well as accessing southbound (e.g. targets) block, file and object storage systems, file systems or services.

    Note that a difference between earlier similar solutions from EMC have been either block based (e.g. Invista, VPLEX, VMAX with third-party storage), or file based. Also note that this means VIPR is not just for VMware or virtual server environments and that it can exist in legacy, virtual or cloud environments.

    ViPR image

    Likewise VIPR is intended to be application agnostic supporting little data, big data, very big data ( VBD) along with Hadoop or other specialized processing. Note that while VIPR will support HDFS in addition to NFS and CIFS file based access, Hadoop will not be running on or in the VIPR controllers as that would live or run elsewhere.

    How will VIPR be deployed and licensed?

    EMC has indicated that the VIPR controller will be delivered as software that installs into a virtual appliance (e.g. VMware) running as a virtual machine (VM) guest. It is not clear when support will exist for other hypervisors (e.g. Microsoft Hyper-V, Citrix/XEN, KVM or if VMware vSphere with vCenter or simply on ESXi free version). As of the announcement pre briefing, EMC had not yet finalized pricing and licensing details. General availability is expected in the second half of calendar 2013.

    Keep in mind that the VIPR controller (software) runs as a VM that can be hosted on a clustered hypervisor for HA. In addition, multiple VIPR controllers can exist in a cluster to further enhance HA.

    Some questions to be addressed among others include:

    • How and where are IOs intercepted?
    • Who can have access to the APIs, what is the process, is there a developers program, SDK along with resources?
    • What network topologies are supported local and remote?
    • What happens when JBOD is used and no advanced data services exist?
    • What are the characteristics of the object access functionality?
    • What if any specific switches or data path devices and tools are needed?
    • How does a host server know to talk with its target and ViPR controller know when to intercept for handling?
    • Will SNIA CDMI be added and when as part of the object access and data services capabilities?
    • Are programmatic bindings available for the object access along with support for other APIs including IOS?
    • What are the performance characteristics including latency under load as well as during a failure or fault scenario?
    • How will EMC place Vplex and its caching model on a local and wide area basis vs. ViPR or will we see those two create some work together, if so, what will that be?

    Bottom line (for now):

    Good move for EMC, now let us see how they execute including driving adoption of their open APIs, something they have had success in the past with Centera and other solutions. Likewise, let us see what other storage vendors become supported or add support along with how pricing and licensing are rolled out. EMC will also have to articulate when and where to use ViPR vs. VPLEX along with other storage systems or management tools.

    Additional related material:
    Are you using or considering implementation of a storage hypervisor?
    Cloud and Virtual Data Storage Networking (CRC)
    Cloud conversations: Public, Private, Hybrid what about Community Clouds?
    Cloud, virtualization, storage and networking in an election year
    Does software cut or move place of vendor lock-in?
    Don’t Use New Technologies in Old Ways
    EMC VPLEX: Virtual Storage Redefined or Respun?
    How many degrees separate you and your information?
    Industry adoption vs. industry deployment, is there a difference?
    Many faces of storage hypervisor, virtual storage or storage virtualization
    People, Not Tech, Prevent IT Convergence
    Resilient Storage Networks (Elsevier)
    Server and Storage Virtualization Life beyond Consolidation
    Should Everything Be Virtualized?
    The Green and Virtual Data Center (CRC)
    Two companies on parallel tracks moving like trains offset by time: EMC and NetApp
    Unified storage systems showdown: NetApp FAS vs. EMC VNX
    backup, restore, BC, DR and archiving
    VMware buys virsto, what about storage hypervisor’s?
    Who is responsible for vendor lockin?

    Ok, nuff said (for now)

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    EMC ViPR software defined object storage part II

    Storage I/O trends

    This is part II in a series of posts pertaining to EMC ViPR software defined storage and object storage. You can read part I here and part III here.

    EMCworld

    Some questions and discussion topics pertaining to ViPR:

    Whom is ViPR for?

    Organizations that need to scale with stability across EMC, third-party or open storage software stacks and commodity hardware. This applies to large and small enterprise, cloud service providers, managed service providers, virtual and cloud environments/

    What this means for EMC hardware/platform/systems?

    They can continue to be used as is, or work with ViPR or other deployment modes.

    Does this mean EMC storage systems are nearing their end of life?

    IMHO for the most part not yet, granted there will be some scenarios where new products will be used vs. others, or existing ones used in new ways for different things.

    As has been the case for years if not decades, some products will survive, continue to evolve and find new roles, kind of like different data storage mediums (e.g. ssd, disk, tape, etc).

    How does ViPR work?

    ViPR functions as a control plane across the data and storage infrastructure supporting both north and southbound. northbound refers to use from or up to application servers (physical machines PM and virtual machines VMs). southbound refers target or destination storage systems. Storage systems can be traditional EMC or third-party (NetApp mentioned as part of first release), appliances, just a bunch of disks (JBOD) or cloud services.

    Some general features and functions:

    • Provisioning and allocation (with automation)
    • Data and storage migration or tiering
    • Leverage scripts, templates and workbooks
    • Support service categories and catalogs
    • Discovery, registration of storage systems
    • Create of storage resource pools for host systems
    • Metering, measuring, reporting, charge or show back
    • Alerts, alarms and notification
    • Self-service portal for access and provisioning

    ViPR data plane (adding data services and value when needed)

    Another part is the data plane for implementing data services and access. For block and file when not needed, ViPR steps out-of-the-way leveraging the underlying storage systems or services.

    object storage
    Object storage access

    When needed, the ViPR data plane can step in to add added services and functionality along with support object based access for little data and big data. For example, Hadoop Distributed File System (HDFS) services can support northbound analytic software applications running on servers accessing storage managed by ViPR.

    Continue reading in part III of this series here including how ViPR works, who it is for and more analysis.

    Ok, nuff said (for now)

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    EMC ViPR virtual physical object and software defined storage (SDS)

    Storage I/O trends

    Introducing EMC ViPR

    This is the first in a three part series, read part II here, and part III here.

    During the recent EMCworld event in Las Vegas among other things, EMC announced ViPR (read announcement here) . Note that this ViPR is not the same EMC Viper project from a few years ago that was focused on data footprint reduction (DFR) including dedupe. ViPR has been in the works for a couple of years taking a step back rethinking how storage is can be used going forward.

    EMCworld

    ViPR is not a technology creation developed in a vacuum instead includes customer feedback, wants and needs. Its core themes are extensible, open and scalable.

    EMCworld

    On the other hand, ViPR addresses plenty of buzzword bingo themes including:

    • Agility, flexibility, multi-tenancy, orchestration
    • Virtual appliance and control plane
    • Data services and storage management
    • IT as a Service (ITaaS) and Infrastructure as a Service (IaaS)
    • Scaling with stability without compromise
    • Software defined storage
    • Public, private, hybrid cloud
    • Big data and little data
    • Block, file and object storage
    • Control plane and data plane
    • Storage hypervisor, virtualization and virtual storage
    • Heterogeneous (third-party) storage support
    • Open API and automation
    • Self-service portals, service catalogs

    Buzzword bingo

    Note that this is essentially announcing the ViPR product and program initiative with general availability slated for second half of 2013.

    What is ViPR addressing?

    IT and data infrastructure (server, storage, IO and networking hardware, software) challenges for traditional, virtual and cloud environments.

    • Data growth, after all, there is no such thing as an information recession with more data being generated, moved, processed, stored and retained for longer periods of time. Then again, people and data are both getting larger and living longer, for both little data and big data along with very big data.

    • Overhead and complexities associated with managing and using an expanding, homogenous (same vendor, perhaps different products) or heterogeneous (different vendors and products) data infrastructure across cloud, virtual and physical, legacy and emerging. This includes add, changes or moves, updates and upgrades, retirement and replacement along with disposition, not to mention protecting data in an expanding footprint.

      • road to cloud

    • Operations and service management, fault and alarm notification, resolution and remediation, rapid provisioning, removing complexity and cost of doing things vs. simply cutting cost and compromising service.

    EMC ViPR

    What is this software defined storage stuff?

    There is the buzzword aspect, and then there is the solution and business opportunity.

    First the buzzword aspect and bandwagon:

    • Software defined marketing (SDM) Leveraging software defined buzzwords.
    • Software defined data centers (SDDC) Leveraging software to derive more value from hardware while enabling agility, flexibility, and scalability and removing complexity. Think the Cloud and Virtual Data Center models including those from VMware among others.
    • Software defined networking (SDN) Rather than explain, simply look at Nicira that VMware bought in 2012.
    • Software defined storage (SDS) Storage software that is independent of any specific hardware, which might be a bit broad, however it is also narrower than saying anything involving software.
    • Software defined BS (SDBS) Something that usually happens as a result when marketers and others jump on a bandwagon, in this case software defined marketing.

    Note that not everything involved with software defined is BS, only some of the marketing spins and overuse. The downside to the software defined marketing and SDBS is the usual reaction of skepticism, cynicism and dismissal, so let us leave the software defined discussion here for now.

    software defined storage

    An example of software defined storage can be storage virtualization, virtual storage and storage hypervisors that are hardware independent. Note that when I say hardware independent, that also means being able to support different vendors systems. Now if you want to have some fun with the software defined storage diehards or purist, tell them that all hardware needs software and all software needs hardware, even if virtual. Further hardware is defined by its software, however lets leave sleeping dogs lay where they rest (at least for now ;)).

    Storage hypervisors were a 2012 popular buzzword bingo topic with plenty of industry adoption and some customer deployment. While 2012 saw plenty of SDM buzz including SDC, SDN 2013 is already seeing an increase including software defined servers, and software defined storage.

    Regardless of what you view of software defined storage, storage hypervisor, storage virtualization and virtual storage is, the primary focus and goal should be addressing business and application needs. Unfortunately, some of the discussions or debates about what is or is not software defined and related themes lose focus of what should be the core goal of enabling business and applications.

    Continue reading in part II of this series here including how ViPR works, who it is for and more analysis.

    Ok, nuff said (for now)

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Welcome to the Cloud Bulk Object Storage Resources Center

    Updated 8/31/19

    Cloud Bulk Big Data Software Defined Object Storage Resources

    server storage I/O trends Object Storage resources

    Welcome to the Cloud, Big Data, Software Defined, Bulk and Object Storage Resources Center Page objectstoragecenter.com.

    This object storage resources, along with software defined, cloud, bulk, and scale-out storage page is part of the server StorageIOblog microsite collection of resources. Software-defined, Bulk, Cloud and Object Storage exist to support expanding and diverse application data demands.

    Other related resources include:

  • Software Defined, Cloud, Bulk and Object Storage Fundamentals
  • Software Defined Data Infrastructure Essentials book (CRC Press)
  • Cloud, Software Defined, Scale-Out, Object Storage News Trends

    • Object storage SDDC SDDI
    Via Software Defined Data Infrastructure Essentials (CRC Press 2017)

    Bulk, Cloud, Object Storage Solutions and Services

    There are various types of cloud, bulk, and object storage including public services such as Amazon Web Services (AWS) Simple Storage Service (S3), Backblaze, Google, Microsoft Azure, IBM Softlayer, Rackspace among many others. There are also solutions for hybrid and private deployment from Cisco, Cloudian, CTERA, Cray, DDN, Dell EMC, Elastifile, Fujitsu, Vantera/HDS, HPE, Hedvig, Huawei, IBM, NetApp, Noobaa, OpenIO, OpenStack, Quantum, Rackspace, Rozo, Scality, Spectra, Storpool, StorageCraft, Suse, Swift, Virtuozzo, WekaIO, WD, among many others.

    Bulk Cloud Object storage SDDC SDDI
    Via Software Defined Data Infrastructure Essentials (CRC Press 2017)

    Cloud products and services among others, along with associated data infrastructures including object storage, file systems, repositories and access methods are at the center of bulk, big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.

    Object Context Matters

    Before discussing Object Storage lets take a step back and look at some context that can clarify some confusion around the term object. The word object has many different meanings and context, both inside of the IT world as well as outside. Context matters with the term object such as a verb being a thing that can be seen or touched as well as a person or thing of action or feeling directed towards.

    Besides a person, place or physical thing, an object can be a software-defined data structure that describes something. For example, a database record describing somebody’s contact or banking information, or a file descriptor with name, index ID, date and time stamps, permissions and access control lists along with other attributes or metadata. Another example is an object or blob stored in a cloud or object storage system repository, as well as an item in a hypervisor, operating system, container image or other application.

    Besides being a verb, an object can also be a noun such as disapproval or disagreement with something or someone. From an IT context perspective, an object can also refer to a programming method (e.g. object-oriented programming [oop], or Java [among other environments] objects and classes) and systems development in addition to describing entities with data structures.

    In other words, a data structure describes an object that can be a simple variable, constant, complex descriptor of something being processed by a program, as well as a function or unit of work. There are also objects unique or with context to specific environments besides Java or databases, operating systems, hypervisors, file systems, cloud and other things.

    The Need For Bulk, Cloud and Object Storage

    There is no such thing as an information recession with more data being generated, moved, processed, stored, preserved and served, granted there are economic realities. Likewise as a society our dependence on information being available for work or entertainment, from medical healthcare to social media and all points in between continues to increase (check out the Human Face of Big Data).

    In addition, people and data are living longer, as well as getting larger (hence little data, big data and very big data). Cloud products and services along with associated object storage, file systems, repositories and access methods are at the center of big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.

    Click here to view (and hear) more content including cloud and object storage fundamentals

    Click here to view software defined, bulk, cloud and object storage trend news

    cloud object storage

    Where to learn more

    The following resources provide additional information about big data, bulk, software defined, cloud and object storage.



    Via InfoStor: Object Storage Is In Your Future
    Via FujiFilm IT Summit: Software Defined Data Infrastructures (SDDI) and Hybrid Clouds
    Via MultiChannel: After ditching cloud business, Verizon inks Virtual Network Services deal with Amazon
    Via MultiChannel: Verizon Digital Media Services now offers integrated Microsoft Azure Storage
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via InfoStor: Cloud Storage Concerns, Considerations and Trends
    Via InfoStor: Object Storage Is In Your Future
    Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
    Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
    Cloud conversations: AWS EBS, Glacier and S3 overview
    AWS (Amazon) storage gateway, first, second and third impressions
    Cloud and Virtual Data Storage Networking (CRC Book)

    View more news, trends and related cloud object storage activity here.

    Videos and podcasts at storageio.tv also available via Applie iTunes.

    Human Face of Big Data
    Human Face of Big Data (Book review)

    Seven Databases in Seven weeks Seven Databases in Seven Weeks (Book review)

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    What This All Means

    Object and cloud storage are in your future, the questions are when, where, with what and how among others.

    Watch for more content and links to be added here soon to this object storage center page including posts, presentations, pod casts, polls, perspectives along with services and product solutions profiles.

    Ok, nuff said, for now.

    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

    Posted on

    Ceph Day in Amsterdam and Sage Weil on Object Storage

    Now also available via

    This is a new episode in the continuing StorageIO industry trends and perspectives pod cast series (you can view more episodes or shows along with other audio and video content here) as well as listening via iTunes or via your preferred means using this RSS feed (https://storageio.com/StorageIO_Podcast.xml)

    StorageIO industry trends cloud, virtualization and big data

    In this episode, I am at the Ceph day in Amsterdam Holland event at the Tobacco Theatre. My guest for this episode is Ceph (Cephalanthera) creator Sage Weil who is also the founder of inktank.com that provides services and support for the open source based Ceph project.

    For those not familiar with Ceph, it is an open source distributed object scale out software platform that can be used for deploying cloud and managed services, general purpose storage for research, commercial, scientific, high performance computing (HPC) or high productivity computing (commercial) along with backup or data protection and archiving destinations.

    During our conversation Sage presents an overview of what Ceph is (e.g. Ceph for non Dummies), where and how it can be used, some history of the project and how it fits in with or provides an alternative to other solutions. Sage also talks about the business or commercial considerations for open source based projects, importance of community and having good business mentors and partners as well as staying busy with his young family.

    If you are a Ceph fan, gain more insight into Sage along with Ceph day sponsors Inktank and 42on. On the other hand, if you new to object storage, open source storage software or cloud storage, listen in to gain perspectives of where technology such as Ceph fits for public, private, hybrid or traditional environments.

    Click here (right-click to download MP3 file) or on the microphone image to listen to the conversation with Sage and myself.

    StorageIO podcast

    Also available via

    Watch (and listen) for more StorageIO industry trends and perspectives audio blog posts pod casts and other upcoming events. Also be sure to heck out other related pod casts, videos, posts, tips and industry commentary at StorageIO.com and StorageIOblog.com.

    Enjoy this episode Ceph Day in Amsterdam with Sage Weil.

    Ok, nuff said.

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2012 StorageIO and UnlimitedIO All Rights Reserved