Tag: bulk

Posted on

Zombie Technology Life after Death Tape Is Still Alive

Zombie Technology Life after Death Tape Is Still Alive

server storage I/O data infrastructure trends

A Zombie Technology is one declared dead yet has Life after Death such as Tape which is still alive.

zombie technology
Image via StorageIO.com (licensed for use from Shutterstock.com)

Tapes Evolving Role

Sure we have heard for decade’s about the death of tape, and someday it will be dead and buried (I mean really dead), no longer used, buried, existing only in museums. Granted tape has been on the decline for some time, and even with many vendors exiting the marketplace, there remains continued development and demand within various data infrastructure environments, including software defined as well as legacy.

data infrastructures

Tape remains viable for some environments as part of an overall memory data storage hierarchy including as a portability (transportable) as well as bulk storage medium.

memory data storage hirearchy classes tiers

Keep in mind that tapes role as a data storage medium also continues to change as does its location. The following table (via Software Defined Data Infrastructure Essentials (CRC Press)) Chapter 10 shows examples of various data movements from source to destination. These movements include migration, replication, clones, mirroring, and backup, copies, among others. The source device can be a block LUN, volume, partition, physical or virtual drive, HDD or SSD, as well as a file system, object, or blob container or bucket. An example of the modes in Table 10.1 include D2D backup from local to local (or remote) disk (HDD or SSD) storage or D2D2D copy from local to local storage, then to the remote.

Mode – Description
D2D – Data gets copied (moved, migrated, replicated, cloned, backed up) from source storage (HDD or SSD) to another device or disk (HDD or SSD)-based device
D2C – Data gets copied from a source device to a cloud device.
D2T – Data gets copied from a source device to a tape device (drive or library).
D2D2D – Data gets copied from a source device to another device, and then to another device.
D2D2T – Data gets copied from a source device to another device, then to tape.
D2D2C   Data gets copied from a source device to another device, then to cloud.
Data Movement Modes from Source to Destination

Note that movement from source to the target can be a copy, rsync, backup, replicate, snapshot, clone, robocopy among many other actions. Also, note that in the earlier examples there are occurrences of tape existing in clouds (e.g. its place) and use changing.  Tip – In the past, “disk” usually referred to HDD. Today, however, it can also mean SSD. Think of D2D as not being just HDD to HDD, as it can also be SSD to SSD, Flash to Flash (F2F), or S2S among many other variations if you prefer (or needed).

Image via Tapestorage.org

For those still interested in tape, check out the Active Archive Alliance recent posts (here), as well as the 2017 Tape Storage Council Memo and State of their industry report (here). While lower end-tape such as LTO (which is not exactly the low-end it was a decade or so ago) continues to evolve, things may not be as persistent for tape at the high-end. With Oracle (via its Sun/StorageTek acquisition) exiting the high-end (e.g. Mainframe focused) tape business, that leaves mainly IBM as a technology provider.

Image via Tapestorage.org

With a single tape device (e.g. drive) vendor at the high-end, that could be the signal for many organizations that it is time to finally either move from tape or at least to LTO (linear tape open) as a stepping stone (e.g. phased migration). The reason not being technical rather business in that many organizations need to have a secondary or competitive offering or go through an exception process.

On the other hand, just as many exited the IBM mainframe server market (e.g. Fujitsu/Amdahl, HDS, NEC), big blue (e.g. IBM) continues to innovate and drive both revenue and margin from those platforms (hardware, software, and services). This leads me to believe that IBM will do what it can to keep its high-end tape customers supported while also providing alternative options.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

What This All Means

I would not schedule the last tape funeral just yet, granted there will continue to be periodic wakes and send off over the coming decade. Tape remains for some environments a viable data storage option when used in new ways, as well as new locations complementing flash SSD and other persistent memories aka storage class memories along with HDD.

Personally, I have been directly tape free for over 14 years. Granted, I have data in some clouds and object storage that may exist on a very cold data storage tier possibly maybe on tape that is transparent to my use. However just because I do not physically have tape, does not mean I do not see the need why others still have to or prefer to use it for different needs.

Also, keep in mind that tape continues to be used as an economic data transport for bulk movement of data for some environments. Meanwhile for those who only want, need or wish tape to finally go away, close your eyes, click your heels together and repeat your favorite tape is not alive chant three (or more) times. Keep in mind that HDDs are keeping tape alive by off loading some functions, while SSDs are keeping HDDs alive handling tasks formerly done by spinning media. Meanwhile, tape can and is still called upon by some organizations to protect or enable bulk recovery for SSD and HDDs even in cloud environments, granted in new different ways.

What this all means is that as a zombie technology having been declared dead for decades yet still live there is life after death for tape, which is still alive, for now.

Ok, nuff said (for now…).

Cheers
Gs

Greg Schulz – Multi-year Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

Software Defined Cloud Bulk Object Storage News Trends

Cloud, Software Defined, Scale-Out, Object Storage News Trends

server storage I/O trends

Updated 1/21/2018

Welcome to the Cloud, Big Data, Software Defined, Bulk and Object Storage News Trends page.

This page contains various links pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage News Trends and resources. Other resources pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage include:

  • www.objectstoragecenter.com
  • Software Defined, Cloud, Bulk and Object Storage Fundamentals
  • Software Defined Data Infrastructure Essentials book (CRC Press)

    • Software defined, Bulk, Cloud and Object Storage exist to support expanding and diverse application data demands. There are various types of cloud, bulk and object storage including public services such as Amazon Web Services (AWS) Simple Storage Service (S3), Google, Microsoft Azure, IBM Softlayer, Rackspace among many others. There are also solutions for hybrid and private deployment from Cisco, DDN, Dell EMC, Fujitsu, HDS, HPE, IBM, NetApp, Noobaa, OpenStack, Quantum, Rackspace, Scality, Seagate, Spectra, Storpool, Suse, Swift and WD among others.

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    Cloud products and services among others, along with associated data infrastructures including object storage, file systems, repositories and access methods are at the center of bulk, big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.



    Via InfoStor: Object Storage Is In Your Future
    Via FujiFilm IT Summit: Software Defined Data Infrastructures (SDDI) and Hybrid Clouds
    Via ChannelPartnersOnline: Selling Software-Defined Storage: Not All File Systems Are the Same
    Via ITProPortal: IBM kills off its first cloud storage platform
    Via MultiChannel: After ditching cloud business, Verizon inks Virtual Network Services deal with Amazon
    Via ITBusinessEdge: Time to Rein in Cloud Storage
    Via BusinessInsider: Kodiak Data Brings Enterprise Storage and Database Services to the Cloud Era with StorageCloud
    Via SerchCloudStorge: Ctera Networks’ file-sharing services gain intelligent cache
    Via StorageIOblog: Who Will Be At Top Of Storage World Next Decade?

    cloud object storage

    The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via CloudComputingAdmin: Cloud Storage Decision Making
    Via Wservernews: Cloud storage considerations
    Via PRNewsWire: Scality Accelerates Multi-Cloud Adoption with New Open-Source Release
    Via EnterpriseStorageForum: Top 10 Tips for Software-Defined Storage Deployment
    Via MultiChannel: Verizon Digital Media Services now offers integrated Microsoft Azure Storage
    Via BroadCstingCable: Quantum’s New StorNext 6 Delivers Unparalleled Combination of Best-in-Class 4K Performance and Advanced Data Management
    Via ShootOnLine: Morro Data Accelerates Media Collaboration With Cache and Sync Appliance
    Via TechCrunch: Backblaze drops download pricing for its B2 storage platform by 60%
    Via PRNewsWire: FileCloud Online Launches an All-In-One File Share, Sync and Backup SaaS Offering in a Challenge to Dropbox and Box

    cloud object storage

    Via HITInfrastructure: Benefits of Object Storage to Health IT Infrastructure
    Via ITBusinessedge: Growing Diversity of Cloud Storage Solutions
    Via Telecompaper: Interoute launches storage zones in Switzerland
    Via TradeCalls:
    Object Storage Market Size, Status and Forecast 2022
    Via OpenStack: Creating object storage container causes user to be logged out, OpenStack Dashboard (Horizon)
    Via RethinkResearch: Nokia claims a 40% cut in cloud DVR Capex, using new Skim Storage
    Via Cloudwards: A Beginner’s Guide on How Cloud Backup With Microsoft Azure Works
    Via ITProPortal: Lack of understanding of data inhibits move to the cloud

    cloud object storage

    Via InfoStor:Cloud Storage Concerns, Considerations and Trends
    Via StreetInsider: IBM Announces New Pricing Model for Cloud Storage and NetApp Object Storage Partnership

    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via ComputerWeekly: Storage 101: Object storage vs block vs file
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via Yahoo Finance: SwiftStack Joins Google Cloud Technology Partner Program
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look

    Via StorageIOblog: HPE Continues Buying Into Server Storage I/O Data Infrastructures
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via TheStack:Google to launch nine new data centres over three new cloud regions
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via DigitalJournal:Global Business Cloud Storage market 2017 share, Segmentation and Industrial Overview forecasts to 2022
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via NextPlatform:How Microsoft Fell Hard for FPGAs
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via NextPlatform:Google Courts Enterprise For Cloud Platform

    cloud object storage

    Via IBM RedBooks: Cloud Object Storage as a
    Service – IBM Cloud Object Storage from Theory to Practice
    Via VentureBeat: Microsoft confirms Azure storage issues around the world
    Via Microsoft Azure: Azure status history
    Via NextPlatform: Windows Server Comes To ARM Chips, But Only For Azure
    Via NextPlatform: ARM And AMD X86 Server Chips Get Mainstream Lift From Microsoft
    Via ITBusinessEdge: Google Expands Enterprise Cloud Service Footprint
    Via ChannelWorld: Google announces three new cloud regions, contract discounts
    Via AWS: AWS Service Health Status
    Via AWS (S3 Feb 28 2017 Port mortem): Summary of the Amazon S3 Service Disruption
    Via SearchCloudStorage: Open source object storage startup OpenIO adds hardware

    cloud object storage

    Via InfoStor: Comments on Cloud Storage Pros and Cons
    Via CRN: StorageCraft Acquires Exablox
    Via TVtechglobal: Signiant CEO: The media industry isnt just one monolithic thing
    Via Computerweekly: Using the cloud as a storage tier
    Via SearchCloudStorage: OpenIO joins object storage cloud scrum
    Via CRN: Dell Partners With Scality On High-Density Object Storage
    Via BroadwayWorld: The World’s Smallest Object Storage Solution, ‘Ceph on Olive’, is Released
    Via SmartDataCollective: Why You Should Be Securing Big Data In The Cloud?
    Via Geekwire: Feds certify Amazon and Microsoft clouds to handle sensitive government data
    Via NewsFactor: SoftNAS Introduces File Services for NetApp StorageGRID
    Via Redmondmag: AWS Cloud Storage Service Turns 10
    Via DropBox Blog: Scaling to exabytes and beyond
    Via Computerworld: Dropbox quits Amazon cloud, takes back 500 PB of data
    Via HPCwire: Seagate Sets Sights on Broader HPC Market

    cloud object storage

    Via Channelnomics.com: Why Samsung’s Joyent buyout could frame its future
    Via GeekyGadgets: Google Drive Cloud Storage Is Now A Little Easier To Organize Thanks To New Tools
    Via ZDnet: Google doubles Cloud Compute local SSD capacity: Now it’s 3TB per VM
    Via AndroidPolice: Barracuda Is Shutting Down The Copy Cloud Storage Service On May 1st
    Via StorageSoup: EMC expands Elastic Cloud (object) Storage
    Via Cloud Review: Access Multiple Cloud Apps from a Unified Interface
    Via CyberSecurity: Cybersecurity and Accessible Storage Trends
    Via eWeek: Cloud Storage Spending Poised for Increase in 2016
    Via InfoStor: Object Storage Is In Your Future
    Via BioScienceTechnology: Six Reasons to Add Object Storage to Your Genomics Lexicon
    Via TechCrunch: Backblaze’s Low-Cost Cloud Storage Service Comes Out Of Beta
    Via CloudComputingAdmin: Cloud Storage Decision Making
    COSbench for cloud and object storage (Various): White Paper (CMU), PDF overview presentation, Get the bits (software) from Github
    Via AWS Blogs: AWS Storage Update – New Lower Cost S3 Storage Option & Glacier Price Reduction
    Via Techcrunch: IBM Acquires Cleversafe To Accelerate Hybrid Cloud Storage Strategy

    cloud object storage

    Via Techcrunch: Developers Can Now Import Data To Google Cloud Storage By Mailing In Their Hard Drives, Tapes And USB Flash Drives
    Via vCloud: VMware vCloud Air Object Storage
    Via SearchStorage: SwiftStack adds new features to object storage software solution
    Via TheVarGuy: Vendor consortium create Ceph advisory board
    Via OpenStack: Whats new in OpenStack Liberty
    Via Microsoft Azure: File access cloud storage in addition to blobs and object storage
    Via AWS: AWS Storage Update – New Lower Cost S3 Storage Option & Glacier Price Reduction
    Via VMware Blogs: Showcasing VMware vCloud Air Object Storage powered by EMC at EMC World!
    Via Venturebeat: Backblaze launches B2 object storage service at 1/4 of the cost of Amazon S3
    Via Techcrunch: Wuala, The Secure P2P Cloud Storage Service Acquired By LaCie, To Shut Down
    Via ChannelworldIN: 13 firms join to open up Ethernet, object-storage to everyone
    Via ITportal: The elephant in the room: The trust deficit issue with public clouds
    Via VCpost: Dropbox latest update: Cloud storage service provider now lets you drag and drop URLs with files
    Via NetworkAsia: HP’s new converged appliance offers a path to hybrid cloud
    Via Techtimes: Google Files For Patent To Search Your Desktop Files, Apps And Cloud Storage

    cloud object storage

    Via CloudTweaks: Questions To Ask Every Cloud Storage Provider
    Via MarketWatch: Linux Foundation Brings Together Industry Leaders to Advance Cloud Object Storage Technologies
    Via CBR: New Kinetic platform targets open source object storage
    Via Techcrunch: Seagate, Toshiba, Scality And Others Launch Kinetic Open Storage Platform For Smart, Low-Overhead Cloud Object Storage
    Via StorageIOblog: July 2015 Newsletter Focus on Content Solution Platforms
    Via ITnewsline: Hitachi Data Systems Announces New Enhancements to the Hitachi Content Platform Portfolio
    Via ITbusinessnet: Caringo Provides Only Software-Based Web-Scale Object Storage for Symantec Enterprise Vault
    Via ITbusinessnet: Caringo Provides Only Software-Based Web-Scale Object Storage for Symantec Enterprise Vault
    Via TopTechNews: Dropbox Adds USB Keys To Boost Security
    Via ZDnet: SUSE brings its management software to the cloud
    Via Forbes: Jumptuit Roadmap Video: A New Search Bar For Your Data At Facebook, Google and Dropbox
    Via Forbes: Panzura Global File System — Disrupting the Storage Industry
    Via TalkinCloud: Barracuda Launches Cloud Storage and Share Service CudaDrive
    Via ITnews: Researchers debut rival to AWS cloud storage
    Via NetworkAsia.net: HP launches new scale-out and scale-up Compute portfolios

    cloud object storage

    Via Server StorageIO: May / June 2015 Newsletter Focus on server storage packaging
    Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
    Via StudioDaily.com: At NAB, Vendors and Users Both Got Serious About 4K Workflow
    Via OpenSource.com: The unofficial guide to OpenStack Summit Vancouver
    Via InsideHPC: Deploying Hadoop on Lustre Storage: Lessons Learned and Best Practices
    Via Yahoo Engineering Blog: Yahoo Cloud Object Store – Object Storage at Exabyte Scale
    Via the Platform: Inside The Ceph Exascale Storage At Yahoo
    Via Swift Summit: Taking the Mystery out of Erasure Codes: A Swift Implementation
    Via GovTech: Storage Costs Cloud Police Cam Issue
    Via BostonHerald: Booting Up: Storage costs cloud police cam issue
    Via StorageIOblog: S3motion Buckets Containers Objects AWS S3 Cloud and EMCcode (with podcast)
    Via StorageIOblog: If focused on cost you might miss other cloud storage benefits
    Via ComputerWorld: Amazon offers network file storage in the cloud
    Via ComputerWeekly: HGST marries helium HDDs and Himalaya in object storage Active Archive
    Via GoogleCloudPlatform Blog: GCS Nearline Online storage at Offline price

    cloud object storage

    Via MarketWatch: Global Data Center Provider CyrusOne Announces Direct Connectivity to Google Cloud Platform
    Via PRNewsWire: Quantum Announces New Archive Solutions Designed To Reduce Unstructured Data Storage Costs
    Via Enterprise Storage Forum: Lustre buying guide
    Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
    Via StreetInsider: Western Digital (WDC) to Acquire Object Storage Software Amplidata
    Via Enterprise Storage Forum: Dell Invests in Object Storage Startup Exablox
    Via Enterprise Storage Forum: Introducing s3motion (S3 and object access docker based appliance)
    Via Computerworld: Quantum enhances their cloud and object storage management with new StorNext software version
    Via HealthIndustry: 2014 IDC MarketScape perspectives
    Via MarketWired: ScaleOut Software Releases Version 5.2 of Its In-Memory Computing Platform
    Via CRN: HP Inks Global Reseller Agreement With Object Storage Startup Scality
    Via NetApp: NetApp Introduces Software-Defined Object Storage for the Hybrid Cloud
    Via ZDnet: NetApp intros software-defined object storage for hybrid cloud adoption
    Via SearchCloudStorage: Are more than five nines of availability really possible?

    cloud object storage

    Via SearchCloudStorage: How do primary storage clouds and cloud for backup differ?
    Via SearchCloudStorage: What should I consider when using SSD cloud?
    Via SearchCloudStorage: Given outages, are you concerned with the security of the cloud?
    Via SearchCloudStorage: Is the cost of cloud storage really cheaper than traditional storage?
    Via HostReview: Panzura Collaborates With Microsoft Azure To Transform Cloud Storage Into A Globally Distributed NAS
    Via ITBusinessNet: Caringo Awarded Patent for Erasure Coding, Replication in Storage Cluster
    Via PRWeb: Western Digital Capital Makes Strategic Investment in Amplidata
    Via ITwire: Dovecot in for the long haul
    Via InfoStor: Top Tips for Object Storage
    Via Enterprise Storage ForumObject Storage Buying Guide
    Via Enterprise Networking Planet: Cloud Gateways 101
    Via eWeek: HP Unveils Smaller, Less Costly Helion Cloud Platform
    Via PCadvisor: CA Technologies, AWS and Riverbed bring mainframe Cloud to big four banks
    Via CMS wire: The Enterprise Cloud is Finally Moving Beyond Storage
    Via Edtech: Microsoft 365’s Massive Cloud Storage Is ‘Icing on Cake’ to University’s Upgrade

    cloud object storage

    Via CRN: Quantum Acquires Symform, Plans To Bring Cloud-Based Backups To Enterprise
    Via BusinessWire: OpenStack Object Storage (Swift) Adds Storage Policies for Enterprise Users and Cloud Operators
    Via SearchStorage: EMC ViPR and Data Services (including Object storage)
    Via Enterprise Storage Forum: Cloud Appliance Buying Guide
    Via Datacenter Dynamics: Verizon, Amplidata and Cloud Object Storage
    Via ZDnet: IBM drops SoftLayer cloud storage costs closer to its rivals
    Via YellowBricks: How long will VSAN rebuilding take with large drives? via Duncan Epping (@DuncanYB) and his Yellow-Bricks site. Note that VMware VSAN uses one of many different forms of object storage (e.g. its not S3 or Swift or CDMI) rather a means for storing VMware vSphere entities and items (e.g. various types of objects)
    Via Computerweekly: Object storage: An architecture for the petabyte era? Via Simon Robinson (@simonrob451) appearing at
    Via MarketWatch: Western Digital/HGST announce alternative to Seagate Kinetic HDD for object storage
    Via EMC: EMC Announces Elastic Cloud Storage Appliance via EMC
    Via TechCrunch: RedHat acquires InkTank provider of Ceph services
    Via CompueterWeekly: DataDirect Networks (DDN) WOS enhancements

    cloud object storage

    Hybrid Clouds: Bridging the Gap Between Public and Private Environments (Recorded for Playback: BrightTalk Webinar)
    Book review: Rethinking Enterprise Storage and Hybrid Clouds by Microsoft/Storsimple and Marc Farley aka (@Gofarley (or what ever name he is currently using ;))
    Goodbye 2013, hello 2014 predictions present and future
    Enterprises should just ignore OpenStack via Virtualizedgeek aka Keith Townsend (@virtualizedgeek)
    OpenStack Swift Object Storage for the Cloud Grows via The Var Guy (@thevarguy) aka Christopher Tozzi
    For Seagate Kinetic to fly…it’s gonna take an ecosystem (Via @Seagate The Storage Effect Blog)
    Object-defined storage (it’s not about object storage) (Via Enrico Signoretti @Juku)
    Seagate Kinetic Cloud and Object Storage I/O platform
    Why You Should Consider Object Storage (Via 21CIT)
    Object Storage Is in Your Future, Even if You Use Files (Via 21CIT)
    The Right Storage Option Is Important for Big Data Success (Via FedTech)
    Hybrid cloud storage with an object storage back end(Via Marc Farley aka @MicroFarley at Microsoft Technet Blogs)
    cloud object storage

    Via SearchStorage: EMC ViPR aimed at service providers, enterprise storage (Video)
    EMC ViPR virtual physical object and software defined storage
    EMC ViPR software defined object storage part II
    EMC ViPR software defined object storage part III
    The rise of cloud-integrated storage and EMC’s ViPR (Via Marc Farley of Microsoft aka @MicroFarley)
    ViPR: Under the covers (Via Chad Sakacc of EMC aka @sakacc)
    Introducing EMC ViPR (Via Chuck Hollis aka @Chuckhollis of EMC)
    EMC Tells their SDS story (e.g. ViPR) (Via the Solutions Architect aka Michael Letschin aka @mletschin)
    Object Paucity (Via Martin Glassborow aka @storagebod)
    Object Storage: You can call it NAS 2.0! (Via Enrico Signoretti aka @esignoretti)
    Object storage is the platform, gateways are the solution (Via Enrico Signoretti aka @esignorettii)
    Files Vs. Objects: the difference is in the value (Via Enrico Signoretti aka @esignorettii)
    Conversation with Justin Stottlemyer of Shutterfly and object storage discussion (Pod cast)

    cloud object storage

    Cloud conversations: AWS EBS, Glacier and S3 overview
    Ceph Day Amsterdam 2012 (Object and cloud storage)
    Ceph Day in Amsterdam and Sage Weil on Object Storage (Podcast)
    Mr. Backup (Curtis Preston) goes back to Ceph School (Video and Podcast)
    AWS (Amazon) storage gateway, first, second and third impressions

    cloud object storage

    Where to learn more

    The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

    Click here to view (and hear) more content including cloud and object storage fundamentals.
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via InfoStor: Cloud Storage Concerns, Considerations and Trends
    Via InfoStor: Object Storage Is In Your Future
    Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
    Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
    Cloud conversations: AWS EBS, Glacier and S3 overview
    AWS (Amazon) storage gateway, first, second and third impressions

    Videos and podcasts at storageio.tv also available via Applie iTunes.

    Human Face of Big Data
    Human Face of Big Data (Book review)

    Seven Databases in Seven weeks
    Seven Databases in Seven Weeks (Book review)

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    Wrap up and summary

    Object and cloud storage are in your future, the questions are when, where, with what and how among others. Watch for more content and links to be added here as well as on the main object storage center page including posts, presentations, pod casts, perspectives along with services and product solutions profiles.

    Ok, nuff said, for now.

    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

    Posted on

    Software Defined, Bulk, Cloud, Scale Out, Object Storage Fundamentals

    Cloud, Bulk, Scale-Out, Object Storage Fundamentals

    data infrastructure sddc object storage fundamentals

    Updated 1/21/2018

    Welcome to the Cloud, Big Data, Software Defined, scale-out, Bulk and Object Storage Fundamentals page.

    This page contains various resources, tips, essential topics pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage Fundamentals. Other resources pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage include:

  • www.objectstoragecenter.com
  • Software Defined Data Infrastructure Essentials book (CRC Press)
  • Cloud, Software Defined, Scale-Out, Object Storage News Trends

    • There are various types of cloud, bulk and object storage including public services such as Amazon Web Services (AWS) Simple Storage Service (S3), Google, Microsoft Microsoft Azure, IBM Softlayer, Rackspace among many others. There are also solutions for hybrid and private deployment from Cisco, Cloudian, Fujifilm, DDN, Dell EMC, Fujitsu, HDS, HPE, IBM, NetApp, Noobaa, OpenStack, Quantum, Rackspace, Scality, Seagate, Spectra, Storpool, Suse, Swift and WD among others.

    Cloud products and services among others, along with associated data infrastructures including object storage, file systems, repositories and access methods are at the center of bulk, big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.

    Cloud Object Storage Fundamentals Access and Architectures

    There are many facets to object storage including technology implementation, products, services, access and architectures for various applications and use scenarios.

    • Project or Account – Top of the hierarchy that can represent the owner or billing information for a service that where buckets are also attached.
    • Region – Location where data is stored that can include one or more data centers also known as Availability Zones.

      • AWS S3 Cross region replication
      Moving and Replicating Buckets/Containers, Subfolders and Objects

    • Availability Zone (AZ) or data center or server that implement durability and accessibility for availability within a region.

      • AWS Regions and Availability Zones AZs
      Example of Regions and Availability Zones (AZs)

    • Bucket or Container – Where objects or sub-folders containing objects are attached and accessed.

      • Object storage fundamentals sddc and cloud software defined

    • Sub-folder – While object storage can be located in a flat namespace for commonality and organization some solutions and service support the notion of sub-folder that resemble traditional directory hierarchy.
    • Object – Byte (or bit) stream that can be as small as one byte to as large as several Tbytes (some solutions and services support up to 5TByte sized objects). The object contains whatever data in any organization along with metadata. Different solutions and services support from a couple hundred KBytes of meta-data to Mbytes worth of meta-data. Regarding what can be stored in an object, anything from files, videos, images, virtual disks (VMDKs, VHDX), ZIP or tar files, backup and archive save sets, executable images or ISO’s, anything you want.
    • End-point – Where or what your software, application or tool and utilities along with gateways attach to for accessing buckets and objects.

      • object storage fundamentals, sddc and cloud storage example

      A common theme for object storage is flexibility, along with scaling (performance, availability, capacity, economics) along with extensibility without compromise or complexity. From those basics, there are many themes and variations from how data is protected (RAID or no RAID, hardware or software), deployed as a service or as tin wrapped software (an appliance), optimized for archiving or video serving or other applications.

      Many facets of cloud and object storage access

      One aspect of object and cloud storage is accessing or using object methods including application programming interfaces (API’s) vs. traditional block (LUN) or NAS (file) based approaches. Keep in mind that many object storage systems, software, and services support NAS file-based access including NFS, CIFS, HDFS  among others for compatibility and ease of use.

      Likewise various API’s can be found across different object solutions, software or services including Amazon Web Services (AWS) Simple Storage Service (S3) HTTP REST based, among others. Other API’s will vary by specific vendor or product however can include IOS (e.g. Apple iPhone and iPad), WebDav, FTP, JSON, XML, XAM, CDMI, SOAP, and DICOM among others. Another aspect of object and cloud storage are expanded  and dynamic metadata.

      While traditional file systems and NAS have simple or fixed metadata, object and cloud storage systems, services and solutions along with some scale-out file systems have ability to support user defined metadata. Specific systems, solutions, software, and services will vary on the amount of metadata that could range on the low-end from 100s of KBytes  to tens or more Mbytes.

      cloud object storage

      Where to learn more

      The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

      Click here to view software defined, bulk, cloud and object storage trend news.


      StorageIO Founder Greg Schulz: File Services on Object Storage with HyperFile

      Via InfoStor: Object Storage Is In Your Future
      Via FujiFilm IT Summit: Software Defined Data Infrastructures (SDDI) and Hybrid Clouds
      Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
      Via InfoStor: Cloud Storage Concerns, Considerations and Trends
      Via InfoStor: Object Storage Is In Your Future
      Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
      Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
      Cloud conversations: AWS EBS, Glacier and S3 overview
      AWS (Amazon) storage gateway, first, second and third impressions
      Cloud and Virtual Data Storage Networking (CRC Book)
      Via ChannelPartnersOnline: Selling Software-Defined Storage: Not All File Systems Are the Same
      Via ITProPortal: IBM kills off its first cloud storage platform
      Via ITBusinessEdge: Time to Rein in Cloud Storage
      Via SerchCloudStorge: Ctera Networks’ file-sharing services gain intelligent cache
      Via StorageIOblog: Who Will Be At Top Of Storage World Next Decade?

      Videos and podcasts at storageio.tv also available via Applie iTunes.

      Human Face of Big Data
      Human Face of Big Data (Book review)

      Seven Databases in Seven weeks
      Seven Databases in Seven Weeks (Book review)

      Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

      Software Defined Data Infrastructure Essentials Book SDDC

      Wrap up and summary

      Object and cloud storage are in your future, the questions are when, where, with what and how among others.

      Watch for more content and links to be added here soon to this object storage center page including posts, presentations, pod casts, polls, perspectives along with services and product solutions profiles.

      Ok, nuff said, for now.

      Gs

      Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

      All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

    Posted on

    HDDs evolve for Content Application servers

    HDDs evolve for Content Application servers

    hdds evolve server storage I/O trends

    Updated 1/23/2018

    Enterprise HDDs evolve for content server platform

    Insight for effective server storage I/O decision making
    Server StorageIO Lab Review

    Which enterprise HDD to use for content servers

    This is the seventh and final post in this multi-part series (read part six here) based on a white paper hands-on lab report I did compliments of Servers Direct and Seagate that you can read in PDF form here. The focus is looking at the Servers Direct (www.serversdirect.com) converged Content Solution platforms with Seagate Enterprise Hard Disk Drive (HDD’s). The focus of this post is comparing how HDD continue to evolve over various generations boosting performance as well as capacity and reliability. This also looks at how there is more to HDD performance than the traditional focus on Revolutions Per Minute (RPM) as a speed indicator.

    Comparing Different Enterprise 10K And 15K HDD Generations

    There is more to HDD performance than RPM speed of the device. RPM plays an important role, however there are other things that impact HDD performance. A common myth is that HDD’s have not improved on performance over the past several years with each successive generation. Table-10 shows a sampling of various generations of enterprise 10K and 15K HDD’s (14) including different form factors and how their performance continues to improve.

    different 10K and 15K HDDs
    Figure-9 10K and 15K HDD performance improvements

    Figure-9 shows how performance continues to improve with 10K and 15K HDD’s with each new generation including those with enhanced cache features. The result is that with improvements in cache software within the drives, along with enhanced persistent non-volatile memory (NVM) and incremental mechanical drive improvements, both read and write performance continues to be enhanced.

    Figure-9 puts into perspective the continued performance enhancements of HDD’s comparing various enterprise 10K and 15K devices. The workload is the same TPC-C tests used earlier in a similar (14) (with no RAID). 100 simulated users are shown in figure-9 accessing a database on each of the different drives all running concurrently. The older 15K 3.5” Cheetah and 2.5” Savio used had a capacity of 146GB which used a database scale factor of 1500 or 134GB. All other drives used a scale factor 3000 or 276GB. Figure-9 also highlights the improvements in both TPS performance as well as lower response time with new HDD’s including those with performance enhanced cache feature.

    The workloads run are same as the TPC-C ones shown earlier, however these drives were not configured with any RAID. The TPC-C activity used Benchmark Factory with similar setup and configuration to those used earlier including on a multi-socket, multi-core Windows 2012 R2 server supporting a Microsoft SQL Server 2012 database with a database for each drive type.

    ENT 10K V3 2.5"

    ENT (Cheetah) 15K 3.5"

    Users

    1

    20

    50

    100

    Users

    1

    20

    50

    100

    TPS (TPC-C)

    14.8

    50.9

    30.3

    39.9

    TPS (TPC-C)

    14.6

    51.3

    27.1

    39.3

    Resp. Time (Sec.)

    0.0

    0.4

    1.6

    1.7

    Resp. Time (Sec.)

    0.0

    0.3

    1.8

    2.1

    ENT 10K 2.5" (with cache)

    ENT (Savio) 15K 2.5"

    Users

    1

    20

    50

    100

    Users

    1

    20

    50

    100

    TPS (TPC-C)

    19.2

    146.3

    72.6

    71.0

    TPS (TPC-C)

    15.8

    59.1

    40.2

    53.6

    Resp. Time (Sec.)

    0.0

    0.1

    0.7

    0.0

    Resp. Time (Sec.)

    0.0

    0.3

    1.2

    1.2

    ENT 15K V4 2.5"

    Users

    1

    20

    50

    100

    TPS (TPC-C)

    19.7

    119.8

    75.3

    69.2

    Resp. Time (Sec.)

    0.0

    0.1

    0.6

    1.0

    ENT 15K (enhanced cache) 2.5"

    Users

    1

    20

    50

    100

    TPS (TPC-C)

    20.1

    184.1

    113.7

    122.1

    Resp. Time (Sec.)

    0.0

    0.1

    0.4

    0.2

    Table-10 Continued Enterprise 10K and 15K HDD performance improvements

    (Note 14) 10K and 15K generational comparisons were run on a separate comparable server to what was used for other test workloads. Workload configuration settings were the same as other database workloads including using Microsoft SQL Server 2012 on a Windows 2012 R2 system with Benchmark Factory driving the workload. Database memory sized was reduced however to only 8GB vs. 16GB used in other tests.

    Where To Learn More

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    What This All Means

    A little bit of flash in the right place with applicable algorithms goes a long way, an example being the Seagate Enterprise HDD’s with enhanced cache feature. Likewise, HDD’s are very much alive complementing SSD and vice versa. For high-performance content application workloads flash SSD solutions including NVMe, 12Gbps SAS and 6Gbps SATA devices are cost effective solutions. HDD’s continue to be cost-effective data storage devices for both capacity, as well as environments that do not need the performance of flash SSD.

    For some environments using a combination of flash and HDD’s complementing each other along with cache software can be a cost-effective solution. The previous workload examples provide insight for making cost-effective informed storage decisions.

    Evaluate today’s HDD’s on their effective performance running workloads as close as similar to your own, or, actually try them out with your applications. Today there is more to HDD performance than just RPM speed, particular with the Seagate Enterprise Performance 10K and 15K HDD’s with enhanced caching feature.

    However the Enterprise Performance 10K with enhanced cache feature provides a good balance of capacity, performance while being cost-effective. If you are using older 3.5” 15K or even previous generation 2.5” 15K RPM and “non-performance enhanced” HDD’s, take a look at how the newer generation HDD’s perform, looking beyond the RPM of the device.

    Fast content applications need fast content and flexible content solution platforms such as those from Servers Direct and HDD’s from Seagate. Key to a successful content application deployment is having the flexibility to hardware define and software defined the platform to meet your needs. Just as there are many different types of content applications along with diverse environments, content solution platforms need to be flexible, scalable and robust, not to mention cost effective.

    Ok, nuff said, for now.

    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.