Tag: Blob

Posted on

Amazon Web Service AWS September 2017 Software Defined Data Infrastructure Updates

Amazon Web Service AWS September 2017 Software Defined Data Infrastructure Updates

server storage I/O data infrastructure trends

Amazon Web Service AWS September 2017 Software Defined Data Infrasture Updates

September was a busy month pertaining to software defined data infrastructure including cloud and related AWS announcements. One of the announcements included VMware partnering to deliver vSphere, vSAN and NSX data infrastructure components for creating software defined data centers (SDDC) also known as multi cloud, and hybrid cloud leveraging AWS elastic bare metal servers (read more here in a companion post). Unlike traditional partner software defined solutions that relied on AWS Elastic Cloud Compute (EC2) instances, VMware is being deployed using private bare metal AWS elastic servers.

What this means is that VMware vSphere (e.g. ESXi) hypervisor, vCenter, software defined storage (vSAN), storage defined network (NSX) and associated vRealize tools are deployed on AWS data infrastructure that can be used for deploying hybrid software defined data centers (e.g. connecting to your existing VMware environment). Learn more about VMware on AWS here or click on the following image.

VMware on AWS via Amazon.com

Additional AWS Updates

Amazon Web Services (AWS) updates include, coinciding with VMworld, the initial availability of VMware on AWS (using virtual private servers e.g. think along the lines of Lightsail, not EC2 instances) was announced. Amazon Web Services (AWS) continues its expansion into database and table services with Relational Data Services (RDS) including various engines (Amazon Auora,MariaDB, MySQL, Oracle, PostgreSQL,and SQL Server along with Database Migration Service (DMS). Note that these RDS are in addition to what you can install and run your self on Elastic Cloud Compute (EC2) virtual machine instances, Lambda serverless containers, or Lightsail Virtual Private Servers (VPS).

AWS has published a guide to database testing on Amazon RDS for Oracle plotting latency and IOPs for OLTP workloads here using SLOB. If you are not familiar with SLOB (Silly Little Oracle Benchmark) here is a podcast with its creator Kevin Closson discussing database performance and related topics. Learn more about SLOB and step by step installation for AWS RDS Oracle here, and for those who are concerned or think that you can not run workloads to evaluate Oracle platforms, have a look at this here.

EC2 enhancements include charging by the second (previous by the hour) for some EC2 instances (see details here including what is or is not currently available) which is a growing trend by private cloud vendors aligning with how serverless containers have been billed. New large memory EC2 instances that for example support up to 3,904GB of DDR4 RAM have been added by AWS. Other EC2 enhancements include updated network performance for some instances, OpenCL development environment to leverage AWS F1 FPGA enabled instances, along with new Elastic GPU enabled instances. Other server and network enhancements include Network Load Balancer for Elastic Load Balancer announced, as well as application load balancer now supports load balancing to IP address as targets for AWS and on premises (e.g. hybrid) resources.

Other updates and announces include data protection backups to AWS via Commvault and AWS Storage Gateway VTL announced. IBM has announced their Spectrum Scale (e.g. formerly known as SONAS aka GPFS) Scale Out Storage solution for high performance compute (HPC) quick start on AWS. Additional AWS enhancements include new edge location in Boston and a third Seattle site, while Direct Connect sites have been added in Boston and Houston along with Canberra Australia. View more AWS announcements and enhancements here.

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

What This All Means

AWS continues to grow and expand, both in terms of number of services, also the extensiveness of them. Likewise AWS continues to add more regions and data center availability zones, enhanced connectivity, along with earlier mentioned service features. The partnership with VMware should enable enterprise organizations to move towards hybrid cloud data infrastructures, while giving AWS an additional reach into those data centers. Overall a good set of enhancements by AWS who continues to evolve their cloud and software defined data infrastructure portfolio of solution offerings.

By the way, if you have not heard, its Blogtober, check out some of the other blogs and posts occurring during October here.

Ok, nuff said, for now.
Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (and vSAN). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

Software Defined Cloud Bulk Object Storage News Trends

Cloud, Software Defined, Scale-Out, Object Storage News Trends

server storage I/O trends

Updated 1/21/2018

Welcome to the Cloud, Big Data, Software Defined, Bulk and Object Storage News Trends page.

This page contains various links pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage News Trends and resources. Other resources pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage include:

  • www.objectstoragecenter.com
  • Software Defined, Cloud, Bulk and Object Storage Fundamentals
  • Software Defined Data Infrastructure Essentials book (CRC Press)

    • Software defined, Bulk, Cloud and Object Storage exist to support expanding and diverse application data demands. There are various types of cloud, bulk and object storage including public services such as Amazon Web Services (AWS) Simple Storage Service (S3), Google, Microsoft Azure, IBM Softlayer, Rackspace among many others. There are also solutions for hybrid and private deployment from Cisco, DDN, Dell EMC, Fujitsu, HDS, HPE, IBM, NetApp, Noobaa, OpenStack, Quantum, Rackspace, Scality, Seagate, Spectra, Storpool, Suse, Swift and WD among others.

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    Cloud products and services among others, along with associated data infrastructures including object storage, file systems, repositories and access methods are at the center of bulk, big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.



    Via InfoStor: Object Storage Is In Your Future
    Via FujiFilm IT Summit: Software Defined Data Infrastructures (SDDI) and Hybrid Clouds
    Via ChannelPartnersOnline: Selling Software-Defined Storage: Not All File Systems Are the Same
    Via ITProPortal: IBM kills off its first cloud storage platform
    Via MultiChannel: After ditching cloud business, Verizon inks Virtual Network Services deal with Amazon
    Via ITBusinessEdge: Time to Rein in Cloud Storage
    Via BusinessInsider: Kodiak Data Brings Enterprise Storage and Database Services to the Cloud Era with StorageCloud
    Via SerchCloudStorge: Ctera Networks’ file-sharing services gain intelligent cache
    Via StorageIOblog: Who Will Be At Top Of Storage World Next Decade?

    cloud object storage

    The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via CloudComputingAdmin: Cloud Storage Decision Making
    Via Wservernews: Cloud storage considerations
    Via PRNewsWire: Scality Accelerates Multi-Cloud Adoption with New Open-Source Release
    Via EnterpriseStorageForum: Top 10 Tips for Software-Defined Storage Deployment
    Via MultiChannel: Verizon Digital Media Services now offers integrated Microsoft Azure Storage
    Via BroadCstingCable: Quantum’s New StorNext 6 Delivers Unparalleled Combination of Best-in-Class 4K Performance and Advanced Data Management
    Via ShootOnLine: Morro Data Accelerates Media Collaboration With Cache and Sync Appliance
    Via TechCrunch: Backblaze drops download pricing for its B2 storage platform by 60%
    Via PRNewsWire: FileCloud Online Launches an All-In-One File Share, Sync and Backup SaaS Offering in a Challenge to Dropbox and Box

    cloud object storage

    Via HITInfrastructure: Benefits of Object Storage to Health IT Infrastructure
    Via ITBusinessedge: Growing Diversity of Cloud Storage Solutions
    Via Telecompaper: Interoute launches storage zones in Switzerland
    Via TradeCalls:
    Object Storage Market Size, Status and Forecast 2022
    Via OpenStack: Creating object storage container causes user to be logged out, OpenStack Dashboard (Horizon)
    Via RethinkResearch: Nokia claims a 40% cut in cloud DVR Capex, using new Skim Storage
    Via Cloudwards: A Beginner’s Guide on How Cloud Backup With Microsoft Azure Works
    Via ITProPortal: Lack of understanding of data inhibits move to the cloud

    cloud object storage

    Via InfoStor:Cloud Storage Concerns, Considerations and Trends
    Via StreetInsider: IBM Announces New Pricing Model for Cloud Storage and NetApp Object Storage Partnership

    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via ComputerWeekly: Storage 101: Object storage vs block vs file
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via Yahoo Finance: SwiftStack Joins Google Cloud Technology Partner Program
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look

    Via StorageIOblog: HPE Continues Buying Into Server Storage I/O Data Infrastructures
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via TheStack:Google to launch nine new data centres over three new cloud regions
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via DigitalJournal:Global Business Cloud Storage market 2017 share, Segmentation and Industrial Overview forecasts to 2022
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via NextPlatform:How Microsoft Fell Hard for FPGAs
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via NextPlatform:Google Courts Enterprise For Cloud Platform

    cloud object storage

    Via IBM RedBooks: Cloud Object Storage as a
    Service – IBM Cloud Object Storage from Theory to Practice
    Via VentureBeat: Microsoft confirms Azure storage issues around the world
    Via Microsoft Azure: Azure status history
    Via NextPlatform: Windows Server Comes To ARM Chips, But Only For Azure
    Via NextPlatform: ARM And AMD X86 Server Chips Get Mainstream Lift From Microsoft
    Via ITBusinessEdge: Google Expands Enterprise Cloud Service Footprint
    Via ChannelWorld: Google announces three new cloud regions, contract discounts
    Via AWS: AWS Service Health Status
    Via AWS (S3 Feb 28 2017 Port mortem): Summary of the Amazon S3 Service Disruption
    Via SearchCloudStorage: Open source object storage startup OpenIO adds hardware

    cloud object storage

    Via InfoStor: Comments on Cloud Storage Pros and Cons
    Via CRN: StorageCraft Acquires Exablox
    Via TVtechglobal: Signiant CEO: The media industry isnt just one monolithic thing
    Via Computerweekly: Using the cloud as a storage tier
    Via SearchCloudStorage: OpenIO joins object storage cloud scrum
    Via CRN: Dell Partners With Scality On High-Density Object Storage
    Via BroadwayWorld: The World’s Smallest Object Storage Solution, ‘Ceph on Olive’, is Released
    Via SmartDataCollective: Why You Should Be Securing Big Data In The Cloud?
    Via Geekwire: Feds certify Amazon and Microsoft clouds to handle sensitive government data
    Via NewsFactor: SoftNAS Introduces File Services for NetApp StorageGRID
    Via Redmondmag: AWS Cloud Storage Service Turns 10
    Via DropBox Blog: Scaling to exabytes and beyond
    Via Computerworld: Dropbox quits Amazon cloud, takes back 500 PB of data
    Via HPCwire: Seagate Sets Sights on Broader HPC Market

    cloud object storage

    Via Channelnomics.com: Why Samsung’s Joyent buyout could frame its future
    Via GeekyGadgets: Google Drive Cloud Storage Is Now A Little Easier To Organize Thanks To New Tools
    Via ZDnet: Google doubles Cloud Compute local SSD capacity: Now it’s 3TB per VM
    Via AndroidPolice: Barracuda Is Shutting Down The Copy Cloud Storage Service On May 1st
    Via StorageSoup: EMC expands Elastic Cloud (object) Storage
    Via Cloud Review: Access Multiple Cloud Apps from a Unified Interface
    Via CyberSecurity: Cybersecurity and Accessible Storage Trends
    Via eWeek: Cloud Storage Spending Poised for Increase in 2016
    Via InfoStor: Object Storage Is In Your Future
    Via BioScienceTechnology: Six Reasons to Add Object Storage to Your Genomics Lexicon
    Via TechCrunch: Backblaze’s Low-Cost Cloud Storage Service Comes Out Of Beta
    Via CloudComputingAdmin: Cloud Storage Decision Making
    COSbench for cloud and object storage (Various): White Paper (CMU), PDF overview presentation, Get the bits (software) from Github
    Via AWS Blogs: AWS Storage Update – New Lower Cost S3 Storage Option & Glacier Price Reduction
    Via Techcrunch: IBM Acquires Cleversafe To Accelerate Hybrid Cloud Storage Strategy

    cloud object storage

    Via Techcrunch: Developers Can Now Import Data To Google Cloud Storage By Mailing In Their Hard Drives, Tapes And USB Flash Drives
    Via vCloud: VMware vCloud Air Object Storage
    Via SearchStorage: SwiftStack adds new features to object storage software solution
    Via TheVarGuy: Vendor consortium create Ceph advisory board
    Via OpenStack: Whats new in OpenStack Liberty
    Via Microsoft Azure: File access cloud storage in addition to blobs and object storage
    Via AWS: AWS Storage Update – New Lower Cost S3 Storage Option & Glacier Price Reduction
    Via VMware Blogs: Showcasing VMware vCloud Air Object Storage powered by EMC at EMC World!
    Via Venturebeat: Backblaze launches B2 object storage service at 1/4 of the cost of Amazon S3
    Via Techcrunch: Wuala, The Secure P2P Cloud Storage Service Acquired By LaCie, To Shut Down
    Via ChannelworldIN: 13 firms join to open up Ethernet, object-storage to everyone
    Via ITportal: The elephant in the room: The trust deficit issue with public clouds
    Via VCpost: Dropbox latest update: Cloud storage service provider now lets you drag and drop URLs with files
    Via NetworkAsia: HP’s new converged appliance offers a path to hybrid cloud
    Via Techtimes: Google Files For Patent To Search Your Desktop Files, Apps And Cloud Storage

    cloud object storage

    Via CloudTweaks: Questions To Ask Every Cloud Storage Provider
    Via MarketWatch: Linux Foundation Brings Together Industry Leaders to Advance Cloud Object Storage Technologies
    Via CBR: New Kinetic platform targets open source object storage
    Via Techcrunch: Seagate, Toshiba, Scality And Others Launch Kinetic Open Storage Platform For Smart, Low-Overhead Cloud Object Storage
    Via StorageIOblog: July 2015 Newsletter Focus on Content Solution Platforms
    Via ITnewsline: Hitachi Data Systems Announces New Enhancements to the Hitachi Content Platform Portfolio
    Via ITbusinessnet: Caringo Provides Only Software-Based Web-Scale Object Storage for Symantec Enterprise Vault
    Via ITbusinessnet: Caringo Provides Only Software-Based Web-Scale Object Storage for Symantec Enterprise Vault
    Via TopTechNews: Dropbox Adds USB Keys To Boost Security
    Via ZDnet: SUSE brings its management software to the cloud
    Via Forbes: Jumptuit Roadmap Video: A New Search Bar For Your Data At Facebook, Google and Dropbox
    Via Forbes: Panzura Global File System — Disrupting the Storage Industry
    Via TalkinCloud: Barracuda Launches Cloud Storage and Share Service CudaDrive
    Via ITnews: Researchers debut rival to AWS cloud storage
    Via NetworkAsia.net: HP launches new scale-out and scale-up Compute portfolios

    cloud object storage

    Via Server StorageIO: May / June 2015 Newsletter Focus on server storage packaging
    Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
    Via StudioDaily.com: At NAB, Vendors and Users Both Got Serious About 4K Workflow
    Via OpenSource.com: The unofficial guide to OpenStack Summit Vancouver
    Via InsideHPC: Deploying Hadoop on Lustre Storage: Lessons Learned and Best Practices
    Via Yahoo Engineering Blog: Yahoo Cloud Object Store – Object Storage at Exabyte Scale
    Via the Platform: Inside The Ceph Exascale Storage At Yahoo
    Via Swift Summit: Taking the Mystery out of Erasure Codes: A Swift Implementation
    Via GovTech: Storage Costs Cloud Police Cam Issue
    Via BostonHerald: Booting Up: Storage costs cloud police cam issue
    Via StorageIOblog: S3motion Buckets Containers Objects AWS S3 Cloud and EMCcode (with podcast)
    Via StorageIOblog: If focused on cost you might miss other cloud storage benefits
    Via ComputerWorld: Amazon offers network file storage in the cloud
    Via ComputerWeekly: HGST marries helium HDDs and Himalaya in object storage Active Archive
    Via GoogleCloudPlatform Blog: GCS Nearline Online storage at Offline price

    cloud object storage

    Via MarketWatch: Global Data Center Provider CyrusOne Announces Direct Connectivity to Google Cloud Platform
    Via PRNewsWire: Quantum Announces New Archive Solutions Designed To Reduce Unstructured Data Storage Costs
    Via Enterprise Storage Forum: Lustre buying guide
    Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
    Via StreetInsider: Western Digital (WDC) to Acquire Object Storage Software Amplidata
    Via Enterprise Storage Forum: Dell Invests in Object Storage Startup Exablox
    Via Enterprise Storage Forum: Introducing s3motion (S3 and object access docker based appliance)
    Via Computerworld: Quantum enhances their cloud and object storage management with new StorNext software version
    Via HealthIndustry: 2014 IDC MarketScape perspectives
    Via MarketWired: ScaleOut Software Releases Version 5.2 of Its In-Memory Computing Platform
    Via CRN: HP Inks Global Reseller Agreement With Object Storage Startup Scality
    Via NetApp: NetApp Introduces Software-Defined Object Storage for the Hybrid Cloud
    Via ZDnet: NetApp intros software-defined object storage for hybrid cloud adoption
    Via SearchCloudStorage: Are more than five nines of availability really possible?

    cloud object storage

    Via SearchCloudStorage: How do primary storage clouds and cloud for backup differ?
    Via SearchCloudStorage: What should I consider when using SSD cloud?
    Via SearchCloudStorage: Given outages, are you concerned with the security of the cloud?
    Via SearchCloudStorage: Is the cost of cloud storage really cheaper than traditional storage?
    Via HostReview: Panzura Collaborates With Microsoft Azure To Transform Cloud Storage Into A Globally Distributed NAS
    Via ITBusinessNet: Caringo Awarded Patent for Erasure Coding, Replication in Storage Cluster
    Via PRWeb: Western Digital Capital Makes Strategic Investment in Amplidata
    Via ITwire: Dovecot in for the long haul
    Via InfoStor: Top Tips for Object Storage
    Via Enterprise Storage ForumObject Storage Buying Guide
    Via Enterprise Networking Planet: Cloud Gateways 101
    Via eWeek: HP Unveils Smaller, Less Costly Helion Cloud Platform
    Via PCadvisor: CA Technologies, AWS and Riverbed bring mainframe Cloud to big four banks
    Via CMS wire: The Enterprise Cloud is Finally Moving Beyond Storage
    Via Edtech: Microsoft 365’s Massive Cloud Storage Is ‘Icing on Cake’ to University’s Upgrade

    cloud object storage

    Via CRN: Quantum Acquires Symform, Plans To Bring Cloud-Based Backups To Enterprise
    Via BusinessWire: OpenStack Object Storage (Swift) Adds Storage Policies for Enterprise Users and Cloud Operators
    Via SearchStorage: EMC ViPR and Data Services (including Object storage)
    Via Enterprise Storage Forum: Cloud Appliance Buying Guide
    Via Datacenter Dynamics: Verizon, Amplidata and Cloud Object Storage
    Via ZDnet: IBM drops SoftLayer cloud storage costs closer to its rivals
    Via YellowBricks: How long will VSAN rebuilding take with large drives? via Duncan Epping (@DuncanYB) and his Yellow-Bricks site. Note that VMware VSAN uses one of many different forms of object storage (e.g. its not S3 or Swift or CDMI) rather a means for storing VMware vSphere entities and items (e.g. various types of objects)
    Via Computerweekly: Object storage: An architecture for the petabyte era? Via Simon Robinson (@simonrob451) appearing at
    Via MarketWatch: Western Digital/HGST announce alternative to Seagate Kinetic HDD for object storage
    Via EMC: EMC Announces Elastic Cloud Storage Appliance via EMC
    Via TechCrunch: RedHat acquires InkTank provider of Ceph services
    Via CompueterWeekly: DataDirect Networks (DDN) WOS enhancements

    cloud object storage

    Hybrid Clouds: Bridging the Gap Between Public and Private Environments (Recorded for Playback: BrightTalk Webinar)
    Book review: Rethinking Enterprise Storage and Hybrid Clouds by Microsoft/Storsimple and Marc Farley aka (@Gofarley (or what ever name he is currently using ;))
    Goodbye 2013, hello 2014 predictions present and future
    Enterprises should just ignore OpenStack via Virtualizedgeek aka Keith Townsend (@virtualizedgeek)
    OpenStack Swift Object Storage for the Cloud Grows via The Var Guy (@thevarguy) aka Christopher Tozzi
    For Seagate Kinetic to fly…it’s gonna take an ecosystem (Via @Seagate The Storage Effect Blog)
    Object-defined storage (it’s not about object storage) (Via Enrico Signoretti @Juku)
    Seagate Kinetic Cloud and Object Storage I/O platform
    Why You Should Consider Object Storage (Via 21CIT)
    Object Storage Is in Your Future, Even if You Use Files (Via 21CIT)
    The Right Storage Option Is Important for Big Data Success (Via FedTech)
    Hybrid cloud storage with an object storage back end(Via Marc Farley aka @MicroFarley at Microsoft Technet Blogs)
    cloud object storage

    Via SearchStorage: EMC ViPR aimed at service providers, enterprise storage (Video)
    EMC ViPR virtual physical object and software defined storage
    EMC ViPR software defined object storage part II
    EMC ViPR software defined object storage part III
    The rise of cloud-integrated storage and EMC’s ViPR (Via Marc Farley of Microsoft aka @MicroFarley)
    ViPR: Under the covers (Via Chad Sakacc of EMC aka @sakacc)
    Introducing EMC ViPR (Via Chuck Hollis aka @Chuckhollis of EMC)
    EMC Tells their SDS story (e.g. ViPR) (Via the Solutions Architect aka Michael Letschin aka @mletschin)
    Object Paucity (Via Martin Glassborow aka @storagebod)
    Object Storage: You can call it NAS 2.0! (Via Enrico Signoretti aka @esignoretti)
    Object storage is the platform, gateways are the solution (Via Enrico Signoretti aka @esignorettii)
    Files Vs. Objects: the difference is in the value (Via Enrico Signoretti aka @esignorettii)
    Conversation with Justin Stottlemyer of Shutterfly and object storage discussion (Pod cast)

    cloud object storage

    Cloud conversations: AWS EBS, Glacier and S3 overview
    Ceph Day Amsterdam 2012 (Object and cloud storage)
    Ceph Day in Amsterdam and Sage Weil on Object Storage (Podcast)
    Mr. Backup (Curtis Preston) goes back to Ceph School (Video and Podcast)
    AWS (Amazon) storage gateway, first, second and third impressions

    cloud object storage

    Where to learn more

    The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

    Click here to view (and hear) more content including cloud and object storage fundamentals.
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via InfoStor: Cloud Storage Concerns, Considerations and Trends
    Via InfoStor: Object Storage Is In Your Future
    Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
    Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
    Cloud conversations: AWS EBS, Glacier and S3 overview
    AWS (Amazon) storage gateway, first, second and third impressions

    Videos and podcasts at storageio.tv also available via Applie iTunes.

    Human Face of Big Data
    Human Face of Big Data (Book review)

    Seven Databases in Seven weeks
    Seven Databases in Seven Weeks (Book review)

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    Wrap up and summary

    Object and cloud storage are in your future, the questions are when, where, with what and how among others. Watch for more content and links to be added here as well as on the main object storage center page including posts, presentations, pod casts, perspectives along with services and product solutions profiles.

    Ok, nuff said, for now.

    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

    Posted on

    Software Defined, Bulk, Cloud, Scale Out, Object Storage Fundamentals

    Cloud, Bulk, Scale-Out, Object Storage Fundamentals

    data infrastructure sddc object storage fundamentals

    Updated 1/21/2018

    Welcome to the Cloud, Big Data, Software Defined, scale-out, Bulk and Object Storage Fundamentals page.

    This page contains various resources, tips, essential topics pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage Fundamentals. Other resources pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage include:

  • www.objectstoragecenter.com
  • Software Defined Data Infrastructure Essentials book (CRC Press)
  • Cloud, Software Defined, Scale-Out, Object Storage News Trends

    • There are various types of cloud, bulk and object storage including public services such as Amazon Web Services (AWS) Simple Storage Service (S3), Google, Microsoft Microsoft Azure, IBM Softlayer, Rackspace among many others. There are also solutions for hybrid and private deployment from Cisco, Cloudian, Fujifilm, DDN, Dell EMC, Fujitsu, HDS, HPE, IBM, NetApp, Noobaa, OpenStack, Quantum, Rackspace, Scality, Seagate, Spectra, Storpool, Suse, Swift and WD among others.

    Cloud products and services among others, along with associated data infrastructures including object storage, file systems, repositories and access methods are at the center of bulk, big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.

    Cloud Object Storage Fundamentals Access and Architectures

    There are many facets to object storage including technology implementation, products, services, access and architectures for various applications and use scenarios.

    • Project or Account – Top of the hierarchy that can represent the owner or billing information for a service that where buckets are also attached.
    • Region – Location where data is stored that can include one or more data centers also known as Availability Zones.

      • AWS S3 Cross region replication
      Moving and Replicating Buckets/Containers, Subfolders and Objects

    • Availability Zone (AZ) or data center or server that implement durability and accessibility for availability within a region.

      • AWS Regions and Availability Zones AZs
      Example of Regions and Availability Zones (AZs)

    • Bucket or Container – Where objects or sub-folders containing objects are attached and accessed.

      • Object storage fundamentals sddc and cloud software defined

    • Sub-folder – While object storage can be located in a flat namespace for commonality and organization some solutions and service support the notion of sub-folder that resemble traditional directory hierarchy.
    • Object – Byte (or bit) stream that can be as small as one byte to as large as several Tbytes (some solutions and services support up to 5TByte sized objects). The object contains whatever data in any organization along with metadata. Different solutions and services support from a couple hundred KBytes of meta-data to Mbytes worth of meta-data. Regarding what can be stored in an object, anything from files, videos, images, virtual disks (VMDKs, VHDX), ZIP or tar files, backup and archive save sets, executable images or ISO’s, anything you want.
    • End-point – Where or what your software, application or tool and utilities along with gateways attach to for accessing buckets and objects.

      • object storage fundamentals, sddc and cloud storage example

      A common theme for object storage is flexibility, along with scaling (performance, availability, capacity, economics) along with extensibility without compromise or complexity. From those basics, there are many themes and variations from how data is protected (RAID or no RAID, hardware or software), deployed as a service or as tin wrapped software (an appliance), optimized for archiving or video serving or other applications.

      Many facets of cloud and object storage access

      One aspect of object and cloud storage is accessing or using object methods including application programming interfaces (API’s) vs. traditional block (LUN) or NAS (file) based approaches. Keep in mind that many object storage systems, software, and services support NAS file-based access including NFS, CIFS, HDFS  among others for compatibility and ease of use.

      Likewise various API’s can be found across different object solutions, software or services including Amazon Web Services (AWS) Simple Storage Service (S3) HTTP REST based, among others. Other API’s will vary by specific vendor or product however can include IOS (e.g. Apple iPhone and iPad), WebDav, FTP, JSON, XML, XAM, CDMI, SOAP, and DICOM among others. Another aspect of object and cloud storage are expanded  and dynamic metadata.

      While traditional file systems and NAS have simple or fixed metadata, object and cloud storage systems, services and solutions along with some scale-out file systems have ability to support user defined metadata. Specific systems, solutions, software, and services will vary on the amount of metadata that could range on the low-end from 100s of KBytes  to tens or more Mbytes.

      cloud object storage

      Where to learn more

      The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

      Click here to view software defined, bulk, cloud and object storage trend news.


      StorageIO Founder Greg Schulz: File Services on Object Storage with HyperFile

      Via InfoStor: Object Storage Is In Your Future
      Via FujiFilm IT Summit: Software Defined Data Infrastructures (SDDI) and Hybrid Clouds
      Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
      Via InfoStor: Cloud Storage Concerns, Considerations and Trends
      Via InfoStor: Object Storage Is In Your Future
      Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
      Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
      Cloud conversations: AWS EBS, Glacier and S3 overview
      AWS (Amazon) storage gateway, first, second and third impressions
      Cloud and Virtual Data Storage Networking (CRC Book)
      Via ChannelPartnersOnline: Selling Software-Defined Storage: Not All File Systems Are the Same
      Via ITProPortal: IBM kills off its first cloud storage platform
      Via ITBusinessEdge: Time to Rein in Cloud Storage
      Via SerchCloudStorge: Ctera Networks’ file-sharing services gain intelligent cache
      Via StorageIOblog: Who Will Be At Top Of Storage World Next Decade?

      Videos and podcasts at storageio.tv also available via Applie iTunes.

      Human Face of Big Data
      Human Face of Big Data (Book review)

      Seven Databases in Seven weeks
      Seven Databases in Seven Weeks (Book review)

      Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

      Software Defined Data Infrastructure Essentials Book SDDC

      Wrap up and summary

      Object and cloud storage are in your future, the questions are when, where, with what and how among others.

      Watch for more content and links to be added here soon to this object storage center page including posts, presentations, pod casts, polls, perspectives along with services and product solutions profiles.

      Ok, nuff said, for now.

      Gs

      Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

      All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

    Posted on

    Cloud and Object storage are in your future, what are some questions?

    Cloud and Object storage are in your future, what are some questions?

    server storage I/O trends

    IMHO there is no doubt that cloud and object storage are in your future, what are some questions?

    Granted, what type of cloud and object storage or service along with for work or entertainment are some questions.

    Likewise, what are your cloud and object storage concerns (assuming you already have heard the benefits)?

    Some other questions include when, where for different applications workload needs, as well as how and with what among others.

    Keep in mind that there are many aspects to cloud storage and they are not all object, likewise, there are many facets to object storage.

    Recently I did a piece over at InfoStor titled Cloud Storage Concerns, Considerations and Trends that looks at the above among other items including:

    • Is cloud storage cheaper than traditional storage?
    • How do you access cloud object storage from legacy block and file applications?
    • How do you implement on-site cloud storage?
    • Is enterprise file sync and share (EFSS) safe and secure?
    • Does cloud storage need to be backed up and protected?
    • What geographic location requirements or regulations apply to you?

    When it comes to cloud computing and, in particular, cloud storage, context matters. Conversations are necessary to discuss concerns, as well as discuss various considerations, options and alternatives. People often ask me questions about the best cloud storage to use, concerns about privacy, security, performance and cost.

    Some of the most common cloud conversations topics involve context :

    • Public, private or hybrid cloud; turnkey subscription service or do it yourself (DIY)?
    • Storage, compute server, networking, applications or development tools?
    • Storage application such as file sync and share like Dropbox?
    • Storage resources such as table, queues, objects, file or block?
    • Storage for applications in the cloud, on-site or hybrid?

    Continue reading Cloud Storage Concerns, Considerations and Trends over at InfoStor.

    Where To Learn More

    Additional related content can be found at:

    What This All Means

    As I mentioned above, cloud and object storage are in your future, granted your future may not rely on just cloud or object storage. Take a few minutes to check out some of the conversation topics, tips and trends in my piece over at InfoStor Cloud Storage Concerns, Considerations and Trends along with more material at www.objectstoragecenter.com.

    Btw, what are your questions, comments, concerns, claims or caveats as part of cloud and object storage conversations?

    Ok, nuff said, for now…

    Cheers
    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, vSAN and VMware vExpert. Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved

    Posted on

    Welcome to the Cloud Bulk Object Storage Resources Center

    Updated 8/31/19

    Cloud Bulk Big Data Software Defined Object Storage Resources

    server storage I/O trends Object Storage resources

    Welcome to the Cloud, Big Data, Software Defined, Bulk and Object Storage Resources Center Page objectstoragecenter.com.

    This object storage resources, along with software defined, cloud, bulk, and scale-out storage page is part of the server StorageIOblog microsite collection of resources. Software-defined, Bulk, Cloud and Object Storage exist to support expanding and diverse application data demands.

    Other related resources include:

  • Software Defined, Cloud, Bulk and Object Storage Fundamentals
  • Software Defined Data Infrastructure Essentials book (CRC Press)
  • Cloud, Software Defined, Scale-Out, Object Storage News Trends

    • Object storage SDDC SDDI
    Via Software Defined Data Infrastructure Essentials (CRC Press 2017)

    Bulk, Cloud, Object Storage Solutions and Services

    There are various types of cloud, bulk, and object storage including public services such as Amazon Web Services (AWS) Simple Storage Service (S3), Backblaze, Google, Microsoft Azure, IBM Softlayer, Rackspace among many others. There are also solutions for hybrid and private deployment from Cisco, Cloudian, CTERA, Cray, DDN, Dell EMC, Elastifile, Fujitsu, Vantera/HDS, HPE, Hedvig, Huawei, IBM, NetApp, Noobaa, OpenIO, OpenStack, Quantum, Rackspace, Rozo, Scality, Spectra, Storpool, StorageCraft, Suse, Swift, Virtuozzo, WekaIO, WD, among many others.

    Bulk Cloud Object storage SDDC SDDI
    Via Software Defined Data Infrastructure Essentials (CRC Press 2017)

    Cloud products and services among others, along with associated data infrastructures including object storage, file systems, repositories and access methods are at the center of bulk, big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.

    Object Context Matters

    Before discussing Object Storage lets take a step back and look at some context that can clarify some confusion around the term object. The word object has many different meanings and context, both inside of the IT world as well as outside. Context matters with the term object such as a verb being a thing that can be seen or touched as well as a person or thing of action or feeling directed towards.

    Besides a person, place or physical thing, an object can be a software-defined data structure that describes something. For example, a database record describing somebody’s contact or banking information, or a file descriptor with name, index ID, date and time stamps, permissions and access control lists along with other attributes or metadata. Another example is an object or blob stored in a cloud or object storage system repository, as well as an item in a hypervisor, operating system, container image or other application.

    Besides being a verb, an object can also be a noun such as disapproval or disagreement with something or someone. From an IT context perspective, an object can also refer to a programming method (e.g. object-oriented programming [oop], or Java [among other environments] objects and classes) and systems development in addition to describing entities with data structures.

    In other words, a data structure describes an object that can be a simple variable, constant, complex descriptor of something being processed by a program, as well as a function or unit of work. There are also objects unique or with context to specific environments besides Java or databases, operating systems, hypervisors, file systems, cloud and other things.

    The Need For Bulk, Cloud and Object Storage

    There is no such thing as an information recession with more data being generated, moved, processed, stored, preserved and served, granted there are economic realities. Likewise as a society our dependence on information being available for work or entertainment, from medical healthcare to social media and all points in between continues to increase (check out the Human Face of Big Data).

    In addition, people and data are living longer, as well as getting larger (hence little data, big data and very big data). Cloud products and services along with associated object storage, file systems, repositories and access methods are at the center of big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.

    Click here to view (and hear) more content including cloud and object storage fundamentals

    Click here to view software defined, bulk, cloud and object storage trend news

    cloud object storage

    Where to learn more

    The following resources provide additional information about big data, bulk, software defined, cloud and object storage.



    Via InfoStor: Object Storage Is In Your Future
    Via FujiFilm IT Summit: Software Defined Data Infrastructures (SDDI) and Hybrid Clouds
    Via MultiChannel: After ditching cloud business, Verizon inks Virtual Network Services deal with Amazon
    Via MultiChannel: Verizon Digital Media Services now offers integrated Microsoft Azure Storage
    Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
    Via InfoStor: Cloud Storage Concerns, Considerations and Trends
    Via InfoStor: Object Storage Is In Your Future
    Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
    Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
    Cloud conversations: AWS EBS, Glacier and S3 overview
    AWS (Amazon) storage gateway, first, second and third impressions
    Cloud and Virtual Data Storage Networking (CRC Book)

    View more news, trends and related cloud object storage activity here.

    Videos and podcasts at storageio.tv also available via Applie iTunes.

    Human Face of Big Data
    Human Face of Big Data (Book review)

    Seven Databases in Seven weeks Seven Databases in Seven Weeks (Book review)

    Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

    Software Defined Data Infrastructure Essentials Book SDDC

    What This All Means

    Object and cloud storage are in your future, the questions are when, where, with what and how among others.

    Watch for more content and links to be added here soon to this object storage center page including posts, presentations, pod casts, polls, perspectives along with services and product solutions profiles.

    Ok, nuff said, for now.

    Gs

    Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.