ceph Archives

March 7, 2017October 18, 2024

Software Defined, Bulk, Cloud, Scale Out, Object Storage Fundamentals

Cloud, Bulk, Scale-Out, Object Storage Fundamentals

Welcome to the Cloud, Big Data, Software Defined, scale-out, Bulk and Object Storage Fundamentals page.

This page contains various resources, tips, essential topics pertaining to Software Defined, scale-out, Cloud, Bulk and Object and blob Storage Fundamentals. Other resources pertaining to Software Defined, scale-out, Cloud, Bulk and Object Storage include:

www.objectstoragecenter.com
Software Defined Data Infrastructure Essentials book (CRC Press)
Cloud, Software Defined, Scale-Out, Object Storage News Trends

There are various types of cloud, bulk and object storage including public services such as Amazon Web Services (AWS) Simple Storage Service (S3), Google, Microsoft Microsoft Azure, IBM Softlayer, Rackspace among many others. There are also solutions for hybrid and private deployment from Cisco, Cloudian, Fujifilm, DDN, Dell EMC, Fujitsu, HDS, HPE, IBM, NetApp, Noobaa, OpenStack, Quantum, Rackspace, Scality, Seagate, Spectra, Storpool, Suse, Swift and WD among others.

Cloud products and services among others, along with associated data infrastructures including object storage, file systems, repositories and access methods are at the center of bulk, big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.

Cloud Object Storage Fundamentals Access and Architectures

There are many facets to object storage including technology implementation, products, services, access and architectures for various applications and use scenarios.

- Project or Account – Top of the hierarchy that can represent the owner or billing information for a service that where buckets are also attached.
- Region – Location where data is stored that can include one or more data centers also known as Availability Zones.

Moving and Replicating Buckets/Containers, Subfolders and Objects

- Availability Zone (AZ) or data center or server that implement durability and accessibility for availability within a region.

Example of Regions and Availability Zones (AZs)

- Bucket or Container – Where objects or sub-folders containing objects are attached and accessed.

Object storage fundamentals sddc and cloud software defined

- Sub-folder – While object storage can be located in a flat namespace for commonality and organization some solutions and service support the notion of sub-folder that resemble traditional directory hierarchy.
- Object – Byte (or bit) stream that can be as small as one byte to as large as several Tbytes (some solutions and services support up to 5TByte sized objects). The object contains whatever data in any organization along with metadata. Different solutions and services support from a couple hundred KBytes of meta-data to Mbytes worth of meta-data. Regarding what can be stored in an object, anything from files, videos, images, virtual disks (VMDKs, VHDX), ZIP or tar files, backup and archive save sets, executable images or ISO’s, anything you want.
- End-point – Where or what your software, application or tool and utilities along with gateways attach to for accessing buckets and objects.

object storage fundamentals, sddc and cloud storage example

A common theme for object storage is flexibility, along with scaling (performance, availability, capacity, economics) along with extensibility without compromise or complexity. From those basics, there are many themes and variations from how data is protected (RAID or no RAID, hardware or software), deployed as a service or as tin wrapped software (an appliance), optimized for archiving or video serving or other applications.

Many facets of cloud and object storage access

One aspect of object and cloud storage is accessing or using object methods including application programming interfaces (API’s) vs. traditional block (LUN) or NAS (file) based approaches. Keep in mind that many object storage systems, software, and services support NAS file-based access including NFS, CIFS, HDFS among others for compatibility and ease of use.

Likewise various API’s can be found across different object solutions, software or services including Amazon Web Services (AWS) Simple Storage Service (S3) HTTP REST based, among others. Other API’s will vary by specific vendor or product however can include IOS (e.g. Apple iPhone and iPad), WebDav, FTP, JSON, XML, XAM, CDMI, SOAP, and DICOM among others. Another aspect of object and cloud storage are expanded and dynamic metadata.

While traditional file systems and NAS have simple or fixed metadata, object and cloud storage systems, services and solutions along with some scale-out file systems have ability to support user defined metadata. Specific systems, solutions, software, and services will vary on the amount of metadata that could range on the low-end from 100s of KBytes to tens or more Mbytes.

cloud object storage

Where to learn more

The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

Click here to view software defined, bulk, cloud and object storage trend news.

StorageIO Founder Greg Schulz: File Services on Object Storage with HyperFile

Via InfoStor: Object Storage Is In Your Future
Via FujiFilm IT Summit: Software Defined Data Infrastructures (SDDI) and Hybrid Clouds
Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
Via InfoStor: Cloud Storage Concerns, Considerations and Trends
Via InfoStor: Object Storage Is In Your Future
Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
Cloud conversations: AWS EBS, Glacier and S3 overview
AWS (Amazon) storage gateway, first, second and third impressions
Cloud and Virtual Data Storage Networking (CRC Book)
Via ChannelPartnersOnline: Selling Software-Defined Storage: Not All File Systems Are the Same
Via ITProPortal: IBM kills off its first cloud storage platform
Via ITBusinessEdge: Time to Rein in Cloud Storage
Via SerchCloudStorge: Ctera Networks’ file-sharing services gain intelligent cache
Via StorageIOblog: Who Will Be At Top Of Storage World Next Decade?

Videos and podcasts at storageio.tv also available via Applie iTunes.

Human Face of Big Data (Book review)

Seven Databases in Seven Weeks (Book review)

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Wrap up and summary

Object and cloud storage are in your future, the questions are when, where, with what and how among others.

Watch for more content and links to be added here soon to this object storage center page including posts, presentations, pod casts, polls, perspectives along with services and product solutions profiles.

Ok, nuff said, for now.

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

January 21, 2017March 7, 2022

SDx Summit London UK (Planning and Enabling Your Journey to Software Defined)

Planning and Enabling Your Journey to Software Defined)

server storage I/O trends

Will 2017 be there year of all software-defined X (e.g. SDx) where X can be everything from data centers (SDDC), data infrastructures (SDDI), infrastructure (SDI), storage (SDS), network (SDN) or marketing (SDM) among others? What about IoT, IoD, ByoD, ByoL (bring your own license), MaaS (metal as a service), clouds, containers, object storage, OpenStack, Mesos, Docker, Kubernetes, NVMe, flash SSD, SCM (Storage Class Memory) among other buzzword bingo terms, technologies and trends, will 2017 be there year for those among others?

What is safe to say is that the above buzzword items, topics, trends, technologies, tools and techniques are in your future, what varies is when, where, how, why, with what and whom to assist you on your journey.

On January 26 2017 join me and others at the Savoy hotel in London UK for the SDx summit organized by Wipro.

My presentation titled Planning and Enabling Your Journey to SDx will have a theme of Transiting from Hype and Marketing Hope to Deployment and Management. In other words, moving beyond SDBS and SDM to how to prepare, plan and what you can do today including hybrid deployments. Some of the topics, themes, trends, technologies, tools and tips in my discussion will include among others:

Software Defined Management and Data Protection
How to pack and prepare for your Software Defined Journey
Be prepared, plan for the unexpected, manage your journey
Learn the local language, expand your trade craft (skills)
Moving and migrating (brownfield) vs. start from scratch (greenfield)
ByoD, DiY, IoD, IoT, Cloud and Container conversations
What you can do today to prepare for your upcoming journey

Where To Learn More

Learn more and register here for the London UK SDx summit.

What This All Means

Regardless of if 2017 will be the year of SDx or any of the other industry popular buzz term trends, technologies and techniques, it is time to start planning as well as preparing. This means identifying questions, concerns and learning about the new tools and technologies that can be used in new ways, while also leveraging old things in new ways to enable a resilient, scalable, flexible as well as cost-effective data infrastructure. For those of you in the London UK area, learn more about the SDx summit organized by Wipro here and hope to see you there.

Ok, nuff said, for now…

Cheers
Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, vSAN and VMware vExpert. Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

May 17, 2016March 7, 2022

2016 Going Dutch Cloud Virtualization Server Storage I/O Seminars

server storage I/O trends

In June 2016 Brouwer Storage Consultancy is organizing their yearly spring seminar workshops in Nijkerk Holland (south of Amsterdam, near Utrecht and Amersfoort) with myself among others presenting.

Brouwer Consultancy

Cloud Virtualization Server Storage I/O Seminars
For this series of seminar workshops, there are four sessions, two being presented by myself, and two others in conjunction with Reduxio as well as Fujitsu & SJ Solutions.

Agenda, How To Register and Where To Learn More

The vendor sponsored sessions will consist of about 50% content being independent presented by myself and Gert Brouwer, the balance by the event sponsors as well as their partners. All presentation and associated content including handouts will be in English.

There will be 4 seminar workshop sessions, two of those are paid sessions dedicated to Greg Schulz and the other two are free (sponsored) sessions where 50% of the content is sponsored (Reduxio, Fujitsu & SJ Solutions) and the other 50% will be independent (Greg Schulz & Gert Brouwer).

Thursday June 9th – Server StorageIO Trends and Updates

Server Storage I/O Fundamental Trends V2.016 and Updates. What’s New, What’s the buzz, what you need to know about. From Speeds and Feeds, Slots and Watts to Who’s doing what. Event Location: Golden Tulip Ampt van Nijkerk Hotel, Berencamperweg 4, 3861MC, Nijkerk. Learn more here (PDF abstract and topics to be covered).

Thursday June 10th – Converged Day

Converged Day – Moving beyond Hyper-Converged Hype and Server Storage I/O Decision Making Strategies. Event Location: Golden Tulip Ampt van Nijkerk Hotel, Berencamperweg 4, 3861MC, Nijkerk. Learn more here (PDF abstract and topics to be covered).

Brouwer and Server StorageIO Seminar Sessions De Roode Schuur

Tuesday June 14th – Round Table Vendor Session with Reduxio

Symposium Workshop – Round Table Vendor Session with Reduxio – Are some solutions really ‘a Paradigm shift’ or ‘new and revolutionary” as they claim to be, or is it just more of the same (e.g. evolutionary)? – Presentations and discussions led by Greg Schulz (StorageIO), Reduxio and Brouwer Storage Consultancy. (Free, sponsored Session, Access for end-users only). Event Location: Hotel & Gasterij De Roode Schuur, Oude Barneveldseweg 98, 3862PS Nijkerk. Learn more here (PDF abstract and topics to be covered).

Wednesday June 15th – Software Defined Data Center Symposium Workshop

Software Defined Data Center Symposium Workshop – Round Table Vendor Session with Fujitsu & SJ Solutions
With subjects like Openstack, Ceph, distributed object storage, Bigdata, Hyper-Converged Infrastructure (HCI), Converged Infrastructure (CI), Software defined storage (SDS) and Network (SDN and NFV), this round table format workshop seminars explores these and other related topics including what to use when, where, why and how. Presentations by Greg Schulz (StorageIO), SJ Solutions & Fujitsu and Brouwer Storage Consultancy. Event Location: Hotel & Gasterij De Roode Schuur, Oude Barneveldseweg 98, 3862PS Nijkerk. Learn more here (PDF abstract and topics to be covered).

For more information, abstracts/agenda, registration and the specific locations for all the above events click here.

Brouwer and Server StorageIO Sessions Ampt van Nijkerk

What This All Means

There is a lot of things occurring in the IT industry from physical to software defined clouds, containers and virtualization, nonvolatile memory (NVM) including flash SSD among others. These series of interactive educational workshop seminars converge on Nijkerk Holland combing content discussions from strategy, planning decision making, to what’s new (and old) that can be used in new ways, as well as some trends, speeds and feeds along with practicality for your environment.

Brouwer Consultancy

I Look forward to seeing you in Nijkerk and Europe during June 2016, in the meantime, contact Brouwer Storage Consultancy for more information on the above sessions as well as to arrange private discussions or meetings.

Ok, nuff said, for now…

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

May 1, 2016March 7, 2022

Ubuntu 16.04 LTS (aka Xenial Xerus) What’s In The Bits and Bytes?

server storage I/O trends

Ubuntu 16.04 LTS (aka Xenial Xerus) was recently released (you can get the bits or software download here). Ubuntu is available in various distributions including as a server, workstation or desktop among others that can run bare metal on a physical machine (PM), virtual machine (VM) or as a cloud instance via services such as Amazon Web Services (AWS) as well as Microsoft Azure among others.

Refresh, What is Ubuntu

For those not familiar or who need a refresh, Ubuntu is an open source Linux distribution with the company behind it called Canonical. The Ubuntu software is a Debian based Linux distribution with Unity (user interface). Ubuntu is available across different platform architecture from industry standard Intel and AMD x86 32bit and 64bit to ARM processors and even the venerable IBM zSeriues (aka zed) mainframe as part of LinuxOne.

As a desktop, some see or use Ubuntu as an open source alternative to desktop interfaces based on those from Microsoft such as Windows or Apple.

As a server Ubuntu can be deployed from traditional applications to cloud, converged and many others including as a docker container, Ceph or OpenStack deployment platform. Speaking of Microsoft and Windows, if you are a *nix bash type person yet need (or have) to work with Windows, bash (and more) are coming to Windows 10. Ubuntu desktop GUI or User Interface options include Unity along with tools such as Compiz and LibreOffice (an alternative to Microsoft Office).

What’s New In the Bits and Bytes (e.g. Software)

Ubuntu 16.04 LTS is based on the Linux 4.4 kernel, that also includes Python 3, Ceph Jewel (block, file and object storage) and OpenStack Mitaka among other enhancements. These and other fixes as well as enhancements include:

Libvirt 1.3.1
Qemu 2.5
Open vSwitch 2.5.0
NginxLX2 2.0
Docker 1.10
PHP 7.9
MySQL 7.0
Juju 2.0
Golang 1.6 toolchain
OpenSSH 7.2p2 with legacy support along with cipher improvements, including 1024 bit diffie-hellman-group1-sha1 key exchange, ssh-dss, ssh-dss-cert
GNU toolchain
Apt 1.2

What About Ubuntu for IBM zSeries Mainframe

Ubuntu runs on 64 bit zSeries architecture with about 95% binary compatibility. If you look at the release notes, there are still a few things being worked out among known issues. However (read the release notes), Ubuntu 16.04 LTS has OpenStack and Ceph, means that those capabilities could be deployed on a zSeries.

Now some of you might think wait, how can Linux and Ceph among others work on a FICON based mainframe?

No worries, keep in mind that FICON the IBM zSeries server storage I/O protocol that co-exists on Fibre Channel along with SCSI_FCP (e.g. FCP) aka what most Open Systems people simply refer to as Fibre Channel (FC) works with the zOS and other operating systems. In the case of native Linux on zSeries, those systems can in fact use SCSI mode for accessing shared storage. In addition to the IBM LinuxOne site, you can learn more about Ubuntu running native on zSeries here on the Ubuntu site.

Where To Learn More

What This All Means

Ubuntu as a Linux distribution continues to evolve and increase in deployment across different environments. Some still view Ubuntu as the low-end Linux for home, hobbyist or those looking for a alternative desktop to Microsoft Windows among others. However Ubuntu is also increasingly being used in roles where other Linux distribution such as Red Hat Enterprise Linux (RHEL), SUSE and Centos among others have gained prior popularity.

In someway’s you can view RHEL as the first generation Linux distribution that gained popular in the enterprise with early adopters, followed by a second wave or generation of those who favored Centos among others such as the cloud crowd. Then there is the Ubuntu wave which is expanding in many areas along with others such as CoreOS. Granted with some people the preference between one Linux distribution vs. another can be as polarizing as Linux vs. Windows, OpenSystems vs. Mainframe vs. Cloud among others.

Having various Ubuntu distributions installed across different servers (in addition to Centos, Suse and others), I found the install and new capabilities of Ubuntu 16.04 LTS interesting and continue to explore the many new features, while upgrading some of my older systems.

Get the Ubuntu 16.04 LTS bits here to give a try or upgrade your existing systems.

Ok, nuff said

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

March 20, 2013June 7, 2022

Welcome to the Cloud Bulk Object Storage Resources Center

Updated 8/31/19

Cloud Bulk Big Data Software Defined Object Storage Resources

server storage I/O trends Object Storage resources

Welcome to the Cloud, Big Data, Software Defined, Bulk and Object Storage Resources Center Page objectstoragecenter.com.

This object storage resources, along with software defined, cloud, bulk, and scale-out storage page is part of the server StorageIOblog microsite collection of resources. Software-defined, Bulk, Cloud and Object Storage exist to support expanding and diverse application data demands.

Other related resources include:

Software Defined, Cloud, Bulk and Object Storage Fundamentals

Software Defined Data Infrastructure Essentials book (CRC Press)

Cloud, Software Defined, Scale-Out, Object Storage News Trends

Via Software Defined Data Infrastructure Essentials (CRC Press 2017)

Bulk, Cloud, Object Storage Solutions and Services

There are various types of cloud, bulk, and object storage including public services such as Amazon Web Services (AWS) Simple Storage Service (S3), Backblaze, Google, Microsoft Azure, IBM Softlayer, Rackspace among many others. There are also solutions for hybrid and private deployment from Cisco, Cloudian, CTERA, Cray, DDN, Dell EMC, Elastifile, Fujitsu, Vantera/HDS, HPE, Hedvig, Huawei, IBM, NetApp, Noobaa, OpenIO, OpenStack, Quantum, Rackspace, Rozo, Scality, Spectra, Storpool, StorageCraft, Suse, Swift, Virtuozzo, WekaIO, WD, among many others.

Bulk Cloud Object storage SDDC SDDI
Via Software Defined Data Infrastructure Essentials (CRC Press 2017)

Object Context Matters

Before discussing Object Storage lets take a step back and look at some context that can clarify some confusion around the term object. The word object has many different meanings and context, both inside of the IT world as well as outside. Context matters with the term object such as a verb being a thing that can be seen or touched as well as a person or thing of action or feeling directed towards.

Besides a person, place or physical thing, an object can be a software-defined data structure that describes something. For example, a database record describing somebody’s contact or banking information, or a file descriptor with name, index ID, date and time stamps, permissions and access control lists along with other attributes or metadata. Another example is an object or blob stored in a cloud or object storage system repository, as well as an item in a hypervisor, operating system, container image or other application.

Besides being a verb, an object can also be a noun such as disapproval or disagreement with something or someone. From an IT context perspective, an object can also refer to a programming method (e.g. object-oriented programming [oop], or Java [among other environments] objects and classes) and systems development in addition to describing entities with data structures.

In other words, a data structure describes an object that can be a simple variable, constant, complex descriptor of something being processed by a program, as well as a function or unit of work. There are also objects unique or with context to specific environments besides Java or databases, operating systems, hypervisors, file systems, cloud and other things.

The Need For Bulk, Cloud and Object Storage

There is no such thing as an information recession with more data being generated, moved, processed, stored, preserved and served, granted there are economic realities. Likewise as a society our dependence on information being available for work or entertainment, from medical healthcare to social media and all points in between continues to increase (check out the Human Face of Big Data).

In addition, people and data are living longer, as well as getting larger (hence little data, big data and very big data). Cloud products and services along with associated object storage, file systems, repositories and access methods are at the center of big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.

Click here to view (and hear) more content including cloud and object storage fundamentals

Click here to view software defined, bulk, cloud and object storage trend news

cloud object storage

Where to learn more

The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

Via InfoStor: Object Storage Is In Your Future
Via FujiFilm IT Summit: Software Defined Data Infrastructures (SDDI) and Hybrid Clouds
Via MultiChannel: After ditching cloud business, Verizon inks Virtual Network Services deal with Amazon
Via MultiChannel: Verizon Digital Media Services now offers integrated Microsoft Azure Storage
Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look
Via InfoStor: Cloud Storage Concerns, Considerations and Trends
Via InfoStor: Object Storage Is In Your Future
Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage
Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements
Cloud conversations: AWS EBS, Glacier and S3 overview
AWS (Amazon) storage gateway, first, second and third impressions
Cloud and Virtual Data Storage Networking (CRC Book)

View more news, trends and related cloud object storage activity here.

Videos and podcasts at storageio.tv also available via Applie iTunes.

Human Face of Big Data (Book review)

Seven Databases in Seven Weeks (Book review)

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

What This All Means

Object and cloud storage are in your future, the questions are when, where, with what and how among others.

Ok, nuff said, for now.

November 26, 2012July 28, 2021

Ceph Day Amsterdam 2012 (Object and cloud storage)

Recently while I was in Europe presenting some sessions at conferences and doing some seminars, I was invited by Ed Saipetch (@edsai) of Inktank.com to attend the first Ceph Day in Amsterdam.

Ceph day image

As luck or fate would turn out, I was in Nijkerk which is about an hour train ride from Amsterdam central station plus a free day in my schedule. After a morning train ride and nice walk from Amsterdam Central I arrived at the Tobacco Theatre (a former tobacco trading venue) where Ceph Day was underway, and in time for lunch of Krokettens sandwich.

Lets take a quick step back and address for those not familiar what is Ceph (Cephalanthera) and why it was worth spending a day to attend this event. Ceph is an open source distributed object scale out (e.g. cluster or grid) software platform running on industry standard hardware.

Ceph is used for deploying object storage, cloud storage and managed services, general purpose storage for research, commercial, scientific, high performance computing (HPC) or high productivity computing (commercial) along with backup or data protection and archiving destinations. Other software similar in functionality or capabilities to Ceph include OpenStack Swift, Basho Riak CS, Cleversafe, Scality and Caringo among others. There are also the tin wrapped software (e.g. appliances or pre-packaged) solutions such as Dell DX (Caringo), DataDirect Networks (DDN) WOS, EMC ATMOS and Centera, Amplidata and HDS HCP among others. From a service standpoint, these solutions can be used to build services similar Amazon S3 and Glacier, Rackspace Cloud files and Cloud Block, DreamHost DreamObject and HP Cloud storage among others.

At the heart of Ceph is RADOS a distributed object store that consists of peer nodes functioning as object storage devices (OSD). Data can be accessed via REST (Amazon S3 like) APIs, Libraries, CEPHFS and gateway with information being spread across nodes and OSDs using a CRUSH based algorithm (note Sage Weil is one of the authors of CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data). Ceph is scalable in terms of performance, availability and capacity by adding extra nodes with hard disk drives (HDD) or solid state devices (SSDs). One of the presentations pertained to DreamHost that was an early adopter of Ceph to make their DreamObjects (cloud storage) offering.

In addition to storage nodes, there are also an odd number of monitor nodes to coordinate and manage the Ceph cluster along with optional gateways for file access. In the above figure (via DreamHost), load balancers sit in front of gateways that interact with the storage nodes. The storage node in this example is a physical server with 12 x 3TB HDDs each configured as a OSD.

In the DreamHost example above, there are 90 storage nodes plus 3 management nodes, the total raw storage capacity (no RAID) is about 3PB (12 x 3TB = 36TB x 90 = 3.24PB). Instead of using RAID or mirroring, each objects data is replicated or copied to three (e.g. N=3) different OSDs (on separate nodes), where N is adjustable for a given level of data protection, for a usable storage capacity of about 1PB.

Note that for more usable capacity and lower availability, N could be set lower, or a larger value of N would give more durability or data protection at higher storage capacity overhead cost. In addition to using JBOD configurations with replication, Ceph can also be configured with a combination of RAID and replication providing more flexibility for larger environments to balance performance, availability, capacity and economics.

One of the benefits of Ceph is the flexibility to configure it how you want or need for different applications. This can be in a cost-effective hardware light configuration using JBOD or internal HDDs in small form factor generally available servers, or high density servers and storage enclosures with optional RAID adapters along with SSD. This flexibility is different from some cloud and object storage systems or software tools which take a stance of not using or avoiding RAID vs. providing options and flexibility to configure and use the technology how you see fit.

Here are some links to presentations from Ceph Day:
Introduction and Welcome by Wido den Hollander
Ceph: A Unified Distributed Storage System by Sage Weil
Ceph in the Cloud by Wido den Hollander
DreamObjects: Cloud Object Storage with Ceph by Ross Turk
Cluster Design and Deployment by Greg Farnum
Notes on Librados by Sage Weil

While at Ceph day, I was able to spend a few minutes with Sage Weil Ceph creator and founder of inktank.com to record a pod cast (listen here) about what Ceph is, where and when to use it, along with other related topics. Also while at the event I had a chance to sit down with Curtis (aka Mr. Backup) Preston where we did a simulcast video and pod cast. The simulcast involved Curtis recording this video with me as a guest discussing Ceph, cloud and object storage, backup, data protection and related themes while I recorded this pod cast.

One of the interesting things I heard, or actually did not hear while at the Ceph Day event that I tend to hear at related conferences such as SNW is a focus on where and how to use, configure and deploy Ceph along with various configuration options, replication or copy modes as opposed to going off on erasure codes or other tangents. In other words, instead of focusing on the data protection protocol and algorithms, or what is wrong with the competition or other architectures, the Ceph Day focused was removing cloud and object storage objections and enablement.

Where do you get Ceph? You can get it here, as well as via 42on.com and inktank.com.

Thanks again to Sage Weil for taking time out of his busy schedule to record a pod cast talking about Ceph, as well 42on.com and inktank for hosting, and the invitation to attend the first Ceph Day in Amsterdam.

Returning to Amsterdam central station after Ceph Day

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

November 12, 2012November 26, 2023

Mr. Backup (Curtis Preston) goes back to Ceph School

Now also available via

This is a new episode in the continuing StorageIO industry trends and perspectives pod cast series (you can view more episodes or shows along with other audio and video content here) as well as listening via iTunes or via your preferred means using this RSS feed (https://storageio.com/StorageIO_Podcast.xml)

In this episode, I am at the Ceph day in Amsterdam Holland event at the Tobacco Theatre hosted by on42.com and inktank.com.

Ceph Day Amsterdam 2012

My guest for this episode is Curtis (Mr. Backup) Preston (@wcpreston) of Backup School and Backup Central fame where we discuss what is Ceph and object storage, cloud storage, file systems, backup and data protection along with dinner we had at an Indonesian restaurant .

Dinner Restaurant Blauw Utrecht Netherlands
Mr Backup getting ready to compress and dedupe dinner

The dinner we are referring to was at Restaurant Blauw in Utrecht Holland (click here) where Curtis and me were joined by Hans De Leenher @hansdeleenher of Veeam (thanks again for the dinner, that was a disclosure btw ;) ).

Note that this is a special episode in that while I’m recording the pod cast, Curtis is recording a video of our discussion for his truebit.tv site that you can view here.

Click here (right-click to download MP3 file) or on the microphone image to listen to the conversation with Curtis and myself.

Also available via

Watch (and listen) for more StorageIO industry trends and perspectives audio blog posts pod casts and other upcoming events. Also be sure to heck out other related pod casts, videos, posts, tips and industry commentary at StorageIO.com and StorageIOblog.com.

Also check out the companion to this pod cast where I meet up with Ceph Creator Sage Weil while at Ceph Day.

Enjoy this episode Mr. Backup (Curtis Preston) goes back to Ceph School.

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

November 3, 2012March 7, 2022

Ceph Day in Amsterdam and Sage Weil on Object Storage

Now also available via

In this episode, I am at the Ceph day in Amsterdam Holland event at the Tobacco Theatre. My guest for this episode is Ceph (Cephalanthera) creator Sage Weil who is also the founder of inktank.com that provides services and support for the open source based Ceph project.

For those not familiar with Ceph, it is an open source distributed object scale out software platform that can be used for deploying cloud and managed services, general purpose storage for research, commercial, scientific, high performance computing (HPC) or high productivity computing (commercial) along with backup or data protection and archiving destinations.

During our conversation Sage presents an overview of what Ceph is (e.g. Ceph for non Dummies), where and how it can be used, some history of the project and how it fits in with or provides an alternative to other solutions. Sage also talks about the business or commercial considerations for open source based projects, importance of community and having good business mentors and partners as well as staying busy with his young family.

If you are a Ceph fan, gain more insight into Sage along with Ceph day sponsors Inktank and 42on. On the other hand, if you new to object storage, open source storage software or cloud storage, listen in to gain perspectives of where technology such as Ceph fits for public, private, hybrid or traditional environments.

Click here (right-click to download MP3 file) or on the microphone image to listen to the conversation with Sage and myself.

Also available via

Enjoy this episode Ceph Day in Amsterdam with Sage Weil.

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

Cloud, Bulk, Scale-Out, Object Storage Fundamentals

Cloud Object Storage Fundamentals Access and Architectures

Many facets of cloud and object storage access

Where to learn more

Wrap up and summary

Share this:

Planning and Enabling Your Journey to Software Defined)

Where To Learn More

What This All Means

Share this:

2016 Going Dutch Cloud Virtualization Server Storage I/O Seminars

Cloud Virtualization Server Storage I/O SeminarsFor this series of seminar workshops, there are four sessions, two being presented by myself, and two others in conjunction with Reduxio as well as Fujitsu & SJ Solutions.

Agenda, How To Register and Where To Learn More

Thursday June 9th – Server StorageIO Trends and Updates

Thursday June 10th – Converged Day

Tuesday June 14th – Round Table Vendor Session with Reduxio

Wednesday June 15th – Software Defined Data Center Symposium Workshop

What This All Means

Share this:

Ubuntu 16.04 LTS (aka Xenial Xerus) What’s In The Bits and Bytes?

Refresh, What is Ubuntu

What’s New In the Bits and Bytes (e.g. Software)

What About Ubuntu for IBM zSeries Mainframe

Where To Learn More

What This All Means

Share this:

Cloud Bulk Big Data Software Defined Object Storage Resources

Bulk, Cloud, Object Storage Solutions and Services

Object Context Matters

The Need For Bulk, Cloud and Object Storage

Where to learn more

What This All Means

Share this:

Share this:

Share this:

Share this:

Cloud Virtualization Server Storage I/O Seminars
For this series of seminar workshops, there are four sessions, two being presented by myself, and two others in conjunction with Reduxio as well as Fujitsu & SJ Solutions.