Tag: Archiving

Posted on

Have you heard about the new CLOUD Act data regulation?

Have you heard about the new CLOUD Act data regulation?

new CLOUD Act data regulation

Have you heard about the new CLOUD Act data regulation?

The new CLOUD Act data regulation became law as part of the recent $1.3 Trillion (USD) omnibus U.S. government budget spending bill passed by Congress on March 23, 2018 and signed by President of the U.S. (POTUS) Donald Trump in March.

CLOUD Act is the acronym for Clarifying Lawful Overseas Use of Data, not to be confused with initiatives such as U.S. federal governments CLOUD First among others which are focused on using cloud, securing and complying (e.g. FedRAMP among others). In other words, the new CLOUD Act data regulation pertains to how data stored by cloud or other service providers can be accessed by law environment officials (LEO).

U.S. Supreme court
Supreme Court of the U.S. (SCOTUS) Image via https://www.supremecourt.gov/

CLOUD Act background and Stored Communications Act

After the signing into law of CLOUD Act, the US Department of Justice (DOJ) has asked the Supreme Court of the U.S. (SCOTUS) to dismiss the pending case against Microsoft (e.g., Azure Cloud). The case or question in front of SCOTUS pertained to whether LEO can search as well as seize information or data that is stored overseas or in foreign counties.

As a refresher, or if you had not heard, SCOTUS was asked to resolve if a service provider who is responding to a warrant based on probable cause under the 1986 era Stored Communications Act, is required to provide data in its custody, control or possession, regardless of if stored inside, or, outside the US.

Microsoft Azure Regions and software defined data infrastructures
Microsoft Azure Regions via Microsoft.com

This particular case in front of SCOTUS centered on whether Microsoft (a U.S. Technology firm) had to comply with a court order to produce emails (as part of an LEO drug investigation) even if those were stored outside of the US. In this particular situation, the emails were alleged to have been stored in a Microsoft Azure Cloud Dublin Ireland data center.

For its part, Microsoft senior attorney Hasan Ali said via FCW “This bill is a significant step forward in the larger global debate on what our privacy laws should look like, even if it does not go to the highest threshold". Here are some additional perspectives via Microsoft Brad Smith on his blog along with a video.

What is CLOUD Act

Clarifying Lawful Overseas Use of Data is the new CLOUD Act data regulation approved by Congress (House and Senate) details can be read here and here respectively with additional perspectives here.

The new CLOUD Act law allows for POTUS to enter into executive agreements with foreign governments about data on criminal suspects. Granted what is or is not a crime in a given country will likely open Pandora’s box of issues. For example, in the case of Microsoft, if an agreement between the U.S. and Ireland were in place, and, Ireland agreed to release the data, it could then be accessed.

Now, for some who might be hyperventilating after reading the last sentence, keep this in mind that if you are overseas, it is up to your government to protect your privacy. The foreign government must have an agreement in place with the U.S. and that a crime has or had been committed, a crime that both parties concur with.

Also, keep in mind that is also appeal processes for providers including that the customer is not a U.S. person and does not reside in the U.S. and the disclosure would put the provider at risk of violating foreign law. Also, keep in mind that various provisions must be met before a cloud or service provider has to hand over your data regardless of what country you reside, or where the data resides.

Where to learn more

Learn more about CLOUD Act, cloud, data protection, world backup day, recovery, restoration, GDPR along with related data infrastructure topics for cloud, legacy and other software defined environments via the following links:

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What this all means and wrap-up

Is the new CLOUD Act data regulation unique to Microsoft Azure Cloud?

No, it also applies to Amazon Web Services (AWS), Google, IBM Softlayer Cloud, Facebook, LinkedIn, Twitter and the long list of other service providers.

What about GDPR?

Keep in mind that the new Global Data Protection Regulations (GDPR) go into effect May 25, 2018, that while based out of the European Union (EU), have global applicability across organizations of all size, scope, and type. Learn more about GDPR, Data Protection and its global impact here.

Thus, if you have not heard about the new CLOUD Act data regulation, now is the time to become aware of it.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

September 2017 Server StorageIO Data Infrastructure Update Newsletter



Server StorageIO September 2017 Data Infrastructure Update Newsletter

Volume 17, Issue IX (September 2017)

Hello and welcome to the September 2017 issue of the Server StorageIO update newsletter.

With September being generally known as back to school month, the two September event bookends were VMware VMworld and Microsoft Ignite with many other things in between. Needless to say, a lot has happened in and around data infrastructure topic areas since the August newsletter (here if you missed it). Here is a post covering some of the things that I participated with during September including presentations at events in Las Vegas (VMworld), New York City (Wipro SDx Summit), SNIA SDC in Santa Clara, Fujifilm Executive Summitt in Seattle, Minneapolis/St. Paul CMG along with other activities.

Software-Defined Data Infrastructure Essentials SDDI SDDC

One of the activities I participated in with while at VMworld in Las Vegas was a book signing event at the VMware bookstore of my new book Software Defined Data Infrastructure Essentials (CRC Press) available at Amazon.com and other global venues.

September has been a busy month pertaining data infrastructure including server storage I/O related trends, activities, news, perspectives and related topics, so let’s have a look at them.

In This Issue

Enjoy this edition of the Server StorageIO data infrastructure update newsletter.

Cheers GS

Data Infrastructure and IT Industry Activity Trends

Some recent Industry Activities, Trends, News and Announcements include:

The month started out with VMworld in Las Vegas (e.g. one of the event bookends for the month). Rather than a long list of announcements in this newsletter, check out this StorageIOblog post covering VMworld, VMware and Dell EMC and related news. As part of VMworld, VMware and Amazon Web Services (AWS) announced news about their partnership. AWS also had several other enhancements and new product announcements during september that can be found in this StorageIOblog post here.

AWS, Dell EMC and VMware were not the only ones making news or announcements during September. Startup NVMe based storage startup Apeiron has announced a Splunk appliance to boost log and analytics processing performance. Gigamon has extended its public cloud monitoring, insight awareness and analytics capabilities including support for Microsoft Azure.

For those looking for the latest new emerging data infrastructure vendors to watch, add Vexta to your list of NVMe based storage systems. Vexta talks a lot about NVMe particular for their backend (e.g. where data stored on NVM based devices accessed via NVMe), access of their storage system is via traditional Fibre Channel (FC) or emerging NVMe over fabric.

Long time data infrastructure server and storage vendor HDS (Hitachi Data Systems) is no more (at least in name) having re branded themselves as Vantara focusing on IoT and Cloud analytics besides their traditional data center focus. Vantara combines what was HDS, Hitachi Insight Group and Pentaho into a single unit effectively based in what was HDS as a new, repackaged, refocused business unit.

Another longtime data infrastructure solution and service provider IBM announced a new Linux only zSeries (ZED) mainframe solution. Some might think the Mainframe is dead, others that it can only run Linux as a virtual guest in a virtual machine. On the other hand some might recall that there are native Linux implementations on the ZED including Ubuntu among others.

Also note that while IBM zOS mainframe operating systems use FICON for storage access, native ZED Linux systems can use open systems based Fibre Channel (FC) e.g. SCSI command set protocols. Is the ZED based Linux for everybody or every environment? Probably not, however for those who have large-scale Linux needs, it might be worth a look to do a total cost of ownership analysis. If nothing else, do your homework, play your cards right and you might have some leverage with the x86 based server crowd when it comes to negotiating leverage.

Cloud storage gateway vendor Nasuni has landed another $38 Million USD in funding, hopefully that will enable them to start landing some new and larger customer revenues growing their business. Meanwhile storage startup Qumulo has announced extending their global file fabric name space to include spanning AWS.

Attala Systems has announced next generation software defined storage for data infrastructures for Telco environments. Percona has added an experimental release of their MySQL engine enhancing performance for high volume, write intensive workloads along with improved cost effectiveness.

Software defined storage vendor Datacore announced enhancements to support fast databases for online transaction processing (OLTP) along with analytics. Meanwhile Linux provider SUSE continues to expand its software defined storage story based around Ceph. Panasas has enhanced its scale out high performance cluster file system global name space for HPC environments with 20 PByte support. Another longtime storage vendor X-IO (formerly known as Xiotech) announced their 4th generation of their Intelligent Storage Element (ISE).

September wrapped up with Microsoft Ignite conference along with many updated, enhancements and new features for Azure, Azure Stack, Windows among others. Read more about those and other Microsoft September announcements here in this StorageIOblog post.

Check out other industry news, comments, trends perspectives here.

Server StorageIO Commentary in the news

Recent Server StorageIO industry trends perspectives commentary in the news.

Via CDW: Comments on Is Your Network About To Fail?
Via EnterpriseStorageForum: Comments on Data Storage and Big Data Analytics
Via InfoGoto: Comments on Cloud FOMO (Fear of missing out)
Via InfoGoto: Comments on Building a Modern Data Strategy
Via InfoGoto: Comments on the future of Multi-Cloud Computing
Via InfoGoto: Comments on AI, Machine Learning and Data management
Via InfoGoto: Comments on Your riskiest data might be in plain sight
Via InfoGoto: Comments on Data Management Too Much To Handle
Via InfoGoto: Comments on Google Cloud Platform Gaining Data Storage Momentum
Via InfoGoto: Comments on Singapore High Rise Data Centers
Via InfoGoto: Comments on New Tape Storage Capacity
Via EnterpriseStorageForum: Comments on 8 ways to save on cloud storage
Via EnterpriseStorageForum: Comments on Google Cloud Platform and Storage

View more Server, Storage and I/O trends and perspectives comments here

Server StorageIOblog Posts

Recent and popular Server StorageIOblog posts include:

In Case You Missed It #ICYMI

View other recent as well as past StorageIOblog posts here

Server StorageIO Data Infrastructure Tips and Articles

Recent Server StorageIO industry trends perspectives commentary in the news.

Via EnterpriseStorageForum: Comments on Who Will Rule the Storage World?
Via InfoGoto: Comments on Google Cloud Platform Gaining Data Storage Momentum
Via InfoGoto: Comments on Singapore High Rise Data Centers
Via InfoGoto: Comments on New Tape Storage Capacity
Via EnterpriseStorageForum: Comments on 8 ways to save on cloud storage
Via EnterpriseStorageForum: Comments on Google Cloud Platform and Storage

View more Server, Storage and I/O trends and perspectives comments here

Server StorageIO Recommended Reading (Watching and Listening) List

In addition to my own books including Software Defined Data Infrastructure Essentials (CRC Press 2017), the following are Server StorageIO recommended reading, watching and listening list items. The list includes various IT, Data Infrastructure and related topics.

Intel Recommended Reading List (IRRL) for developers is a good resource to check out.

Its October which means that it is also Blogtober, check out some of the blogs and posts occurring during October here.

Preston De Guise aka @backupbear is Author of several books has an interesting new site Foolsrushin.info that looks at topics including Ethics in IT among others. Check out his new book Data Protection: Ensuring Data Availability (CRC Press 2017).

Brendan Gregg has a great site for Linux performance related topics here.

Greg Knieriemen has a must read weekly blog, post, column collection of whats going on in and around the IT and data infrastructure related industries, Check it out here.

Interested in file systems, CIFS, SMB, SAMBA and related topics then check out Chris Hertels book on implementing CIFS here at Amazon.com

For those involved with VMware, check out Frank Denneman VMware vSphere 6.5 host resource guide-book here at Amazon.com.

I often mention in presentations a must have for anybody involved with software defined anything, or programming for that matter which is the Niklaus Wirth classic Algorithms + Data Structures = Programs that you can get on Amazon.com here.

Another great book to have is Seven Databases in Seven Weeks which not only provides an overview of popular NoSQL databases such as Cassandra, Mongo, HBASE among others, lots of good examples and hands on guides. Get your copy here at Amazon.com.

Watch for more more items to be added to the book shelf soon.

Events and Activities

Recent and upcoming event activities.

Nov. 2, 2017 – Webinar – Modern Data Protection for Hyper-Convergence
Sep. 21, 2017 – MSP CMG – Minneapolis MN
Sep. 20, 2017 – Webinar – BC, DR and Business Resiliency (BR) tips
Sep. 14, 2017 – Fujifilm IT Executive Summit – Seattle WA
Sep. 12, 2017 – SNIA Software Developers Conference (SDC) – Santa Clara CA
Sep. 7, 2017 – Wipro SDX – Enabling, Planning Your Software Defined Journey
August 28-30, 2017 – VMworld – Las Vegas

See more webinars and activities on the Server StorageIO Events page here.

Server StorageIO Industry Resources and Links

Useful links and pages:
Microsoft TechNet – Various Microsoft related from Azure to Docker to Windows
storageio.com/links – Various industry links (over 1,000 with more to be added soon)
objectstoragecenter.com – Cloud and object storage topics, tips and news items
OpenStack.org – Various OpenStack related items
storageio.com/downloads – Various presentations and other download material
storageio.com/protect – Various data protection items and topics
thenvmeplace.com – Focus on NVMe trends and technologies
thessdplace.com – NVM and Solid State Disk topics, tips and techniques
storageio.com/converge – Various CI, HCI and related SDS topics
storageio.com/performance – Various server, storage and I/O benchmark and tools
VMware Technical Network – Various VMware related items

Ok, nuff said, for now.

Cheers
Gs

Greg Schulz – Multi-year Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio.

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

Announcing Software Defined Data Infrastructure Essentials Book by Greg Schulz

New SDDI Essentials Book by Greg Schulz of Server StorageIO

Cloud, Converged, Virtual Fundamental Server Storage I/O Tradecraft

server storage I/O data infrastructure trends

Update 1/21/2018

Over the past several months I have posted, commenting, presenting and discussing more about Software Defined Data Infrastructure Essentials aka SDDI or SDDC and SDI. Now it is time to announce my new book (my 4th solo project), Software Defined Data Infrastructure Essentials Book (CRC Press). Software Defined Data Infrastructure Essentials is now generally available at various global venues in hardcopy, hardback print as well as various electronic versions including via Amazon and CRC Press among others. For those attending VMworld 2017 in Las Vegas, I will be doing a book signing, meet and greet at 1PM Tuesday August 29 in the VMworld book store, as well as presenting at various other fall industry events.

Software Defined Data Infrastructure Essentials Book Announcement

(Via Businesswire) Stillwater, Minnesota – August 23, 2017  – Server StorageIO, a leading independent IT industry advisory and consultancy firm, in conjunction with publisher CRC Press, a Taylor and Francis imprint, announced the release and general availability of “Software-Defined Data Infrastructure Essentials,” a new book by Greg Schulz, noted author and Server StorageIO founder.

Software Defined Data Infrastructure Essentials

The Software Defined Data Infrastructure Essentials book covers physical, cloud, converged (and hyper-converged), container, and virtual server storage I/O networking technologies, revealing trends, tools, techniques, and tradecraft skills.

Data Infrastructures Protect Preserve Secure and Serve Information
Various IT and Cloud Infrastructure Layers including Data Infrastructures

From cloud web scale to enterprise and small environments, IoT to database, software-defined data center (SDDC) to converged and container servers, flash solid state devices (SSD) to storage and I/O networking,, the book helps develop or refine hardware, software, services and management experiences, providing real-world examples for those involved with or looking to expand their data infrastructure education knowledge and tradecraft skills.

Software Defined Data Infrastructure Essentials book topics include:

  • Cloud, Converged, Container, and Virtual Server Storage I/O networking
  • Data protection (archive, availability, backup, BC/DR, snapshot, security)
  • Block, file, object, structured, unstructured and data value
  • Analytics, monitoring, reporting, and management metrics
  • Industry trends, tools, techniques, decision making
  • Local, remote server, storage and network I/O troubleshooting
  • Performance, availability, capacity and  economics (PACE)

Where To Purchase Your Copy

Order via Amazon.com and CRC Press along with Google Books among other global venues.

What People Are Saying About Software Defined Data Infrastructure Essentials Book

“From CIOs to operations, sales to engineering, this book is a comprehensive reference, a must-read for IT infrastructure professionals, beginners to seasoned experts,” said Tom Becchetti, advisory systems engineer.

"We had a front row seat watching Greg present live in our education workshop seminar sessions for ITC professionals in the Netherlands material that is in this book. We recommend this amazing book to expand your converged and data infrastructure knowledge from beginners to industry veterans."

Gert and Frank Brouwer – Brouwer Storage Consultancy

"Software-Defined Data Infrastructures provides the foundational building blocks to improve your craft in several areas including applications, clouds, legacy, and more.  IT professionals, as well as sales professionals and support personal, stand to gain a great deal by reading this book."

Mark McSherry- Oracle Regional Sales Manager

"Greg Schulz has provided a complete ‘toolkit’ for storage management along with the background and framework for the storage or data infrastructure professional (or those aspiring to become one)."
Greg Brunton – Experienced Storage and Data Management Professional

“Software-defined data infrastructures are where hardware, software, server, storage, I/O networking and related services converge inside data centers or clouds to protect, preserve, secure and serve applications and data,” said Schulz.  “Both readers who are new to data infrastructures and seasoned pros will find this indispensable for gaining and expanding their knowledge.”

SDDI and SDDC components

More About Software Defined Data Infrastructure Essentials
Software Defined Data Infrastructures (SDDI) Essentials provides fundamental coverage of physical, cloud, converged, and virtual server storage I/O networking technologies, trends, tools, techniques, and tradecraft skills. From webscale, software-defined, containers, database, key-value store, cloud, and enterprise to small or medium-size business, the book is filled with techniques, and tips to help develop or refine your server storage I/O hardware, software, Software Defined Data Centers (SDDC), Software Data Infrastructures (SDI) or Software Defined Anything (SDx) and services skills. Whether you are new to data infrastructures or a seasoned pro, you will find this comprehensive reference indispensable for gaining as well as expanding experience with technologies, tools, techniques, and trends.

Software Defined Data Infrastructure Essentials SDDI SDDC content

This book is the definitive source providing comprehensive coverage about IT and cloud Data Infrastructures for experienced industry experts to beginners. Coverage of topics spans from higher level applications down to components (hardware, software, networks, and services) that get defined to create data infrastructures that support business, web, and other information services. This includes Servers, Storage, I/O Networks, Hardware, Software, Management Tools, Physical, Software Defined Virtual, Cloud, Docker, Containers (Docker and others) as well as Bulk, Block, File, Object, Cloud, Virtual and software defined storage.

Additional topics include Data protection (Availability, Archiving, Resiliency, HA, BC, BR, DR, Backup), Performance and Capacity Planning, Converged Infrastructure (CI), Hyper-Converged, NVM and NVMe Flash SSD, Storage Class Memory (SCM), NVMe over Fabrics, Benchmarking (including metrics matter along with tools), Performance Capacity Planning and much more including whos doing what, how things work, what to use when, where, why along with current and emerging trends.

Book Features

ISBN-13: 978-1498738156
ISBN-10: 149873815X
Hardcover: 672 pages
(Available in Kindle and other electronic formats)
Over 200 illustrations and 70 plus tables
Frequently asked Questions (and answers) along with many tips
Various learning exercises, extensive glossary and appendices
Publisher: Auerbach/CRC Press Publications; 1 edition (June 19, 2017)
Language: English

SDDI and SDDC toolbox

Where To Learn More

Learn more about related technology, trends, tools, techniques, and tips with the following links.

Data Infrastructures Protect Preserve Secure and Serve Information
Various IT and Cloud Infrastructure Layers including Data Infrastructures

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

Data Infrastructures exist to protect, preserve, secure and serve information along with the applications and data they depend on. With more data being created at a faster rate, along with the size of data becoming larger, increased application functionality to transform data into information means more demands on data infrastructures and their underlying resources.

Software-Defined Data Infrastructure Essentials: Cloud, Converged, and Virtual Fundamental Server Storage I/O Tradecraft is for people who are currently involved with or looking to expand their knowledge and tradecraft skills (experience) of data infrastructures. Software-defined data centers (SDDC), software data infrastructures (SDI), software-defined data infrastructure (SDDI) and traditional data infrastructures are made up of software, hardware, services, and best practices and tools spanning servers, I/O networking, and storage from physical to software-defined virtual, container, and clouds. The role of data infrastructures is to enable and support information technology (IT) and organizational information applications.

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

Everything is not the same in business, organizations, IT, and in particular servers, storage, and I/O. This means that there are different audiences who will benefit from reading this book. Because everything and everybody is not the same when it comes to server and storage I/O along with associated IT environments and applications, different readers may want to focus on various sections or chapters of this book.

If you are looking to expand your knowledge into an adjacent area or to understand whats under the hood, from converged, hyper-converged to traditional data infrastructures topics, this book is for you. For experienced storage, server, and networking professionals, this book connects the dots as well as provides coverage of virtualization, cloud, and other convergence themes and topics.

This book is also for those who are new or need to learn more about data infrastructure, server, storage, I/O networking, hardware, software, and services. Another audience for this book is experienced IT professionals who are now responsible for or working with data infrastructure components, technologies, tools, and techniques.

Learn more here about Software Defined Data Infrastructure (SDDI) Essentials book along with cloud, converged, and virtual fundamental server storage I/O tradecraft topics, order your copy from Amazon.com or CRC Press here, and thank you in advance for learning more about SDDI and related topics.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Part II Revisting AWS S3 Storage Gateway (Test Drive Deployment)

server storage I/O trends

Part II Revisiting AWS S3 Storage Gateway (Test Drive Deployment)

This Amazon Web Service (AWS) Storage Gateway Revisited posts is a follow-up to the AWS Storage Gateway test drive and review I did a few years ago (thus why it’s called revisited). As part of a two-part series, the first post looks at what AWS Storage Gateway is, how it has improved since my last review of AWS Storage Gateway along with deployment options. The second post in the series looks at a sample test drive deployment and use.

What About Storage Gateway Costs?

Costs vary by region, type of storage being used (files stored in S3, Volume Storage, EBS Snapshots, Virtual Tape storage, Virtual Tape storage archive), as well as type of gateway host, along with how access and used. Request pricing varies including data written to AWS storage by gateway (up to maximum of $125.00 per month), snapshot/volume delete, virtual tape delete, (prorate fee for deletes within 90 days of being archived), virtual tape archival, virtual tape retrieval. Note that there are also various data transfer fees that also vary by region and gateway host. Learn more about pricing here.

What Are Some Storage Gateway Alternatives

AWS and S3 storage gateway access alternatives include those from various third-party (including that are in the AWS marketplace), as well as via data protection tools (e.g. backup/restore, archive, snapshot, replication) and more commonly storage systems. Some tools include Cloudberry, S3FS, S3 motion, S3 Browser among many others.

Tip is when a vendor says they support S3, ask them if that is for their back-end (e.g. they can access and store data in S3), or front-end (e.g. they can be accessed by applications that speak S3 API). Also explore what format the application, tool or storage system stores data in AWS storage, for example, are files mapped one to one to S3 objects along with corresponding directory hierarchy, or are they stored in a save set or other entity.

AWS Storage Gateway Deployment and Management Tips

Once you have created your AWS account (if you did not already have one) and logging into the AWS console (note the link defaults to US East 1 Region), go to the AWS Services Dashboard and select Storage Gateway (or click here which goes to US East 1). You will be presented with three options (File, Volume or VTL) modes.

What Does Storage Gateway and Install Look Like

The following is what installing a AWS Storage Gateway for file and then volume looks like. First, access the AWS Storage Gateway main landing page (it might change by time you read this) to get started. Scroll down and click on the Get Started with AWS Storage Gateway button or click here.

AWS Storage Gateway Landing Page

Select type of gateway to create, in the following example File is chosen.

Select type of AWS storage gateway

Next select the type of file gateway host (EC2 cloud hosted, or on-premises VMware). If you choose VMware, an OVA will be downloaded (follow the onscreen instructions) that you deploy on your ESXi system or with vCenter. Note that there is a different VMware VM gateway OAV for File Gateway and another for Volume Gateway. In the following example VMware ESXi OVA is selected and downloaded, then accessed via VMware tools such as vSphere Web Client for deployment.

AWS Storage Gateway select download

Once your VMware OVA file is downloaded from AWS, install using your preferred VMware tool, in this case I used the vSphere Web Client.

AWS Storage Gateway VM deploy

Once you have deployed the VMware VM for File Storage Gateway, it is time to connect to the gateway using the IP address assigned (static or DHCP) for the VM. Note that you may need to allocate some extra VMware storage to the VM if prompted (this mainly applies to Volume Gateway). Also follow directions about setting NTP time, using paravirtual adapters, thick vs. thin provisioning along with IP settings. Also double-check to make sure your VM and host are set for high-performance power setting. Note that the default username is sguser and password is sgpassword for the gateway.

AWS Storage Gateway Connect

Once you successfully connect to the gateway, next step will be to configure file share settings.

AWS Storage Gateway Configure File Share

Configure file share by selecting which gateway to use (in case you have more than one), name of an S3 bucket name to create, type of storage (S3 Standard or IA), along with Access Management security controls.

AWS Storage Gateway Create Share

Next step is to complete file share creation, not the commands provided for Linux and Windows for accessing the file share.

AWS Storage Gateway Review Share Settings

Review file share settings

AWS Storage Gateway access from Windows

Now lets use the file share by accessing and mounting to a Windows system, then copy some files to the file share.

AWS Storage Gateway verify Bucket Items

Now let’s go to the AWS console (or in our example use S3 Browser or your favorite tool) and look at the S3 bucket for the file share and see what is there. Note that each file is an object, and the objects simply appear as a file. If there were sub-directory those would also exist. Note that there are other buckets that I have masked out as we are only interested in the one named awsgwydemo that is configured using S3 Standard storage.

AWS Storage Gateway Volume

Now lets look at using the S3 Storage Gateway for Volumes. Similar to deploying for File Gateway, start out at the AWS Storage Gateway page and select Volume Gateway, then select what type of host (EC2 cloud, VMware or Hyper-V (2008 R2 or 2012) for on-premises deployment). Lets use the VMware Gateway, however as mentioned above, this is a different OVA/OVF than the File Gateway.

AWS Storage Gateway Configure Volume

Download the VMware OVA/OVF from AWS, and then install using your preferred VMware tools making sure to configure the gateway per instructions. Note that the Volume Gateway needs a couple of storage devices allocated to it. This means you will need to make sure that a SCSI adapter exists (or add one) to the VM, along with the disks (HDD or SSD) for local storage. Refer to AWS documentation about how to size, for my deployment I added a couple of small 80GB drives (you can choose to put on HDD or SSD including NVMe). Note that when connecting to the gateway if you get an error similar to below, make sure that you are in fact using the Volume Gateway and not mistakenly using the File Gateway OVA (VM). Note that the default username is sguser and password is sgpassword for the gateway.

AWS Storage Gateway Connect To Volume

Now connect to the local Volume Storage Gateway and notice the two local disks allocated to it.

AWS Storage Gateway Cached Volume Deploy

Next its time to create the Gateway which are deploying a Volume Cached below.

AWS Storage Gateway Volume Create

Next up is creating a volume, along with its security and access information.

AWS Storage Gateway Volume Settings

Volume configuration continued.

AWS Storage Gateway Volume CHAP

And now some additional configuration of the volume including iSCSI CHAP security.

AWS Storage Gateway Windows Access

Which leads us up to some Windows related volume access and configuration.

AWS Storage Gateway Using iSCSI Volume

Now lets use the new iSCSI based AWS Storage Gateway Volume. On the left you can see various WIndows command line activity, along with corresponding configuration information on the right.

AWS Storage Gateway Being Used by Windows

And there you have it, a quick tour of AWS Storage Gateway, granted there are more options that you can try yourself.

AWS

Where To Learn More

What This All Means

Overall I like the improvements that AWS has made to the Storage Gateway along with the different options it provides. Something to keep in mind is that if you are planning to use the AWS Storage Gateway File serving sharing mode that there are caveats to multiple concurrent writers to the same bucket. I would not be surprised if some other gateway or software based tool vendors tried to throw some fud towards the Storage Gateway, however ask them then how they coordinate multiple concurrent updates to a bucket while preserving data integrity.

Which Storage Gateway variant from AWS to use (e.g. File, Volume, VTL) depends on what your needs are, same with where the gateway is placed (Cloud hosted or on-premises with VMware or Hyper-V). Keep an eye on your costs, and more than just the storage space capacity. This means pay attention to your access and requests fees, as well as different service levels, along with data transfer fees.

You might wonder what about EFS and why you would want to use AWS Storage Gateway? Good question, at the time of this post EFS has evolved from being internal (e.g. within AWS and across regions) to having an external facing end-point however there is a catch. That catch which might have changed by time you read this is that the end-point can only be accessed from AWS Direct Connect locations.

This means that if your servers are not in a AWS Direct Connect location, without some creative configuration, EFS is not an option. Thus Storage Gateway File mode might be an option in place of EFS as well as using AWS storage access tools from others. For example I have some of my S3 buckets mounted on Linux systems using S3FS for doing rsync or other operations from local to cloud. In addition to S3FS, I also have various backup tools that place data into S3 buckets for backup, BC and DR as well as archiving.

Check out AWS Storage Gateway yourself and see what it can do or if it is a fit for your environment.

Ok, nuff said (for now…).

Cheers
Gs

Greg Schulz – Multi-year Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Watch for the spring 2017 release of his new book "Software-Defined Data Infrastructure Essentials" (CRC Press).

Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.

Posted on

Some August 2015 Amazon Web Services (AWS) and Microsoft Azure Cloud Updates

Storage I/O trends

Some August 2015 Amazon Web Services (AWS) and Microsoft Azure Cloud Updates

Cloud Services Providers continue to extend their feature, function and capabilities and the following are two examples. Being a customer of both Amazon Web Services (AWS) as well as Microsoft Azure (among others), I receive monthly news updates about service improvements along with new features. Here are a couple of examples involving recent updates from AWS and Azure.

Azure enhancements

Microsoft Azure customer update

Azure Premium Storage generally available in Japan East

Solid State Device (SSD) based Azure Premium Storage is now available in Japan East region. Add up to 32 TB and more than 64,000 IOPs (read operations) per virtual machine with  Azure Premium Storage. Learn more about Azure storage and pricing here.

Azure Data Factory generally available

Data Factory is a cloud based data integration service for automated management as well as movement and transformation of data, learn more and view pricing options here.

AWS Partner Updates

Recent Amazon Web Services (AWS) customer update included the following pertaining to partner storage solutions.

AWS partner updates

AWS Partner Network APN

Learn more about AWS Partner Network (APN) here or click on the above image.

AWS APN competency programs include:

  • Storage
  • Healthcare
  • Life Sciences
  • SAP Solutions
  • Microsoft Solutions
  • Oracle Solutions
  • Marketing and Commerce
  • Big Data
  • Security
  • Digital Media

AWS Partner Network (APN) Solutions for Storage include:

Archiving to AWS Glacier

  • Commvault
  • NetApp (AltaVault)

    • Backup to AWS using S3

  • CloudBerry Lab
  • Commvault
  • Ctera
  • Druva
  • NetApp (AltaVault)

    • Primary Cloud File and NAS storage complementing on-premises (e.g. your local) storage

  • Avere
  • Ctera
  • NetApp (Cloud OnTap)
  • Panzura
  • SoftNAS
  • Zadara

    • Secure File Transfer

  • Aspera
  • Signiant

    • Note that the above are those listed on the AWS Storage Partner Page as of this being published and subject to change. Likewise other solutions that are not part of the AWS partner program may not be listed.

    Where to read, watch and learn more

    Storage I/O trends

    What this all means and wrap up

    Cloud Service Providers (CSP) continue to enhance their capabilities, as well as their footprints as part of growth. In addition to technology, tools and number of regions, sites and data centers, the CSPs are also expanding their partner networks both about how many partners, also in the scope of those partnerships. Some of these partnerships are in the scope of the cloud as a destination, others are for enabling hybrid where public clouds become an extension complementing traditional IT. Everything is not the same in most environments and one type of cloud approach does not have to suit or fit all needs, hence the value of hybrid cloud deployment and usage.

    Ok, nuff said, for now…

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Data Protection Diaries: Are your restores ready for World Backup Day 2015?

    Data Protection Diaries: Are your restores ready for World Backup Day 2015?

    This is part of an ongoing data protection diaries series of post about, well, cloud and data protection and what I’m doing pertaining to World Backup Day 2015 along with related topics.

    In case you forgot or did not know, World Backup Day is March 31 2015 (@worldbackupday) so now is a good time to be ready. The only challenge that I have with the World Backup Day (view their site here) that has gone on for a few years know is that it is a good way to call out the importance of backing up or protecting data. However its time to also put more emphasis and focus on being able to make sure those backups or protection copies actually work.

    By this I mean doing more than making sure that your data can be read from tape, disk, SSD or cloud service actually going a step further and verifying that restored data can actually be used (read, written, etc).

    The Problem, Issue, Challenge, Opportunity and Need

    The problem, issue and challenges are simple, are your applications, systems and data protected as well as can you use those protection copies (e.g. backups, snapshots, replicas or archives) when as well as were needed?

    storage I/O data protection

    The opportunity is simple, avoiding downtime or impact to your business or organization by being proactive.

    Understanding the challenge and designing a strategy

    The following is my preparation checklist for World Backup Data 2015 (e.g. March 31 2015) which includes what I need or want to protect, as well as some other things to be done including testing, verification, address (remediate or fix) known issues while identifying other areas for future enhancements. Thus perhaps like yours, data protection for my environment which includes physical, virtual along with cloud spanning servers to mobile devices is constantly evolving.

    collect TPM metrics from SQL Server with hammerdb
    My data protection preparation, checklist and to do list

    Finding a solution

    While I already have a strategy, plan and solution that encompasses different tools, technologies and techniques, they are also evolving. Part of the evolving is to improve while also exploring options to use new and old things in new ways as well as eat my down dog food or walk the talk vs. talk the talk. The following figure provides a representation of my environment that spans physical, virtual and clouds (more than one) and how different applications along with systems are protected against various threats or risks. Key is that not all applications and data are the same thus enabling them to be protected in different ways as well as over various intervals. Needless to say there is more to how, when, where and with what different applications and systems are protected in my environment than show, perhaps more on that in the future.

    server storageio and unlimitedio data protection
    Some of what my data protection involves for Server StorageIO

    Taking action

    What I’m doing is going through my checklist to verify and confirm the various items on the checklist as well as find areas for improvement which is actually an ongoing process.

    Do I find things that need to be corrected?

    Yup, in fact found something that while it was not a problem, identified a way to improve on a process that will once fully implemented enabler more flexibility both if a restoration is needed, as well as for general everyday use not to mention remove some complexity and cost.

    Speaking of lessons learned, check this out that ties into why you want 4 3 2 1 based data protection strategies.

    Storage I/O trends

    Where to learn more

    Here are some extra links to have a look at:

    Data Protection Diaries
    Cloud conversations: If focused on cost you might miss other cloud storage benefits
    5 Tips for Factoring Software into Disaster Recovery Plans
    Remote office backup, archiving and disaster recovery for networking pros
    Cloud conversations: Gaining cloud confidence from insights into AWS outages (Part II)
    Given outages, are you concerned with the security of the cloud?
    Data Archiving: Life Beyond Compliance
    My copies were corrupted: The 3-2-1 rule
    Take a 4-3-2-1 approach to backing up data
    Cloud and Virtual Data Storage Networks – Chapter 8 (CRC/Taylor and Francis)

    What this all means and wrap-up

    Be prepared, be proactive when it comes to data protection and business resiliency vs. simply relying reacting and recovering hoping that all will be ok (or works).

    Take a few minutes (or longer) and test your data protection including backup to make sure that you can:

    a) Verify that in fact they are working protecting applications and data in the way expected

    b) Restore data to an alternate place (verify functionality as well as prevent a problem)

    c) Actually use the data meaning it is decrypted, inflated (un-compressed, un-de duped) and security certificates along with ownership properties properly applied

    d) Look at different versions or generations of protection copies if you need to go back further in time

    e) Identify area of improvement or find and isolate problem issues in advance vs. finding out after the fact

    Time to get back to work checking and verifying things as well as attending to some other items.

    Ok, nuff said, for now…

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Ceph Day Amsterdam 2012 (Object and cloud storage)

    StorageIO industry trends cloud, virtualization and big data

    Recently while I was in Europe presenting some sessions at conferences and doing some seminars, I was invited by Ed Saipetch (@edsai) of Inktank.com to attend the first Ceph Day in Amsterdam.

    Ceph day image

    As luck or fate would turn out, I was in Nijkerk which is about an hour train ride from Amsterdam central station plus a free day in my schedule. After a morning train ride and nice walk from Amsterdam Central I arrived at the Tobacco Theatre (a former tobacco trading venue) where Ceph Day was underway, and in time for lunch of Krokettens sandwich.

    Attendees at Ceph Day

    Lets take a quick step back and address for those not familiar what is Ceph (Cephalanthera) and why it was worth spending a day to attend this event. Ceph is an open source distributed object scale out (e.g. cluster or grid) software platform running on industry standard hardware.

    Dell server supporting ceph demoSketch of ceph demo configuration

    Ceph is used for deploying object storage, cloud storage and managed services, general purpose storage for research, commercial, scientific, high performance computing (HPC) or high productivity computing (commercial) along with backup or data protection and archiving destinations. Other software similar in functionality or capabilities to Ceph include OpenStack Swift, Basho Riak CS, Cleversafe, Scality and Caringo among others. There are also the tin wrapped software (e.g. appliances or pre-packaged) solutions such as Dell DX (Caringo), DataDirect Networks (DDN) WOS, EMC ATMOS and Centera, Amplidata and HDS HCP among others. From a service standpoint, these solutions can be used to build services similar Amazon S3 and Glacier, Rackspace Cloud files and Cloud Block, DreamHost DreamObject and HP Cloud storage among others.

    Ceph cloud and object storage architecture image

    At the heart of Ceph is RADOS a distributed object store that consists of peer nodes functioning as object storage devices (OSD). Data can be accessed via REST (Amazon S3 like) APIs, Libraries, CEPHFS and gateway with information being spread across nodes and OSDs using a CRUSH based algorithm (note Sage Weil is one of the authors of CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data). Ceph is scalable in terms of performance, availability and capacity by adding extra nodes with hard disk drives (HDD) or solid state devices (SSDs). One of the presentations pertained to DreamHost that was an early adopter of Ceph to make their DreamObjects (cloud storage) offering.

    Ceph cloud and object storage deployment image

    In addition to storage nodes, there are also an odd number of monitor nodes to coordinate and manage the Ceph cluster along with optional gateways for file access. In the above figure (via DreamHost), load balancers sit in front of gateways that interact with the storage nodes. The storage node in this example is a physical server with 12 x 3TB HDDs each configured as a OSD.

    Ceph dreamhost dreamobject cloud and object storage configuration image

    In the DreamHost example above, there are 90 storage nodes plus 3 management nodes, the total raw storage capacity (no RAID) is about 3PB (12 x 3TB = 36TB x 90 = 3.24PB). Instead of using RAID or mirroring, each objects data is replicated or copied to three (e.g. N=3) different OSDs (on separate nodes), where N is adjustable for a given level of data protection, for a usable storage capacity of about 1PB.

    Note that for more usable capacity and lower availability, N could be set lower, or a larger value of N would give more durability or data protection at higher storage capacity overhead cost. In addition to using JBOD configurations with replication, Ceph can also be configured with a combination of RAID and replication providing more flexibility for larger environments to balance performance, availability, capacity and economics.

    Ceph dreamhost and dreamobject cloud and object storage deployment image

    One of the benefits of Ceph is the flexibility to configure it how you want or need for different applications. This can be in a cost-effective hardware light configuration using JBOD or internal HDDs in small form factor generally available servers, or high density servers and storage enclosures with optional RAID adapters along with SSD. This flexibility is different from some cloud and object storage systems or software tools which take a stance of not using or avoiding RAID vs. providing options and flexibility to configure and use the technology how you see fit.

    Here are some links to presentations from Ceph Day:
    Introduction and Welcome by Wido den Hollander
    Ceph: A Unified Distributed Storage System by Sage Weil
    Ceph in the Cloud by Wido den Hollander
    DreamObjects: Cloud Object Storage with Ceph by Ross Turk
    Cluster Design and Deployment by Greg Farnum
    Notes on Librados by Sage Weil

    Presentations during ceph day

    While at Ceph day, I was able to spend a few minutes with Sage Weil Ceph creator and founder of inktank.com to record a pod cast (listen here) about what Ceph is, where and when to use it, along with other related topics. Also while at the event I had a chance to sit down with Curtis (aka Mr. Backup) Preston where we did a simulcast video and pod cast. The simulcast involved Curtis recording this video with me as a guest discussing Ceph, cloud and object storage, backup, data protection and related themes while I recorded this pod cast.

    One of the interesting things I heard, or actually did not hear while at the Ceph Day event that I tend to hear at related conferences such as SNW is a focus on where and how to use, configure and deploy Ceph along with various configuration options, replication or copy modes as opposed to going off on erasure codes or other tangents. In other words, instead of focusing on the data protection protocol and algorithms, or what is wrong with the competition or other architectures, the Ceph Day focused was removing cloud and object storage objections and enablement.

    Where do you get Ceph? You can get it here, as well as via 42on.com and inktank.com.

    Thanks again to Sage Weil for taking time out of his busy schedule to record a pod cast talking about Ceph, as well 42on.com and inktank for hosting, and the invitation to attend the first Ceph Day in Amsterdam.

    View of downtown Amsterdam on way to train station to return to Nijkerk
    Returning to Amsterdam central station after Ceph Day

    Ok, nuff said.

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2012 StorageIO and UnlimitedIO All Rights Reserved

    Posted on

    The Differences Between Singapore and Houston in May

    Storage I/O trends

    In addition to participating at the Techtarget (TT) spring 2008 edition of Storage Decisions (SD) event in Chicago this past week, I was also briefly in Houston Texas this week to do a keynote talk on the Wide World of Archiving Life Beyond Compliance at the Omni Hotel . Between the venue, temperature and humidity, I thought I was in Singapore with a sudden craving for pepper crab at Jumbo’s Seafood in the East Coast Seafood Center. While the Houston venue and those in Singapore were similar as was the temperature and humidity, the real difference was that I remind in the central time zone with a 2.5 hour flight vs. 25 hours of flights and changing planes. Both locales have nice people who speak English and have great food as well. If in Houston, check out the Omni Hotel and for archiving, the Mobius folks are great as well.

    Ok, nuff said.

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Wide World of Archiving – Life Beyond Compliance

    Earlier this week I did a keynote talk at a TechTarget event in the New York city area titled the “Wide World of Archiving – Life Beyond Compliance” with the basic theme that archiving and data preservation for future or possible future use is not unique or exclusive to SARBOX, HIPPA, CFR, PCI, OHSAS, ISO or other members of the common alphabet soup of governmental or industry regulatory compliance needs.

    The basic theme is that archiving can be used to discuss many IT and business pain points and issues from preserving project oriented or seasonal data to off-loading un-used or seldom used data to free up resources to meet power, cooling, floor space and environmental (PCFE) issues
    “aka Green”     along with boosting performance for on-line access as well as backup, BC and DR.

    The challenge however is that archiving while a powerful technique, is also complex in that it requires hardware and mediums to park your data onto, software to find and then execute policies defined by someone to move data to the archive medium and if applicable, delete or cleanup data that has been moved all of which has cost and application specific issues. Then the human side which is more involved than simply throwing head count at the tasks and avoiding the mistakes of the Mythical Man Month.

    The human side of archiving is the glue to make it work in that similar to cleaning out your garage, attic, basement or store-room, you can have someone come and do the real work, however do they have the insight to know what to keep and what to discard? Sure that’s an overly simple example, and there are plenty of search and discovery software management tool vendors who will be more than happy to show you a demo of their wares that will discovery and classify and categorize and index what data you have as well as interface with policy managers, data movers and archiving devices.

    However who is going to tell the management tools what policies are applicable and the different variances for your different business segments or activities? Consequently the key to making archiving work particularly on a broader basis is to get internal personal familiar with your business, IT personal, as well as external subject matter experts involved all of which leads to a challenge and dilemma of is it cheaper to just buy more energy-efficient, space-saving storage than to pay the fees to find, manage, move and archive data. Talking with one of the attendees who brought up some good points that this all makes sense however there is a scaling challenge and when dealing with 100’s of TBytes or PBytes, the complexity increases.

    This is where the notion of scaling with stability comes into play in that many solutions exist to address different functionality for example archiving, de-duping, compression, server or storage virtualization, thin-provisioning among many others however how do they scale with stability. That is, how stable or reliable do the solutions remain when scaling from 10s to 100’s to 10,000’s or even 100,000’s users, email boxes, sessions, streams or from 10’s of TBytes to 10’s of PBytes? How does the performance hold up, how does the availability hold up, how does the management and on-going care and feeding change for the better or worse? Concerns around scaling is a common issue I hear from IT organizations pertaining to both hardware and software tools in that what works great during a WebEx demo or PowerPoint or pdf slide show may be different from real-world performance, management, reliability and complexity concerns. After all, have you ever seen a WebEx or live office or PowerPoint or PDF slide deck showing a hardware or software based solution that could not scale or provide transparent interoperability? That would be akin to finding a used car sales rep who gives you a tour of how a car was refurbished inside and out after it was declared totaled by the previous owners insurance company after the last great flood or hurricane.

    Getting back to archiving, and not trying to conquer all of your data at one time, take a divide and conquer approach, go for some low hanging fruit where your chances of success go up that you can then build some momentum and perhaps a business case to do a larger project. Also, one solution particularly one archiving software solution may not be applicable to all of your needs in that you may need a tool specialized for email, one for databases and another general purpose tool. Likewise you may need to engage different subject matter experts to help you with policy definition and establishing rules to meet different requirements which is where business partners can come into play with either their in-house staff, partners, or associates that they work with for different issues and needs.

    Look beyond the hardware and software, look at the people or human and knowledge side of archiving as well as look beyond archiving for compliance as there is a much bigger wide world of archiving and opportunity. If you remember the ABC sports TV show “Wide World of Sports” you may recall Jim McCay saying “Spanning the globe to bring you the constant variety of sports… the thrill of victory… and the agony of defeat… the human drama of athletic competition… This is “ABC’s Wide World of Sports!”.

    From an archiving perspective, keep this in mind in that there is a wide world of opportunities for archiving, the thrill of victory are the benefits, the agony of defeat are the miss-steps, lack of scaling, out of control costs or complexity, the human drama is how to make or break a solution, this is the “Wide World of Archiving”…

    Rest assured, some form of archiving structured database, semi-structured email with attachments and un-structured word, PowerPoint, PDF, MP3 and other data is in your future, it’s a matter of when. Archiving is just one of many tools available for effectively managing your data and addressing data footprint sprawl particular for data that you can not simply delete and ignore, if you need it to go forward, you need to keep it. Or, as a friend of mine says You can’t go forward unless you can go back. Likewise, you can’t manage what you don’t know about; you can’t move and delete what you can’t manage.

    Look for solution providers who are not looking to simply get you to buy the latest and greatest archiving storage device, or, the slickest archiving management tool with a Uhi Gui that rivals those on an Wii or Xbox, or, that is looking to simply run up billable hours. That’s a balancing act requires investing time with different business solution providers to see where their core business is, how they can scale, where and how they make their money to help you decide where and how they are fit as opposed to simply adding complexity to your environment and existing issues.

    Ok, nuff said.

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    StorageIO Spring Keynote and Speaking tour V2.008

    Several new keynote and speaking engagements involving myself have been added to the StorageIO events page including among others:

    April 8th, 2008 – SNW Orlando FL
    Beyond Green-Wash:
    IT Data Center Power, Cooling, Floor Space and Environmental (PCFE) Topics and Trends V2.008
    This talk will move past what are the issues and reasons for going green and get right to the point of what you can do today leveraging various technologies, techniques and best practices to address PCFE and green environmental issues including EHS, low power and economic sustainment in an environmental friendly manner as well as what to include in a long term green strategy for your data center.

    Chicago, May 13th-15th – StorageDecisions
    Clustered Storage:
    From SMB, to Scientific, to Social Networking and Web 2.0
    The growth of structured and unstructured data continues at an explosive rate in most environments resulting in a constantly expanding data footprint requiring data and storage management resources. Similarly, the relative ease of use of NFS and Windows CIFS file sharing based storage, also known as Network Attached Storage (NAS), has led to a proliferation of NAS and Windows file servers which are not all that different from how the ease of use of personal computers (PCs) resulted in desktop and server sprawl. With the focus of many IT organizations today to do more with less, or, do more with what you have, clustered storage and clustered file serving have become a popular option to support modular, scalable and flexible growth. Clustered storage including clustered file serving, grid and web 2.0 based storage solutions are no longer confined to the specific high performance scientific applications they are commonly associated. Clustered storage serving is commonly being deployed to support a wide diversity of applications including commercial, entertainment or media, Web 2.0 and social networking along with grid, cloud and traditional scientific needs.

    This session takes a look at among other topics:
    ? Look at what different clustered storage vendors are claiming and how their solutions differ
    ? Fact vs. Fiction, Myths and Realties of clustered storage
    o Grid vs. Clusters, Cluster vs. Grid, what?s the differences
    o Clustered storage is only for ultra large environments like Google
    o Clustered file serving is only for high performance (HPC) environments
    o SMBs and bulk storage applications can not benefit from clustered storage
    ? What are the caveats to be aware of when deploying clustered storage?
    ? What are some emerging trends and solutions to keep an eye on for clustered storage
    ? What are some questions that some vendors do not want you to ask about their solutions!

    Green and Environmental Friendly Storage:
    Practical Ways to Achieve Energy Efficiency
    Green is in-and every storage vendor out there has a green story to tell. Despite the vendor and industry hyperbole about the environmental benefits of their products, there are still no standard metrics by which to measure and compare power consumption or energy efficiency claims. The challenge is sorting out and closing the gap between vendor green messaging and IT data center issues including power, cooling, floor space and other environmental topics including RoHS and e-waste disposal. This session looks at several practical techniques and technologies that you can leverage today to achieve an energy efficiency data center to sustain business growth in an economical and ecological friendly manner.

    Topics that will be covered include among others:
    ? How truthful are vendor claims and what is ?Green wash?
    ? Facts and Fiction, Myths and Realities:
    o Storage is cheaper to buy than to power
    o Power avoidance vs. energy efficiency
    o Are Solid State Devices (SSD) the silver bullet?
    o Dedupe vs. Archive vs. Compression vs. Consolidation
    ? What?s real and achievable today, what are your options?
    ? Measuring and determining energy efficiency with emerging metrics
    ? How to do more with what you have and avoid forklift upgrades
    ? Who is the ?Greenest of them all? and where to learn more

    I will also be keynoting at several TechTarget seminar series events around the U.S. including
    StorageIO events page located here.

    Cheers
    GS