For those of you who may not know Tom, he has been in the IT, data center, data infrastructure, server and storage (as well as data protection) industry for many years (ok decades) as a customer and vendor in various roles. Not surprising our data infrastructure discussion involves server, software, storage, big data, backup, data protection, big data protection, CMG (Computer Measurement Group @mspcmg), copy data management, cloud, containers, fundamental tradecraft skills among other related topics.
Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO. All Rights Reserved.
In case you forgot or did not know, World Backup Day is March 31 2015 (@worldbackupday) so now is a good time to be ready. The only challenge that I have with the World Backup Day (view their site here) that has gone on for a few years know is that it is a good way to call out the importance of backing up or protecting data. However its time to also put more emphasis and focus on being able to make sure those backups or protection copies actually work.
By this I mean doing more than making sure that your data can be read from tape, disk, SSD or cloud service actually going a step further and verifying that restored data can actually be used (read, written, etc).
The Problem, Issue, Challenge, Opportunity and Need
The problem, issue and challenges are simple, are your applications, systems and data protected as well as can you use those protection copies (e.g. backups, snapshots, replicas or archives) when as well as were needed?
The opportunity is simple, avoiding downtime or impact to your business or organization by being proactive.
Understanding the challenge and designing a strategy
The following is my preparation checklist for World Backup Data 2015 (e.g. March 31 2015) which includes what I need or want to protect, as well as some other things to be done including testing, verification, address (remediate or fix) known issues while identifying other areas for future enhancements. Thus perhaps like yours, data protection for my environment which includes physical, virtual along with cloud spanning servers to mobile devices is constantly evolving.
My data protection preparation, checklist and to do list
Finding a solution
While I already have a strategy, plan and solution that encompasses different tools, technologies and techniques, they are also evolving. Part of the evolving is to improve while also exploring options to use new and old things in new ways as well as eat my down dog food or walk the talk vs. talk the talk. The following figure provides a representation of my environment that spans physical, virtual and clouds (more than one) and how different applications along with systems are protected against various threats or risks. Key is that not all applications and data are the same thus enabling them to be protected in different ways as well as over various intervals. Needless to say there is more to how, when, where and with what different applications and systems are protected in my environment than show, perhaps more on that in the future.
Some of what my data protection involves for Server StorageIO
Taking action
What I’m doing is going through my checklist to verify and confirm the various items on the checklist as well as find areas for improvement which is actually an ongoing process.
Do I find things that need to be corrected?
Yup, in fact found something that while it was not a problem, identified a way to improve on a process that will once fully implemented enabler more flexibility both if a restoration is needed, as well as for general everyday use not to mention remove some complexity and cost.
Speaking of lessons learned, check this out that ties into why you want 4 3 2 1 based data protection strategies.
Be prepared, be proactive when it comes to data protection and business resiliency vs. simply relying reacting and recovering hoping that all will be ok (or works).
Take a few minutes (or longer) and test your data protection including backup to make sure that you can:
a) Verify that in fact they are working protecting applications and data in the way expected
b) Restore data to an alternate place (verify functionality as well as prevent a problem)
c) Actually use the data meaning it is decrypted, inflated (un-compressed, un-de duped) and security certificates along with ownership properties properly applied
d) Look at different versions or generations of protection copies if you need to go back further in time
e) Identify area of improvement or find and isolate problem issues in advance vs. finding out after the fact
Time to get back to work checking and verifying things as well as attending to some other items.
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
Different threat risks and reasons to protect your digital assets (data)
March 31 is World Backup Day which means you should make sure that your data and digital assets (photos, videos, music or audio, scanned items) along with other digital documents are protected. Keep in mind that various reasons for protecting, preserving and serving your data regardless of if you are a consumer with needs to protect your home and personal information, or a large business, institution or government agency.
Why World Backup Day and Data Protection Focus
By being protected this means making sure that there are copies of your documents, data, files, software tools, settings, configurations and other digital assets. These copies can be in different locations (home, office, on-site, off-site, in the cloud) as well as for various points in time or recovery point objective (RPO) such as monthly, weekly, daily, hourly and so forth.
Having different copies for various times (e.g. your protection interval) gives you the ability to go back to a specific time to recover or restore lost, stolen, damaged, infected, erased, or accidentally over-written data. Having multiple copies is also a safeguard incase either the data, files, objects or items being backed up or protected are bad, or the copy is damaged, lost or stolen.
Restore Test Time
While the focus of world backup data is to make sure that you are backing up or protecting your data and digital assets, it is also about making sure what you think is being protected is actually occurring. It is also a time to make sure what you think is occurring or know is being done can actually be used when needed (restore, recover, rebuild, reload, rollback among other things that start with R). This means testing that you can find the files, folders, volumes, objects or data items that were protected, use those copies or backups to restore to a different place (you don’t want to create a disaster by over-writing your good data).
In addition to making sure that the data can be restored to a different place, go one more step to verify that the data can actually be used which means has it be decrypted or unlocked, have the security or other rights and access settings along with meta data been applied. While that might seem obvious it is often the obvious that will bite you and cause problems, hence take some time to test that all is working, not to mention get some practice doing restores.
Data Protection and Backup 3 2 1 Rule and Guide
Recently I did a piece based on my own experiences with data protection including Backup as well as Restore over at Spiceworks called My copies were corrupted: The 3-2-1 rule. For those not familiar, or as a reminder 3 2 1 means have more than three copies or better yet, versions stored on at least two different devices, systems, drives, media or mediums in at least one different location from the primary or main copy.
Following is an excerpt from the My copies were corrupted: The 3-2-1 rule piece:
Not long ago I had a situation where something happened to an XML file that I needed. I discovered it was corrupted, and I needed to do a quick restore.
“No worries,” I thought, “I’ll simply copy the most recent version that I had saved to my file server.” No such luck. That file had been just copied and was damaged.
“OK, no worries,” I thought. “That’s why I have a periodic backup copy.” It turns out that had worked flawlessly. Except there was a catch — it had backed up the damaged file. This meant that any and all other copies of the file were also damaged as far back as to when the problem occurred.
Yes I eat my own dog food meaning that I practice what I talk about (e.g. walking the talk) leveraging not just a 3 2 1 approach, actually more of a 4 3 2 1 hybrid which means different protection internals, various retention’s and frequencies, not all data gets treated the same, using local disk, removable disk to go off-site as well as cloud. I also test candidly more often by accident using the local, removable and cloud copies when I accidentally delete something, or save the wrong version.
Some of my data and applications are protected throughout the day, others on set schedules that vary from hours to days to weeks to months or more. Yes, some of my data such as large videos or other items that are static do not change, so why backup them up or protect every day, week or month? I also align the type of protection, frequency, retention to meet different threat risks, as well as encrypt data. Part of actually testing and using the restores or recoveries is also determining what certificates or settings are missing, as well as where opportunities exist or needed to enhance data protection.
Closing comments (for now)
Take some time to learn more about data protection including how you can improve or modernize while rethinking what to protect, when, where, why how and with what.
In addition to having copies from different points in time and extra copies in various locations, also make sure that they are secured or encrypted AND make sure to protect your encryption keys. After all, try to find a digital locksmith to unlock your data who is not working for a government agency when you need to get access to your data ;)…
Also check out the collection of technology and vendor / product neutral data protection and backup/restore content at BackupU (disclosure: sponsored by Dell Data Protection Software) that includes various webinars and Google+ hangout sessions that I have been involved with.
Watch for more data protection conversations about related trends, themes, technologies, techniques perspectives in my ongoing data protection diaries discussions as well as read more about Backup and other related items at www.storageioblog.com/data-protection-diaries-main/.
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
This is a series of posts about data protection which includes security (logical and physical), backup/restore, business continuance (BC), disaster recovery (DR), business resiliency (BR) along with high availability (HA), archiving and related topic themes, technologies and trends.
Think of data protection like protect, preserve and serve information across cloud, virtual and physical environments spanning traditional servers, storage I/O networking along with mobile (ok, some IoT as well), SOHO/SMB to enterprise.
Getting started, taking a step back
Recently I have done a series of webinars and Google+ hangouts as part of the BackupU initiative brought to you by Dell Software (that’s a disclosure btw ;) ) that are vendor and technology neutral. Instead of the usual vendor product or technology focused seminars and events, these are about getting back to the roots, the fundamentals of what to protect when and why, then decide your options as well as different approaches (e.g. what tools to use when).
In addition over the past year (ok, years) I have also been doing other data protection related events, seminars, workshops, articles, tips, posts across cloud, virtual and physical from SOHO/SMB to enterprise. These are in addition to the other data infrastructure server and storage I/O stuff (e.g. SSD, object storage, software defined, big data, little data, buzzword bingo and others).
Keep in mind that in the data center or information factory everything is not the same as there are different applications, threat risk scenarios, availability and durability among other considerations. In this series like the cloud conversations among others, I’m going to be pulling various data protection themes together hopefully to make it easier for others to find, as well as where I know where to get them.
Some notes for an upcoming post in this series using my Livescribe about data protection
Data protection topics, trends, technologies and related themes
BackupU resources (Sponsored by Dell): Various Application, Virtualization and Related Data Protection links and resources (vendor and product neutral)
BackupU resources (Sponsored by Dell): Various Application, Virtualization and Related Data Protection webinars and G+ chats (vendor and product neutral)
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
I recently came across a piece by Carl Brooks over at IT Tech News Daily that caught my eye, title was Cloud Storage Often Results in Data Loss. The piece has an effective title (good for search engine: SEO optimization) as it stood out from many others I saw on that particular day.
What caught my eye on Carls piece is that it reads as if the facts based on a quick survey point to clouds resulting in data loss, as opposed to being an opinion that some cloud usage can result in data loss.
My opinion is that if not used properly including ignoring best practices, any form of data storage medium or media could result or be blamed for data loss. For some people they have lost data as a result of using cloud storage services just as other people have lost data or access to information on other storage mediums and solutions. For example, data has been lost on tape, Hard Disk Drives (HDDs), Solid State Devices (SSD), Hybrid HDDs (HHDD), RAID and non RAID, local and remote and even optical based storage systems large and small. In some cases, there have been errors or problems with the medium or media, in other cases storage systems have lost access to, or lost data due to hardware, firmware, software, or configuration including due to human error among other issues.
Technology failure: Not if, rather when and how to decrease impact Any technology regardless of what it is or who it is from along with its architecture design and implementation can fail. It is not if, rather when and how gracefully along with what safeguards to decrease the impact, in addition to containing or isolating faults differentiates various products or solutions. How they automatically repair and self heal to keep running or support accessibility and maintain data integrity are important as is how those options are used. Granted a failure may not be technology related per say, rather something associated with human intervention, configuration, change management (or lack thereof) along with accidental or intentional activities.
I follow my advice and best practices when selecting cloud providers looking for good value, service level agreements (SLAs) and service level objectives (SLOs) over low cost or for free services.
In the several years of using cloud based storage and services there has been some loss of access, however no loss of data. Those service disruptions or loss of access to data and services ranged from a few minutes to a little over an hour. In those scenarios, if I could not have waited for cloud storage to become accessible, I could have accessed a local copy if it were available.
Had a major disruption occurred where it would have been several days before I could gain access to that information, or if it were actually lost, I have a data insurance policy. That data insurance policy is part of my business continuance (BC) and disaster recovery (DR) strategy. My BC and DR strategy is a multi layered approach combining local, offline and offsite as along with online cloud data protection and archiving.
Assuming my cloud storage service could get data back to a given point (RPO) in a given amount of time (RTO), I have some options. One option is to wait for the service or information to become available again assuming a local copy is no longer valid or available. Another option is to start restoration from a master gold copy and then roll forward changes from the cloud services as that information becomes available. In other words, I am using cloud storage as another resource that is for both protecting what is local, as well as complimenting how I locally protect things.
Minimize or cut data loss or loss of access Anything important should be protected locally and remotely meaning leveraging cloud and a master or gold backup copy.
To cut the cost of protecting information, I also leverage archives, which mean not all data gets protected the same. Important data is protected more often reducing RPO exposure and speed up RTO during restoration. Other data that is not as important is protected, however on a different frequency with other retention cycles, in other words, tiered data protection. By implementing tiered data protection, best practices, and various technologies including data footprint reduction (DFR) such as archive, compression, dedupe in addition to local disk to disk (D2D), disk to disk to cloud (D2D2C), along with routine copies to offline media (removable HDDs or RHDDs) that go offsite, Im able to stretch my data protection budget further. Not only is my data protection budget stretched further, I have more options to speed up RTO and better detail for recovery and enhanced RPOs.
If you are looking to avoid losing data, or loss of access, it is a simple equation in no particular order:
Strategy and design
Best practices and processes
Various technologies
Quality products
Robust service delivery
Configuration and implementation
SLO and SLA management metrics
People skill set and knowledge
Usage guidelines or terms of service (ToS)
Unfortunately, clouds like other technologies or solutions get a bad reputation or blamed when something goes wrong. Sometimes it is the technology or service that fails, other times it is a combination of errors that resulted in loss of access or lost data. With clouds as has been the case with other storage mediums and systems in the past, when something goes wrong and if it has been hyped, chances are it will become a target for blame or finger pointing vs. determining what went wrong so that it does not occur again. For example cloud storage has been hyped as easy to use, don’t worry, just put your data there, you can get out of the business of managing storage as the cloud will do that magically for you behind the scenes.
The reality is that while cloud storage solutions can offload functions, someone is still responsible for making decisions on its usage and configuration that impact availability. What separates various providers is their ability to design in best practices, isolate and contain faults quickly, have resiliency integrated as part of a solution along with various SLAs aligned to what the service level you are expecting in an easy to use manner.
Does that mean the more you pay the more reliable and resilient a solution should be? No, not necessarily, as there can still be risks including how the solution is used.
Does that mean low cost or for free solutions have the most risk? No, not necessarily as it comes down to how you use or design around those options. In other words, while cloud storage services remove or mask complexity, it still comes down to how you are going to use a given service.
Shared responsibility for cloud (and non cloud) storage data protection Anything important enough that you cannot afford to lose, or have quick access to should be protected in different locations and on various mediums. In other words, balance your risk. Cloud storage service provider toned to take responsibility to meet service expectations for a given SLA and SLOs that you agree to pay for (unless free).
As the customer you have the responsibility of following best practices supplied by the service provider including reading the ToS. Part of the responsibility as a customer or consumer is to understand what are the ToS, SLA and SLOs for a given level of service that you are using. As a customer or consumer, this means doing your homework to be ready as a smart educated buyer or consumer of cloud storage services.
If you are a vendor or value added reseller (VAR), your opportunity is to help customers with the acquisition process to make informed decision. For VARs and solution providers, this can mean up selling customers to a higher level of service by making them aware of the risk and reward benefits as opposed to focus on cost. After all, if a order taker at McDonalds can ask Would you like to super size your order, why cant you as a vendor or solution provider also have a value oriented up sell message.
Additional related links to read more and sources of information:
Poll: Who is responsible for cloud storage data loss?
Taking action, what you should (or not) do Dont be scared of clouds, however do your homework, be ready, look before you leap and follow best practices. Look into the service level agreements (SLAs) associated with a given cloud storage product or service. Follow best practices about how you or someone else will protect what data is put into the cloud.
For critical data or information, consider having a copy of that data in the cloud as well as at or in another place, which could be in a different cloud or local or offsite and offline. Keep in mind the theme for critical information and data is not if, rather when so what can be done to decrease the risk or impact of something happening, in other words, be ready.
Data put into the cloud can be lost, or, loss of access to it can occur for some amount of time just as happens with using non cloud storage such as tape, disk or ssd. What impacts or minimizes your risk of using traditional local or remote as well as cloud storage are the best practices, how configured, protected, secured and managed. Another consideration is the type and quality of the storage product or cloud service can have a big impact. Sure, a quality product or service can fail; however, you can also design and configure to decrease those impacts.
Wrap up Bottom line, do not be scared of cloud storage, however be ready, do your homework, review best practices, understand benefits and caveats, risk and reward. For those who want to learn more about cloud storage (public, private and hybrid) along with data protection, data management, data footprint reduction among other related topics and best practices, I happen to know of some good resources. Those resources in addition to the links provided above are titled Cloud and Virtual Data Storage Networking (CRC Press) that you can learn more about here as well as find at Amazon among other venues. Also, check out Enterprise Systems Backup and Recovery: A Corporate Insurance Policy by Preston De Guise (aka twitter @backupbear ) which is a great resource for protecting data.
This web cast session takes a look at the state of public, private and hybrid cloud storage solutions and services including what you need to know to be prepared for a successful deployment. Topics to be covered include best practices, management and data protection in addition to navigating the hype and FUD associated with cloud storage today.
Check out the web cast either live or the replay later.
Have you heard or read the reports and speculation that VTLs (Virtual Tape Libraries) are dead?
It seems that in IT the all to popular trend is to declare something dead so that your new product or technology can have a chance of making it in to the market or perhaps seen in a better light.
Sometimes this approach works to temporary freeze the market until common sense and clarity returns to the market or until something else fun to talk about comes along and in other cases, the messages can fall on deft ears.
The approach of declaring something dead tends to play well for those who like shiny new toys (SNT) or new shiny toys (NST) and being on the popular, cool trendy bandwagon.
Not surprisingly, while some actual IT customers can fall into the SNT or NST syndrome, its often the broader industry including media, bloggers, analysts, consultants and other self proclaimed or anointed pundits as well as vendors who latch on to the declare it dead movement. After all, who wants to talk about something that is old, boring and already being sold to paying customers who are using it. Now this is not a bad thing as we need a balance of up and coming challengers to keep the status quo challenged, likewise we need a balance of the new to avoid death grips on the old and what is working.
Likewise, many IT customers particularly larger ones tend to be very risk averse and conservative with their budgets protecting their investments thus they may only go leading bleeding edge if there is a dual redundant blood bank with a backup on hot standby (thats some HA humor BTW).
Another reason that declaring items dead in support of SNT and NST is that while many of the commonly declared dead items are on the proverbial plateau of productivity for IT customers, that also can mean that they are on the plateau of profitability for the vendors.
However, not all good things last and at sometime, there is the need to transition from the old to the new and this is where things like virtualization including virtual tape libraries or virtual disk libraries or virtual storage library or what ever you want to call a VxL (more on what a VxL is in a moment) can come into play.
I realize that for some, particularly those who like to grasp on to SNT, NST and ride the dead pool bandwagons this will probably appear as snarky or cynical which is fine, after all, for some, you should be laughing to the bank and if not, you may in fact be missing out on an opportunity for playing in the dead pool marketing game.
Now back to VxL.
In the case of VTLs, for some it is the T word that bothers them, you know T as in Tape which is not a SNT or NST in an age where SSD has supposedly killed the disk drive which allegedly terminated tape (yeah right). Sure tape is not being used as much for backup as it has in the past with its role shifting to that of longer term retention, something that it is well suited for.
For tape fans (or cynics) you can read more here, here and here. However there is still a large amount of backup/restore along with other data protection or preservation (e.g. archiving) processing (software tools, processes, procedures, skill sets, management tools) that still expects to see tape.
Hence this is where VTLs or VxLs come into play leveraging virtualization in an Life Beyond Consolidation (and here) scenario providing abstraction, transparency, agility and emulation and IMHO are still very much alive and evolving.
Ok, for those who do not like or believe in or of its continued existence and evolving role, substitute the T (tape) with X and you get a VxL. That is, plug in what ever X word that makes you happy or marketable or a Shiny New TLA. For example Virtual Disk Library, Virtual Storage Library, Virtual Backup Library, Virtual Compression Library, Virtual Dedupe Library, Virtual ILM Library, Virtual Archive Library, Virtual Cloud Library and so forth. Granted some VxLs only emulate tape and hence are VTLs while others support NAS and other protocols (or personalities) not to mention functionality ranging from replication, DFR as well as automated policy management.
However, keep in mind that if your preference is VTL, VxL or what ever other buzzword bingo name that you want to use or come up with, look at how virtualization in the form of abstraction, transparency and emulation can bridge the gap between the new (disk based data protection) combined with DFR (Data Footprint Reduction) and the old (existing backup/restore, archive or other management tools and processes.
Here are some additional links pertaining to VTLs (excuse me, VxLs):
Virtual tape libraries: Old backup technology holdover or gateway to the future?
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
