Data security Archives

January 30, 2014March 7, 2022

Part II Until the focus expands to data protection – What to do about it

Storage I/O trends

Part II – Until the focus expands to data protection – What to do about it

This is the second of a three-part series (read part I here) about how vendors are keeping backup alive, however what they can and should do to shift and expand the conversation to data protection and related themes.

Modernizing data protection and what to do about it

Building off of what was mentioned in the first post, lets take a look at what can be done including expanding the conversation around data protection in support of business continuance (BC), disaster recovery (DR), high availability (HA), business resiliency (BR) not to mention helping backup to actually retire (someday). Now when I backup retire, I’m not necessarily talking about a technology such as hardware, software or a service including clouds, rather when, where, why and how data gets protected. What I mean by this is to step back from looking at the tools and technologies to how they are used and can be used in new and different ways moving forward.

StorageIO people convergence
Converged people and technology teams

All to often I see where new technologies or tools get used in old ways which while providing some near-term relief, the full capabilities of what is being used may not be fully realized. This also ties into the theme of people not technologies can be a barrier to convergence and transformation that you can read more about here and here.

Whats your data protection strategy, business or technology focused?

expand focus beyond tools
Data protection strategy evolving beyond tools looking for a problem to solve

Part of modernizing data protection is getting back to the roots or fundamentals including revisiting business needs, requirements along with applicable threat risks to then align application tools, technologies and techniques. This means expanding focus from just the technology, however also more importantly how to use different tools for various scenarios. In other words having a tool-box and know how to use it vs. everything looking like a nail as all you have is a hammer. Check out various webinars, Google+ hangouts and other live events that I’m involved with on the StorageIO.com events page on data protection and related data infrastructure themes including BackupU (getting back to the basics and fundamentals).

data protection options

Everything is not the same, leverage different data protection approaches to different situations

Wrap up (for now)

Continue reading part three of this series here to see what can be done (taking action) about shifting the conversation about modernizing data protection. Also check out conversations about trends, themes, technologies, techniques perspectives in my ongoing data protection diaries discussions (e.g. www.storageioblog.com/data-protection-diaries-main/).

Ok, nuff said

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

January 30, 2014January 23, 2019

Until the focus expands to data protection, backup is staying alive!

Storage I/O trends

Until the focus expands to data protection, backup is staying alive!

This is the first of a three-part series discussing how and why vendors are keeping backup alive, read part two here.

Some vendors, Value Added Resellers (VARs), pundits (consultants, analysts, media, bloggers) and their followers want backup to not only be declared dead, they also want to attend (or send flowers) to the wake and funeral not to mention proof of burial so to speak.

Yet many of these same vendors, VARs and their pundits also are helping or causing backup to staying alive.

Sure there are plenty of discussion including industry adoption and customer deployment around modernizing backup and data protection that are also tied to disaster recovery (DR), business continuance (BC), high availability (HA) and business resiliency (BR).

On the other hand the usual themes are around talking about product or technology deployment to modernize backup by simply swapping out hardware (e.g. disk for tape, cloud for disk), applying data footprint reduciton (DFR) including archiving, compression and dedupe or, another common scenario of switching from one vendors tool to another.

How vendors are helping backup staying alive?

One of the routine things I hear from vendors among others is that backup needs to move from the 70’s or 80’s or 90’s to the current era when the John Travolta and Oliva Newton John movie Saturday Night Fever and the Bee Gees song "Stayin Alive" appeared (click here to hear the song via Amazon).

Stayin Alive Image via Amazon.com

Some vendors keep talking and using the term backup instead of expanding the conversation to data protection that includes backup/restore, business continuance (BC), disaster recovery (DR) along with archiving and security. Now let’s be that we can not expect something like backup to be removed from the vocabulary overnight as its been around for decades, hence it will take time.

IMHO: The biggest barrier to moving away from backup is the industry including vendors, their pundits, press/media, vars and customers who continue to insist on using or referring to back up vs. expanding the conversation to data protection. – GS @StorageIO

Until there’s a broad focus on shifting to and using the term data protection including backup, BC, DR and archiving, people will simply keep referring to what they know, read or hear (e.g. backup). On the other hand if the industry starts putting more focus on using data protection with backup, people will stat following suit using the two and over time backup as a term can fade away.

Taking a step back to move forward

Some of the modernizing backup discussions is actually focused on take a step back to reconsider why, when, where, how and with what different applications, systems and data gets protected. certainly there are the various industry trends, challenges and opportunities some of which are shown below including more facts to protect, preserve and service for longer periods of time.

Likewise there are various threat risks or scenarios to protect information assets from or against, not all of which are head-line news making event situations.

data protection threat risk scenarios

Not all threat risks are headline news making events

There is an old saying in and around backup/restore, BC, DR, BR and HA of never letting a disaster go to waste. What this means is that if you have never noticed, there is usually a flurry of marketing and awareness activity including conversations about why you should do something BC, DR and other data protection activities right around, or shortly after a disaster scenario. However not all disasters or incidents are headline news making events and hence there should be more awareness every day vs. just during disaster season or situations. In addition, this also means expanding the focus on other situations that are likely to occur including among others those in the following figure.

data protection headline news and beyond

Continue reading part two of this series here to see what can be done about shifting the conversation about modernizing data protection. Also check out conversations about trends, themes, technologies, techniques perspectives in my ongoing data protection diaries discussions (e.g. www.storageioblog.com/data-protection-diaries-main/).

Ok, nuff said

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

January 30, 2014March 7, 2022

Part III Until the focus expands to data protection – Taking action

Storage I/O trends

Part III – Until the focus expands to data protection – Taking action

This is the thrid of a three-part series (read part II here) about how vendors are keeping backup alive, however what they can and should do to shift and expand the conversation to data protection and related themes.

Modernizing is more than simply swapping one technology for another

As I have said for a couple of years now, modernizing data protection, or data protection modernization if you prefer is more than simply deduping or swapping out media, mediums, tape, disk, clouds, software or services like a recurring flat tire on an automobile. If you keep getting flat tires, instead of treating the symptom, find and fix the problem which means for backup, taking a step back and realizing that what is really being done is protecting data (e.g. data protection).

Granted the security people may not like sharing the term data protection as some of them prefer to keep that unique, just like some of the compliance people want to keep archiving exclusive to their focus areas, however lets move on.

On the other hand, data protection also means that, protect, preserve and enable data and information to be accessed and served when and were needed in a cost-effective way with consistency and coherency.

Sure there is still the act of making a copy or a backup at time intervals (frequency) with various coverage (how much gets copied) to multiple locations (copies) with versions kept for different amounts of time (retention) to support RTO and RPO, not to mention SLA and SLO for ITSM (how’s that for some buzzword bingo ;).

Buzzword bingo

This means using copies, sync (or rsync), snapshots, replication and CDP, discrete copies such as backups along with all the other buzzword bingo enabling tools, technologies and techniques (e.g. Agent or Agent less, Archive, Availability zones. Not to mention Bare metal, virtual bare metal, Block based, CDP, Compression, Consolidation, Deletion, Data management, Dedupe, eDiscovery, durability, erasure coding/parity, file level, meta data and policy management, replication, snapshots, RAID, plugin, object storage, NAS, VTL, disk, tape, cloud, virtual among others). In addition to taking a step back, this also means rethinking why, how, when, where data (and information) gets protected to meet various threat risks as well as diverse business requirements.

Storage I/O toolbox
No tools in the toolbox (physical, virtual or cloud)

Part of the rethinking is expanding the focus from what are the tools, who makes what’s, how do they work, their features and functions to how to use the tool or technology for different things.

Storage I/O backup and data protection tools
Various tools (hardware, software, services) for different physical, virtual and cloud tasks

This is like going into a store like Lowe’s or Home Depot and talking to the sales people their (ok, associates or team members) who can tell you everything thing there is to know about the tool or technology, however they can’t tell you how to use it.

Sometimes you can get lucky and there will be somebody working at the tool (hardware or software) store who will ask you what you are trying to do and give you advice based on their experience of a different approach with another tool or tools and some supporting material or parts and supplies.

Does this sound familiar to data infrastructure or IT in general, not to mention server, storage, backup and data protection among other areas of interest?

If all you have, or know how to use is a hammer, then everything or situation starts to look like a nail. Expand your toolbox with more tools AND learn how to use or apply them in new and different ways. Align the right tool, technology and technique to the task at hand!

Expand from talking new technology to using new (and old) things in new ways

In addition to focusing on new tools and technology along with their associated terminologies across physical, virtual and cloud environments, it is also time to expand the discussion and awareness to using new (and old) things in new ways. This also means expanding the terminology from backup/restore to more comprehensive data protection as part of modernizing your environment.

For example some people (and vendors) use the term or phrase "Modernizing Data Protection" to mean swap out tape for disk, or disk for cloud, or one cloud for another cloud, or upgrade from one software version to another, or simply swap one vendors software or tool for another, yet continue to use it for all practical purposes in the same way.

Sure, moving from hourly or daily copies to tape over to direct to disk and then either redeploying tape where it is better suited (streaming large amounts of data, powering off to save energy, e.g. deep cold archive). This also means leveraging fast random access to small files that need to be recovered (usually within first hours or days of being protected).

technology alignment
Aligning tools, technologies, techniques to various threat risk scenarios

Modernizing data protection (also known as transformation) also means recognizing that not everything is the same in the data center or information factory regardless of size, and that there are also different and evolving data access patterns. Another reason and trend to consider is that there is no such thing as an information recession and that people plus data are living longer as well as getting larger.

Expand your awareness and focus beyond simply knowing what the tools are and who makes them to how, when, where, why along with pros/cons of using them to discuss different situations. This means having multiple tools in your data protection toolbox as well as knowing how to use different tools for various tasks instead of always using a hammer. – GS @StorageIO

The data protection continuum, more than tools and technoligiues

Call to action, stop talking about it, start walking the talk

If you or somebody else is tired of hearing about backup, then stop complaining about it and take some action. Following are some things to expand your thinking, awareness, discussions and activities around modernizing data protection (and moving beyond traditional backup).

Take a step back and check the basics or fundamentals of data protection which when enabled, allows your organization to move forward after a small or big incident (or disaster).
Start thinking beyond backup tools and technologies (hardware, software, services) particular how its been done, to why it needs to be done, how can it be done differently.
Revisit why you are protecting different things, realize that not everything is the same, so does that mean you have to protect everything the same way?
Learn about how to use different tools and technologies which is different from learning about the tools, features and functions.
Also keep in mind that a barrier is often people and process (along with organizational politics) that also result in new (and old) technologies being used in old ways.
Think about using different tools and technologies in different e.g. hybrid ways.
This means start using new (and old) tools, techniques, techniques in new ways, start to apply your return on innovation by using things to discuss issues, vs. simply using them for the sake of using them.

In addition to the above items, here are some added links on various topics and themes mentioned here:

BackupU – Vendor and technology neutral series of on-line webinars, Google+ hangouts, book chapter downloads and other content (Sponsored by Dell Data Protection Software, that’s a disclosure btw )

Via StorageIOblog – Only You Can Prevent Cloud Data Loss,
Cloud conversations: confidence, certainty and confidentiality,
Modernizing data protection with certainty,
More Data Footprint Reduction (DFR) Material,
More modernizing data protection, virtualization and clouds with certainty,
EMC Evolves Enterprise Data Protection with Enhancements and Data protection modernization, more than swapping out media.

Via StorageIO Reports/Resources Page – backup, restore, BC, DR and archiving available here including presentations and book chapter downloads

Via Internet evolution – People, Not Tech, Prevent IT Convergence.

Closing comments (for now)

Now having said all of that, It would be unrealistic to think that we can simply overnight drop the term backup and switch to data protection, after all, we need backwards compatibility. However until the industry which means from vendors, their pundits (analyst, bloggers, consultants, evangelists), press/media, vars, investors and customers start thinking and speaking in the broader context of data protection, life beyond backup, guess what, we will still be talking about backup. Start calling it (e.g. backup) data protection and perhaps within a generation (or sooner), the term backup will have been ILM, compressed, deduped, tiered, spun down, put into deep cold archive storage to take a long REST on object storage with a NAS interface in a software defined hybrid virtualized cloud ;).

Watch for more data protection conversations about related trends, themes, technologies, techniques perspectives in my ongoing data protection diaries discussions (e.g. www.storageioblog.com/data-protection-diaries-main/).

Ok, nuff said

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

January 30, 2014April 2, 2022

Data Protection Diaries – My data protection needs and wants

Storage I/O trends

Blog post: Data Protection Diaries – My data protection needs and wants

Update 1/10/18

Rather than talking about what others should do or consider for their data protection needs, for this post I wrote down some notes using my Livescribe about what I need and want for my environment. As part of walking the talk in future posts I’m going to expand a bit more on what I’m doing as well as considering for enhancements to my environment for data protection which consists of cloud, virtual and physical.

Why and what am I Protecting?

live scribe example
Livescribe notes that I used for creating the following content

What is my environment

Server and StorageIO (aka StorageIO) is a small business that is focused in and around data infrastructures which includes data protection as a result, have lots of data including videos, audio, images, presentations, reports, research as well, file serving as back-office applications. Then there are websites, blog, email and related applications, some of which are cloud based that are also part of my environment that have different availability, durable, and accessibility requirements.

My environment includes local on-site physical as well as virtual systems, mobile devices, as well as off-site resources including a dedicated private server (DPS) at a service provider. On one hand as a small business, I could easily move most if not everything into the cloud using an as a service model. However, I also have a lab and research environment for doing various things involving data infrastructure including data protection so why not leverage those for other things.

Why do I need to protect my information and data infrastructure?

Protect and preserve the business along with associated information as well as assets
Compliance (self and client based, PCI and other)
Security (logical and physical) and privacy to guard against theft, loss, instrusions
Logical (corruption, virus, accidental deletion) and physical damage to systems, devices, applications and data
Isolate and contain faults of hardware, software, networks, people actions from spreading to disasters
Guard against on-site or off-site incidents, acts of man or nature, head-line news and non head-line news
Address previous experience, incidents and situations, preventing future issues or problems
Support growth while enabling agility, flexibity
Walk the talk, research, learning increasing experience

My wants – What I would like to have

Somebody else pay for it all, or exist in world where there are no threat risks to information (yeh right ;) )
Cost effective and value (not necessarily the cheapest, I also want it to work)
High availability and durability to protect against different threat risks (including myself)
Automated, magically to take care of everything enabled by unicorns and pixie dust ;).

My requirements – What I need (vs. want):

Support mix of physical, virtual and cloud applications, systems and data
Different applications and data, local and some that are mobile
Various operating environments including Windows and Linux
NOT have to change my environment to meet limits of a particular solution or approach
Need a solution (s) that fit my needs and that can scale, evolve as well as enable change when my environment does
Also leverage what I have while supporting new things

Data protection topics, trends, technologies and related themes

Part 1 – Data Infrastructure Data Protection Fundamentals
Part 2 – Reliability, Availability, Serviceability ( RAS) Data Protection Fundamentals
Part 3 – Data Protection Access Availability RAID Erasure Codes ( EC) including LRC
Part 4 – Data Protection Recovery Points (Archive, Backup, Snapshots, Versions)
Part 5 – Point In Time Data Protection Granularity Points of Interest
Part 6 – Data Protection Security Logical Physical Software Defined
Part 7 – Data Protection Tools, Technologies, Toolbox, Buzzword Bingo Trends
Part 8 – Data Protection Diaries Walking Data Protection Talk
Part 9 – who’s Doing What ( Toolbox Technology Tools)
Part 10 – Data Protection Resources Where to Learn More
Data Protection Diaries series
Data Infrastructure server storage I/O network Recommended Reading List Book Shelf
Software Defined Data Infrastructure Essentials (CRC 2017) Book

Wrap and summary (for now)

Taking a step back to look at a high-level of what my data protection needs are involves looking at business requirements along with various threat risks, not to mention technical considerations. In a future post I will outline what I am doing as well as considering for enhancements or other changes along with different tools, technologies used in hybrid ways. Watch for more posts in this ongoing series of the data protection dairies via www.storageioblog.com/data-protection-diaries-main/.

Ok, nuff said (for now)

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

January 21, 2014March 7, 2022

Securing your information assets and data, what about your storage?

Storage I/O trends

Securing your information assets and data, what about your storage?

Recently I did a piece over at the site Information Security Buzz title How Secure Is Your Data Storage? that takes a cursory look at securing your digital assets from a storage perspective. Keep in mind that data protection can mean many things to different people from various focus or technology domain perspectives. Likewise there are various threat risks to protect against and, not all of them are head-line news making events.

data protection threat risk scenarios

Protecting data and data protection

Protecting your data or data protection is a diverse topic and not exclusive to just backup/restore, business continuance (BC), disaster recovery (DR), high availability (HA), durability, archiving, privacy and compliance (PCI, Hippa, High-tech, Sarbox, etc) or security (logical [encryption, access control, identity management] and physical).

In the broader scope and context of information infrastructures and data infrastructures, think of data protection as part of or enabling protect, process, preserve and serving of information in an effective way that does not introduce complexity or compromise your digital and physical assets.

Following is an excerpt from the piece over at Information Security Buzz:

The usual belief is that information behind firewall’s and on storage attached to servers that have rights access control and find access, all is safe; hence no need to encrypt the real storage device.

There is a couple of other usual comments or statements that people make to me about encrypting storage devices that it is too difficult due to lack of good key management, and the other is that people say the encryption algorithms are no good. Both can be valid points, particular given what we are hearing with the NSA and other government activities. My usual response is a) have spare keys placed in safe trusted locations and b) do you lock the doors and windows on your home as somebody who really wants to get in probably can, hence need for multiple rings of security, however the encryption will deter the casual or more typical adversary.

Click to read more

Additional data protection topics and links

In addition to the above, also check out the following related items on the many difference faces or facets of data protection.

Various StorageIO tips and articles from different venues: Via StateTech Magazine – 5 Tips for Factoring Software into Disaster Recovery Plans and Via the StorageIO fall November 2013 news letter, Cloud and data protection perspectives.

Also via StorageIOblog: Data protection modernization, more than swapping out media and Cloud conversations: Has Nirvanix shutdown caused cloud confidence and data protection concerns? along with In the data center or information factory not everything is the same plus Securing data at rest and fast secure erase with SED’s.

Also check out BackupU (www.software.dell.com/backupU) series of webinars and Google+ hangouts that I’m involved with about modernizing and rethinking data protection. Note that while Dell is the sponsor of these events, they are also vendor and technology neutral, that’s a disclosure btw fwiw ;) ).

Closing perspective, for now…

Only you can prevent data loss as it is a shared responsibility!

Ok, nuff said (for now)

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

January 13, 2014November 26, 2023

Some Windows Server Storage I/O related commands

Storage I/O trends

Some Windows Server Storage I/O related commands

The following are some commands and tools for Microsoft Windows environments that are useful for storage I/O activities (among others).

Finding a Windows physical disk, SSD or storage system device name

So you may know and how to find out the more familiar Windows storage device (Solid State Device – SSD, Hard Disk Drives – HDD among others) names such as A:, B:, C:, D: as what you can view from the Windows Explorer, Computer or Admin tools.

Windows storage devices

However what if you need to find out a physical name for raw (not mounted) and mounted devices for configuration? For example, if you have a tool that wants the physical name for your C: drive that might be \\.\PhysicalDrive0\?

No worries, use the command WMIC DISKDRIVE LIST BRIEF

WIndows physical device name

Need more detail about the devices beyond what is shown above?

Then use WMIC DISKDRIVE LIST or as in the above example, direct the output to a file with the results shown below (scroll to the left or right to see more detail information).

        Availability  BytesPerSector  Capabilities  CapabilityDescriptions                 CompressionMethod  ConfigManagerErrorCode  ConfigManagerUserConfig  DefaultBlockSize  Description  DeviceID            ErrorCleared  ErrorDescription  ErrorMethodology  Index  InstallDate  InterfaceType  LastErrorCode  Manufacturer            MaxBlockSize  MaxMediaSize  MediaLoaded  MediaType              MinBlockSize  Model                                  Name                NeedsCleaning  NumberOfMediaSupported  Partitions  PNPDeviceID                                                  PowerManagementCapabilities  PowerManagementSupported  SCSIBus  SCSILogicalUnit  SCSIPort  SCSITargetId  SectorsPerTrack  Signature   Size           Status  StatusInfo  SystemName  TotalCylinders  TotalHeads  TotalSectors  TotalTracks  TracksPerCylinder  
              512             {3, 4}        {"Random Access", "Supports Writing"}                     0                       FALSE                                      Disk drive   \\.\PHYSICALDRIVE2                                                    2                   SCSI                          (Standard disk drives)                              TRUE         Fixed hard disk media                ATA ST3000DM001-1CH1 SCSI Disk Device  \\.\PHYSICALDRIVE2                                         0           SCSI\DISK&VEN_ATA&PROD_ST3000DM001-1CH1\5&3626375C&0&000600                                                         0        0                3         6             63               0           3000590369280  OK                  DBIOTEST    364801          255         5860528065    93024255     255                
              512             {3, 4}        {"Random Access", "Supports Writing"}                     0                       FALSE                                      Disk drive   \\.\PHYSICALDRIVE3                                                    3                   SCSI                          (Standard disk drives)                              TRUE         Fixed hard disk media                SEAGATE ST600MP0034 SCSI Disk Device   \\.\PHYSICALDRIVE3                                         0           SCSI\DISK&VEN_SEAGATE&PROD_ST600MP0034\5&3626375C&0&000A00                                                          0        0                3         10            63                           600124654080   OK                  DBIOTEST    72961           255         1172118465    18605055     255                
              512             {3, 4}        {"Random Access", "Supports Writing"}                     0                       FALSE                                      Disk drive   \\.\PHYSICALDRIVE4                                                    4                   SCSI                          (Standard disk drives)                              TRUE         Fixed hard disk media                SEAGATE ST600MX0004 SCSI Disk Device   \\.\PHYSICALDRIVE4                                         0           SCSI\DISK&VEN_SEAGATE&PROD_ST600MX0004\5&3626375C&0&000C00                                                          0        0                3         12            63                           600124654080   OK                  DBIOTEST    72961           255         1172118465    18605055     255                
              512             {3, 4}        {"Random Access", "Supports Writing"}                     0                       FALSE                                      Disk drive   \\.\PHYSICALDRIVE1                                                    1                   SCSI                          (Standard disk drives)                              TRUE         Fixed hard disk media                SEAGATE ST9300603SS SCSI Disk Device   \\.\PHYSICALDRIVE1                                         0           SCSI\DISK&VEN_SEAGATE&PROD_ST9300603SS\5&3626375C&0&000400                                                          0        0                3         4             63                           299992412160   OK                  DBIOTEST    36472           255         585922680     9300360      255                
              512             {3, 4}        {"Random Access", "Supports Writing"}                     0                       FALSE                                      Disk drive   \\.\PHYSICALDRIVE0                                                    0                   SCSI                          (Standard disk drives)                              TRUE         Fixed hard disk media                VMware Virtual disk SCSI Disk Device   \\.\PHYSICALDRIVE0                                         2           SCSI\DISK&VEN_VMWARE&PROD_VIRTUAL_DISK\5&1982005&1&000000                                                           0        0                2         0             63               -873641784  64420392960    OK                  DBIOTEST    7832            255         125821080     1997160      255

Remembering (or learning) Xcopy

Some of you might be familiar with Xcopy and if not, it is a handy tool for copying files, folders and directories to local as well as networked storage. Some handy Xcopy command switches include:

/j = use un-buffered IO for large files
/y = suppress prompting
/c = continue if error
/E = copy sub directories
/H = copy hidden files
/Q = quiet mode (don’t list files being copied)

In the following example the content of the Videos folder and its sub-directories (83.5GB) are copied to another destination. Note the Time /T command that is also shown which is useful for timing how long the copy takes (e.g. subtract start-time from end-time and you have elapsed time). In this example 83.5GB are copied from one place to another on the same SSD device and using the results of the Time /T command the elapsed time was about 12 minutes.

Windows SSD TRIM
Xcopy command example

Diskpart, don’t be scared, however be careful

Ever have a Windows storage device or system that failed to boot, or a problem with a partition, volume or other issue?

How about running into a situation where you are not able to format a device that you know and can confirm is ok to erase, yet you get a message that the volume is write protected or read only?

Diskpart is handy, powerful and potentially dangerous tool if you are not careful as you could mistakenly drop a good volume or partition (e.g. the importance of having good backups). However Diskpart can be used to help repair storage devices that have boot problems, or for clearing read only attributes among other tasks. If you are prefer GUI interfaces, many of the Diskpart functions can also be done via Disk Management interface (e.g. Control Panel -> All Control Panel Items -> Administrative Tools -> Computer Management -> Storage -> Disk Management ). Note that Diskpart to do certain functions will need to be run as Administrator.

windows diskpart

In the above example the LIST DISK command shows what disks are present (on-line or off-line) which means that you may see devices here that do not show up elsewhere. Also shown is selecting a disk and then listing partitions, selecting a partition and showing attributes. The Attribute command can be used for clearing Read Only modes should a partition become write protected.

Hint, ever have a device that was once had VMware installed on it, then you move it to Windows and try to reformat for use and get a Read Only error? If so, you will want to have a look at Diskpart and the Attribute commands. However BE CAREFULL and pay attention which disk, partition and volumes you are working with as you can easily cause a problem that would result in testing how good your backups are.

Is SATA SSD TRIM Enabled?

If you have a SATA SSD the TRIM command is a form of garbage collection that is supported with Windows 7 (SAS drives use the SCSI UNMAP). Not sure if your system has TRIM enabled? Try the following command as administrator. Note that if you see a result of "0" then TRIM is enabled while a value of "1" means that it is disabled for your system.

Windows SSD TRIM

Want to learn more about TRIM, check out this piece from Intel as well as this Microsoft Windows item.

Having issues with collecting CPU and performance statistics?

Have an issue or problem collecting your system statistics, or when running a benchmark, workload generation tool such as vdbench and getting an "Unable to obtain CPU statistics"?

Try the Lodctr /R command (as administrator), however read this Microsoft Tip first to learn more.

Windows Lodctr /R

Sdelete and drive erase

Like its name implies, if you do not have this tool, you can download it here from Microsoft to not only delete files, folders, as well as write "0" patterns across a disk to secure erase it. You can specify the number of times you want to run the write "0" patterns across a disk to meet your erasure requirements.

There is also another use for Sdelete which is if you need or want to pre-condition a SSD or other device such as for testing, you can run a pre-conditioning pass using Sdelete.

Some command options include -p #n where "n" is the number of times to run, -s recursive to process sub-directories, -z to write "0" or zero out the space on the device, -c for clean, -a to process read-only attributes. Learn more and get your copy of Sdelete from Microsoft here.

Rufus, Seatools, Samsung Disk Magician and Cyberduck

A handy tool available from Seagate (may only work with Seagate and their partner devices) is SeaTools that can give drive information, health and status as well as perform various tests including SMART.

Seagate Seatools example

Different HDD and SSD as well as storage system vendors give tools for configuration, monitoring, management and in some cases data movement with their solutions. Samsung SSD Magician is a tool I have installed for managing my SSDs (830 and 840 Pros) that has features for updating firmware, drive health as well as performance optimization. Other hand tools include the Samsung copy tool based on Clonix as Acronis among other clone or data migration utilities (more on those in a future post).

Samsung SSD Magician

While the Microsoft WIndows USB Tool is handy for dealing with Microsoft ISO, however for creating USB’s with ISO’s such as for installing VMware or Linux on bare metal systems, Rufus is a handy tool to have in the tool-box.

Rufus ISO to USB tool

Another useful tool that functions as an SSH and FTP utility is Cyberduck that also supports access to Amazon S3 among other cloud services.

There are many other tools for server, storage I/O and other activities on WIndows, not to mention other platforms, however hopefully you find the above useful.

How about it, what’s your favorite Windows server, storage I/O tools and commands?

Ok, nuff said (for now)

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

January 6, 2014November 26, 2023

Goodbye 2013, hello 2014, predictions past, present and future

Storage I/O trends

Good by 2013 and hello 2014 along with predictions past, present and future

First, for those who may have missed this, thanks to all who helped make 2013 a great year!

Looking back at 2013 I saw a continued trend of more vendors and their media public relations (PR) people reaching out to have their predictions placed in articles, posts, columns or trends perspectives pieces.

Hmm, maybe a new trend is predictions selfies? ;)

Not to worry, this is not a wrapper piece for a bunch of those pitched and placed predictions requests that I received in 2013 as those have been saved for a rainy or dull day when we need to have some fun ;) .

What about 2013 server storage I/O networking, cloud, virtual and physical?

2013 end up with some end of year spree’s including Avago acquiring storage I/O and networking vendor LSI for about $6.6B USD (e.g. SSD cards, RAID cards, cache cards, HBA’s (Host Bus Adapters), chips and other items) along with Seagate buying Xyratex for about $374M USD (a Seagate suppliers and a customer partner).

Xyratex is known by some for making the storage enclosures that house hard disk drive (HDD’s) and Solid State Device (SSD) drives that are used by many well-known, and some not so well-known systems and solution vendors. Xyratex also has other pieces of their business such as appliances that combine their storage enclosures for HDD and SSD’s along with server boards, along with a software group focus on High Performance Compute (HPC) Lustre. There is another part of the Xyratex business that is not as well-known which is the test equipment used by disk drive manufacturers such as Seagate as part of their manufacturing process. Thus the Seagate acquisition moves them up market with more integrated solutions to offer to their (e.g. Seagate and Xyratex) joint customers, as well as streamline their own supply chain and costs (not to mention sell equipment to the other remaining drive manufactures WD and Toshiba).

Storage I/O trends

Other 2013 acquisitions included (Whiptail by Cisco, Virident by WD (who also bought several other companies), Softlayer by IBM) along with various mergers, company launches, company shutdowns (cloud storage Nirvanix and SSD maker OCZ bankruptcy filing), and IPO’s (some did well like Nimble while Violin not so well), while earlier high-flying industry darlings such as FusionIO are now the high-flung darling targets of the shareholder sock lawsuit attorneys.

2013 also saw the end of SNW (Storage Network World), jointly produced by SNIA and Computerworld Storage in the US after more than a decade. Some perspectives from the last US SNW held October 2013 can be found in the Fall 2013 StorageIO Update Newsletter here, granted those were before the event was formal announced as being terminated.

Speaking of events, check out the November 2013 StorageIO Update Newsletter here for perspectives from attending the Amazon Web Services (AWS) re:Invent conference which joins VMworld, EMCworld and a bunch of other vendor world events.

Lets also not forget Dell buying itself in 2013.

StorageIO in the news

Click on the following links read (and here) more about various 2013 industry perspectives trends commentary of mine in various venues, along with tips, articles, newsletters, events, pod cast, videos and other items.

What about 2014?

Perhaps 2014 will build on the 2013 momentum of the annual rights of pages refereed to as making meaningless future year trends and predictions as being passe?

Not that there is anything wrong with making predictions for the coming year, particular if they actually have some relevance, practicality not to mention track record.

However that past few years seems to have resulted in press releases along with product (or services) plugs being masked as predictions, or simply making the same predictions for the coming year that did not come to be for the earlier year (or the one before that or before that and so forth).

On the other hand, from an entertainment perspective, perhaps that’s where we will see annual predictions finally get classified and put into perspectives as being just that.

Storage I/O trends

Now for those who still cling to as well as look forward to annual predictions, ok, simple, we will continue in 2014 (and beyond) from where we left off in 2013 (and 2012 and earlier) meaning more (or continued):

Software defined "x" (replace "x" with your favorite topic) industry discussion adoption yet customer adoption or deployment question conversations.

Cloud conversations shifted from lets all go to the cloud as the new shiny technology to questioning the security, privacy, stability, vendor or service viability not to mention other common sense concerns that should have been discussed or looked into earlier. I have also heard from people who say Amazon (as well as Verizon, Microsoft, Blue host, Google, Nirvanix, Yahoo and the list goes on) outages are bad for the image of clouds as they shake people’s confidences. IMHO people confidence needs to be shaken to that of having some common sense around clouds including don’t be scared, be ready, do your homework and basic due diligence. This means cloud conversations over concerns set the stage for increased awareness into decision-making, usage, deployment and best practices (all of which are good things for continued cloud deployments). However if some vendors or pundits feel that people having basic cloud concerns that can be addressed is not good for their products or services, I would like to talk with them because they may be missing an opportunity to create long-term confidence with their customers or prospects.

VDI as a technology being deployed continues to grow (e.g. customer adoption) while the industry adoption (buzz or what’s being talked about) has slowed a bit which makes sense as vendors jump from one bandwagon to the new software defined bandwagon.

Continued awareness around modernizing data protection including backup/restore, business continuance (BC), disaster recovery (DR), high availability, archiving and security means more than simply swapping out old technology for new, yet using it in old ways. After all, in the data center and information factory not everything is the same. Speaking of data protection, check out the series of technology neutral webcast and video chats that started last fall as part of BackupU brought to you by Dell. Even though Dell is the sponsor of the series (that’s a disclosure btw ;) ) the focus of the sessions is on how to use different tools, technologies and techniques in new ways as well as having the right tools for different tasks. Check out the information as well as register to get a free Data Protection chapter download from my book Cloud and Virtual Data Storage Networking (CRC Press) at the BackupU site as well as attend upcoming events.

The nand flash solid state devices (SSD) cash-dash (and shakeout) continues with some acquisitions and IPO’s, as well as disappearances of some weaker vendors, while appearance of some new. SSD is showing that it is real in several ways (despite myths, fud and hype some of which gets clarified here) ranging from some past IPO vendors (e.g. FusiuonIO) seeing exit of their CEO and founders while their stock plummets and arrival of shareholder investor lawsuits, to Violins ho-hum IPO. What this means is that the market is real, it has a very bright future, however there is also a correction occurring showing that reality may be settling in for the long run (e.g. next couple of decades) vs. SSD being in the realm of unicorns.

Storage I/O trends

Internet of Things (IoT) and Internet of Devices (IoD) may give some relief for Big Data, BYOD, VDI, Software Defined and Cloud among others that need a rest after they busy usage that past few years. On the other hand, expect enhanced use of earlier buzzwords combined with IoT and IOD. Of course that also means plenty of questions around what is and is not IoD along with IoT and if actually relevant to what you are doing.

Also in 2014 some will discover storage and related data infrastructure topics or some new product / service thus having a revolutionary experience that storage is now exciting while others will have a DejaVu moment that it has been exciting for the past several years if not decades.

More big data buzz as well as realization by some that a pragmatic approach opens up a bigger broader market, not to mention customers more likely to realize they have more in common with big data than it simply being something new forcing them to move cautiously.

To say that OpenStack and related technologies will continue to gain both industry and customer adoption (and deployment) status building off of 2013 in 2014 would be an understatement, not to mention too easy to say, or leave out.

While SSD’s continue to gain in deployment, after the question is not if, rather when, where, with what and how much nand flash SSD is in your future, HDD’s continue to evolve for physical, virtual and cloud environments. This also includes Seagate announcing a new (Kinetic) Ethernet attached HDD (note that this is not a NAS or iSCSI device) that uses a new key value object storage API for storing content data (more on this in 2014).

This also means realizing that large amounts of little data can result in back logs of lots of big data, and that big data is growing into very fast big data, not to mention realization by some that HDFS is just another distributed file system that happens to work with Hadoop.

SOHO’s and lower end of SMB begin to get more respect (and not just during the week of Consumer Electronic Show – CES).

Realization that there is a difference between Industry Adoption and Customer Deployment, not to mention industry buzz and traction vs. customer adoption.

server storage I/O trends

What about beyond 2014?

That’s easy, many of the predictions and prophecies that you hear about for the coming year have also been pitched in prior years, so it only makes sense that some of those will be part of the future.

If you have seen or experienced something you are more likely to have DejaVu.
Otoh if you have not seen or experienced something you are more likely to have a new and revolutionary moment!
Start using new (and old) things in new ways vs. simply using new things in old ways.
Barrier to technology convergence, not to mention new technology adoption is often people or their organizations.
Convergence is still around, cloud conversations around concerns get addressed leading to continued confidence for some.
Realization that data infrastructure span servers, storage I/O networking, cloud, virtual, physical, hardware, software and services.
That you can not have software defined without hardware and hardware defined requires software.
And it is time for me to get a new book project (or two) completed in addition to helping others with what they are working on, more on this in the months to come…

Here’s my point

The late Jim Morrison of the Doors said "There are things known and things unknown and in between are the doors.".

Above image and link via Amazon.com

Hence there is what we know about 2013 or will learn about the past in the future, then there is what will be in 2014 as well as beyond, hence lets step through some doors and see what will be. This means learn and leverage lessons from the past to avoid making the same or similar mistakes in the future, however doing so while looking forward without a death grip clinging to the past.

Needless to say there will be more to review, preview and discuss throughout the coming year and beyond as we go from what is unknown through doors and learn about the known.

Thanks to all who made 2013 a great year, best wishes to all, look forward to seeing and hearing from you in 2014!

Ok, nuff said (for now)

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

December 26, 2013January 23, 2019

Small Medium Business (SMB) IT continues to gain respect, what about SOHO?

Storage I/O trends

Blog post: Small Medium Business (SMB) IT continues to gain respect, what about SOHO?

Note that in Information Technology (IT) conversations there are multiple meanings for SMB including Server Message Block aka Microsoft Windows CIFS (Common Internet File System) along with its SAMBA implementation, however for this piece the context is Small Medium Business.

A decade or so ago, mention SMB (Small Medium Business) to many vendors, particular those who were either established or focused on the big game enterprise space and you might have gotten a condescending look or answer if not worse.

In other words, a decade ago the SMB did not get much respect from some vendors and those who followed or covered them.

Fast forward to today and many of those same vendors along with their pundits and media followers have now gotten their SMB grove, lingo, swagger or social media footsteps, granted for some that might be at the higher end of SMB also known as SME (Small Medium Enterprise).

Today in general the SMB is finally getting respect and in some circles its down right cool and trendy vs. being perceived as old school, stodgy large enterprise. Likewise the Remote Office Branch Office (ROBO) gained more awareness and coverage a few years back which while the ROBO buzz has subsided, the market and opportunities are certainly there.

What about Small Office Home Office (SOHO) today?

I assert that SOHO today is getting the same lack of respect that SMB in general received a decade ago.

IMHO the SOHO environment and market today is being treated with a similar lack of respect that the larger SMB received a decade ago.

Granted there are some vendors and their followings who are seeing the value and opportunity, not to mention market size potential of expanding their portfolios, not to mention routes to markets to meet their different needs of the SOHO.

relative enterprise sme smb soho positioning

What is the SOHO market or environment

One of the challenges with SMB, SOHO among other classifications are just that, the classifications.

Some classificaitons are based on number of employees, others on number of servers or workstations, while others are based on revenue or even physical location.

Meanwhile some are based on types of products, technologies or tools while others are tied to IT or general technology spending.

Some confuse the SOHO space with the consumer market space or sector which should not be a surprise if you view market segments as enterprise, SMB and consumer. However if you take a more pragmatic approach, between true consumer and SMB space, there lies the SOHO space. For some the definitions of what is consumer, SOHO, SMB, SME and enterprise (among others) will be based on number of employees, or revenue amount. Yet for others the categories may be tied to IT spending (e.g. price bands), number of workstations, servers, storage space capacity or some other metric. On the other hand some definitions of what is consumer vs. SOHO vs. SMB vs. SME or enterprise will be based on product capabilities, size, feature function and cost among other attributes.

Storage I/O trends

Understanding the SOHO

Keep in mind that SOHO can also overlap with Remote Office Branch Office (ROBO), not to mention blend with high-end consumer (prosumer) or lower bounds of SMB.

Part of the challenge (or problem) is that many confuse the Home Office or HO aspect of SOHO as being consumer.

Likewise many also confuse the Small Office or SO part of SOHO as being just the small home office or the virtual office of a mobile worker.

The reality is that just as the SMB space has expanded, there is also a growing area just above where consumer markets exist and where many place the lower-end of SMB (e.g. the bottom limits of where the solutions fit).

First keep in mind that many put too much focus and mistakenly believe that the HO or Home Office part of SOHO means that this is just a consumer focused space.

The reality is that while the HO gets included as part of SOHO, there is also the SO or Small Office which is actually the low-end of the SMB space.

Keep in mind that there are more:
SOHO than SMB
SMB than SME
SME than enterprise
F500 (Fortune 500) than F100
F100 than F10 and so forth.

Here is my point

SOHO does not have to be the Rodney Dangerfield of IT (e.g. gets no respect)!

If you jumped on the SMB bandwagon a decade ago, start paying attention to what’s going on with the SOHO or lower-end SMB sector. The reasons are simple, just as SMBs can grow up to be larger SMBs or SME or enterprise, SOHOs can also evolve to become SMBs either in business size, or in IT and data infrastructure needs, requirements.

For those who prefer (at least for now) look down upon or ignore the SOHO similar to what was done with SMB before converting to SMBism, do so at your own risk.

However let me be clear, this does not mean ignore or shift focus and thus disrupt or lose coverage of other areas, rather, extend, expand and at least become aware of what is going on in the SOHO space.

Ok, nuff said (for now)

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

November 20, 2013March 7, 2022

November 2013 Server and StorageIO Update Newsletter & AWS reinvent info

Welcome to the November 2013 edition of the StorageIO Update (newsletter) containing trends perspectives on cloud, virtualization and data infrastructure topics. Fall (here in North America) has been busy with in-person, on-line live and virtual events along with various client projects, research, time in the StorageIO cloud, virtual and physical lab test driving, validating and doing proof of concept research among other tasks. Check out the industry trends perspectives articles, comments and blog posts below that covers some activity over the past month.

Last week I had the chance to attend the second annual AWS re:Invent event in Las Vegas, see my comments, perspectives along with a summary of announcements from that conference below.

Watch for future posts, commentary, perspectives and other information down the road (and in the not so distant future) pertaining to information and data infrastructure topics, themes and trends across cloud, virtual, legacy server, storage, networking, hardware and software. Also check out our backup, restore, BC, DR and archiving (Under the resources section on StorageIO.com) for various presentation, book chapter downloads and other content.

Enjoy this edition of the StorageIO Update newsletter.

Ok, nuff said (for now)

Cheers gs

StorageIO Industry Trends and Perspectives

Industry trends: Amazon Web Services (AWS) re:Invent

Last week I attended the AWS re:Invent event in Las Vegas. This was the second annual AWS re:Invent conference which while having an AWS and cloud theme, it is also what I would describe as a data infrastructure event.

As a data infrastructure event AWS re:Invent spans traditional legacy IT and applications to newly invented, re-written, re-hosted or re-platformed ones from existing and new organizations. By this I mean a mix of traditional IT or enterprise people as well as cloud and virtual geek types (said with affection and all due respect of course) across server (operating system, software and tools), storage (primary, secondary, archive and tools), networking, security, development tools, applications and architecture.

That also means management from application and data protection spanning High Availability (HA), Business Continuance (BC), Disaster Recovery (DR), backup/restore, archiving, security, performance and capacity planning, service management among other related themes across public, private, hybrid and community cloud environments or paradigms. Hmm, I think I know of a book that covers the above and other related topic themes, trends, technologies and best practices called Cloud and Virtual Data Storage Networking (CRC Press) available via Amazon.com in print and Kindle (among other) versions.

During the event AWS announced enhanced and new services including:

WorkSpaces (Virtual Desktop Infrastructure – VDI) announced as a new service for cloud based desktops across various client devices including laptops, Kindle Fire, iPad and Android tablets using PCoIP.
Kinesis which is a managed service for real-time processing of streaming (e.g. Big) data at scale including ability to collect and process hundreds of GBytes of data per second across hundreds of thousands of data sources. On top of Kinesis you can build your big data applications or conduct analysis to give real-time key performance indicator dashboards, exception and alarm or event notification and other informed decision-making activity.
EC2 C3 instances provide Intel Xeon E5 processors and Solid State Device (SSD) based direct attached storage (DAS) like functionality vs. EBS provisioned IOPs for cost-effective storage I/O performance and compute capabilities.
Another EC2 enhancement are G2 instance that leverage high performance NVIDIA GRID GPU with 1,536 parallel processing cores. This new instance is well suited for 3D graphics, rendering, streaming video and other related applications that need large-scale parallel or high performance compute (HPC) also known as high productivity compute.
Redshift (cloud data warehouse) now supports cross region snapshots for HA, BC and DR purposes.
CloudTrail records AWS API calls made via the management console for analytics and logging of API activity.
Beta of Trusted Advisor dashboard with cost optimization saving estimates including EBS and provisioned IOPs
Relational Database Service (RDS) support for PostgresSQL including multi-AZ deployment.
Ability to discover and launch various software from AWS Marketplace via the EC2 Console. The AWS Marketplace for those not familiar with it is a catalog of various software or application titles (over 800 products across 24 categories) including free and commercial licensed solutions that include SAP, Citrix, Lotus Notes/Domino among many others.
AppStream is a low latency (STX protocol based) service for streaming resource (e.g. compute, storage or memory) intensive applications and games from AWS cloud to various clients, desktops or mobile devices. This means that the resource intensive functionality can be shifted to the cloud, while providing a low latency (e.g. fast) user experience off-loading the client from having to support increased compute, memory or storage capabilities. Key to AppStream is the ability to stream data in a low-latency manner including over networks normally not suited for high quality or bandwidth intensive applications. IMHO AppStream while focused initially on mobile app’s and gaming, being a bit streaming technology has the potential to be used for other similar functions that can leverage download speed improvements.

When I asked an AWS person if or what role AppStream might have or related to WorkSpaces their only response was a large smile and no comment. Does this mean WorkSpaces leverages AppStream? Candidly I don’t know, however if you look deeper into AppStream and expand your horizons, see what you can think up in terms of innovation. Updated 11/21/13 AWS has provided clarification that WorkSpaces is based on PCoIP while AppStream uses the STX protocols.

Check out AWS Sr. VP Andy Jassy keynote presentation here.

Overall I found the AWS re:Invent event to be a good conference spanning many aspects and areas of focus which means I will be putting it on my must attend list for 2014.

Industry trends tips, commentary, articles and blog posts
What is being seen, heard and talked about while out and about

The following is a synopsis of some StorageIOblog posts, articles and comments in different venues on various industry trends, perspectives and related themes about clouds, virtualization, data and storage infrastructure topics among related themes.

Storage I/O posts

Recent industry trends, perspectives and commentary by StorageIO Greg Schulz in various venues:

NetworkComputing: Comments on Software-Defined Storage Startups Win Funding

Digistor: Comments on SSD and flash storage
InfoStor: Comments on data backup and virtualization software

ITbusinessEdge: Comments on flash SSD and hybrid storage environments

NetworkComputing: Comments on Hybrid Storage Startup Nimble Storage Files For IPO

InfoStor: Comments on EMC’s Light to Speed: Flash, VNX, and Software-Defined

InfoStor: Data Backup Virtualization Software: Four Solutions

ODSI: Q&A With Greg Schulz – A Quick Roundup of Data Storage Industry

Recent StorageIO Tips and Articles in various venues:

FedTechMagazine: 3 Tips for Maximizing Tiered Hypervisors
InfoStor:
RAID Remains Relevant, Really!

Storage I/O trends

Recent StorageIO blog post:

EMC announces XtremIO General Availability (Part I) – Announcement analysis of the all flash SSD storage system
Part II: EMC announces XtremIO General Availability, speeds and feeds – Part two of two part series with analysis
What does gaining industry traction or adoption mean too you? – There is a difference between buzz and deployment
Fall 2013 (September and October) StorageIO Update Newsletter – In case you missed the fall edition, here it is

StorageIO Industry Trends and Perspectives

Check out our objectstoragecenter.com page where you will find a growing collection of information and links on cloud and object storage themes, technologies and trends.

StorageIO activities (out and about)

Seminars, symposium, conferences, webinars
Live in person and recorded recent and upcoming events

While 2013 is winding down, the StorageIO calendar continues to evolve, here are some recent and upcoming activities.

December 11, 2013	Backup.U	Data Protection for Cloud 201	Backup.U Google+ hangout
December 3, 2013	Backup.U	Data Protection for Cloud 101	Backup.U Online Webinar
November 19, 2013	Backup.U	Data Protection for Virtualization 201	Backup.U Google+ hangout
November 12-13, 2013	AWS re:Invent	AWS re:Invent event	Las Vegas, NV
November 5, 2013	Backup.U	Data Protection for Virtualization 101	Backup.U Online Webinar
October 22, 2013	Backup.U	Data Protection for Applications 201	Backup.U Google+ hangout

Click here to view other upcoming along with earlier event activities. Watch for more 2013 events to be added soon to the StorageIO events calendar page. Topics include data protection modernization (backup/restore, HA, BC, DR, archive), data footprint reduction (archive, compression, dedupe), storage optimization, SSD, object storage, server and storage virtualization, big data, little data, cloud and object storage, performance and management trends among others.

Vendors, VAR’s and event organizers, give us a call or send an email to discuss having us involved in your upcoming pod cast, web cast, virtual seminar, conference or other events.

If you missed the Fall (September and October) 2013 StorageIO update newsletter, click here to view that and other previous editions as HTML or PDF versions. Subscribe to this newsletter (and pass it along)

and click here to subscribe to this news letter. View archives of past StorageIO update news letters as well as download PDF versions at: www.storageio.com/newsletter

Ok, nuff said (for now).
Cheers Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)

September 28, 2013November 26, 2023

Cloud conversations: Has Nirvanix shutdown caused cloud confidence concerns?

Storage I/O trends

Cloud conversations: Has Nirvanix shutdown caused cloud confidence concerns?

Recently seven plus year old cloud storage startup Nirvanix announced that they were finally shutting down and that customers should move their data.

nirvanix customer message

Nirvanix has also posted an announcement that they have established an agreement with IBM Softlayer (read about that acquisition here) to help customers migrate to those services as well as to those of Amazon Web Services (AWS), (read more about AWS in this primer here), Google and Microsoft Azure.

Cloud customer concerns?

With Nirvanix shutting down there has been plenty of articles, blog posts, twitter tweets and other conversations asking if Clouds are safe.

Btw, here is a link to my ongoing poll where you can cast your vote on what you think about clouds.

IMHO clouds can be safe if used in safe ways which includes knowing and addressing your concerns, not to mention following best practices, some of which pre-date the cloud era, sometimes by a few decades.

Nirvanix Storm Clouds

More on this in a moment, however lets touch base on Nirvanix and why I said they were finally shutting down.

The reason I say finally shutting down is that there were plenty of early warning signs and storm clouds circling Nirvanix for a few years now.

What I mean by this is that in their seven plus years of being in business, there have been more than a few CEO changes, something that is not unheard of.

Likewise there have been some changes to their business model ranging from selling their software as a service to a solution to hosting among others, again, smart startups and establishes organizations will adapt over time.

Nirvanix also invested heavily in marketing, public relations (PR) and analyst relations (AR) to generate buzz along with gaining endorsements as do most startups to get recognition, followings and investors if not real customers on board.

In the case of Nirvanix, the indicator signs mentioned above also included what seemed like a semi-annual if not annual changing of CEOs, marketing and others tying into business model adjustments.

cloud storage

It was only a year or so ago that if you gauged a company health by the PR and AR news or activity and endorsements you would have believed Nirvanix was about to crush Amazon, Rackspace or many others, perhaps some actually did believe that, followed shortly there after by the abrupt departure of their then CEO and marketing team. Thus just as fast as Nirvanix seemed to be the phoenix rising in stardom their aura started to dim again, which could or should have been a warning sign.

This is not to solo out Nirvanix, however given their penchant for marketing and now what appears to some as a sudden collapse or shutdown, they have also become a lightning rod of sort for clouds in general. Given all the hype and fud around clouds when something does happen the distract ors will be quick to jump or pile on to say things like "See, I told you, clouds are bad".

Meanwhile the cloud cheerleaders may go into denial saying there are no problems or issues with clouds, or they may go back into a committee meeting to create a new stack, standard, API set marketing consortium alliance. ;) On the other hand, there are valid concerns with any technology including clouds that in general there are good implementations that can be used the wrong way, or questionable implementations and selections used in what seem like good ways that can go bad.

This is not to say that clouds in general whether as a service, solution or product on a public, private or hybrid bases are any riskier than traditional hardware, software and services. Instead what this should be is a wake up call for people and organizations to review clouds citing their concerns along with revisiting what to do or can be done about them.

Clouds: Being prepared

Ben Woo of Neuralytix posted this question comment to one of the Linked In groups Collateral Considerations If You Were/Are A Nirvanix Customer which I posted some tips and recommendations including:

1) If you have another copy of your data somewhere else (which you should btw), how will your data at Nirvanix be securely erased, and the storage it resides on be safely (and secure) decommissioned?

2) if you do have another copy of your data elsewhere, how current is it, can you bring it up to date from various sources (including update from Nirvanix while they stay online)?

3) Where will you move your data to short or near term, as well as long-term.

4) What changes will you make to your procurement process for cloud services in the future to protect against situations like this happening to you?

5) As part of your plan for putting data into the cloud, refine your strategy for getting it out, moving it to another service or place as well as having an alternate copy somewhere.

Fwiw any data I put into a cloud service there is also another copy somewhere else which even though there is a cost, there is a benefit, The benefit is that ability to decide which to use if needed, as well as having a backup/spare copy.

Storage I/O trends

Cloud Concerns and Confidence

As part of cloud procurement as services or products, the same proper due diligence should occur as if you were buying traditional hardware, software, networking or services. That includes checking out not only the technology, also the companies financial, business records, customer references (both good and not so good or bad ones) to gain confidence. Part of gaining that confidence also involves addressing ahead of time how you will get your data out of or back from that services if needed.

Keep in mind that if your data is very important, are you going to keep it in just one place? For example I have data backed-up as well as archived to cloud providers, however I also have local copies either on-site or off.

Likewise there is data I have local kept at alternate locations including cloud. Sure that is costly, however by not treating all of my data and applications the same, I’m able to balance those costs out, plus use cost advantages of different services as well as on-site to be effective. I may be spending no less on data protection, in fact I’m actually spending a bit more, however I also have more copies and versions of important data and in multiple locations. Data that is not changing often does not get protected as often, however there are multiple copies to meet different needs or threat risks.

Storage I/O trends

Don’t be scared of clouds, be prepared

While some of the other smaller cloud storage vendors will see some new customers, I suspect that near to mid-term, it will be the larger, more established and well funded providers that gain the most from this current situation. Granted some customers are looking for alternatives to the mega cloud providers such as Amazon, Google, HP, IBM, Microsoft and Rackspace among others, however there are a long list of others some of which who are not so well-known that should be such as Centurylink/Savvis, Verizon/Terremark, Sungurd, Dimension Data, Peak, Bluehost, Carbonite, Mozy (owned by EMC), Xerox ACS, Evault (owned by Seagate) not to mention a long list of many others.

Something to be aware of as part of doing your due diligence is determining who or what actually powers a particular cloud service. The larger providers such as Rackspace, Amazon, Microsoft, HP among others have their own infrastructure while some of the smaller service providers may in fact use one of the larger (or even smaller) providers as their real back-end. Hence understanding who is behind a particular cloud service is important to help decide the viability and stability of who it is you are subscribed to or working with.

Something that I have said for the past couple of years and a theme of my book Cloud and Virtual Data Storage Networking (CRC Taylor & Francis) is do not be scared of clouds, however be ready, do your homework.

This also means having cloud concerns is a good thing, again don’t be scared, however find what those concerns are along with if they are major or minor. From that list you can start to decide how or if they can be worked around, as well as be prepared ahead of time should you either need all of your cloud data back quickly, or should that service become un-available.

Also when it comes to clouds, look beyond lowest cost or for free, likewise if something sounds too good to be true, perhaps it is. Instead look for value or how much do you get per what you spend including confidence in the service, service level agreements (SLA), security, and other items.

Keep in mind, only you can prevent data loss either on-site or in the cloud, granted it is a shared responsibility (With a poll).

Ok, nuff said.

Cheers
Gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)

July 18, 2013January 29, 2020

Seagate provides proof of life: Enterprise HDD enhancements

Storage I/O trends

Proof of life: Enterprise Hard Disk Drives (HDD’s) are enhanced

Last week while hard disk drive (HDD) competitor Western Digital (WD) was announcing yet another (Velobit) in a string of acquisitions ( e.g. earlier included Stec, Arkeia) and investments (Skyera), Seagate announced new enterprise class HDD’s to their portfolio. Note that it was only two years ago that WD acquired Hitachi Global Storage Technologies (HGST) the disk drive manufacturing business of Hitachi Ltd. (not to be confused with HDS).

Similar to WD expanding their presence in the growing nand flash SSD market, Seagate also in May of this year extended their existing enterprise class SSD portfolio. These enhancements included new drives with 12Gbs SAS interface, along with a partnership (and investment) with PCIe flash card startup vendor Virident. Other PCIe flash SSD card vendors (manufacturers and OEMs) include Cisco, Dell, EMC, FusionIO, HP, IBM, LSI, Micron, NetApp and Oracle among others.

These new Seagate enterprise class HDD’s are designed for use in cloud and traditional data center servers and storage systems. A month or two ago Seagate also announced new ultra-thin (5mm) client (aka desktop) class HDD’s along with a 3.5 inch 4TB video optimized HDD. The video optimized HDD’s are intended for Digital Video Recorders (DVR’s), Set Top Boxes (STB’s) or other similar applications.

What was announced?

Specifically what Seagate announced were two enterprise class drives, one for performance (e.g. 1.2TB 10K) and the other for space capacity (e.g. 4TB).

	Enterprise High Performance 10K.7 (aka formerly known as Savio)	Enterprise Terascale (aka formerly known as constellation)
Class/category	Enterprise / High Performance	Enterprise High Capacity
Form factor	2.5” Small Form Factor (SFF)	3.5”
Interface	6Gbs SAS	6Gbs SATA
Space capacity	1,200GB (1.2TB)	4TB
RPM speed	10,000	5,900
Average seek	2.9 ms	12 ms
DRAM cache	64MB	64MB
Power idle / operating	4.8 watts	5.49 / 6.49 watts
Intelligent Power Management (IPM)	Yes – Seagate PowerChoice	Yes – Seagate PowerChoice
Warranty	Limited 5 years	Limited 3 years
Instant Secure Erase (ISE)	Yes	Optional
Other features	RAID Rebuild assist, Self-Encrypting Device (SED)	Advanced Format (AF) 4K block in addition to standard 512 byte sectors
Use cases	Replace earlier generation 3.5” 15K SAS and Fibre Channel HDD’s for higher performance applications including file systems, databases where SSD are not practical fit.	Backup and data protection, replication, copy operations for erasure coding and data dispersal, active in dormant archives, unstructured NAS, big data, data warehouse, cloud and object storage.

Note the Seagate Terascale has a disk rotation speed of 5,900 (5.9K RPM) which is not a typo given the more traditional 5.4K RPM drives. This slight increase in performance from 5.4K to 5.9K should give when combined with other enhancements (e.g. firmware, electronics) to boost performance for higher capacity workloads.

Let us watch for some performance numbers to be published by Seagate or others. Note that I have not had a chance to try these new drives yet, however look forward to getting my hands on them (among others) sometime in the future for a test drive to add to the growing list found here (hey Seagate and WD, that’s a hint ;) ).

What this all means?

Storage I/O trends

Wait, weren’t HDD’s supposed to be dead or dying?

Some people just like new and emerging things and thus will declare anything existing or that they have lost interest in (or their jobs need it) as old, boring or dead.

For example if you listen to some, they may say nand flash SSD are also dead or dying. For what it is worth, imho nand flash-based SSDs still have a bright future in front of them even with new technologies emerging as they will take time to mature (read more here or listen here).

However, the reality is that for at least the next decade, like them or not, HDD’s will continue to play a role that is also evolving. Thus, these and other improvements with HDD’s will be needed until current nand flash or emerging PCM (Phase Change Memory) among other forms of SSD are capable of picking up all the storage workloads in a cost-effective way.

Btw, yes, I am also a fan and user of nand flash-based SSD’s, in addition to HDD’s and see roles for both as being viable complementing each other for traditional, virtual and cloud environments.

In short, HDD’s will keep spinning (pun intended) for some time granted their roles and usage will also evolve similar to that of tape summit resources.

Storage I/O trends

With this announcement by Seagate along with other enhancements from WD show that the HDD will not only see its 60th birthday, (and here), it will probably also easily see its 70th and not from the comfort of a computer museum. The reason is that there is yet another wave of HDD improvements just around the corner including Shingled Magnetic Recording (SMR) (more info here) along with Heat Assisted Magnetic Recording (HAMR) among others. Watch for more on HAMR and SMR in future posts. With these and other enhancements, we should be able to see a return to the rapid density improvements with HDD’s observed during the mid to late 2000 era when Perpendicular recording became available.

What is up with this ISE stuff is that the same as what Xiotech (e.g. XIO) had?

Is this the same technology that Xiotech (now Xio) referred to the ISE the answer is no. This Seagate ISE is for fast secure erase of data on disk. The benefit of Instant Secure Erase (ISE) is to cut from hours or days the time required to erase a drive for secure disposal to seconds (or less). For those environments that already factor drives erase time as part of those overall costs, this can increase the useful time in service to help improve TCO and ROI.

Wait a minute, aren’t slower RPM’s supposed to be lower performance?

Some of you might be wondering or asking the question of wait, how can a 10,000 revolution per minute (10K RPM) HDD be considered fast vs. a 15K HDD, let alone SSD?

Storage I/O trends

There is a trend occurring with HDD’s that the old rules of IOPS or performance being tied directly to the size and rotational speed (RPM’s) of drives, along with their interfaces. This comes down to being careful to judge a book or in this case a drive by its cover. While RPM’s do have an impact on performance, new generation drives at 10K such as some 2.5” models are delivering performance equal to or better than earlier generation 3.5” 15K device’s.

Likewise, there are similar improvements with 5.4K devices vs. previous generation 7.2K models. As you will see in some of the results found here, not all the old rules of thumbs when it comes to drive performance are still valid. Likewise, keep those metrics that matter in the proper context.

Click on above image to see various performance results

For example as seen in the results (above), the more DRAM or DDR cache on the drives has a positive impact on sequential reads which can be good news if that is what your applications need. Thus, do your homework and avoid judging a device simply by its RPM, interface or form factor.

Other considerations, temperature and vibration

Another consideration is that with increased density of more drives being placed in a given amount of space, some of which may not have the best climate controls, humidity and vibration are concerns. Thus, the importance of drives having vibration dampening or safeguards to keep up performance are important. Likewise, even though drive heads and platters are sealed, there are also considerations that need to be taken care of for humidity in data center or cloud service providers in hot environments near the equator.

If this is not connecting with you, think about how close parts of Southeast Asia and the India subcontinent are to the equator along with the rapid growth and low-cost focus occurring there. Your data center might be temperature and humidity controlled, however others who very focused on cost cutting may not be as concerned with normal facilities best practices.

What type of drives should be used for cloud, virtual and traditional storage?

Good question and one where the answer should be it depends upon what you are trying or need to do (e.g. see previous posts here or here and here (via Seagate)).For example here are some tips for big data storage and storage making decisions in general.

Disclosure

Seagate recently invited me along with several other industry analysts to their cloud storage analyst summit in San Francisco where they covered roundtrip coach airfare, lodging, airport transfers and a nice dinner at the Epic Roast house.

hdd image

I also have received in the past a couple of Momentus XT HHDD (aka SSHD) from Seagate. These are in addition to those that I bought including various Seagate, WD along with HGST, Fujitsu, Toshiba and Samsung (SSD and HDD’s) that I use for various things.

Ok, nuff said (for now).

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio

January 29, 2013November 26, 2023

In the data center or information factory, not everything is the same

StorageIO Industry trends and perspectives image

Sometimes what should be understood, or that is common sense or that you think everybody should know needs to be stated. After all, there could be somebody who does not know what some assume as common sense or what others know for various reasons. At times, there is simply the need to restate or have a reminder of what should be known.

Consequently, in the data center or information factory, either traditional, virtual, converged, private, hybrid or public cloud, everything is not the same. When I say not everything is the same, is that different applications with various service level objectives (SLO’s) and service level agreements (SLA’s). These are based on different characteristics from performance, availability, reliability, responsiveness, cost, security, privacy among others. Likewise, there are different size and types of organizations with various requirements from enterprise to SMB, ROBO and SOHO, business or government, education or research.

There are also different threat risks for various applications or information services within in an organization, or across different industry sectors. Thus various needs for meeting availability SLA’s, recovery time objectives (RTO’s) and recovery point objectives (RPO’s) for data protection ranging from backup/restore, to high-availability (HA), business continuance (BC), disaster recovery (DR) and archiving. Let us not forget about logical and physical security of information, assets and people, processes and intellectual property.

Some data centers or information factories are compute intensive while others are data centric, some are IO or activity intensive with a mix of compute and storage. On the other hand, some data centers such as a communications hub may be network centric with very little data sticking or being stored.

Even within in a data center or information factory, various applications will have different profiles, protection requirements for big data and little data. There can also be a mix of old legacy applications and new systems developed in-house, purchased, open-source based or accessed as a service. The servers and storage may be software defined (a new buzzword that has already jumped the shark), virtualized or operated in a private, hybrid or community cloud if not using a public service.

Here are some related posts tied to everything is not the same:
Optimize Data Storage for Performance and Capacity
Is SSD only for performance?
Cloud conversations: Gaining cloud confidence from insights into AWS outages
Data Center Infrastructure Management (DCIM) and IRM
Saving Money with Green IT: Time To Invest In Information Factories
Everything Is Not Equal in the Datacenter, Part 1
Everything Is Not Equal in the Datacenter, Part 2
Everything Is Not Equal in the Datacenter, Part 3

Thus, not all things are the same in the data center, or information factories, both those under traditional management paradigms, as well as those supporting public, private, hybrid or community clouds.

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

January 8, 2013November 26, 2023

Congratulations Imation and Nexsan, are there any independent storage vendors left?

StorageIO industry trends cloud, virtualization and big data

Last week Imation, the company that is known for making CDs, DVDs, magnetic tape and in the past floppy disk (diskettes) bought Nexsan, a company known for the SATA and SAS storage products.

Imation is also (or should be) owns the TDK and Memorex names (remember is it real or is it Memorex? If not Google it). They also have had for several years removable hard disk drive (RHDD) products including the Odyssey (I am in the process of retiring mine), as well as partnership with the former ProStor for RDX and having acquired some of the assets of ProStor namely their RDX based InifiVault storage appliance. Imation has also been involved in some other things including USB and other forms of flash-based solid state devices (SSD), as well as a couple of years (2007) they launched cloud backup with DataGuard before cloud backup had become a popular buzzword topic.

Imation has also divested parts of its business over past several years including some medical related (X-ray stuff) to Kodak who occupies part of the headquarter building in Oakdale MN, or at least last time I looked when driving by there on way from the airport. They also divested their SAN lab with some of the staff going to Glasshouse and other pieces going to Lion bridge (an independent test lab company). Beyond traditional of data protection, backup/restore and archiving media or mediums from consumer to large-scale enterprise, Imation has also been involved in other areas involving recording. Imation also has done some other recent acquisitions around dedupe (Nine Technologies).

For its part, Nexsan has extended their portfolio from SATA and SAS products, AutoMaid Intelligent Power Management (IPM) which gives benefits of variable power and performance without the penalties of first generation MAID type products. Read more about IPM and related themes here, here and here. Nexsan also supports NAS and iSCSI solutions in addition to their archive and content or object storage focused Assureon product they bought a few years ago.

This is a good acquisition for both companies as it gives Imation a new set of products to sell into their existing accounts and channels. It also can leverage Nexsan’s channel and solution selling skills giving them (Nexsan) a bigger brand and large parent for credibility (not that they did not have that in the past).

Here are is a link to a piece done by Dave Raffo that includes some comments and perspectives from me. To say that the synergy here is about archiving or selling SSD or storage would be too easy and miss a bigger potential. That potential is Imation has been in the business of selling consumable accessories for protecting and preserving data. Notice I said consumable accessories which in the past has meant manufacturing consumable media (e.g. Floppy disks or discs, CD, DVDs, magnetic tapes) as well as partnering around flash and HDDs.

In many environments from small to large to super-sized cloud and service providers, some types of storage systems including some of those that Nexsan sells can be considered a consumable media or medium taking over the role that tape, CDs or DVDs have been used in the pat. Instead of using tape or CDs or DVDs to protect the HDDs and SSDs based data, HDD based solutions are being used for disk-to-disk (D2D) protection (part of modernizing data protection). D2D is being done as appliances, or in conjunction with cloud and object storage system software stacks such as OpenStack swift, Basho Riak CS, CloudStack, Cleversafe, Ceph, Caringo and a list of others, in addition to appliances such as EMC ATMOS among others than can support 3rd party storage device as consumable mediums. Keep in mind that there is no such thing as a data or information recession, and people and data are living longer and getting larger, both for big data and little data.

The big if in this acquisition which IMHO is a fair price for both parties based on realistic valuations is if they can collective execute on it. This means that Imation and Nexsan need to leverage each other’s strengths, address any weakness, close gaps and expand into each other’s markets, channels and sell the entire portfolio as opposed to becoming singular focused on a particular area tool or technology. If Imation can execute on this and Nexsan leverages their new parent, the result should be moving from the roughly $85M USD sales to $100M+ then $125M then $150M and so forth over the next couple of years.

Even if Imation keeps maintains revenues or a slight increase, which would also be a good deal for them, granted the industry pundits may not agree, so let us see where this is in a few years. However if Imation can grow the Nexsan business, then it would become a very good deal. Thus, IMHO the price valuation for the deal has the risk built into, something like when NetApp bought the Engenio business unit from LSI back in 2011 for about $480M USD. At that time, Engenio was doing about $705M USD in revenue and seen by many industry pundits as being on the decline, thus a lower valuation. For its part, NetApp, has been executing maintaining the revenue of that business unit with some expansion, thus their execution so far is being rewarding for taking the risk.

Let us see if Imation can do the same thing.

Now, does that mean that Nexsan was the last of the independent storage vendors left?

Hardly, after all there is still Xiotech, excuse me, Xio as they changed their name as part of a repackaging, relaunch and downsizing. There is DotHill who supplies partners such HP, or Dothills former partner supplier InfoTrend. If you are an Apple fan then you might know about Promise, if not, you should. Lets not forget about Data Direct Networks (DDN) that is still independent and at around $200M (give or take several million) in revenue, are very much still around.

How about Xyratex, sure they make the enclosures and appliances that many others use in their solutions, however they also have a storage solutions business focused on scale out, clustered and grid NAS based on Lustre. There are some others that I am drawing a blank on now (if you read this and are one of them, chime in) in addition to all the new or current generation of startups (you can chime in as well to let people know who you are to be bought).

There is still consolidation taking place, both of smaller vendors by mid-sized vendors, mid-sized vendors by big vendors, big vendors by mega vendors, and startups by established.

Again congratulations to both Imation and Nexsan, let us see who or what is next on the 2013 mergers and acquisition list, as well as who will join the where are they now club.

Disclosure: Nexsan has been a StorageIO client in the past; however, Imation has not been a client, although they have bought me lunch before here in the Stillwater, MN area.

With Imation having their own brand name and identity, not to mention TDK and Memorex, now I have to wonder will Nexsan be real or Memorex or something else? ;)

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

January 7, 2013December 24, 2020

Cloud conversations: Gaining cloud confidence from insights into AWS outages (Part II)

StorageIO industry trends cloud, virtualization and big data

This is the second in a two-part industry trends and perspective looking at learning from cloud incidents, view part I here.

There is good information, insight and lessons to be learned from cloud outages and other incidents.

Sorry cynics no that does not mean an end to clouds, as they are here to stay. However when and where to use them, along with what best practices, how to be ready and configure for use are part of the discussion. This means that clouds may not be for everybody or all applications, or at least today. For those who are into clouds for the long haul (either all in or partially) including current skeptics, there are many lessons to be learned and leveraged.

In order to gain confidence in clouds, some questions that I routinely am asked include are clouds more or less reliable than what you are doing? Depends on what you are doing, and how you will be using the cloud services. If you are applying HA and other BC or resiliency best practices, you may be able to configure and isolate from the more common situations. On the other hand, if you are simply using the cloud services as a low-cost alternative selecting the lowest price and service class (SLAs and SLOs), you might get what you paid for. Thus, clouds are a shared responsibility, the service provider has things they need to do, and the user or person designing how the service will be used have some decisions making responsibilities.

Keep in mind that high availability (HA), resiliency, business continuance (BC) along with disaster recovery (DR) are the sum of several pieces. This includes people, best practices, processes including change management, good design eliminating points of failure and isolating or containing faults, along with how the components or technology used (e.g. hardware, software, networks, services, tools). Good technology used in goods ways can be part of a highly resilient flexible and scalable data infrastructure. Good technology used in the wrong ways may not leverage the solutions to their full potential.

While it is easy to focus on the physical technologies (servers, storage, networks, software, facilities), many of the cloud services incidents or outages have involved people, process and best practices so those need to be considered.

These incidents or outages bring awareness, a level set, that this is still early in the cloud evolution lifecycle and to move beyond seeing clouds as just a way to cut cost, and seeing the importance and value HA, resiliency, BC and DR. This means learning from mistakes, taking action to correct or fix errors, find and cut points of failure are part of a technology maturing or the use of it. These all tie into having services with service level agreements (SLAs) with service level objectives (SLOs) for availability, reliability, durability, accessibility, performance and security among others to protect against mayhem or other things that can and do happen.

Images licensed for use by StorageIO via
Atomazul / Shutterstock.com

The reason I mentioned earlier that AWS had another incident is that like their peers or competitors who have incidents in the past, AWS appears to be going through some growing, maturing, evolution related activities. During summer 2012 there was an AWS incident that affected Netflix (read more here: AWS and the Netflix Fix?). It should also be noted that there were earlier AWS outages where Netflix (read about Netflix architecture here) leveraged resiliency designs to try and prevent mayhem when others were impacted.

Is AWS a lightning rod for things to happen, a point of attraction for Mayhem and others?

Granted given their size, scope of services and how being used on a global basis AWS is blazing new territory and experiences, similar to what other information services delivery platforms did in the past. What I mean is that while taken for granted today, open systems Unix, Linux, Windows-based along with client-server, midrange or distributed systems, not to mention mainframe hardware, software, networks, processes, procedures, best practices all went through growing pains.

There are a couple of interesting threads going on over in various LinkedIn Groups based on some reporters stories including on speculation of what happened, followed with some good discussions of what actually happened and how to prevent recurrence of them in the future.

Over in the Cloud Computing, SaaS & Virtualization group forum, this thread is based on a Forbes article (Amazon AWS Takes Down Netflix on Christmas Eve) and involves conversations about SLAs, best practices, HA and related themes. Have a look at the story the thread is based on and some of the assertions being made, and ensuing discussions.

Also over at LinkedIn, in the Cloud Hosting & Service Providers group forum, this thread is based on a story titled Why Netflix’ Christmas Eve Crash Was Its Own Fault with a good discussion on clouds, HA, BC, DR, resiliency and related themes.

Over at the Virtualization Practice, there is a piece titled Is Amazon Ruining Public Cloud Computing? with comments from me and Adrian Cockcroft (@Adrianco) a Netflix Architect (you can read his blog here). You can also view some presentations about the Netflix architecture here.

What this all means

Saying you get what you pay for would be too easy and perhaps not applicable.

There are good services free, or low-cost, just like good free content and other things, however vice versa, just because something costs more, does not make it better.

Otoh, there are services that charge a premium however may have no better if not worse reliability, same with content for fee or perceived value that is no better than what you get free.

Additional related material

Some closing thoughts:

Clouds are real and can be used safely; however, they are a shared responsibility.
Only you can prevent cloud data loss, which means do your homework, be ready.
If something can go wrong, it probably will, particularly if humans are involved.
Prepare for the unexpected and clarify assumptions vs. realities of service capabilities.
Leverage fault isolation and containment to prevent rolling or spreading disasters.
Look at cloud services beyond lowest cost or for cost avoidance.
What is your organizations culture for learning from mistakes vs. fixing blame?
Ask yourself if you, your applications and organization are ready for clouds.
Ask your cloud providers if they are ready for you and your applications.
Identify what your cloud concerns are to decide what can be done about them.
Do a proof of concept to decide what types of clouds and services are best for you.

Do not be scared of clouds, however be ready, do your homework, learn from the mistakes, misfortune and errors of others. Establish and leverage known best practices while creating new ones. Look at the past for guidance to the future, however avoid clinging to, and bringing the baggage of the past to the future. Use new technologies, tools and techniques in new ways vs. using them in old ways.

Ok, nuff said.

Cheers gs

Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

Part II – Until the focus expands to data protection – What to do about it

Modernizing data protection and what to do about it

Whats your data protection strategy, business or technology focused?

Wrap up (for now)

Share this:

Until the focus expands to data protection, backup is staying alive!

How vendors are helping backup staying alive?

Taking a step back to move forward

Not all threat risks are headline news making events

Share this:

Part III – Until the focus expands to data protection – Taking action

Modernizing is more than simply swapping one technology for another

Expand from talking new technology to using new (and old) things in new ways

Call to action, stop talking about it, start walking the talk

Closing comments (for now)

Share this:

Blog post: Data Protection Diaries – My data protection needs and wants

Why and what am I Protecting?

What is my environment

Why do I need to protect my information and data infrastructure?

My wants – What I would like to have

My requirements – What I need (vs. want):

Data protection topics, trends, technologies and related themes

Wrap and summary (for now)

Share this:

Securing your information assets and data, what about your storage?

Protecting data and data protection

Additional data protection topics and links

Closing perspective, for now…

Share this:

Some Windows Server Storage I/O related commands

Finding a Windows physical disk, SSD or storage system device name

Remembering (or learning) Xcopy

Diskpart, don’t be scared, however be careful

Is SATA SSD TRIM Enabled?

Having issues with collecting CPU and performance statistics?

Sdelete and drive erase

Rufus, Seatools, Samsung Disk Magician and Cyberduck

Share this:

Good by 2013 and hello 2014 along with predictions past, present and future

What about 2013 server storage I/O networking, cloud, virtual and physical?

What about 2014?

What about beyond 2014?

Here’s my point

Share this:

Blog post: Small Medium Business (SMB) IT continues to gain respect, what about SOHO?

What about Small Office Home Office (SOHO) today?

What is the SOHO market or environment

Understanding the SOHO

Here is my point

Share this:

November 2013 Server and StorageIO Update Newsletter & AWS reinvent info

Industry trends: Amazon Web Services (AWS) re:Invent

Share this:

Cloud conversations: Has Nirvanix shutdown caused cloud confidence concerns?

Cloud customer concerns?

Nirvanix Storm Clouds

Clouds: Being prepared

Cloud Concerns and Confidence

Don’t be scared of clouds, be prepared

Share this:

Proof of life: Enterprise Hard Disk Drives (HDD’s) are enhanced

What was announced?

What this all means?

Wait, weren’t HDD’s supposed to be dead or dying?

What is up with this ISE stuff is that the same as what Xiotech (e.g. XIO) had?

Wait a minute, aren’t slower RPM’s supposed to be lower performance?

Other considerations, temperature and vibration

What type of drives should be used for cloud, virtual and traditional storage?

Disclosure

Share this:

Share this:

Share this:

Share this: