Server StorageIO Data Infrastructure Insights and Analysis
Data footprint reduciton including archiving (email, database, files), compression (on-line and device based), de-duplicaiton (aka single instance storage, commonality factoring, differencing, etc.).
|
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
Amazon Web Services (AWS) recently among other enhancements announced new Simple Storage Service (S3) cross-region replication of objects from a bucket (e.g. container) in one region to a bucket in another region. AWS also recently enhanced Elastic Block Storage (EBS) increasing maximum performance and size of Provisioned IOPS (SSD) and General Purpose (SSD) volumes. EBS enhancements included ability to store up to 16 TBytes of data in a single volume and do 20,000 input/output operations per second (IOPS). Read more about EBS and other recent AWS server, storage I/O and application enhancements here.
The challenge is being able to move data (e.g. objects) stored in AWS buckets in one region to another in a safe, secure, timely, automated, cost-effective way.
Even though AWS has a global name-space, buckets and their objects (e.g. files, data, videos, images, bit and byte streams) are stored in a specific region designated by the customer or user (AWS S3, EBS, EC2, Glacier, Regions and Availability Zone primer can be found here).
The following diagram shows the challenge and how to copy or replicate objects in an S3 bucket in one region to a destination bucket in a different region. While objects can be copied or replicated without S3 cross-region replication, that involves essentially reading your objects pulling that data out via the internet and then writing to another place. The catch is that this can add extra costs, take time, consume network bandwidth and need extra tools (Cloudberry, Cyberduck, S3fuse, S3motion, S3browser, S3 tools (not AWS) and a long list of others).
Highlights of AWS S3 Cross-region replication include:
To activate, you simply enable versioning on a bucket, enable cross-region replication, indicate source bucket (or prefix of objects in bucket), specify destination region and target bucket name (or create one), then create or select an IAM (Identify Access Management) role and objects should be replicated.
There are several regions around the world and up until today AWS customers could copy, sync or replicate S3 bucket contents between AWS regions manually (or via automation) using various tools such as Cloudberry, Cyberduck, S3browser and S3motion to name just a few as well as via various gateways and other technologies. Some of those tools and technologies are open-source or free, some are freemium and some are premium for a few that also vary by interface (some with GUI, others with CLI or APIs) including ability to mount an S3 bucket as a local network drive and use tools to sync or copy.
However a catch with the above mentioned tools (among others) and approaches is that to replicate your data (e.g. objects in a bucket) can involve other AWS S3 fees. For example reading data (e.g. a GET which has a fee) from one AWS region and then copying out to the internet has fees. Likewise when copying data into another AWS S3 region (e.g. a PUT which are free) there is also the cost of storage at the destination.
For my first hands on (first look) experience with AWS cross-region replication today I enabled a bucket in the US Standard region (e.g. Northern Virginia) and created a new target destination bucket in the EU Ireland. Setup and configuration was very quick, literally just a few minutes with most of the time spent reading the text on the new AWS S3 dashboard properties configuration displays.
I selected an existing test bucket to replicate and noticed that nothing had replicated over to the other bucket until I realized that new objects would be replicated. Once some new objects were added to the source bucket within a matter of moments (e.g. few minutes) they appeared across the pond in my EU Ireland bucket. When I deleted those replicated objects from my EU Ireland bucket and switched back to my view of the source bucket in the US, those new objects were already deleted from the source. Yes, just like regular mirroring or replication, pay attention to how you have things configured (e.g. synchronized vs. contribute vs. echo of changes etc.).
While I was not able to do a solid quantifiable performance test, simply based on some quick copies and my network speed moving via S3 cross-region replication was faster than using something like s3motion with my server in the middle.
It also appears from some initial testing today that a benefit of AWS S3 cross-region replication (besides being bundled and part of AWS) is that some fees to pull data out of AWS and transfer out via the internet can be avoided.
Here are some links to learn more about AWS S3 and related topics
For those who are looking for a way to streamline replicating data (e.g. objects) from an AWS bucket in one region with a bucket in a different region you now have a new option. There are potential cost savings if that is your goal along with performance benefits in addition to using what ever might be working in your environment. Replicating objects provides a way of expanding your business continuance (BC), business resiliency (BR) and disaster recovery (DR) involving S3 across regions as well as a means for content cache or distribution among other possible uses.
Overall, I like this ability for moving S3 objects within AWS, however I will continue to use other tools such as S3motion and s3sfs for moving data in and out of AWS as well as among other public cloud serves and local resources.
Ok, nuff said, for now..
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
This is part of an ongoing data protection diaries series of post about, well, cloud and data protection and what I’m doing pertaining to World Backup Day 2015 along with related topics.
In case you forgot or did not know, World Backup Day is March 31 2015 (@worldbackupday) so now is a good time to be ready. The only challenge that I have with the World Backup Day (view their site here) that has gone on for a few years know is that it is a good way to call out the importance of backing up or protecting data. However its time to also put more emphasis and focus on being able to make sure those backups or protection copies actually work.
By this I mean doing more than making sure that your data can be read from tape, disk, SSD or cloud service actually going a step further and verifying that restored data can actually be used (read, written, etc).
The problem, issue and challenges are simple, are your applications, systems and data protected as well as can you use those protection copies (e.g. backups, snapshots, replicas or archives) when as well as were needed?
The opportunity is simple, avoiding downtime or impact to your business or organization by being proactive.
The following is my preparation checklist for World Backup Data 2015 (e.g. March 31 2015) which includes what I need or want to protect, as well as some other things to be done including testing, verification, address (remediate or fix) known issues while identifying other areas for future enhancements. Thus perhaps like yours, data protection for my environment which includes physical, virtual along with cloud spanning servers to mobile devices is constantly evolving.
My data protection preparation, checklist and to do list
While I already have a strategy, plan and solution that encompasses different tools, technologies and techniques, they are also evolving. Part of the evolving is to improve while also exploring options to use new and old things in new ways as well as eat my down dog food or walk the talk vs. talk the talk. The following figure provides a representation of my environment that spans physical, virtual and clouds (more than one) and how different applications along with systems are protected against various threats or risks. Key is that not all applications and data are the same thus enabling them to be protected in different ways as well as over various intervals. Needless to say there is more to how, when, where and with what different applications and systems are protected in my environment than show, perhaps more on that in the future.
Some of what my data protection involves for Server StorageIO
What I’m doing is going through my checklist to verify and confirm the various items on the checklist as well as find areas for improvement which is actually an ongoing process.
Do I find things that need to be corrected?
Yup, in fact found something that while it was not a problem, identified a way to improve on a process that will once fully implemented enabler more flexibility both if a restoration is needed, as well as for general everyday use not to mention remove some complexity and cost.
Speaking of lessons learned, check this out that ties into why you want 4 3 2 1 based data protection strategies.
Here are some extra links to have a look at:
Data Protection Diaries
Cloud conversations: If focused on cost you might miss other cloud storage benefits
5 Tips for Factoring Software into Disaster Recovery Plans
Remote office backup, archiving and disaster recovery for networking pros
Cloud conversations: Gaining cloud confidence from insights into AWS outages (Part II)
Given outages, are you concerned with the security of the cloud?
Data Archiving: Life Beyond Compliance
My copies were corrupted: The 3-2-1 rule
Take a 4-3-2-1 approach to backing up data
Cloud and Virtual Data Storage Networks – Chapter 8 (CRC/Taylor and Francis)
Be prepared, be proactive when it comes to data protection and business resiliency vs. simply relying reacting and recovering hoping that all will be ok (or works).
Take a few minutes (or longer) and test your data protection including backup to make sure that you can:
a) Verify that in fact they are working protecting applications and data in the way expected
b) Restore data to an alternate place (verify functionality as well as prevent a problem)
c) Actually use the data meaning it is decrypted, inflated (un-compressed, un-de duped) and security certificates along with ownership properties properly applied
d) Look at different versions or generations of protection copies if you need to go back further in time
e) Identify area of improvement or find and isolate problem issues in advance vs. finding out after the fact
Time to get back to work checking and verifying things as well as attending to some other items.
Ok, nuff said, for now…
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
It’s that time of the year again for award season:
Eric has run for several years now an annual top VMware, Virtualization, Storage and related blogs voting now taking place until March 16th 2015 (click on the image below). You will find a nice mix of new school, old school and a few current or future school theme blogs represented with some being more VMware specific. However there are also many blogs at the vpad site that have a cloud, virtual, server, storage, networking, software defined, development and other related themes.
Click on the above image to cast your vote for favorite:
My StorageIOblog.com has been on the vLaunchPad site for several years now as well as having syndicated content that also appears via some of the other venues listed there.
In addition to my StorageIOblog and podcast, you will also find many of my fellow VMware vExperts among others at the vLaunchpad site so check them out as well.
This is a people’s choice process (yes it is a popularity process of sorts as well) however also a way of rewarding or thanking those who take time to create and share content with you and others. If you take time to read various blogs, listen to podcasts as well as consume other content, please take a few moments and cast your vote here (thank you in advance) which I hope includes StorageIOblog.com as part of the top ten, as well as being nominated in the Storage, Podcast and Independent blogger categories.
Ok, nuff said, for now…
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
|
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
Green IT is about enabling efficient, effective and productive information services delivery. There is a growing green gap between green hype messaging or green washing and IT pain point issues including limits on availability or rising costs of power, cooling, floor-space as well as e-waste and environmental health and safety (PCFE). To close the gap will involve addressing green messaging and rhetoric closer to where IT organizations pain points are and where budget dollars exists that can address PCFE and other green related issues as a by-product. The green gap will also be narrowed as awareness of broader green related topics coincide with IT data center pain points, in other words, alignment of messaging with IT issues that have or will have budget dollars allocated towards them to sustain business and economic growth via IT resource usage efficiency. Read more here.
There are many aspects to "Green" Information Technology including servers, storage, networks and associated management tools and techniques. The reasons and focus of "Green IT" including "Green Data Storage ", "Green Computing" and related focus areas are varied to discuss diverse needs, issues and requirements including among others:
While much of the IT industry focuses on CO2 emissions footprints, data management software and electrical power consumption, cooling and ventilation of IT data centers is an area of focus associated with "Green IT" as well as a means to discuss more effective use of electrical energy that can yield rapid results for many environments. Large tier-1 vendors including HP and IBM among others who have an IT and data center wide focus have services designed to do quick assessments as well as detailed analysis and re-organization of IT data center physical facilities to improve air flow and power consumption for more effective cooling of IT technologies including servers, storage, networks and other equipment.
Similar to your own residence, basic steps to improve your cooling effectiveness can lead to use of less energy to cut your budget impact, or, enable you to do more with what you already have with your cooling capacity to support growth, acquisitions and or consolidation initiatives. Vendors are also looking at means and alternatives for cooling IT equipment ranging from computer assisted computational fluid dynamics (CFD) software analysis of data center cooling and ventilation to refrigerated cooling racks some leveraging water or inert liquid cooling.
Various metrics exists and others are evolving for measuring, estimating, reporting, analyzing and discussing IT Data Center infrastructure resource topics including servers, storage, networks, facilities and associated software management tools from a power, cooling and green environmental standpoint. The importance of metrics is to focus on the larger impact of a piece of IT equipment that includes its cost and energy consumption that factors in cooling and other hosting or site environmental costs. Naturally energy costs and CO2 (carbon offsets) will vary by geography and region along with type of electrical power being used (Coal, Natural Gas, Nuclear, Wind, Thermo, Solar, etc) and other factors that should be kept in perspective as part of the big picture.
Consequently your view and needs or interests around "Green" IT may be from an electrical power conservation perspective to maximize your power consumption or to adapt to a given power footprint or ceiling. Your focus around "Green" Data Centers and Green Storage may be from a carbon savings standpoint or proper disposition of old and retired IT equipment or from a data center cooling standpoint. Another area of focus may be that you are looking to cut your data footprint to align with your power, cooling and green footprint while enhancing application and data service delivery to your customers.
The following are useful links to related efficient, effective, productive, flexible, scalable and resilient IT data center along with server storage I/O networking hardware and software that supports cloud and virtual green data centers.
Various IT industry vendor and service provider links
Green and Virtual Data Center: Productive Economical Efficient Effective Flexible
Green and Virtual Data Center links
Are large storage arrays dead at the hands of SSD?
Closing the Green Gap
Energy efficient technology sales depend on the pitch
The result of a green and virtual data center is that of a flexible, agile, resilient, scalable information factory that is also economical, productive, efficient, productive as well as sustainable.
Ok, nuff said (for now)
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
A Green and Virtual IT Data Center (e.g. an information factory) means an environment comprising:
Green and Virtual Data Center Resources
| ||
Productive Efficient Effective Economical Flexible Agile and SustainableGreen hype and green washing may be on the endangered species list and going away, however, green IT for servers, storage, networks, facilities as well as related software and management techniques that address energy efficiency including power and cooling along with e-waste, environmental health and safety related issues are topics that wont be going away anytime soon. There is a growing green gap between green hype messaging or green washing and IT pain point issues including limits on availability or rising costs of power, cooling, floor-space as well as e-waste and environmental health and safety (PCFE). To close the gap will involve addressing green messaging and rhetoric closer to where IT organizations pain points are and where budget dollars exists that can address PCFE and other green related issues as a by-product. The green gap will also be narrowed as awareness of broader green related topics coincide with IT data center pain points, in other words, alignment of messaging with IT issues that have or will have budget dollars allocated towards them to sustain business and economic growth via IT resource usage efficiency. Read more here. |
The following are useful links to related efficient, effective, productive, flexible, scalable and resilient IT data center along with server storage I/O networking hardware and software that supports cloud and virtual green data centers.
Various IT industry vendor and service provider links
Green and Virtual Data Center Primer
Green and Virtual Data Center links
Are large storage arrays dead at the hands of SSD?
Closing the Green Gap
Energy efficient technology sales depend on the pitch
EPA Energy Star for Data Center Storage Update
EPA Energy Star for data center storage draft 3 specification
Green IT Confusion Continues, Opportunities Missed!
Green IT deferral blamed on economic recession might be result of green gap
How much SSD do you need vs. want?
How to reduce your Data Footprint impact (Podcast)
Industry trend: People plus data are aging and living longer
In the data center or information factory, not everything is the same
More storage and IO metrics that matter
Optimizing storage capacity and performance to reduce your data footprint
Performance metrics: Evaluating your data storage efficiency
PUE, Are you Managing Power, Energy or Productivity?
Saving Money with Green Data Storage Technology
Saving Money with Green IT: Time To Invest In Information Factories
Shifting from energy avoidance to energy efficiency
SNIA Green Storage Knowledge Center
Speaking of speeding up business with SSD storage
SSD and Green IT moving beyond green washing
Storage Efficiency and Optimization: The Other Green
Supporting IT growth demand during economic uncertain times
The Green and Virtual Data Center Book (CRC Press, Intel Recommended Reading)
The new Green IT: Efficient, Effective, Smart and Productive
The other Green Storage: Efficiency and Optimization
What is the best kind of IO? The one you do not have to do
Watch for more links and resources to be added soon.
The result of a green and virtual data center is that of a flexible, agile, resilient, scalable information factory that is also economical, productive, efficient, productive as well as sustainable.
Ok, nuff said (for now)
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
Updated 10/25/2017
Green hype and green washing may be on the endangered species list and going away, however, green IT for servers, storage, networks, facilities as well as related software and management techniques that address energy efficiency including power and cooling along with e-waste, environmental health and safety related issues are topics that wont be going away anytime soon.
There is a growing green gap between green hype messaging or green washing and IT pain point issues including limits on availability or rising costs of power, cooling, floor-space as well as e-waste and environmental health and safety (PCFE).
To close the gap will involve addressing green messaging and rhetoric closer to where IT organizations pain points are and where budget dollars exists that can address PCFE and other green related issues as a by-product. The green gap will also be narrowed as awareness of broader green related topics coincide with IT data center pain points, in other words, alignment of messaging with IT issues that have or will have budget dollars allocated towards them to sustain business and economic growth via IT resource usage efficiency. Read more here.
The following are useful links to related efficient, effective, productive, flexible, scalable and resilient IT data center along with server storage I/O networking hardware and software that supports cloud and virtual green data centers.
Various IT industry vendors and other links
Via StorageIOblog – Happy Earth Day 2016 Eliminating Digital and Data e-Waste
Green and Virtual Data Center Primer
Green and Virtual Data Center: Productive Economical Efficient Effective Flexible
Are large storage arrays dead at the hands of SSD?
Closing the Green Gap
Energy efficient technology sales depend on the pitch
EPA Energy Star for Data Center Storage Update
EPA Energy Star for data center storage draft 3 specification
Green IT Confusion Continues, Opportunities Missed!
Green IT deferral blamed on economic recession might be result of green gap
How much SSD do you need vs. want?
How to reduce your Data Footprint impact (Podcast)
Industry trend: People plus data are aging and living longer
In the data center or information factory, not everything is the same
More storage and IO metrics that matter
Optimizing storage capacity and performance to reduce your data footprint
Performance metrics: Evaluating your data storage efficiency
PUE, Are you Managing Power, Energy or Productivity?
Saving Money with Green Data Storage Technology
Saving Money with Green IT: Time To Invest In Information Factories
Shifting from energy avoidance to energy efficiency
SNIA Green Storage Knowledge Center
Speaking of speeding up business with SSD storage
SSD and Green IT moving beyond green washing
Storage Efficiency and Optimization: The Other Green
Supporting IT growth demand during economic uncertain times
The Green and Virtual Data Center Book (CRC Press, Intel Recommended Reading)
The new Green IT: Efficient, Effective, Smart and Productive
The other Green Storage: Efficiency and Optimization
What is the best kind of IO? The one you do not have to do
 | Click here to learn about "The Green and Virtual Data Center" book (CRC Press) for enabling efficient , productive IT data centers. This book covers cloud, virtualization, servers, storage, networks, software, facilities and associated management topics, technologies and techniques including metrics that matter. This book by industry veteran IT advisor and author Greg Schulz is the definitive guide for enabling economic efficiency and productive next generation data center strategies. Read more here and order your copyhere. Also check out Cloud and Virtual Data Storage Networking (CRC Press) a new book by Greg Schulz. |
White papers, analyst reports and perspectivesBusiness benefits of data footprint reduction (archiving, compression, de-dupe) Data center I/O and performance issues – Server I/O and storage capacity gap Analysis of EPA Report to Congress (Law 109-431) The Many Faces of MAID Storage Technology Achieving Energy Efficiency with FLASH based SSD MAID 2.0: Energy Savings without Performance Compromises Articles, Tips, Blogs, Webcasts and Podcasts AP – SNIA Green Emerald Program and measurements AP – Southern California heat wave strains electrical system Ars Technica – EPA: Power usage in data centers could double by 2011 Ars Technica – Meet the climate savers: Major tech firms launch war on energy-inefficient PCs – Article Askageek.com – Buying an environmental friendly laptop – November 2008 Baseline – Examining Energy Consumption in the Data Center Baseline – Burts Bees: What IT Means When You Go Green Bizcovering – Green architecture for the masses Broadstuff – Are Green 2.0 and Enterprise 2.0 Incompatible? Business Week – CEO Guide to Technology Business Week – Computers’ elusive eco factor Business Week – Clean Energy – Its Getting Affordable Byte & Switch – Keeping it Green This Summer – Don’t be "Green washed" Byte & Switch – IBM Sees Green in Energy Certificates Byte & Switch – Users Search for power solutions Byte & Switch – DoE issues Green Storage Warning CBR – The Green Light for Green IT CBR – Big boxes make greener data centers CFO – Power Scourge Channel Insider – A 12 Step Program to Dispose of IT Equipment China.org.cn – China publishes Energy paper CIO – Green Storage Means Money Saved on Power CIO – Data center designers share secrets for going green CIO – Best Place to Build a Data Center in North America CIO Insight – Clever Marketing or the Real Thing? Cleantechnica – Cooling Data Centers Could Prevent Massive Electrical Waste – June 2008 Climatebiz – Carbon Calculators Yield Spectrum of Results: Study CNET News – Linux coders tackle power efficiency CNET News – Research: Old data centers can be nearly as ‘green’ as new ones CNET News – Congress, Greenpeace move on e-wast CNN Money – A Green Collar Recession CNN Money – IBM creates alliance with industry leaders supporting new data center standards Communication News – Utility bills key to greener IT Computerweekly – Business case for green storage Computerweekly – Optimising data centre operations Computerweekly – Green still good for IT, if it saves money Computerweekly – Meeting the Demands for storage Computerworld – Wells Fargo Free Data Center Cooling System Computerworld – Seven ways to get green and save money Computerworld – Build your data center here: The most energy-efficient locations Computerworld – EPA: U.S. needs more power plants to support data centers Computerworld – GreenIT: A marketing ploy or new technology? Computerworld – Gartner Criticizes Green Grid Computerworld – IT Skills no longer sufficient for data center execs. Computerworld – Meet MAID 2.0 and Intelligent Power Management Computerworld – Feds to offer energy ratings on servers and storage Computerworld – Greenpeace still hunting for truly green electronics Computerworld – How to benchmark data center energy costs ComputerworldUK – Datacenters at risk from poor governance ComputerworldUK – Top IT Leaders Back Green Survey ComputerworldMH – Lean and Green CTR – Strategies for enhancing energy efficiency CTR – Economies of Scale – Green Data Warehouse Appliances Datacenterknowledge – Microsoft to build Illinois datacenter Data Center Strategies – Storage The Next Hot Topic Earthtimes – Fujitsu installs hydrogen fuel cell power eChannelline – IBM Goes Green(er) Ecoearth.info – California Moves To Speed Solar, Wind Power Grid Connections Ecogeek – Solar power company figures they can power 90% of America Economist – Cool IT Electronic Design – How many watts in that Gigabyte eMazzanti – Desktop virtualization movement creeping into customer sites ens-Newswire – Western Governors Ask Obama for National Green Energy Plan Environmental Leader – Best Place to Build an Energy Efficient Data Center Environmental Leader – New Guide Helps Advertisers Avoid Greenwash Complaints Enterprise Storage Forum – Power Struggles Take Center Stage at SNW Enterprise Storage Forum – Pace Yourself for Storage Power & Cooling Needs Enterprise Storage Forum – Storage Power and Cooling Issues Heat Up – StorageIO Article Enterprise Storage Forum – Score Savings With A Storage Power Play Enterprise Storage Forum – I/O, I/O, Its off to Virtual Work I Go Enterprise Storage Forum – Not Just a Flash in the Pan – Various SSD options Enterprise Storage Forum – Closing the Green Gap – Article August 2008 EPA Report to Congress and Public Law 109-431 – Reports & links eWeek – Saving Green by being Green eWeek – ‘No Cooling Necessary’ Data Centers Coming? eWeek – How the ‘Down’ Macroeconomy Will Impact the Data Storage Sector ExpressComputer – In defense of Green IT ExpressComputer – What data center crisis Forbes – How to Build a Quick Charging Battery GCN – Sun launches eco data center GreenerComputing – New Code of Conduct to Establish Best Practices in Green Data Centers GreenerComputing – Silicon valley’s green detente GreenerComputing – Majority of companies plan to green their data centers GreenerComputing – Citigroup to spend $232M on Green Data Center GreenerComputing – Chicago and Quincy, WA Top Green Data Center Locations GreenerComputing – Using airside economizers to chill data center cooling bills GreenerComputing – Making the most of asset disposal GreenerComputing – Greenpeace vendor rankings GreenerComputing – Four Steps to Improving Data Center Efficiency without Capital Expenditures GreenerComputing – Enabling a Green and Virtual Data Center Green-PC – Strategic Steps Down the Green Path Greeniewatch – BBC news chiefs attack plans for climate change campaign Greeniewatch – Warmest year predictions and data that has not yet been measured GoverenmentExecutive – Public Private Sectors Differ on "Green" Efforts HPC Wire – How hot is your code Industry Standard – Why green data centers mean partner opportunities InformationWeek – It could be 15 years before we know what is really green InformationWeek – Beyond Server Consolidaiton InformationWeek – Green IT Beyond Virtualization: The Case For Consolidation InfoWorld – Sun celebrates green datacenter innovations InfoWorld – Tech’s own datacenters are their green showrooms InfoWorld – 2007: The Year in Green InfoWorld – Green Grid Announces Tech Forum in Feb 2008 InfoWorld – SPEC seeds future green-server benchmarks InfoWorld – Climate Savers green catalog proves un-ripe InfoWorld – Forester: Eco-minded activity up among IT pros InfoWorld – Green ventures in Silicon Valley, Mass reaped most VC cash in ’07 InfoWorld – Congress misses chance to see green-energy growth InfoWorld – Unisys pushes green envelope with datacenter expansion InfoWorld – No easy green strategy for storage Internet News – Storage Technologies for a Slowing Economy Internet News – Economy will Force IT to Transform ITManagement – Green Computing, Green Revenue itnews – Data centre chiefs dismiss green hype itnews – Australian Green IT regulations could arrive this year IT Pro – SNIA Green storage metrics released ITtoolbox – MAID discussion Linux Power – Saving power with Linux on Intel platforms MSNBC – Microsoft to build data center in Ireland National Post – Green technology at the L.A. Auto Show Network World – Turning the datacenter green Network World – Color Interop Green Network World – Green not helpful word for setting environmental policies NewScientistEnvironment – Computer servers as bad for climate as SUVs Newser – Texas commission approves nation’s largest wind power project New Yorker – Big Foot: In measuring carbon emissions, it’s easy to confuse morality and science NY Times – What the Green Bubble Will Leave Behind PRNewswire – Al Gore and Cisco CEO John Chambers to debate climate change Processor – More than just monitoring Processor – The new data center: What’s hot in Data Center physical infrastructure: Processor – Liquid Cooling in the Data Center Processor – Curbing IT Power Usage Processor – Services To The Rescue – Services Available For Today’s Data Centers Processor – Green Initiatives: Hire A Consultant? Processor – Energy-Saving Initiatives Processor – The EPA’s Low Carbon Campaig Processor – Data Center Power Planning SAN Jose Mercury – Making Data Centers Green SDA-Asia – Green IT still a priority despite Credit Crunch SearchCIO – EPA report gives data centers little guidance SearchCIO – Green IT Strategies Could Lead to hefty ROIs SearchCIO – Green IT In the Data Center: Plenty of Talk, not much Walk SearchCIO – Green IT Overpitched by Vendors, CIOs beware SearchDataCenter – Study ranks cheapest places to build a data center SearchDataCenter – Green technology still ranks low for data center planners SearchDataCenter – Green Data Center: Energy Effiecnty Computing in the 21st Century SearchDataCenter – Green Data Center Advice: Is LEED Feasible SearchDataCenter – Green Data Centers Tackle LEED Certification SearchDataCenter – PG&E invests in data center effieicny SearchDataCenter – A solar powered datacenter SearchSMBStorage – Improve your storage energy efficiency SearchSMBStorage – SMB capacity planning: Focusing on energy conservation SearchSMBStorage – Data footprint reduction for SMBs SearchSMBStorage – MAID & other energy-saving storage technologies for SMBs SearchStorage – How to increase your storage energy efficiency SearchStorage – Is storage now top energy hog in the data center SearchStorage – Storage eZine: Turning Storage Green SearchStorage – The Green Storage Gap SearchStorageChannel – Green Data Storage Projects Silicon.com – The greening of IT: Cooling costs SNIA – SNIA Green Storage Overview SNIA – Green Storage SNW – Beyond Green-wash SNW Spring 2008 Beyond Green-wash State.org – Why Texas Has Its Own Power Grid StorageDecisions – Different Shades of Green Storage Magazine – Storage still lacks energy metrics StorageIOblog – Posts pertaining to Green, power, cooling, floor-space, EHS (PCFE) Storage Search – Various postings, news and topics pertaining to Green IT Technology Times – Revealed: the environmental impact of Google searches TechTarget – Data center power efficiency TechTarget – Tip for determining power consumption Techworld – Inside a green data center Techworld – Box reduction – Low hanging green datacenter fruit Techworld – Datacentere used to heat swimming pool Theinquirer – Spansion and Virident flash server farms Theinquirer – Storage firms worry about energy efficiency How green is the valley TheRegister – Data Centre Efficiency, the good, the bad and the way to hot TheRegister – Server makers snub whalesong for serious windmill abuse TheRegister – Green data center threat level: Not Green The Standard – Growing cynicism around going Green ThoughtPut – Energy Central Thoughtput – Power, Cooling, Green Storage and related industry trends Wallstreet Journal – Utilities Amp Up Push To Slash Energy Use Wallstreet Journal – The IT in Green Investing Wallstreet Journal – Tech’s Energy Consumption on the Rise Washingtonpost – Texas approves major new wind power project WhatPC – Green IT: It doesnt have to cost the earth WHIRnews – SingTel building green data center Wind-watch.org – Loss of wind causes Texas power grid emergency WyomingNews – Overcoming Greens Stereotype Yahoo – Washington Senate Unviel Green Job Plan ZDnet – Will supercomputer speeds hit a plateau? Are data centers causing climate change News and Press Releases Business Wire – The Green and Virtual Data Center Enterprise Storage Forum – Intel and HGST (Hitachi) partner on FLASH SSD PCworld – Intel and HP describe Green Strategy DoE – To Invest Approximately $1.3 Billion to Commercialize CCS Technology Yahoo – Shell Opens Los Angeles’ First Combined Hydrogen and Gasoline Station DuPont – DuPont Projects Save Enough Energy to Power 25,000 Homes Gartner – Users Are Becoming Increasingly Confused About the Issues and Solutions Surrounding Green IT Websites and Tools Various power, cooling, emmisions and device configuration tools and calculators Solar Action Alliance web site SNIA Emerald program Carbon Disclosure Project The Chicago Climate Exchange Climate Savers Data Center Decisions Electronic Industries Alliance (EIA) EMC – Digital Life Calculator Energy Star Energy Star Data Center Initiatives Greenpeace – Technology ranking website also here GlobalActionPlan KyotoPlanet LBNL High Tech Data centers Millicomputing RoHS & WEE News Storage Performance Council (SPC) SNIA Green Technical Working Group SPEC Transaction Processing Council (TPC) The Green Grid The Raised Floor Terra Pass Carbon Offset Credits – Website with CO2 calculators Energy Information Administration – EIA (US and International Electrical Information) U.S. Department of Energy and related information U.S. DOE Energy Efficient Industrial Programs U.S. EPA server and storage energy topics Zerofootprint – Various "Green" and environmental related links and calculators Vendor Centric and Marketing Website Links and toolsVendors and organizations have different types of calculators some with focus on power, cooling, floor space, carbon offsets or emissions, ROI, TCO and other IT data center infrastructure resource management. Following is an evolving list and by no means definitive even for a particular vendors as different manufactures may have multiple different calculators for different product lines or areas of focus. Brocade – Green website Cisco – Green and Environmental websites here, here and here Dell – Green website EMC – EMC Energy, Power and Cooling Related Website HDS – How to be green – HDS Positioning White Paper HP – HP Green Website IBM – Green Data Center – IBM Positioning White Paper IBM – Green Data Center for Education – IBM Positioning White Paper Intel – What is an Efficient Data Center and how do I measure it? LSI – Green site and white paper NetApp – Press Release and related information Sun – Various articles and links Symantec – Global 2000 Struggle to Adopt "Green" Data Centers – Announcement of Survey results ACTON Adinfa APC Australian Conservation Foundation Avocent BBC Brocade Carbon Credit Calculator UK Carbon Footprint Site Carbon Planet Carbonify CarbonZero Cassatt CO2 Stats Site Copan Dell DirectGov UK Acton Diesel Service & Supply Power Calculator & Converter Eaton Powerware Ecobusinesslinks Ecoscale EMC Power Calculator EMC Web Power Calculator EMC Digital Life Calculator EPA Power Profiler EPA Related Tools EPEAT Google UK Green Footprint Green Grid Calculator HP and more here HVAC Calculator IBM Logicalis Kohler Power (Business and Residential) Micron MSN Carbon Footprint Calculator National Wildlife Foundation NEF UK NetApp Rackwise Platespin Safecom Sterling Planet Sun and more here and here and here Tandberg TechRepublic TerraPass Carbon Offset Credits Thomas Kreen AG Toronto Hydro Calculator 80 Plus Calculator VMware 42u Green Grid PUE DCiE calculator 42u energy calculator Green and Virtual Tools
Green and Virtual Metrics
Disclaimer and notesDisclaimer and note: URL’s submitted for inclusion on this site will be reviewed for consideration and to be in generally accepted good taste in regards to the theme of this site. Best effort has been made to validate and verify the URLs that appear on this page and web site however they are subject to change. The author and/or maintainer’s) of this page and web site make no endorsement to and assume no responsibility for the URLs and their content that are listed on this page. |
The result of a green and virtual data center is that of a flexible, agile, resilient, scalable information factory that is also economical, productive, efficient, productive as well as sustainable.
Ok, nuff said (for now)
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
This is part one of a two-part series about Amazon Web Services (AWS) re:Invent 2014 and other recent cloud updates, read part two here.
A few weeks ago I attended Amazon Web Service (AWS) re:Invent 2014 in Las Vegas for a few days. For those of you who have not yet attended this event, I recommend adding it to your agenda. If you have interest in compute servers, networking, storage, development tools or management of cloud (public, private, hybrid), virtualization and related topic themes, you should check out AWS re:invent.
AWS made several announcements at re:invent including many around development tools, compute and data storage services. One of those to keep an eye on is cloud based Aurora relational database service that complement existing RDS tools. Aurora is positioned as an alternative to traditional SQL based transactional databases commonly found in enterprise environments (e.g. SQL Server among others).
Using the AWS Management Portal for vCenter adds a plug-in within your VMware vCenter to manage your AWS infrastructure. The vCenter for AWS plug-in includes support for AWS EC2 and Virtual Machine (VM) import to migrate your VMware VMs to AWS EC2, create VPC (Virtual Private Clouds) along with subnet’s. There is no cost for the plug-in, you simply pay for the underlying AWS resources consumed (e.g. EC2, EBS, S3). Learn more about AWS Management Portal for vCenter here, and download the OVA plug-in for vCenter here.
AWS Andy Jassy (Image via AWS)
November 12, 2014 (Day 1) Keynote (highlight video, full keynote). This is the session where AWS SVP Andy Jassy made several announcements including Aurora relational database that complements existing RDS (Relational Data Services). In addition to Andy, the key-note sessions also included various special guests ranging from AWS customers, partners and internal people in support of the various initiatives and announcements.
Amazon.com CTO Werner Vogels (Image via AWS)
November 13, 2014 (Day 2) Keynote (highlight video, full keynote). In this session, Amazon.com CTO Werner Vogels appears making announcements about the new Container and Lambda services.
Announcements and enhancements made by AWS during re:Invent include:
Hardware security module (HSM) based key managed service for creating and control of encryption keys to protect security of digital assets and their keys. Integration with AWS EBS and others services including S3 and Redshift along with CloudTrail logs for regulatory, compliance and management. Learn more about AWS KMS here
For those who are not familiar, AWS has a suite of database related services including SQL and no SQL based, simple to transactional to Petabyte (PB) scale data warehouses for big data and analytics. AWS offers the Relational Database Service (RDS) which is a suite of different database types, instances and services. RDS instance and types include SimpleDB, MySQL, Postgress, Oracle, SQL Server and the new AWS Aurora offering (read more below). Other little data database and big data repository related offerings include DynamoDB (a non-SQL database), ElasticCache (in memory cache repository) and Redshift (large-scale data warehouse and big data repository).
In addition to database services offered by AWS, you can also combine various AWS resources including EC2 compute, EBS and other storage offerings to create your own solution. For example there are various Amazon Machine Images (AMI’s) or pre-built operating systems and database tools available with EC2 as well as via the AWS Marketplace , such as MongoDB and Couchbase among others. For those not familiar with MongoDB, Couchbase, Cassandra, Riak along with other non SQL or alternative databases and key value repositories, check out Seven Databases in Seven Weeks in my book review of it here.
Seven Databases in Seven Weeks and NoSQL movement available from Amazon.com
Aurora is a new relational database offering part of the AWS RDS suite of services. Positioned as an alternative to commercial high-end database, Aurora is a cost-effective database engine compatible with MySQL. AWS is claiming 5x better performance than standard MySQL with Aurora while being resilient and durable. Learn more about Aurora which will be available in early 2015 and its current preview here.
AWS will be adding a new C4 instance as a next generation of EC2 compute instance based on Intel Xeon E5-2666 v3 (Haswell) processors. The Intel Xeon E5-2666 v3 processors run at a clock speed of 2.9 GHz providing the highest level of EC2 performance. AWS is targeting traditional High Performance Computing (HPC) along with other compute intensive workloads including analytics, gaming, and transcoding among others. Learn more AWS EC2 instances here, and view this Server and StorageIO EC2, EBS and associated AWS primer here.
Containers such as those via Docker have become popular to support developers rapidly build as well as deploy scalable applications. AWS has added a new feature called EC2 Container Service that supports Docker using simple API’s. In addition to supporting Docker, EC2 Container Service is a high performance scalable container management service for distributed applications deployed on a cluster of EC2 instances. Similar to other EC2 services, EC2 Container Service leverages security groups, EBS volumes and Identity Access Management (IAM) roles along with scheduling placement of containers to meet your needs. Note that AWS is not alone in adding container and docker support with Microsoft Azure also having recently made some announcements, learn more about Azure and Docker here. Learn more about EC2 container service here and more about Docker here.
Continue reading about re:Invent 2014 and other recent AWS enhancements here in part two of this two-part series.
Ok, nuff said (for now)
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
|
Ok, nuff said (for now)
Cheers
Gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
Scaling means different things to various people depending on the context or what it is referring to.
For example, scaling can me having or doing more of something, or less as well as referring to how more, or less of something is implemented.
Scaling occurs in a couple of different dimensions and ways:
Often I hear people talk about scaling only in the context of space capacity. However there are aspects including performance, availability as well as scaling-up or scaling-out. Scaling from application workloads perspectives include four main group themes which are performance, availability, capacity and economics (as well as energy).
The latter of the above items should be thought of more in terms of a by-product, result or goal for implementing scaling. Scaling should not result in a compromise of some other attribute such as increasing performance and loss of capacity or increased complexity. Scaling with stability also means that as you scale in some direction, or across some attribute (e.g. PACE), there should not be a corresponding increase in complexity of management, or loss of performance and availability. To use a popular buzz-term scaling with stability means performance, availability, capacity, economics should scale linear with their capabilities or perhaps cost less.
Some examples of scaling in different directions include:
Of course you can combine the above in various combinations such as the example of scaling up and out, as well as apply different names and nomenclature to see your needs or preferences. The following are a closer look at the above with some simple examples.
Example of scaling up (vertically)
Example of scaling-down (e.g. for smaller scenarios)
Example of scaling-out (horizontally)
Example of scaling-out and up(horizontally and vertical)
There are many aspects to scaling, as well as side-effects or impacts as a result of scaling.
Scaling can refer to different workload attributes as well as how to support those applications.
Regardless of what you view scaling as meaning, keep in mind the context of where and when it is used and that others might have another scale view of scale.
Ok, nuff said (for now)…
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
The CompTIA folks are looking for some comments and feedback from those who are involved with data storage in various ways as part of planning for their upcoming enhancements to the Storage+ certification testing.
As a point of disclosure, I am member of the CompTIA Storage+ certification advisory committee (CAC), however I don’t get paid or receive any other type of renumeration for contributing my time to give them feedback and guidance other than a thank, Atta boy for giving back and playing it forward to help others in the IT community similar to what my predecessors did.
I have been asked to pass this along to others (e.g. you or who ever forwards it on to you).
Please take a few moments and feel free to share with others this link here to the survey for CompTIA Storage+.
What they are looking for is to validate the exam blueprint generated from a recent Job Task Analysis (JTA) process.
In other words, does the certification exam show real-world relevance to what you and your associates may be doing involved with data storage.
This is opposed to being aligned with those whose’s job it is to create test questions and may not understand what it is you the IT pro involved with storage does or does not do.
If you have ever taken a certification exam test and scratched your head or wondered out why some questions that seem to lack real-world relevance were included, vs. ones of practical on-the-job experience were missing, here’s your chance to give feedback.
Note that you will not be rewarded with an Amex or Amazon gift card, Starbucks or Dunkin Donuts certificates, free software download or some other incentive to play and win, however if you take the survey let me know and will be sure to tweet you an Atta boy or Atta girl! However they are giving away a free T-Shirt to every 10 survey takers.
Btw, if you really need something for free, send me a note (I’m not that difficult to find) as I have some free copies of Resilient Storage Networking (RSN): Designing Flexible Scalable Data Infrastructures (Elsevier) you simply pay shopping and handling. RSN can be used to help prepare you for various storage testing as well as other day-to-day activities.
CompTIA is looking for survey takers who have some hands-on experience or involved with data storage (e.g. can you spell SAN, NAS, Disk or SSD and work with them hands-on then you are a candidate ;).
Welcome to the CompTIA Storage+ Certification Job Task Analysis (JTA) Survey Here is what (and whom) CompTIA is looking for feedback from: As a small token of CompTIA appreciation for your participation, they will provide an official CompTIA T-shirt to every tenth (1 of every 10) person who completes this survey. Go here for official rules. Click here to complete the CompTIA Storage+ survey Contact CompTIA with any survey issues, research@comptia.org |
What say you, take a few minutes like I did and give some feedback, you will not be on the hook for anything, and if you do get spammed by the CompTIA folks, let me know and I in turn will spam them back for spamming you as well as me.
Ok, nuff said
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
| ||||||||||||||||||||||||||||||||
August 2014 Industry trend and perspectives
The following is a synopsis of some StorageIOblog posts, articles and comments in different venues on various industry trends, perspectives and related themes about clouds, virtualization, data and storage infrastructure topics among related themes. StorageIO comments and perspectives in the news
Virtual Desktop Infrastructures (VDI) remains a popular industry and IT customer topic, not to mention being one of the favorite themes of Solid State Device (SSD) vendors. SSD component and system solution vendors along with their supporters love VDI as the by-product of aggregation (e.g. consolidation) which applies to VDI is aggravation. Aggravation is the result of increased storage I/O performance (IOP’s, bandwidth, response time) from consolidating the various desktops. It should not be a surprise that some of the biggest fans encouraging organizations to adopt VDI are the SSD vendors. Read some of my comments and perspectives on VDI here at FedTech Magazine. Speaking of virtualizing the data center, software defined data centers (SDDC) along with software defined networking (SDN) and software defined storage (SDS) remain popular including some software defined marketing (SDM). Here are some of my comments and perspectives moving beyond the hype of SDDC.
Recently the Fibre Channel Industry Association (FCIA) who works with the T11 standards body of both legacy or classic Fibre Channel (FC) as well as newer FC over Ethernet (FCoE) made some announcements. These announcements including enhancements such as Fibre Channel Back Bone version 6 (FC-BB-6) among others. Both FC and FCoE are alive and doing well, granted one has been around longer (FC) and can be seen at its plateau while the other (FCoE) continues to evolve and grow in adoption. In some ways, FCoE is in a similar role today to where FC was in the late 90s and early 2000s ironically facing some common fud. You can read my comments here as part of a quote in support of the announcement , along with my more industry trend perspectives in this blog post here. Buyers guides are popular with both vendors, VAR’s as well as IT organizations (e.g. customers) following are some of my comments and industry trend perspectives appearing in Enterprise Storage Forum. Here are perspectives on buyers guides for Enterprise File Sync and Share (EFSS), Unified Data Storage and Object Storage. EMC has come under pressure as mentioned in earlier StorageIO update newsletters to increase its shareholder benefit including spin-off of VMware. Here are some of my comments and perspectives that appeared in CruxialCIO. Read more industry trends perspectives comments on the StorageIO news page. StorageIO video and audio pod casts
StorageIOblog posts and perspectives
Despite being declared dead, traditional or classic Fibre Channel (FC) along with FC over Ethernet (FCoE) continues to evolve with FC-BB-6, read more here. VMworld 2014 took place this past week and included announcements about EVO:Rack and Rail (more on this in a future edition). You can get started learning about EVO:Rack and RAIL at Duncan Epping (aka @DuncanYB) Yellow Bricks site. VMware Virtual SAN (VSAN) is at the heart of EVO which you can read an overview here in this earlier StorageIO update newsletter (March 2014).
Also watch for some extra content that I’m working on including some video podcasts articles and blog posts from my trip to VMworld 2014. However one of the themes in the background of VMworld 2014 is the current beta of VMware vSphere V6 along with Virtual Volumes aka VVOL’s. The following are a couple of my recent posts including primer overview of VVOL’s along with a poll you can cast your vote. Check out Are VMware VVOL’s in your virtual server and storage I/O future? and VMware VVOL’s and storage I/O fundamentals (Part 1) along with (Part 2). | ||||||||||||||||||||||||||||||||
StorageIO events and activities The StorageIO calendar continues to evolve including several new events being added for September and well into the fall with more in the works including upcoming Dutch European sessions the week of October 6th in Nijkerk Holland (learn more here). The following are some upcoming September events. These include live in-person seminars, conferences, keynote and speaking activities as well as on-line webinars, twitter chats, Google+ hangouts among others.
Note: Dates, times, venues and subject contents subject to change, refer to events page for current status Click here to view other upcoming along with earlier event activities. Watch for more 2014 events to be added soon to the StorageIO events calendar page. Topics include data protection modernization (backup/restore, HA, BC, DR, archive), data footprint reduction (archive, compression, dedupe), storage optimization, SSD, object storage, server and storage virtualization, software defined, big data, little data, cloud and object storage, performance and management trends among others. Vendors, VAR’s and event organizers, give us a call or send an email to discuss having us involved in your upcoming pod cast, web cast, virtual seminar, conference or other events. | ||||||||||||||||||||||||||||||||
Server and StorageIO Technology Tips and Tools In addition to the industry trends and perspectives comments in the news mentioned above, along with the StorageIO blog posts, the following are some of my recent articles and tips that have appeared in various industry venues.
Over at the new Storage Acceleration site I have a couple of pieces, the first is What, When, Why & How to Accelerate Storage and the other is Tips for Measuring Your Storage Acceleration. | ||||||||||||||||||||||||||||||||
StorageIO Update Newsletter ArchivesClick here to view earlier StorageIO Update newsletters (HTML and PDF versions) at www.storageio.com/newsletter. Subscribe to this newsletter (and pass it along) by clicking here (Via Secure Campaigner site). View archives of past StorageIO update news letters as well as download PDF versions at: www.storageio.com/newsletter | ||||||||||||||||||||||||||||||||
Ok, nuff said (for now) Cheers gs Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier) All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved | ||||||||||||||||||||||||||||||||
Note that this is a three part series with the first piece here (e.g. Are VMware VVOL’s in your virtual server and storage I/O future?), the second piece here (e.g.VMware VVOL’s and storage I/O fundamentals Part 1) and the third piece here (e.g. VMware VVOL’s and storage I/O fundamentals Part 2).
Picking up from where we left off in the first part of the VMware VVOL’s and storage I/O fundamentals, lets take a closer look at VVOL’s.
First however lets be clear that while VMware uses terms including object and object storage in the context of VVOL’s, its not the same as some other object storage solutions. Learn more about object storage here at www.objectstoragecenter.com
No, VVOL’s are accessed via the VMware software and associated API’s that are supported by various storage providers. VVOL’s are not LUN’s like regular block (e.g. DAS or SAN) storage that use SAS, iSCSI, FC, FCoE, IBA/SRP, nor are they NAS volumes like NFS mount points. Likewise VVOL’s are not accessed using any of the various object storage access methods mentioned above (e.g. AWS S3, Rest, CDMI, etc) instead they are an application specific implementation. For some of you this approach of an applications specific or unique storage access method may be new, perhaps revolutionary, otoh, some of you might be having a DejaVu moment right about now.
VVOL is not a LUN in the context of what you may know and like (or hate, even if you have never worked with them), likewise it is not a NAS volume like you know (or have heard of), neither are they objects in the context of what you might have seen or heard such as S3 among others.
Keep in mind that what makes up a VMware virtual machine are the VMK, VMDK and some other files (shown in the figure below), and if enough information is known about where those blocks of data are or can be found, they can be worked upon. Also keep in mind that at least near-term, block is the lowest common denominator that all file systems and object repositories get built-up.
VMware ESXi storage I/O, IOPS and data store basics
Here is the thing, while VVOL’s will be accessible via a block interface such as iSCSI, FC or FCoE or for that matter, over Ethernet based IP using NFS. Think of these storage interfaces and access mechanisms as the general transport for how vSphere ESXi will communicate with the storage system (e.g. their data path) under vCenter management.
What is happening inside the storage system that will be presented back to ESXi will be different than a normal SCSI LUN contents and only understood by VMware hypervisor. ESXi will still tell the storage system what it wants to do including moving blocks of data. The storage system however will have more insight and awareness into the context of what those blocks of data mean. This is how the storage systems will be able to more closely integrate snapshots, replication, cloning and other functions by having awareness into which data to move, as opposed to moving or working with an entire LUN where a VMDK may live. Keep in mind that the storage system will still function as it normally would, just think of VVOL as another or new personality and access mechanism used for VMware to communicate and manage storage.
VMware VVOL concepts (in general) with VMDK being pushed down into the storage system
Think in terms of the iSCSI (or FC or something else) for block or NFS for NAS as being the addressing mechanism to communicate between ESXi and the storage array, except instead of traditional SCSI LUN access and mapping, more work and insight is pushed down into the array. Also keep in mind that with a LUN, it is simply an address from what to use Logical Block Numbers or Logical Block Addresses. In the case of a storage array, it in turn manages placement of data on SSD or HDDs in turn using blocks aka LBA/LBN’s In other words, a host that does not speak VVOL would get an error if trying to use a LUN or target on a storage system that is a VVOL, that’s assuming it is not masked or hidden ;).
The Storage Provider aka SP is created by the, well, the provider of the storage system or appliance leveraging a VMware API (hint, sign up for the beta and there is an SDK). Simply put, the SP is a two-way communication mechanism leveraging VASA for reporting information, configuration and other insight up to VMware ESXi hypervisor, vCenter and other management tools. In addition the storage provider receives VASA configuration information from VMware about how to configure the storage system (e.g. storage containers). Keep in mind that the SP is the out of band management interface between the storage system supporting and presenting VVOL’s and VMware hypervisors.
This is a storage pool created on the storage array or appliance (e.g. VMware vCenter works with array and storage provider (SP) to create) in place of using a normal LUN. With a SP and PE, the storage container becomes visible to ESXi hosts, VVOL’s can be created in the storage container until it runs out of space. Also note that the storage container takes on the storage profile assigned to it which are inherited by the VVOLs in it. This is in place of presenting LUN’s to ESXi that you can then create VMFS data stores (or use as raw) and then carve storage to VMs.
The PE provides visibility for the VMware hypervisor to see and access VMDK’s and other objects (e.g. .vmx, swap, etc) stored in VVOL’s. The protocol endpoint (PE) manages or directs I/O received from the VM enabling scaling across many virtual volumes leveraging multipathing of the PE (inherited by the VVOL’s.). Note that for storage I/O operations, the PE is simply a pass thru mechanism and does not store the VMDK or other contents. If using iSCSI, FC, FCoE or other SAN interface, then the PE works on a LUN basis (again not actually storing data), and if using NAS NFS, then with a mount point. Key point is that the PE gets out-of-the-way.
What are you VVOL plans, view results and cast your vote here
There certainly are many more details to VVOL’s. that you can get a preview of in the beta, a well as via various demos, webinars, VMworld sessions as more becomes public. However for now, hope you found this quick overview on VVOL’s. of use, since VVOL’s. at the time of this writing are not yet released, you will need to wait for more detailed info, or join the beta or poke around the web (for now). Also if you have not seen the first part overview to this piece, check it out here as I give some more links to get you started to learn more about VVOL’s.
Keep an eye on and learn more about VVOL’s. at VMworld 2014 as well as in various other venues.
IMHO VVOL’s. are or will be in your future, however the question will be is there going to be a back to the future moment for some of you with VVOL’s.?
What VVOL questions, comments and concerns are in your future and on your mind?
Ok, nuff said (for now)
Cheers gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved
