Category: review

Posted on

Which Enterprise HDD for Content Applications Different File Size Impact

Which HDD for Content Applications Different File Size Impact

Different File Size Impact server storage I/O trends

Updated 1/23/2018

Which enterprise HDD to use with a content server platform different file size impact.

Insight for effective server storage I/O decision making
Server StorageIO Lab Review

Which enterprise HDD to use for content servers

This is the fifth in a multi-part series (read part four here) based on a white paper hands-on lab report I did compliments of Servers Direct and Seagate that you can read in PDF form here. The focus is looking at the Servers Direct (www.serversdirect.com) converged Content Solution platforms with Seagate Enterprise Hard Disk Drive (HDD’s). In this post the focus looks at large and small file I/O processing.

File Performance Activity

Tip, Content solutions use files in various ways. Use the following to gain perspective how various HDD’s handle workloads similar to your specific needs.

Two separate file processing workloads were run (12), one with a relative small number of large files, and another with a large number of small files. For the large file processing (table-3), 5 GByte sized files were created and then accessed via 128 Kbyte (128KB) sized I/O over a 10 hour period with 90% read using 64 threads (workers). Large file workload simulates what might be seen with higher definition video, image or other content streaming.

(Note 12) File processing workloads were run using Vdbench 5.04 and file anchors with sample script configuration below. Instead of vdbench you could also use other tools such as sysbench or fio among others.

VdbenchFSBigTest.txt
# Sample script for big files testing
fsd=fsd1,anchor=H:,depth=1,width=5,files=20,size=5G
fwd=fwd1,fsd=fsd1,rdpct=90,xfersize=128k,fileselect=random,fileio=random,threads=64
rd=rd1,fwd=fwd1,fwdrate=max,format=yes,elapsed=10h,interval=30

vdbench -f VdbenchFSBigTest.txt -m 16 -o Results_FSbig_H_060615

VdbenchFSSmallTest.txt
# Sample script for big files testing
fsd=fsd1,anchor=H:,depth=1,width=64,files=25600,size=16k
fwd=fwd1,fsd=fsd1,rdpct=90,xfersize=1k,fileselect=random,fileio=random,threads=64
rd=rd1,fwd=fwd1,fwdrate=max,format=yes,elapsed=10h,interval=30

vdbench -f VdbenchFSSmallTest.txt -m 16 -o Results_FSsmall_H_060615

The 10% writes are intended to reflect some update activity for new content or other changes to content. Note that 128KB per second translates to roughly 1 Gbps streaming content such as higher definition video. However 4K video (not optimized) would require a higher speed as well as resulting in larger file sizes. Table-3 shows the performance during the large file access period showing average read /write rates and response time, bandwidth (MBps), average open and close rates with response time.

Avg. File Read Rate

Avg. Read Resp. Time
Sec.

Avg. File Write Rate

Avg. Write Resp. Time
Sec.

Avg.
CPU %
Total

Avg. CPU % System

Avg. MBps
Read

Avg. MBps
Write

ENT 15K R1

580.7

107.9

64.5

19.7

52.2

35.5

72.6

8.1

ENT 10K R1

455.4

135.5

50.6

44.6

34.0

22.7

56.9

6.3

ENT CAP R1

285.5

221.9

31.8

19.0

43.9

28.3

37.7

4.0

ENT 10K R10

690.9

87.21

76.8

48.6

35.0

21.8

86.4

9.6

Table-3 Performance summary for large file access operations (90% read)

Table-3 shows that for two-drive RAID 1, the Enterprise 15K are the fastest performance, however using a RAID 10 with four 10K HDD’s with enhanced cache features provide a good price, performance and space capacity option. Software RAID was used in this workload test.

Figure-4 shows the relative performance of various HDD options handling large files, keep in mind that for the response line lower is better, while for the activity rate higher is better.

large file processing
Figure-4 Large file processing 90% read, 10% write rate and response time

In figure-4 you can see the performance in terms of response time (reads larger dashed line, writes smaller dotted line) along with number of file read operations per second (reads solid blue column bar, writes green column bar). Reminder that lower response time, and higher activity rates are better. Performance declines moving from left to right, from 15K to 10K Enterprise Performance with enhanced cache feature to Enterprise Capacity (7.2K), all of which were hardware RAID 1. Also shown is a hardware RAID 10 (four x 10K HDD’s).

Results in figure-4 above and table-4 below show how various drives can be configured to balance their performance, capacity and costs to meet different needs. Table-4 below shows an analysis looking at average file reads per second (RPS) performance vs. HDD costs, usable capacity and protection level.

Table-4 is an example of looking at multiple metrics to make informed decisions as to which HDD would be best suited to your specific needs. For example RAID 10 using four 10K drives provides good performance and protection along with large usable space, however that also comes at a budget cost (e.g. price).

Avg.
File Reads Per Sec. (RPS)

Single Drive Cost per RPS

Multi-Drive Cost per RPS

Single Drive Cost / Per GB Capacity

Cost / Per GB Usable (Protected) Cap.

Drive Cost (Multiple Drives)

Protection Overhead (Space Capacity for RAID)

Cost per usable GB per RPS

Avg. File Read Resp. (Sec.)

ENT 15K R1

580.7

$1.02

$2.05

$ 0.99

$0.99

$1,190

100%

$2.1

107.9

ENT 10K R1

455.5

1.92

3.84

0.49

0.49

1,750

100%

3.8

135.5

ENT CAP R1

285.5

1.40

2.80

0.20

0.20

798

100%

2.8

271.9

ENT 10K R10

690.9

1.27

5.07

0.49

0.97

3,500

100%

5.1

87.2

Table-4 Performance, capacity and cost analysis for big file processing

Small File Size Processing

To simulate a general file sharing environment, or content streaming with many smaller objects, 1,638,464 16KB sized files were created on each device being tested (table-5). These files were spread across 64 directories (25,600 files each) and accessed via 64 threads (workers) doing 90% reads with a 1KB I/O size over a ten hour time frame. Like the large file test, and database activity, all workloads were run at the same time (e.g. test devices were concurrently busy).

Avg. File Read Rate

Avg. Read Resp. Time
Sec.

Avg. File Write Rate

Avg. Write Resp. Time
Sec.

Avg.
CPU %
Total

Avg. CPU % System

Avg. MBps
Read

Avg. MBps
Write

ENT 15K R1

3,415.7

1.5

379.4

132.2

24.9

19.5

3.3

0.4

ENT 10K R1

2,203.4

2.9

244.7

172.8

24.7

19.3

2.2

0.2

ENT CAP R1

1,063.1

12.7

118.1

303.3

24.6

19.2

1.1

0.1

ENT 10K R10

4,590.5

0.7

509.9

101.7

27.7

22.1

4.5

0.5

Table-5 Performance summary for small sized (16KB) file access operations (90% read)

Figure-5 shows the relative performance of various HDD options handling large files, keep in mind that for the response line lower is better, while for the activity rate higher is better.

small file processing
Figure-5 Small file processing 90% read, 10% write rate and response time

In figure-5 you can see the performance in terms of response time (reads larger dashed line, writes smaller dotted line) along with number of file read operations per second (reads solid blue column bar, writes green column bar). Reminder that lower response time, and higher activity rates are better. Performance declines moving from left to right, from 15K to 10K Enterprise Performance with enhanced cache feature to Enterprise Capacity (7.2K RPM), all of which were hardware RAID 1. Also shown is a hardware RAID 10 (four x 10K RPM HDD’s) that has higher performance and capacity along with costs (table-5).

Results in figure-5 above and table-5 below show how various drives can be configured to balance their performance, capacity and costs to meet different needs. Table-6 below shows an analysis looking at average file reads per second (RPS) performance vs. HDD costs, usable capacity and protection level.

Table-6 is an example of looking at multiple metrics to make informed decisions as to which HDD would be best suited to your specific needs. For example RAID 10 using four 10K drives provides good performance and protection along with large usable space, however that also comes at a budget cost (e.g. price).

Avg.
File Reads Per Sec. (RPS)

Single Drive Cost per RPS

Multi-Drive Cost per RPS

Single Drive Cost / Per GB Capacity

Cost / Per GB Usable (Protected) Cap.

Drive Cost (Multiple Drives)

Protection Overhead (Space Capacity for RAID)

Cost per usable GB per RPS

Avg. File Read Resp. (Sec.)

ENT 15K R1

3,415.7

$0.17

$0.35

$0.99

$0.99

$1,190

100%

$0.35

1.51

ENT 10K R1

2,203.4

0.40

0.79

0.49

0.49

1,750

100%

0.79

2.90

ENT CAP R1

1,063.1

0.38

0.75

0.20

0.20

798

100%

0.75

12.70

ENT 10K R10

4,590.5

0.19

0.76

0.49

0.97

3,500

100%

0.76

0.70

Table-6 Performance, capacity and cost analysis for small file processing

Looking at the small file processing analysis in table-5 shows that the 15K HDD’s on an apples to apples basis (e.g. same RAID level and number of drives) provide the best performance. However when also factoring in space capacity, performance, different RAID level or other protection schemes along with cost, there are other considerations. On the other hand the Enterprise Capacity 2TB HDD’s have a low cost per capacity, however do not have the performance of other options, assuming your applications need more performance.

Thus the right HDD for one application may not be the best one for a different scenario as well as multiple metrics as shown in table-5 need to be included in an informed storage decision making process.

Where To Learn More

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

File processing are common content applications tasks, some being small, others large or mixed as well as reads and writes. Even if your content environment is using object storage, chances are unless it is a new applications or a gateway exists, you may be using NAS or file based access. Thus the importance of if your applications are doing file based processing, either run your own applications or use tools that can simulate as close as possible to what your environment is doing.

Continue reading part six in this multi-part series here where the focus is around general I/O including 8KB and 128KB sized IOPs along with associated metrics.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Which Enterprise HDD for Content Applications General I/O Performance

Which HDD for Content Applications general I/O Performance

hdd general i/o performance server storage I/O trends

Updated 1/23/2018

Which enterprise HDD to use with a content server platform general I/O performance Insight for effective server storage I/O decision making
Server StorageIO Lab Review

Which enterprise HDD to use for content servers

This is the sixth in a multi-part series (read part five here) based on a white paper hands-on lab report I did compliments of Servers Direct and Seagate that you can read in PDF form here. The focus is looking at the Servers Direct (www.serversdirect.com) converged Content Solution platforms with Seagate Enterprise Hard Disk Drive (HDD’s). In this post the focus is around general I/O performance including 8KB and 128KB IOP sizes.

General I/O Performance

In addition to running database and file (large and small) processing workloads, Vdbench was also used to collect basic small (8KB) and large (128KB) sized I/O operations. This consisted of random and sequential reads as well as writes with the results shown below. In addition to using vdbench, other tools that could be used include Microsoft Diskspd, fio, iorate and iometer among many others.

These workloads used Vdbench configured (13) to do direct I/O to a Windows file system mounted device using as much of the available disk space as possible. All workloads used 16 threads and were run concurrently similar to database and file processing tests.

(Note 13) Sample vdbench configuration for general I/O, note different settings were used for various tests

Table-7 shows workload results for 8KB random IOPs 75% reads and 75% writes including IOPs, bandwidth and response time.

 

ENT 15K RAID1

ENT 10K RAID1

ENT CAP RAID1

ENT 10K R10
(4 Drives)

ECAP SW RAID (5 Drives)

 

75% Read

25% Read

75% Read

25% Read

75% Read

25% Read

75% Read

25% Read

75% Read

25% Read

I/O Rate (IOPs)

597.11

559.26

514

475

285

293

979

984

491

644

MB/sec

4.7

4.4

4.0

3.7

2.2

2.3

7.7

7.7

3.8

5.0

Resp. Time (Sec.)

25.9

27.6

30.2

32.7

55.5

53.7

16.3

16.3

32.6

24.8

Table-7 8KB sized random IOPs workload results

Figure-6 shows small (8KB) random I/O (75% read and 25% read) across different HDD configurations. Performance including activity rates (e.g. IOPs), bandwidth and response time for mixed reads / writes are shown. Note how response time increases with the Enterprise Capacity configurations vs. other performance optimized drives.

general 8K random IO
Figure-6 8KB random reads and write showing IOP activity, bandwidth and response time

Table-8 below shows workload results for 8GB sized I/Os 100% sequential with 75% reads and 75% writes including IOPs, MB/sec and response time in seconds.

ENT 15K RAID1

ENT 10K RAID1

ENT CAP RAID1

ENT 10K R10
(4 Drives)

ECAP SW RAID (5 Drives)

75% Read

25% Read

75% Read

25% Read

75% Read

25% Read

75% Read

25% Read

75% Read

25% Read

I/O Rate (IOPs)

3,778

3,414

3,761

3,986

3,379

1,274

11,840

8,368

2,891

1,146

MB/sec

29.5

26.7

29.4

31.1

26.4

10.0

92.5

65.4

22.6

9.0

Resp. Time (Sec.)

2.2

3.1

2.3

2.4

2.7

10.9

1.3

1.9

5.5

14.0

Table-8 8KB sized sequential workload results

Figure-7 shows small 8KB sequential mixed reads and writes (75% read and 75% write), while the Enterprise Capacity 2TB HDD has a large amount of space capacity, its performance in a RAID 1 vs. other similar configured drives is slower.

8KB Sequential
Figure-7 8KB sequential 75% reads and 75% write showing bandwidth activity

Table-9 shows workload results for 100% sequential, 100% read and 100% write 128KB sized I/Os including IOPs, bandwidth and response time.

ENT 15K RAID1

ENT 10K RAID1

ENT CAP RAID1

ENT 10K R10
(4 Drives)

ECAP SW RAID (5 Drives)

Read

Write

Read

Write

Read

Write

Read

Write

Read

Write

I/O Rate (IOPs)

1,798

1,771

1,716

1,688

921

912

3,552

3,486

780

721

MB/sec

224.7

221.3

214.5

210.9

115.2

114.0

444.0

435.8

97.4

90.1

Resp. Time (Sec.)

8.9

9.0

9.3

9.5

17.4

17.5

4.5

4.6

19.3

20.2

Table-9 128KB sized sequential workload results

Figure-8 shows sequential or streaming operations of larger I/O (100% read and 100% write) requests sizes (128KB) that would be found with large content applications. Figure-8 highlights the relationship between lower response time and increased IOPs as well as bandwidth.

128K Sequential
Figure-8 128KB sequential reads and write showing IOP activity, bandwidth and response time

Where To Learn More

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

Some content applications are doing small random I/Os for database, key value stores or repositories as well as meta data processing while others are doing large sequential I/O. 128KB sized I/O may be large for your environment, on the other hand, with an increasing number of applications, file systems, software defined storage management tools among others, 1 to 10MB or even larger I/O sizes are becoming common. Key is selecting I/O sizes and read write as well as random sequential along with I/O or queue depths that align with your environment.

Continue reading part seven the final post in this multi-part series here where the focus is around how HDD’s continue to evolve including performance beyond traditional RPM based execrations along with wrap up.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

HDDs evolve for Content Application servers

HDDs evolve for Content Application servers

hdds evolve server storage I/O trends

Updated 1/23/2018

Enterprise HDDs evolve for content server platform

Insight for effective server storage I/O decision making
Server StorageIO Lab Review

Which enterprise HDD to use for content servers

This is the seventh and final post in this multi-part series (read part six here) based on a white paper hands-on lab report I did compliments of Servers Direct and Seagate that you can read in PDF form here. The focus is looking at the Servers Direct (www.serversdirect.com) converged Content Solution platforms with Seagate Enterprise Hard Disk Drive (HDD’s). The focus of this post is comparing how HDD continue to evolve over various generations boosting performance as well as capacity and reliability. This also looks at how there is more to HDD performance than the traditional focus on Revolutions Per Minute (RPM) as a speed indicator.

Comparing Different Enterprise 10K And 15K HDD Generations

There is more to HDD performance than RPM speed of the device. RPM plays an important role, however there are other things that impact HDD performance. A common myth is that HDD’s have not improved on performance over the past several years with each successive generation. Table-10 shows a sampling of various generations of enterprise 10K and 15K HDD’s (14) including different form factors and how their performance continues to improve.

different 10K and 15K HDDs
Figure-9 10K and 15K HDD performance improvements

Figure-9 shows how performance continues to improve with 10K and 15K HDD’s with each new generation including those with enhanced cache features. The result is that with improvements in cache software within the drives, along with enhanced persistent non-volatile memory (NVM) and incremental mechanical drive improvements, both read and write performance continues to be enhanced.

Figure-9 puts into perspective the continued performance enhancements of HDD’s comparing various enterprise 10K and 15K devices. The workload is the same TPC-C tests used earlier in a similar (14) (with no RAID). 100 simulated users are shown in figure-9 accessing a database on each of the different drives all running concurrently. The older 15K 3.5” Cheetah and 2.5” Savio used had a capacity of 146GB which used a database scale factor of 1500 or 134GB. All other drives used a scale factor 3000 or 276GB. Figure-9 also highlights the improvements in both TPS performance as well as lower response time with new HDD’s including those with performance enhanced cache feature.

The workloads run are same as the TPC-C ones shown earlier, however these drives were not configured with any RAID. The TPC-C activity used Benchmark Factory with similar setup and configuration to those used earlier including on a multi-socket, multi-core Windows 2012 R2 server supporting a Microsoft SQL Server 2012 database with a database for each drive type.

ENT 10K V3 2.5"

ENT (Cheetah) 15K 3.5"

Users

1

20

50

100

Users

1

20

50

100

TPS (TPC-C)

14.8

50.9

30.3

39.9

TPS (TPC-C)

14.6

51.3

27.1

39.3

Resp. Time (Sec.)

0.0

0.4

1.6

1.7

Resp. Time (Sec.)

0.0

0.3

1.8

2.1

ENT 10K 2.5" (with cache)

ENT (Savio) 15K 2.5"

Users

1

20

50

100

Users

1

20

50

100

TPS (TPC-C)

19.2

146.3

72.6

71.0

TPS (TPC-C)

15.8

59.1

40.2

53.6

Resp. Time (Sec.)

0.0

0.1

0.7

0.0

Resp. Time (Sec.)

0.0

0.3

1.2

1.2

ENT 15K V4 2.5"

Users

1

20

50

100

TPS (TPC-C)

19.7

119.8

75.3

69.2

Resp. Time (Sec.)

0.0

0.1

0.6

1.0

ENT 15K (enhanced cache) 2.5"

Users

1

20

50

100

TPS (TPC-C)

20.1

184.1

113.7

122.1

Resp. Time (Sec.)

0.0

0.1

0.4

0.2

Table-10 Continued Enterprise 10K and 15K HDD performance improvements

(Note 14) 10K and 15K generational comparisons were run on a separate comparable server to what was used for other test workloads. Workload configuration settings were the same as other database workloads including using Microsoft SQL Server 2012 on a Windows 2012 R2 system with Benchmark Factory driving the workload. Database memory sized was reduced however to only 8GB vs. 16GB used in other tests.

Where To Learn More

Additional learning experiences along with common questions (and answers), as well as tips can be found in Software Defined Data Infrastructure Essentials book.

Software Defined Data Infrastructure Essentials Book SDDC

What This All Means

A little bit of flash in the right place with applicable algorithms goes a long way, an example being the Seagate Enterprise HDD’s with enhanced cache feature. Likewise, HDD’s are very much alive complementing SSD and vice versa. For high-performance content application workloads flash SSD solutions including NVMe, 12Gbps SAS and 6Gbps SATA devices are cost effective solutions. HDD’s continue to be cost-effective data storage devices for both capacity, as well as environments that do not need the performance of flash SSD.

For some environments using a combination of flash and HDD’s complementing each other along with cache software can be a cost-effective solution. The previous workload examples provide insight for making cost-effective informed storage decisions.

Evaluate today’s HDD’s on their effective performance running workloads as close as similar to your own, or, actually try them out with your applications. Today there is more to HDD performance than just RPM speed, particular with the Seagate Enterprise Performance 10K and 15K HDD’s with enhanced caching feature.

However the Enterprise Performance 10K with enhanced cache feature provides a good balance of capacity, performance while being cost-effective. If you are using older 3.5” 15K or even previous generation 2.5” 15K RPM and “non-performance enhanced” HDD’s, take a look at how the newer generation HDD’s perform, looking beyond the RPM of the device.

Fast content applications need fast content and flexible content solution platforms such as those from Servers Direct and HDD’s from Seagate. Key to a successful content application deployment is having the flexibility to hardware define and software defined the platform to meet your needs. Just as there are many different types of content applications along with diverse environments, content solution platforms need to be flexible, scalable and robust, not to mention cost effective.

Ok, nuff said, for now.

Gs

Greg Schulz – Microsoft MVP Cloud and Data Center Management, VMware vExpert 2010-2017 (vSAN and vCloud). Author of Software Defined Data Infrastructure Essentials (CRC Press), as well as Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press), Resilient Storage Networks (Elsevier) and twitter @storageio. Courteous comments are welcome for consideration. First published on https://storageioblog.com any reproduction in whole, in part, with changes to content, without source attribution under title or without permission is forbidden.

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO. All Rights Reserved. StorageIO is a registered Trade Mark (TM) of Server StorageIO.

Posted on

Software Defined Storage Virtual Hard Disk (VHD) Algorithms + Data Structures

Software Defined Storage Virtual Hard Disk (VHD) Algorithms + Data Structures

server storage I/O trends

For those who are into, or simply like to talk about software defined storage (SDS), APIs, Windows, Virtual Hard Disks (VHD) or VHDX, or Hyper-V among other related themes, have you ever actually looked at the specification for VHDX? If not, here is the link to the open specification that Microsoft published (this one dates back to 2012).

Microsoft VHDX specification document
Click on above image to download the VHDX specification from Microsoft.com

How about Algorithms + Data Structures = Programs by Niklaus Wirth, some of you might remember that from the past, if not, it’s a timeless piece of work that has many fundamental concepts for understanding software defined anything. I came across Algorithms + Data Structures = Programs back in Graduate School when I was getting my masters degree in Software Engineering at night, while working during the day in an IT environment on servers, storage, I/O networking hardware and software.


Algorithms + Data Structures = Programs on Amazon.com

In addition to the Amazon.com link above, here is a link to a free (legitimate PDF) copy.

The reason I mention Software Defined, Virtual Hard Disk and Algorithms + Data Structures = Programs is that they are all directly related, or at a minimum can help demystify things.

Inside a VHD and VHDX

The following is an excerpt from the Microsoft VHDX specification document mentioned above that shows a logical view of how a VHDX is defined as a data structure, as well as how algorithms should use and access them.

Microsoft VHDX specification

Keep in mind that anything software defined is a collection of data structures that describe how bits, bytes, blocks, blobs or other entities are organized and then accessed by algorithms that are defined how to use those data structures. Thus the connection to Algorithms + Data Structures = Programs mentioned above.

In the case of a Virtual Hard Disk (VHD) or VHDX they are the data structures defined (see the specification here) and then used by various programs (applications or algorithms) such as Windows or other operating systems, hypervisors or utilities.

A VHDX (or VMDK or VVOL or qcow or other virtual disk for that matter) is a file whose contents are organized e.g. the data structures per a given specification (here).

The VHDX can then be moved around like another file and used for booting some operating systems, as well as simply mounting and using like any other disk or device.

This also means that you can nest putting a VHDX inside of a VHDX and so forth.

Where to learn more

Continue reading with the following links about Virtual Hard Disks pertaining to Microsoft Windows, Hyper-V, VMware among others.

  • Algorithms + Data Structures = Programs on Amazon.com
  • Microsoft Technet Virtual Hard Disk Sharing Overview
  • Download the VHDX specification from Microsoft.com
  • Microsoft Technet Hyper-V Virtual Hard Disk (VHD) Format Overview
  • Microsoft Technet Online Virtual Hard Disk Resizing Overview
  • VMware Developer Resource Center (VDDK for vSphere 6.0)
  • VMware VVOLs and storage I/O fundamentals (Part 1)

    • What this all means

    Applications and utilities or basically anything that is algorithms working with data structures is a program. Software Defined Storage or Software Defined anything involves defining data structures that describes various entities, along with the algorithms to work with and use those data structures.

    Sharpen, refresh or expand your software defined data center, software defined network, software defined storage or software defined storage management as well as software defined marketing game by digging a bit deeper into the bits and bytes. Who knows, you might just go from talking the talk to walking the talk, if nothing else, talking the talk better..

    Ok, nuff said (for now)

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(R) and UnlimitedIO All Rights Reserved

    Posted on

    Server StorageIO October 2015 Update Newsletter


    Server and StorageIO Update Newsletter

    Volume 15, Issue X – Industry Trends, M&A, PTSA

    Hello and welcome to this October 2015 Server StorageIO update newsletter. Fall has arrived here in the northern hemisphere which means its spring in the southern hemisphere, and getting colder here. While fall means cooler out-door temperature with winter just around the corner, in the IT/ITC industry, particular the data infrastructure sector (server, storage, I/O networking, hardware, software, cloud, physical, software defined virtual) things are very hot. Sure the various industry and vendor focused conferences, road shows and mini-events with associated new product, technology or services announcements (PTSA. There are also the various merger and acquisitions (M&A) that have occurred throughout the year including the recent Dell buying EMC, and Western Digital (WD) buying SANdisk among others.

    This edition of the Server StorageIO update newsletter has a focus on industry trends perspectives including recent M&A and PTSA activity. In addition to industry fall industry M&A and PTSA activity, there also plenty of conference, seminars, workshops, webinars and other events some of which you can see here on the Server StorageIO events page.

    On a slightly different note, for those interested and not aware of the European Union (EU) ruling earlier this month on data privacy (e.g. Safe Harbor), here and here are a couple of links to stories discussing the new ruling changes between the EU and US (among other countries). The EU data privacy rulings involve personal data being moved out of EU countries to US data centers such as cloud and application services firms.

    Enjoy this edition of the Server StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and podcast’s along with in the news commentary appearing soon.

    Cheers GS

    In This Issue

  • Feature Topic
  • Industry Trends News
  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts
  • Videos and Podcasts
  • Events and Webinars
  • Recommended Reading List
  • Industry Activity Trends
  • Server StorageIO Lab reports
  • New and Old Vendor Update
  • Resources and Links

    • Feature Topic – TBD

    This months feature topic theme is industry trends perspectives including M&A activity.

    Some M&A, IPO and divestiture activity includes:

    Continue reading more about NVM, NVMe, NAND flash, SSD Server and storage I/O related topics at www.thessdplace.com as well as about I/O performance, monitoring and benchmarking tools at www.storageperformance.us.

     

    StorageIOblog Posts

    Recent and popular Server StorageIOblog posts include:

    View other recent as well as past blog posts here

    Server Storage I/O Industry Activity Trends (Cloud, Virtual, Physical)

    StorageIO news (image licensed for use from Shutterstock by StorageIO)

    Some new Products Technology Services Announcements (PTSA) include:

    • Amazon Web Service (AWS) Simple Storage Service (S3) Infrequent Access (IA) storage class for inactive data with immediate access vs. Glacier cold or frozen (dormant) data with slow or time delayed access. AWS also announced Snowball bulk data import/export 50TB appliance service in addition to their earlier offered capabilities.
    • EMC Rexray (part of EMCcode) and Mesosphere (for Mesos data center operating system) have joined to enable persistent Docker volumes for Mesos (e.g. data center operating system platform).
    • Microsoft Azure recent enhancements include file access of cloud storage (on-premises and within Azure cloud) leveraging SMB interfaces. Here is a primer on Azure cloud storage service offerings. View other recent Azure Cloud Storage, Compute, Database and Data Analytics service offerings here. In addition to Microsoft Azure cloud offerings or Windows 10 desktop operating system, you can also download WIndows Server 2016 Technical Preview 3 (TP3) and see what’s new here. Some of the features include Storage Spaces Direct (e.g. DAS storage) and replication among other features.

    View other recent news and industry trends here

    StorageIO Commentary in the news

    StorageIO news (image licensed for use from Shutterstock by StorageIO)
    Recent Server StorageIO commentary and industry trends perspectives about news, activities tips, and announcements.

    • NetworkComputing: Dell buying EMC: The Storage Ramifications
    • EnterpriseTech: VMware Targets Synergies in Dell EMC Deal 
    • HPCwire: Dell to Buy EMC for $67B
    • EnterpriseStorageForum: Data Storage: Do We Really Need to Store Everything?
    • EnterpriseStorageForum: Why Hard Drives Are Here to Stay (For Now)
    • EnterpriseStorageForum: Top Ten Ways to Use OpenStack for Storage
    • EnterpriseStorageForum: Are We Heading for Storage Armageddon?

    View more Server, Storage and I/O hardware as well as software trends comments here

    Vendors you may not have heard of

    Various vendors (and service providers) you may not know or heard about recently.

    • Hedvig – Converged server storage software management tools
    • Infinidat – Another Moshe Yanai Storage System Startup
    • Mesosphere – Mesos Data Center Operating System management tools
    • Plexxi – Networking startup with former EMC executive Rich Napolitano as CEO
    • ScaleMP – Scale-out server aggregation management tools

    Check out more vendors you may know, have heard of, or that are perhaps new on the Server StorageIO Industry Links page here (over 1,000 entries and growing).

    StorageIO Tips and Articles

    Recent Server StorageIO articles appearing in different venues include:

    • Virtual Blocks (VMware Blogs):  EVO:RAIL – What Is It And Why Does It Matter?
      This is the first of a multi-part series looking at Converged Infrastructures (CI), Hyper-Converged Infrastructures (HCI), Cluster in Box (CiB) and other unified solution bundles. There is a trend of industry adoption talking about CI, HCI, CiB and other bundled solutions, along with growing IT customer adoption and deployment. Different sized organizations are looking at various types of CI solutions to meet various application and workloads needs. Read more here.
    • WServerNews.com:  Cloud (Microsoft Azure) storage considerations
      Let’s say that you have been tasked with, or decided that it is time to use (or try) public cloud storage such as Microsoft Azure. Ok, now what do you do and what decisions need to be made? Keep in mind that Microsoft Azure like many other popular public clouds provides many difference services available for fee (subscription) along with free trials. These services include applications, compute, networking, storage along with development and management platform tools. Read more here.
    • NetworkComputing:  Selecting Storage: Buzzword Bingo
      The storage industry is rife with buzzwords. Here are some of the popular ones storage buyers need to navigate carefully to find storage products that truly meet their needs. Read more here.

    • InfoStor:  What’s The Best Storage Benchmark? It Depends…

    • EnterpriseStorageForum:  NAND, DRAM, SAS/SCSI & SATA/AHCI: Not Dead, Yet!

    Check out these resources and links technology, techniques, trends as well as tools. View more tips and articles here

    StorageIO Videos and Podcasts

    StorageIO podcasts are also available via and at StorageIO.tv

    StorageIO Webinars and Industry Events

    Deltaware Emerging Technology Summit November 10, 2015

    Dell Data Protection Summit Nov 4, 2015 7AM PT

    Microsoft MVP Summit Nov 2-5, 2015

    Server Storage I/O Dutch Workshop Seminar Series
    Nijkerk Netherlands October 13-16 2015

    October 13 – Symposium: Software Defined Storage Management
    October 14 – Server Storage I/O Fundamental Trends
    October 15 – Symposium – Data Center Infrastructure Management (DCIM)
    October 16 – “Converged Day” Server and Storage Decision making

    Learn more and register at the Brouwer Consultancy website here.

    September 23 – Webinar Redmond Magazine & Dell Data Protection
    The New World Order of Data Protection – Focus on Recovery
    Learn more about the 9Rs of data protection and recovery

    See more webinars and other activities on the Server StorageIO Events page here.

    From StorageIO Labs

    Research, Reviews and Reports

    Quick Look: SATA and NVMe Flash SSD Performance
    SATA and NVMe flash SSD performance

    Non-Volatile Memory (NVM) Express (NVMe) continues to evolve as a technology for enabling and improving server storage I/O for NVM including nand flash SSD storage. NVMe streamlines performance enabling more work to be done (e.g. IOPs), data to be moved (bandwidth) at a lower response time using less CPU. The above figure is a quick look comparing nand flash SSD being accessed via SATA III (6Gbps) on the left and NVMe (x4) on the right. As with any server storage I/O performance comparisons there are many variables and take them with a grain of salt. While IOPs and bandwidth are often discussed, keep in mind that with the new protocol, drivers and device controllers with NVMe that streamline I/O less CPU is needed. Learn more about NVM, NVMe, flash, SSD and related topics at www.thessdplace.com.

    View other StorageIO lab review reports here

    Server StorageIO Recommended Reading List

    The following are various recommended reading including books, blogs and videos. If you have not done so recently, also check out the Intel Recommended Reading List (here) where you will also find a couple of my books.

    Seven Databases in Seven Weeks guide to no SQL via Amazon.com

    The Human Face of Big Data book review. To say this is a big book would be an understatement, then again, big data is a big topic with a lot of diversity if you open your eyes and think in a pragmatic way, which once you open and see the pages you will see. This is physically a big book (11x 14 inches) with lots of pictures, texts, stories, factoids and thought stimulating information of the many facets and dimensions of big data across 224 pages. The Human Face of Big Data is more than a coffee table or picture book as it is full of with information, factoids and perspectives how information and data surround us every day. Open up a copy of The Human Face of Big Data and you will see examples of how data and information are all around us, and our dependence upon it. Read more here.

    Server StorageIO Industry Resources and Links

    Check out these useful links and pages:

    storageio.com/links
    objectstoragecenter.com
    storageioblog.com/data-protection-diaries-main/
    storageperformance.us
    thenvmeplace
    thessdplace.com
    storageio.com/raid
    storageio.com/ssd

    Ok, nuff said

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    August Server StorageIO Update Newsletter – NVM and Flash SSD Focus

    Volume 15, Issue VIII

    Hello and welcome to this August 2015 Server StorageIO update newsletter. Summer is wrapping up here in the northern hemisphere which means the fall conference season has started, holidays in progress as well as getting ready for back to school time. I have been spending my summer working on various things involving servers, storage, I/O networking hardware, software, services from cloud to containers, virtual and physical. This includes OpenStack, VMware vCloud Air, AWS, Microsoft Azure, GCS among others, as well as new versions of Microsoft Windows and Servers, Non Volatile Memory (NVM) including flash SSD, NVM Express (NVMe), databases, data protection, software defined, cache, micro-tiering and benchmarking using various tools among other things (some are still under wraps).

    Enjoy this edition of the Server StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and podcast’s along with in the news commentary appearing soon.

    Cheers GS

    In This Issue

  • Feature Topic
  • Industry Trends News
  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts
  • Videos and Podcasts
  • Events and Webinars
  • Recommended Reading List
  • Industry Activity Trends
  • Server StorageIO Lab reports
  • New and Old Vendor Update
  • Resources and Links

    • Feature Topic – Non Volatile Memory including NAND flash SSD

    Via Intel History of Memory
    Via Intel: Click above image to view history of memory

    This months feature topic theme is Non Volatile Memory (NVM) which includes technologies such as NAND flash commonly used in Solid State Devices (SSDs) storage today, as well as in USB thumb drive, mobile and hand-held devices among many other uses. NVM spans servers, storage, I/O devices along with mobile and handheld among many other technologies. In addition to NAND flash, other forms of NVM include Non Volatile Random Access Memory (NVRAM), Read Only Memory (ROM) along with some emerging new technologies including the recently announced Intel and Micron 3D XPoint among others.

    • NVMe: The Golden Ticket for Faster Flash Storage? (Via EnterpriseStorageForum)
    • What should I consider when using SSD cloud? (Via SearchCloudStorage)
    • MSP CMG, Sept. 2014 Presentation (Flash back to reality – Myths and Realities
    • Flash and SSD Industry trends perspectives plus benchmarking tips) – PDF
    • Selecting Storage: Start With Requirements (Via NetworkComputing)
    • Spot The Newest & Best Server Trends (Via Processor)
    • Market ripe for embedded flash storage as prices drop (Via Powermore (Dell))

    Continue reading more about NVM, NVMe, NAND flash, SSD Server and storage I/O related topics at www.thessdplace.com as well as about I/O performance, monitoring and benchmarking tools at www.storageperformance.us.

     

    StorageIOblog Posts

    Recent and popular Server StorageIOblog posts include:

    View other recent as well as past blog posts here

    Server Storage I/O Industry Activity Trends (Cloud, Virtual, Physical)

    StorageIO news (image licensed for use from Shutterstock by StorageIO)

    • PMC Announces NVMe SSD Controllers (Via TomsITpro)
    • New SATA SSD powers elastic cloud agility for CSPs (Via Cbronline)
    • Toshiba Solid-State Drive Family Features PCIe Technology (Via Eweek)
    • SanDisk aims CloudSpeed Ultra SSD at cloud providers (Via ITwire)
    • Everspin & Aupera reveal MRAM Module M.2 Form Factor (Via BusinessWire)
    • PMC-Sierra Scales Storage with PCIe, NVMe (Via EEtimes)
    • Seagate Grows Its Nytro Enterprise Flash Storage Line (Via InfoStor)
    • New SAS Solid State Drive From Seagate Micron Alliance (Via Seagate)
    • Samsung ups the SSD ante with faster, higher capacity drives (Via ITworld)

    View other recent news and industry trends here

    StorageIO Commentary in the news

    StorageIO news (image licensed for use from Shutterstock by StorageIO)
    Recent Server StorageIO commentary and industry trends perspectives about news, activities tips, and announcements.

    • Processor: Comments on Spot The Newest & Best Server Trends
    • Processor: Comments on A Snapshot Strategy For Backups & Data Recovery
    • EnterpriseStorageForum: Comments on Defining the Future of DR Storage
    • EnterpriseStorageForum: Comments on Top Ten Tips for DR as a Service
    • EnterpriseStorageForum: Comments on NVMe: Golden Ticket for Faster Storage

    View more Server, Storage and I/O hardware as well as software trends comments here

    Vendors you may not have heard of

    Various vendors (and service providers) you may not know or heard about recently.

    • Scala – Scale out storage management software tools
    • Reduxio – Enterprise hybrid storage with data services
    • Jam TreeSize Pro – Data discovery and storage resource analysis and reporting

    Check out more vendors you may know, have heard of, or that are perhaps new on the Server StorageIO Industry Links page here (over 1,000 entries and growing).

    StorageIO Tips and Articles

    Recent Server StorageIO articles appearing in different venues include:

    • IronMountain:  Information Lifecycle Management: Which Data Types Have Value?
      It’s important to keep in mind that on a fundamental level, there are three types of data: information that has value, information that does not have value and information that has unknown value. Data value can be measured along performance, availability, capacity and economic attributes, which define how the data gets managed across different tiers of storage. In general data can have value, unknown value or no value. Read more here.
    • EnterpriseStorageForum:  Is Future Storage Converging Around Hyper-Converged?
      Depending on who you talk or listen to, hyper-converged storage is either the future of storage, or it is a hype niche market that is not for everybody, particular not larger environments. How converged is the hyper-converged market? There are many environments that can leverage CI along with HCI, CiB or other bundles solutions. Granted, not all of those environments will converge around the same CI, CiB and HCI or pod solution bundles as everything is not the same in most IT environments and data centers. Not all markets, environments or solutions are the same. Read more here.

    Check out these resources and links technology, techniques, trends as well as tools. View more tips and articles here

    StorageIO Videos and Podcasts

    StorageIO podcasts are also available via and at StorageIO.tv

    StorageIO Webinars and Industry Events

    Server Storage I/O Workshop Seminars
    Nijkerk Netherlands October 13-16 2015

    VMworld August 30-September 3 2015

    See additional webinars and other activities on the Server StorageIO Events page here.

    From StorageIO Labs

    Research, Reviews and Reports

    Enmotus FuzeDrive (Server based Micro-Tiering)
    Enmotus FuzeDrive
    • Micro-teiring of reads and writes
    • FuzeDrive for transparent tiering
    • Dynamic tiering with selectable options
    • Monitoring and diagnostics tools
    • Transparent to operating systems
    • Hardware transparent (HDD and SSD)
    • Server I/O interface agnostic
    • Optional RAM cache and file pinning
    • Maximize NVM flash SSD investment
    • Compliment other SDS solutions
    • Use for servers or workstations

    Enmotus FuzeDrive provides micro-tiering boosting performance (reads and writes) of storage attached to physical bare metal servers, virtual and cloud instances including Windows and Linux operating systems across various applications. In the simple example above five separate SQL Server databases (260GB each) were placed on a single 6TB HDD. A TPCC workload was run concurrently against all databases with various numbers of users. One workload used a single 6TB HDD (blue) while the other used a FuzeDrive (green) comprised of a 6TB HDD and a 400GB SSD showing basic micro-tiering improvements.

    View other StorageIO lab review reports here

    Server StorageIO Recommended Reading List

    The following are various recommended reading including books, blogs and videos. If you have not done so recently, also check out the Intel Recommended Reading List (here) where you will also find a couple of my books.

    Get Whats Yours via Amazon.com
    While not a technology book, you do not have to be at or near retirement age to be planning for retirement. Some of you may already be at or near retirement age, for others, its time to start planning or refining your plans. A friend recommended this book and I’m recommending it to others. Its pretty straight forward and you might be surprised how much money people may be leaving on the table! Check it out here at Amazon.com.

    Server StorageIO Industry Resources and Links

    Check out these useful links and pages:

    storageio.com/links
    objectstoragecenter.com
    storageioblog.com/data-protection-diaries-main/
    storageperformance.us
    thenvmeplace
    thessdplace.com
    storageio.com/raid
    storageio.com/ssd

    Ok, nuff said

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Supermicro CSE-M14TQC Use your media bay to add 12 Gbps SAS SSD drives to your server

    Storage I/O trends

    Supermicro CSE-M14TQC Use your media bay to add 12 Gbps SAS SSD drives to your server

    Do you have a computer server, workstation or mini-tower PC that needs to have more 2.5" form factor hard disk drive (HDD), solid state device (SSD) or hybrid flash drives added yet no expansion space?

    Do you also want or need the HDD or SSD drive expansion slots to be hot swappable, 6 Gbps SATA3 along with up to 12 Gbps SAS devices?

    Do you have an available 5.25" media bay slot (e.g. where you can add an optional CD or DVD drive) or can you remove your existing CD or DVD drive using USB for software loading?

    Do you need to carry out the above without swapping out your existing server or workstation on a reasonable budget, say around $100 USD plus tax, handling, shipping (your prices may vary)?

    If you need implement the above, then here is a possible solution, or in my case, an real solution.

    Via StorageIOblog Supermicro 4 x 2.5 12Gbps SAS enclosure CSE-M14TQC
    Supermicro CSE-M14TQC with hot swap canister before installing in one of my servers

    In the past I have used a solution from Startech that supports up to 4 x 2.5" 6 Gbps SAS and SATA drives in a 5.25" media bay form factor installing these in my various HP, Dell and Lenovo servers to increase internal storage bays (slots).

    Via Amazon.com StarTech SAS and SATA expansion
    Via Amazon.com StarTech 4 x 2.5" SAS and SATA internal enclosure

    I still use the StarTech device shown (read earlier reviews and experiences here, here and here) above in some of my servers which continue to be great for 6Gbps SAS and SATA 2.5" HDDs and SSDs. However for 12 Gbps SAS devices, I have used other approaches including external 12 Gbps SAS enclosures.

    Recently while talking with the folks over at Servers Direct, I mentioned how I was using StarTech 4 x 2.5" 6Gbps SAS/SATA media bay enclosure as a means of boosting the number of internal drives that could be put into some smaller servers. The Servers Direct folks told me about the Supermicro CSE-M14TQC which after doing some research, I decided to buy one to complement the StarTech 6Gbps enclosures, as well as external 12 Gbps SAS enclosures or other internal options.

    What is the Supermicro CSE-M14TQC?

    The CSE-M14TQC is a 5.25" form factor enclosure that enables four (4) 2.5" hot swappable (if your adapter and OS supports hot swap) 12 Gbps SAS or 6 Gbps SATA devices (HDD and SSD) to fit into the media bay slot normally used by CD/DVD devices in servers or workstations. There is a single Molex male power connector on the rear of the enclosure that can be used to attach to your servers available power using applicable connector adapters. In addition there are four seperate drive connectors (e.g. SATA type connectors) that support up to 12 Gbps SAS per drive which you can attach to your servers motherboard (note SAS devices need a SAS controller), HBA or RAID adapters internal ports.

    Cooling is provided via a rear mounted 12,500 RPM 16 cubic feet per minute fan, each of the four drives are hot swappable (requires operating system or hypervisor support) contained in a small canister (provided with the enclosure). Drives easily mount to the canister via screws that are also supplied as part of the enclosure kit. There is also a drive activity and failure notification LED for the devices. If you do not have any available SAS or SATA ports on your servers motherboard, you can use an available PCIe slot and add a HBA or RAID card for attaching the CSE-M14TQC to the drives. For example, a 12 Gbps SAS (6 Gbps SATA) Avago/LSI RAID card, or a 6 Gbps SAS/SATA RAID card.

    Via Supermicro CSE-M14TQC rear details (4 x SATA and 1 Molex power connector)

    Via StorageIOblog Supermicro 4 x 2.5 rear view CSE-M14TQC 12Gbps SAS enclosure
    CSE-M14TQCrear view before installation

    Via StorageIOblog Supermicro CSE-M14TQC 12Gbps SAS enclosure cabling
    CSE-M14TQC ready for installation with 4 x SATA (12 Gbps SAS) drive connectors and Molex power connector

    Tip: In the case of the Lenovo TS140 that I initially installed the CSE-M14TQC into, there is not a lot of space for installing the drive connectors or Molex power connector to the enclosure. Instead, attach the cables to the CSE-M14TQC as shown above before installing the enclosure into the media bay slot. Simply attach the connectors as shown and feed them through the media bay opening as you install the CSE-M14TQC enclosure. Then attach the drive connectors to your HBA, RAID card or server motherboard and the power connector to your power source inside the server.

    Note and disclaimer, pay attention to your server manufactures power loading and specification along with how much power will be used by the HDD or SSD’s to be installed to avoid electrical power or fire issues due to overloading!

    Via StorageIOblog Supermicro CSE-M14TQC enclosure Lenovo TS140
    CSE-M14TQC installed into Lenovo TS140 empty media bay

    Via StorageIOblog Supermicro CSE-M14TQC drive enclosure Lenovo TS140

    CSE-M14TQC installed with front face plated installed on Lenovo TS140

    Where to read, watch and learn more

    Storage I/O trends

    What this all means and wrap up

    If you have a server that simply needs some extra storage capacity by adding some 2.5" HDDs, or boosting performance with fast SSDs yet do not have any more internal drive slots or expansion bays, leverage your media bay. This applies to smaller environments where you might have one or two servers, as well as for environments where you want or need to create a scale out software defined storage or hyper-converged platform using your own hardware. Another option is that if you have a lab or test environment for VMware vSphere ESXi Windows, Linux, Openstack or other things, this can be a cost-effective approach to adding both storage space capacity as well as performance and leveraging newer 12Gbps SAS technologies.

    For example, create a VMware VSAN cluster using smaller servers such as Lenovo TS140 or equivalent where you can install a couple of 6TB or 8TB higher capacity 3.5" drive in the internal drive bays, then adding a couple of 12 Gbps SAS SSDs along with a couple of 2.5" 2TB (or larger) HDDs along with a RAID card, and high-speed networking card. If VMware VSAN is not your thing, how about setting up a Windows Server 2012 R2 failover cluster including Scale Out File Server (SOFS) with Hyper-V, or perhaps OpenStack or one of many other virtual storage appliances (VSA) or software defined storage, networking or other solutions. Perhaps you need to deploy more storage for a big data Hadoop based analytics system, or cloud or object storage solution? On the other hand, if you simply need to add some storage to your storage or media or gaming server or general purpose server, the CSE-M14TQC can be an option along with other external solutions.

    Ok, nuff said

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    July 2015 Server StorageIO Update Newsletter

    Volume 15, Issue VII

    Hello and welcome to this July 2015 Server StorageIO update newsletter. Its mid summer here in the northern hemisphere which for many means vacations or holidays.

    Content Solution Platforms

    Thus this months newsletter has a focus on content solution platforms including hardware and software that get defined to support various applications. Content solutions span from video (4K, HD and legacy streaming, pre-/post-production and editing), audio, imaging (photo, seismic, energy, healthcare, etc.) to security surveillance (including Intelligent Video Surveillance [ISV] as well as Intelligence Surveillance and Reconnaissance [ISR]).

    StorageIOblog posts

    In case you missed it:

    View other recent as well as past blog posts here

    From StorageIO Labs

    Research, Reviews and Reports

    Servers Direct Content Platform
    Servers Direct Content Solution Platform

    An industry and customer trend is leveraging converged platforms based on multi-socket processors with dozens of cores and threads (logical processors) to support parallel or high-concurrent threaded content based applications.

    Recently I had the opportunity by Servers Direct to get some hands-on test time with one of their 2U Content Solution platforms. In addition to big fast data, other content solution applications include: content distribution network (CDN) content caching, network function virtualization (NFV), software-defined network (SDN), cloud rich unstructured big fast media data, analytics and little data (e.g. SQL and NoSQL database, key-value stores, repositories and meta-data) among others.

    Read more about content solution platforms including those Intel powered platforms from Servers Direct in this Server StorageIO Industry Trends Perspective solution brief here.

    View other Server StorageIO lab review reports here

    Closing Comments

    Enjoy this edition of the Server StorageIO update newsletter and watch for new tips, articles, StorageIO lab report reviews, blog posts, videos and podcast’s along with in the news commentary appearing soon.

    Cheers gs

    Greg Schulz – @StorageIO

    Microsoft MVP File System Storage
    VMware vExpert

    In This Issue

  • Industry Trends News
  • Commentary in the news
  • Tips and Articles
  • StorageIOblog posts
  • Server StorageIO Lab reviews
  • Events and Webinars
  • Resources and Links

    • StorageIO Commentary in the news

    StorageIO news (image licensed for use from Shutterstock by StorageIO)
    Recent Server StorageIO commentary and industry trends perspectives about news, activities and announcements.

    Processor: A Look At Object-Based Storage
    Processor: Newest and best server trends
    PowerMore: Flash not just for performance
    SearchVirtualStorage: Containers and storage
    BizTechMagazine: Simplify with virtualization
    EnterpriseStorageForum: Future DR Storage
    EnterpriseStorageForum: 10 Tips for DRaaS
    EnterpriseStorageForum: NVMe planning

    View more trends comments here

    StorageIO Tips and Articles

    A common question I am asked is, “What is the best storage technology?” My routine answer is, “It depends!” During my recent Interop Las Vegas session “Smart Shopping for Your Storage Strategy” I addressed this very question. Read more in my tip Selecting Storage: Start With Requirements over at Network Computing.

    Check out these resources and links on server storage I/O performance and benchmarking tools. View more tips and articles here

    Various Industry Events

    Server Storage I/O Workshop Seminars
    Nijkerk Netherlands October 13-16 2015

    VMworld August 30-September 3 2015

    Flash Memory Summit August 11-13

    View other recent and upcoming events here

    Webinars

    BrightTalk Webinar – June 23 2015 9AM PT
    Server Storage I/O Innovation v2.015: Protect Preserve & Serve Your Information

    Resources and Links

    Check out these useful links and pages:
    storageio.com/links
    objectstoragecenter.com
    storageioblog.com/data-protection-diaries-main/
    storageperformance.us
    thessdplace.com
    storageio.com/raid
    storageio.com/ssd

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 Server StorageIO(TM) and UnlimitedIO All Rights Reserved

    Posted on

    VMware vCloud Air Server StorageIOlab Test Drive with videos

    Server Storage I/O trends

    VMware vCloud Air Server StorageIOlab Test Drive with videos

    Recently I was invited by VMware vCloud Air to do a free hands-on test drive of their actual production environment. Some of you may already being using VMware vSphere, vRealize and other software defined data center (SDDC) aka Virtual Server Infrastructure (VSI) or Virtual Desktop Infrastructure (VDI) tools among others. Likewise some of you may already be using one of the many cloud compute or Infrastructure as a Service (IaaS) such as Amazon Web Services (AWS) Elastic Cloud Compute (EC2), Centurylink, Google Cloud, IBM Softlayer, Microsoft Azure, Rackspace or Virtustream (being bought by EMC) among many others.

    VMware vCloud Air provides a platform similar to those just mentioned among others for your applications and their underlying resource needs (compute, memory, storage, networking) to be fulfilled. In addition, it should not be a surprise that VMware vCloud Air shares many common themes, philosophies and user experiences with the traditional on-premises based VMware solutions you may be familiar with.

    VMware vCloud Air overview

    You can give VMware vCloud Air a trial for free while the offer lasts by clicking here (service details here). Basically if you click on the link and register a new account for using VMware vCloud Air they will give you up to $500 USD in service credits to use in the real production environment while the offer lasts which iirc is through end of June 2015.

    Server StorageIO test drive VMware vCloud Air video I
    Click on above image to view video part I

    Server StorageIO test drive VMware vCloud Air part II
    Click on above image to view video part II

    What this means is that you can go and setup some servers with as many CPUs or cores, memory, Hard Disk Drive (HDD) or flash Solid State Devices (SSD) storage, external IP networks using various operating systems (Centos, Ubuntu, Windows 2008, 20012, 20012 R2) for free, or until you use up the service credits.

    Speaking of which, let me give you a bit of a tip or hint, even though you can get free time, if you provision a fast server with lots of fast SSD storage and leave it sit idle over night or over a weekend, you will chew up your free credits rather fast. So the tip which should be common sense is if you are going to do some proof of concepts and then leave things alone for a while, power the virtual cloud servers off to stretch your credits further. On the other hand, if you have something that you want to run on a fast server with fast storage over a weekend or longer, give that a try, just pay attention to your resource usage and possible charges should you exhaust your service credits.

    My Server StorageIO test drive mission objective

    For my test drive, I created a new account by using the above link to get the service credits. Note that you can use your regular VMware account with vCloud Air, however you wont get the free service credits. So while it is a few minutes of extra work, the benefit was worth it vs. simply using my existing VMware account and racking up more cloud services charges on my credit card. As part of this Server StorageIOlab test drive, I created two companion videos part I here and part II here that you can view to follow along and get a better idea of how vCloud works.

    VMware vCloud Air overview
    Phase one, create the virtual data center, database server, client servers and first setup

    My goal was to set up a simple Virtual Data Center (VDC) that would consist of five Windows 2012 R2 servers, one would be a MySQL database server with the other four being client application servers. You can download MySQL from here at Oracle as well as via other sources. For applications to simplify things I used Hammerdb as well as Benchmark Factory that is part of the Quest Toad tool set for database admins. You can download a free trial copy of Benchmark Factory here, and HammerDB here. Another tool that I used for monitoring the servers is Spotlight on Windows (SoW) which is also free here. Speaking of tools, here is a link to various server and storage I/O performance as well as monitoring tools.

    Links to tools that I used for this test-drive included:

    Setting up a virtual data center vdc
    Phase one steps and activity summary

    Summary of phase one of vdc
    Recap of what was done in phase one, watch the associated video here.

    After the initial setup (e.g. part I video here), the next step was to add some more virtual machines and take a closer look at the environment. Note that most of the work in setting up this environment was Windows, MySQL, Hammerdb, Benchmark Factory, Spotlight on Windows along with other common tools so their installation is not a focus in these videos or this post, perhaps a future post will dig into those in more depth.

    Summary of phase two of the vdc
    What was done during phase II (view the video here)

    VMware vCloud Air vdc trest drive

    There is much more to VMware vCloud Air and on their main site there are many useful links including overviews, how-too tutorials, product and service offering details and much more here. Besides paying attention to your resource usage and avoid being surprised by service charges, two other tips I can pass along that are also mentioned in the videos (here and here) is to pay attention what region you setup your virtual data centers in, second is have your network thought out ahead of time to streamline setting up the NAT and firewall as well as gateway configurations.

    Where to learn more

    Learn more about data protection and related topics, themes, trends, tools and technologies via the following links:

    Server Storage I/O trends

    What this all means and wrap-up

    Overall I like the VMware vCloud Air service which if you are VMware centric focused will be a familiar cloud option including integration with vCloud Director and other tools you may already have in your environment. Even if you are not familiar with VMware vSphere and associated vRealize tools, the vCloud service is intuitive enough that you can be productive fairly quickly. On one hand vCloud Air does not have the extensive menu of service offerings to choose from such as with AWS, Google, Azure or others, however that also means a simpler menu of options to choose from and simplify things.

    I had wanted to spend some time actually using vCloud and the offer to use some free service credits in the production environment made it worth making the time to actually setup some workloads and do some testing. Even if you are not a VMware focused environment, I would recommend giving VMware vCloud Air a test drive to see what it can do for you, as opposed to what you can do for it…

    Ok, nuff said for now

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    S3motion Buckets Containers Objects AWS S3 Cloud and EMCcode

    Storage I/O trends

    S3motion Buckets Containers Objects AWS S3 Cloud and EMCcode

    It’s springtime in Kentucky and recently I had the opportunity to have a conversation with Kendrick Coleman to talk about S3motion, Buckets, Containers, Objects, AWS S3, Cloud and Object Storage, node.js, EMCcode and open source among other related topics which are available in a podcast here, or video here and available at StorageIO.tv.

    In this Server StorageIO industry trends perspective podcast episode, @EMCcode (Part of EMC) developer advocate Kendrick Coleman (@KendrickColeman) joins me for a conversation. Our conversation spans spring-time in Kentucky (where Kendrick lives) which means Bourbon and horse racing as well as his blog (www.kendrickcoleman.com).

    Btw, in the podcast I refer to Captain Obvious and Kendrick’s beard, for those not familiar with who or what @Captainobvious is that is made reference to, click here to learn more.


    @Kendrickcoleman     & @Captainobvious

    What about Clouds Object Storage Programming and other technical stuff?

    Of course we also talk some tech including what is EMCcode, EMC Federation, Cloud Foundry, clouds, object storage, buckets, containers, objects, node.js, Docker, Openstack, AWS S3, micro services, and the S3motion tool that Kendrick developed.

    Cloud and Object Storage Access
    Click to view video

    Kendrick explains the motivation behind S3motion along with trends in and around objects (including GET, PUT vs. traditional Read, Write) as well as programming among related topic themes and how context matters.

    S3motion for AWS S3 Google and object storage
    Click to listen to podcast

    I have used S3motion for moving buckets, containers and objects around including between AWS S3, Google Cloud Storage (GCS) and Microsoft Azure as well as to/from local. S3motion is a good tool to have in your server storage I/O tool box for working with cloud and object storage along with others such as Cloudberry, S3fs, Cyberduck, S3 browser among many others.

    You can get S3motion free from git hub here.

    Amazon Web Services AWS

    Where to learn more

    Here are some links to learn more about AWS S3, Cloud and Object Storage along with related topics

    Also available on

    What this all means and wrap-up

    Context matters when it comes to many things particular about objects as they can mean different things. Tools such as S3motion make it easy for moving your buckets or containers along with objects from one cloud storage system, solution or service to another. Also check out EMCcode to see what they are doing on different fronts from supporting new and greenfield development with Cloud Foundry and PaaS to Openstack to bridging current environments to the next generation of platforms. Also check out Kendricks blog site as he has a lot of good technical content as well as some other fun stuff to learn about. Look forward to having Kendrick on as a guest again soon to continue our conversations. In the meantime, check out S3motion to see how it can fit into your server storage I/O tool box.

    Ok, nuff said, for now..

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Microsoft Diskspd (Part II): Server Storage I/O Benchmark Tools

    Microsoft Diskspd (Part II): Server Storage I/O Benchmark Tools

    server storage I/O trends

    This is part-two of a two-part post pertaining Microsoft Diskspd.that is also part of a broader series focused on server storage I/O benchmarking, performance, capacity planning, tools and related technologies. You can view part-one of this post here, along with companion links here.

    Microsoft Diskspd StorageIO lab test drive

    Server and StorageIO lab

    Talking about tools and technologies is one thing, installing as well as trying them is the next step for gaining experience so how about some quick hands-on time with Microsoft Diskspd (download your copy here).

    The following commands all specify an I/O size of 8Kbytes doing I/O to a 45GByte file called diskspd.dat located on the F: drive. Note that a 45GByte file is on the small size for general performance testing, however it was used for simplicity in this example. Ideally a larger target storage area (file, partition, device) would be used, otoh, if your application uses a small storage device or volume, then tune accordingly.

    In this test, the F: drive is an iSCSI RAID protected volume, however you could use other storage interfaces supported by Windows including other block DAS or SAN (e.g. SATA, SAS, USB, iSCSI, FC, FCoE, etc) as well as NAS. Also common to the following commands is using 16 threads and 32 outstanding I/Os to simulate concurrent activity of many users, or application processing threads.
    server storage I/O performance
    Another common parameter used in the following was -r for random, 7200 seconds (e.g. two hour) test duration time, display latency ( -L ) disable hardware and software cache ( -h), forcing cpu affinity (-a0,1,2,3). Since the test ran on a server with four cores I wanted to see if I could use those for helping to keep the threads and storage busy. What varies in the commands below is the percentage of reads vs. writes, as well as the results output file. Some of the workload below also had the -S option specified to disable OS I/O buffering (to view how buffering helps when enabled or disabled). Depending on the goal, or type of test, validation, or workload being run, I would choose to set some of these parameters differently.

    diskspd -c45g -b8K -t16 -o32 -r -d7200 -h -w0 -L -a0,1,2,3 F:\diskspd.dat >> SIOWS2012R203_Eiscsi_145_noh_write000.txt

    diskspd -c45g -b8K -t16 -o32 -r -d7200 -h -w50 -L -a0,1,2,3 F:\diskspd.dat >> SIOWS2012R203_Eiscsi_145_noh_write050.txt

    diskspd -c45g -b8K -t16 -o32 -r -d7200 -h -w100 -L -a0,1,2,3 F:\diskspd.dat >> SIOWS2012R203_Eiscsi_145_noh_write100.txt

    diskspd -c45g -b8K -t16 -o32 -r -d7200 -h -S -w0 -L -a0,1,2,3 F:\diskspd.dat >> SIOWS2012R203_Eiscsi_145_noSh_test_write000.txt

    diskspd -c45g -b8K -t16 -o32 -r -d7200 -h -S -w50 -L -a0,1,2,3 F:\diskspd.dat >> SIOWS2012R203_Eiscsi_145_noSh_write050.txt

    diskspd -c45g -b8K -t16 -o32 -r -d7200 -h -S -w100 -L -a0,1,2,3 F:\diskspd.dat >> SIOWS2012R203_Eiscsi_145_noSh_write100.txt

    The following is the output from the above workload command.
    Microsoft Diskspd sample output
    Microsoft Diskspd sample output part 2
    Microsoft Diskspd sample output part 3

    Note that as with any benchmark, workload test or simulation your results will vary. In the above the server, storage and I/O system were not tuned as the focus was on working with the tool, determining its capabilities. Thus do not focus on the performance results per say, rather what you can do with Diskspd as a tool to try different things. Btw, fwiw, in the above example in addition to using an iSCSI target, the Windows 2012 R2 server was a guest on a VMware ESXi 5.5 system.

    Where to learn more

    The following are related links to read more about server (cloud, virtual and physical) storage I/O benchmarking tools, technologies and techniques.

    Drew Robb’s benchmarking quick reference guide
    Server storage I/O benchmarking tools, technologies and techniques resource page
    Server and Storage I/O Benchmarking 101 for Smarties.
    Microsoft Diskspd download and Microsoft Diskspd overview (via Technet)
    I/O, I/O how well do you know about good or bad server and storage I/Os?
    Server and Storage I/O Benchmark Tools: Microsoft Diskspd (Part I and Part II)

    Comments and wrap-up

    What I like about Diskspd (Pros)

    Reporting including CPU usage (you can’t do server and storage I/O without CPU) along with IOP’s (activity), bandwidth (throughout or amount of data being moved), per thread and total results along with optional reporting. While a GUI would be nice particular for beginners, I’m used to setting up scripts for different workloads so having an extensive options for setting up different workloads is welcome. Being associated with a specific OS (e.g. Windows) the CPU affinity and buffer management controls will be handy for some projects.

    Diskspd has the flexibility to use different storage interfaces and types of storage including files or partitions should be taken for granted, however with some tools don’t take things for granted. I like the flexibility to easily specify various IO sizes including large 1MByte, 10MByte, 20MByte, 100MByte and 500MByte to simulate application workloads that do large sequential (or random) activity. I tried some IO sizes (e.g. specified by -b parameter larger than 500MB however, I received various errors including "Could not allocate a buffer bytes for target" which means that Diskspd can do IO sizes smaller than that. While not able to do IO sizes larger than 500MB, this is actually impressive. Several other tools I have used or with have IO size limits down around 10MByte which makes it difficult for creating workloads that do large IOP’s (note this is the IOP size, not the number of IOP’s).

    Oh, something else that should be obvious however will state it, Diskspd is free unlike some industry de-facto standard tools or workload generators that need a fee to get and use.

    Where Diskspd could be improved (Cons)

    For some users a GUI or configuration wizard would make the tool easier to get started with, on the other hand (oth), I tend to use the command capabilities of tools. Would also be nice to specify ranges as part of a single command such as stepping through an IO size range (e.g. 4K, 8K, 16K, 1MB, 10MB) as well as read write percentages along with varying random sequential mixes. Granted this can easily be done by having a series of commands, however I have become spoiled by using other tools such as vdbench.

    Summary

    Server and storage I/O performance toolbox

    Overall I like Diskspd and have added it to my Server Storage I/O workload and benchmark tool-box

    Keep in mind that the best benchmark or workload generation technology tool will be your own application(s) configured to run as close as possible to production activity levels.

    However when that is not possible, the an alternative is to use tools that have the flexibility to be configured as close as possible to your application(s) workload characteristics. This means that the focus should not be as much on the tool, as opposed to how flexible is a tool to work for you, granted the tool needs to be robust.

    Having said that, Microsoft Diskspd is a good and extensible tool for benchmarking, simulation, validation and comparisons, however it will only be as good as the parameters and configuration you set it up to use.

    Check out Microsoft Diskspd and add it to your benchmark and server storage I/O tool-box like I have done.

    Ok, nuff said (for now)

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Server storage I/O Intel NUC nick knack notes – First impressions

    Storage I/O trends

    Server storage I/O Intel NUC nick knack notes – First impressions

    This is the first of a two-part (part II here) series of my experiences (and impressions) using an Intel NUC ( a 4th generation model) for various things about cloud, virtual, physical and software defined server storage I/O networking.

    The NUC has been around new for a few years and continues to evolve and recently I bought my first one (e.g. a 4th generation model) to join some other servers that I have. My reason for getting a NUC is to use it as a simple low-power platform to run different software on including bare-metal OS, hypervisors, cloud, virtual and software defined server storage and networking applications on that might otherwise be on an old laptop or mini-tower.

    Intel® NUC with Intel® Core™ i5 Processor and 2.5-Inch Drive Support (NUC5i5RYH) via Intel.com

    Introducing Intel Next Unit Computing aka NUC

    For those not familiar, NUC is a series of products from Intel called Next Unit Computing that offer an alternative to traditional mini-desktop or even laptop and notebooks. There are several different NUC models available including the newer 5th generation models (click here to see various models and generations). The NUCs are simple, small units of computing with an Intel processor and room for your choice of memory, persistent storage (e.g. Hard Disk Drive (HDD) or flash Solid State Device (SSD), networking, video, audio and other peripheral device attachment.

    software (not supplied) is defined by what you choose to use such as a Windows or *nix operating system, VMware ESXi, Microsoft Hyper-V, KVM or Xen hypervisor, or some other applications. The base NUC package includes front and rear-side ports for attaching various devices. In terms of functionality, think of a laptop without a keyboard or video screen, or in terms of a small head-less (e.g. no monitor) mini-tower desktop workstation PC.

    Which NUC to buy?

    If you need to be the first with anything new, then jump direct to the recently released 5th generation models.

    On the other hand, if you are looking for a bargain, there are some good deals on 4th generation or older. likewise depending on your processor speed and features needed along with available budget, those criteria and others will direct you to a specific NUC model.

    I went with a 4th generation NUC realizing that the newer models were just around the corner as I figured could always get another (e.g. create a NUC cluster) newer model when needed. In addition I also wanted a model that had enough performance to last a few years of use and the flexibility to be reconfigured as needed. My choice was a model D54250WYK priced around $352 USD via Amazon (prices may vary by different venues).

    Whats included with a NUC?

    My first NUC is a model D54250WYK (e.g. BOXD54250WYKH1 ) that you can view the specific speeds and feeds here at the Intel site along with ordering info here at Amazon (or your other preferred venue).

    View and compare other NUC models at the Intel NUC site here.

    The following images show the front-side two USB 3.0 ports along with head-phone (or speaker) and microphone jacks. Looking at the rear-side of the NUC there are a couple of air vents, power connector port (external power supply), mini-display and HDMI video port, GbE LAN, and two USB 3.0 ports.

    NUC front viewRear ports of NUC
    Left is front view of my NUC model 54250 and Right is back or rear view of NUC

    NUC ModelBOXD54250WYKH1 (speeds/feeds vary by specific model)
    Form factor1.95" tall
    ProcessorIntel Core i5-4250U with active heat sink fan
    MemoryTwo SO-DIMM DDR3L (e.g. laptop) memory, up to 16GB (e.g. 2x8GB)
    DisplayOne mini DisplayPort with audio
    One mini HDMI port with audio
    AudioIntel HD Audio, 8 channel (7.1) digital audio via HDMI and DisplayPort, also headphone jack
    LANIntel Gigabit Ethernet (GbE) (I218)
    Peripheral and storageTwo USB 3.0 (e.g. blue) front side
    Two USB 3.0 rear side
    Two USB 2.0 (internal)

    One SATA port (internal 2.5 inch drive bay)

    Consumer infrared sensor (front panel)
    ExpansionOne full-length mini PCI Express slot with mSATA support
    One half-length mini PCI Express slot
    Included in the boxLaptop style 19V 65W power adapter (brick) and cord, VESA mounting bracket (e.g. for mounting on rear of video monitor), integration (installation) guide, wireless antennae (integrated into chassis), Intel Core i5 logo
    Warranty3-year limited

    Processor Speeds and Feeds

    There are various Intel Core i3 and i5 processors available depending on specific NUC model, such as my 54250WYK has a two core (1.3Ghz each) 4th generation i5-4250U (click here to see Intel speeds and feeds) which includes Intel Visual BIOS, Turbo Boost, Rapid Start and virtualization support among other features.

    Note that features vary by processor type, along with other software, firmware or BIOS updates. While the 1.3Ghz two core (e.g. max 2.6Ghz) is not as robust as faster quad (or more) cores running at 3.0Ghz (or faster), for most applications including as a first virtual lab or storage sand box among other uses, it will be fast enough or comparable to a lower-mid range laptop capabilities.

    What this all means

    In general I like the NUC so much that I bought one (model 54250) and would consider adding another in the future for somethings, however also see the need to continue using my other compute servers for different workloads.

    This wraps up part I of this two-part series and what this means is that I like the idea of a Intel NUC I bought one. Continue reading in part-two here where I cover the options that I added to my NUC, initial configuration, deployment, use and additional impressions.

    Ok, nuff said for now, check out part-two here.

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    Server storage I/O Intel NUC nick knack notes – Second impressions

    Storage I/O trends

    Server storage I/O Intel NUC nick knack notes – Second impressions

    This is the second of a two-part series about my first and second impressions of the Intel NUC (Next Unit Computing). In the first post (here) I give an overview and my first impressions while in this post lets look at options added to my NUC model 54250, first deployment use and more impressions.

    Intel® NUC with Intel® Core™ i5 Processor and 2.5-Inch Drive Support (NUC5i5RYH) via Intel.com

    What you will want to add to a NUC

    Since the NUC is a basic brick with a processor mounted on its mother board, you will need to add memory, some type of persistent storage device (mSATA, SATA or USB based) and optionally a WiFi card.

    One of the nice things about the NUC is that in many ways it is the equivalent functionality of a laptop or mini-tower without the extra overhead (cost, components, packaging) enabling you to customize as needed for your specific requirements. For example there is no keyboard, mouse, video screen, WiFi, Hard Disk Drive (HDD) or flash Solid State Device (SSD) included with an operating system pre-installed. There is no least memory required enabling you to decide how much to configure while using compatible laptop style memory. Video and monitors attach via HDMI or mini-port including VGA devices via an adapter cable. Keyboard and mouse if needed are handled via USB ports.

    Here is what I added to my NUC model 5420.

    1Crucial 16GB Kit (2 x 8GB) DDR3 1600 (PC3-12800) SODIMM 204-Pin Notebook Memory
    1Intel Network 7260 WiFi Wireless-AC 7260 H/T Dual Band 2×2 AC+Bluetooth HMC. Here is link to Intel site for various drivers.
    1500GB Samsung Electronics 840 EVO mSATA 0.85-Inch Solid State Drive
    1SATA HDD, SSD or HHDD/SSHD (I used one of my existing drives)

    Note that you will also need to supply some type of Keyboard Video Mouse (KVM), in my case I used a HDMI to VGA adapter cable to attach the NUC via HDMI (for video) and USB (keyboard and mouse) to my Startech KVM switch.

    Following images show on the left Intel WiFi card installed and on the right, a Samsung 840 EVO mSATA 500GB flash SSD installed above the WiFi card. Also notice on the far right of the images the two DDR3 "notebook" class DRAM DIMM slots.

    NUC WiFi cardmSATA SSD
    Left: Intel WiFi card installed and Right Samsung EVO mSATA SSD card (sits above WiFi card)

    Note that the NUC (as do many laptops) accepts 9mm or smaller thin 7mm height HDDs and SSDs in its SATA drive bay. I mention this because some of the higher-capacity 2TB 2.5" SFF drives are taller than 9m as shown in the above image and do not fit in the NUC internal SATA drive bay. While many devices and systems support 2.5" drive slots for HDD, SSD or HHDD/SSHDs, pay attention to the height and avoid surprises when something does not fit like it was assumed to.

    2.5 HDD and SSDs
    Low-profile and tall-profile 2.5" SFF HDDs

    Additional drives and devices can be attached using external USB 3.0 ports including HDDs, SSDs or even USB to GbE adapters if needed. You will need to supply your own operating system, hypervisor, storage, networking or other software, such as Windows, *nix, VMware ESXi, Hyper-V, KVM, Xen, OpenStack or any of the various ZFS based (among others) storage appliances.

    Unpacking and physical NUC installation

    Initial setup and physical configuration of the NUC is pretty quick with the only tool needed being a Philips screw driver.

    NUC and components ready for installation
    Intel NUC 5420 and components ready for installation

    With all the components including the NUC itself laid out for a quick inventory including recording serial numbers (see image above), the next step is to open up the NUC by removing four Philip screws from the bottom. Once the screws are removed and bottom plate removed, the SATA drive bay opens up to reach the slots of memory, mSATA SSD and WiFi card (see images below). Once the memory, mSATA and WiFi cards are installed, the SATA drive bay coverage those components and it is time to install a 2.5" standard height HDD or SSD. For my first deployment I installed temporarily installed on of my older HHDDs a 750GB Seagate Momentus XT that will be replaced by something newer soon.

    NUC internal HDD/SSD slotNUC internal HDD installed
    View of NUC with bottom cover removed, Left empty SATA drive bay, Right HDD installed

    After the components are installed, it is time to replace the bottom cover plate of the NUC securing in place with the four screws previously removed. Next up is attaching any external devices via USB and other ports including KVM and LAN network connection. Once the hardware is ready, its time to power up the NUC and checkout the Visual BIOS (or UEFI) as shown below.

    Intel NUC Visual BIOSIntel NUC Visual BIOS display
    NUC VisualBIOS screen shot examples

    At this point unless you have already installed an operating system, hypervisor or other software on a HDD, SSD or USB device, it is time to install your prefered software.

    Windows 7

    First up was Windows 7 as I already had an image built on the HHDD that required some drivers to be added. specifically, a visit to the Intel resources site (See NUC resources and links section later in this post) was made to get a LAN GbE, WiFi and USB drivers. Once those were installed the on-board GbE LAN port worked good as did the WiFi. Another driver that needed to be download was for a USB-GbE adapter to add another LAN connection. Also a couple of reboots were required for other Windows drivers and configuration changes to take place to correct some transient problems including KVM hangs which eventually cleared themselves up.

    Windows 2012 R2

    Following Windows 7, next up was a clean install of Windows 2012 R2 which also required some drivers and configuration changes. One of the challenges is that Windows 2012 R2 is not officially supported on the NUC with its GbE LAN and WiFi cards. However after doing some searches and reading a few posts including this and this, a solution was found and Windows 2012 R2 and its networking are working good.

    Ubuntu and Clonezilla

    Next up was a quick install of Ubuntu 14.04 which went pretty smooth, as well as using Clonezilla to do some drive maintenance, move images and partitions among other things.

    VMware ESXi 5.5U2

    My first attempt at installing a standard VMware ESXi 5.5U2 image ran into problems due to the GbE LAN port not being seen. The solution is to use a different build, or custom ISO that includes the applicable GbE LAN driver (e.g. net-e1000e-2.3.2.x86_64.vib) and some useful information at Florian Grehl site (@virten) and over at Andreas Peetz site (@VFrontDe) including SATA controller driver for xahci. Once the GbE driver was added (same driver that addresses other Intel NIC I217/I218 based systems) along with updating the SATA driver, VMware worked fine.

    Needless to say there are many other things I plan on doing with the NUC both as a standalone bare-metal system as well as a virtual platform as I get more time and projects allow.

    What about building your NUC alternative?

    In addition to the NUC models available via Intel and its partners and accessorizing as needed, there are also special customized and ruggedized NUC versions similar to what you would expect to find with laptop, notebooks, and other PC based systems.

    MSI Probox rear viewMSI Probox front view
    Left MSI ProBox rear-view Right MSI ProBox front view

    If you are looking to do more than what Intel and its partners offer, then there are some other options such as to increase the number of external ports among other capabilities. One option which I recently added to my collection of systems is an DIY (Do It Yourself) MSI ProBox (VESA mountable) such as this one here.

    MSI Probox internal view
    Internal view MSI ProBox (no memory, processor or disks)

    With the MSI ProBox, they are essentially a motherboard with an empty single cpu socket (e.g. LGA 1150 up to 65W) for supporting various processors, two empty DDR3 DIMM slots, 2 empty 2.5" SATA ports among other capabilities. Enclosures such as the MSI ProBox give you flexibility creating something more robust beyond a basic NUC yet smaller than a traditional server depending on your specific needs.

    Looking for other small form factor modular and ruggedized server options as an alternative to a NUC, than check out those from Xi3, Advantech, Cadian Networks, and Logic Supply among many others.

    Storage I/O trends

    First NUC impressions

    Overall I like the NUC and see many uses for it from consumer, home including entertainment and media systems, video security surveillance as well as a small server or workstation device. In addition, I can see a NUC being used for smaller environments as desktop workstations or as a lower-power, lower performance system including as a small virtualization host for SOHO, small SMB and ROBO environments. Another usage is for home virtual lab as well as gaming among other scenarios including simple software defined storage proof of concepts. For example, how about creating a small cluster of NUCs to run VMware VSAN, or Datacore, EMC ScaleIO, Starwind, Microsoft SOFS or Hyper-V as well as any of the many ZFS based NAS storage software applications.

    Pro’s – Features and benefits

    Small, low-power, self-contained with flexibility to choose my memory, WiFi, storage (HDD or SSD) without the extra cost of those items or software being included.

    Con’s – Caveats or what to look out for

    Would be nice to have another GbE LAN port however I addressed that by adding a USB 3.0 to GbE cable, likewise would be nice if the 2.5" SATA drive bay supported tall height form-factor devices such as the 2TB devices. The work around for adding larger capacity and physically larger storage devices is to use the USB 3.0 ports. The biggest warning is if you are going to venture outside of the official supported operating system and application software realm be ready to load some drivers, possibly patch and hack some install scripts and then plug and pray it all works. So far I have not run into any major show stoppers that were not addressed with some time spent searching (google will be your friend), then loading the drivers or making configuration changes.

    Additional NUC resources and links

    Various Intel products support search page
    Intel NUC support and download links
    Intel NUC model 54250 page, product brief page (and PDF version), and support with download links
    Intel NUC home theater solutions guide (PDF)
    Intel HCL for NUC page and Intel Core i5-4250U processor speeds and feeds
    VMware on NUC tips
    VMware ESXi driver for LAN net-e1000e-2.3.2.x86_64
    VMware ESXi SATA xahci driver
    Server storage I/O Intel NUC nick knack notes – First impressions
    Server Storage I/O Cables Connectors Chargers & other Geek Gifts (Part I and Part II)
    Software defined storage on a budget with Lenovo TS140

    Storage I/O trends

    What this all means

    Intel NUC provides a good option for many situations that might otherwise need a larger mini-tower desktop workstations or similar systems both for home, consumer and small office needs. NUC can also be used for specialized pre-configured application specific situations that need low-power, basic system functionality and expansion options in a small physical footprint. In addition NUC can also be a good option for adding to an existing physical and virtual LAB or as a basis for starting a new one.

    So far I have found many uses for NUC which free up other systems to do other tasks while enabling some older devices to finally be retired. On the other hand like most any technology, while the NUC is flexible, its low power and performance are not enough to support other applications. However the NUC gives me flexibility to leverage the applicable unit of compute (e.g. server, workstation, etc.) that is applicable to a given task or put another way, use the right technology tool for the task at hand.

    For now I only need a single NUC to be a companion to my other HP, Dell and Lenovo servers as well as MSI ProBox, however maybe there will be a small NUC cluster, grid or ring configured down the road.

    What say you, do you have a NUC if so, how is it being used and tips, tricks or hints to share with others?

    Ok, nuff said for now.

    Cheers
    Gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2023 StorageIO and UnlimitedIO LLC All Rights Reserved

    Posted on

    DIY converged server software defined storage on a budget using Lenovo TS140

    Attention DIY Converged Server Storage Bargain Shoppers

    Software defined storage on a budget with Lenovo TS140

    server storage I/O trends

    Recently I put together a two-part series of some server storage I/O items to get a geek for a gift (read part I here and part II here) that also contain items that can be used for accessorizing servers such as the Lenovo ThinkServer TS140.

    Image via Lenovo.com

    Likewise I have done reviews of the Lenovo ThinkServer TS140 in the past which included me liking them and buying some (read the reviews here and here), along with a review of the larger TD340 here.

    Why is this of interest

    Do you need or want to do a Do It Yourself (DIY) build of a small server compute cluster, or a software defined storage cluster (e.g. scale-out), or perhaps a converged storage for VMware VSAN, Microsoft SOFS or something else?

    Do you need a new server, second or third server, or expand a cluster, create a lab or similar and want the ability to tailor your system without shopping or a motherboard, enclosure, power supply and so forth?

    Are you a virtualization or software defined person looking to create a small VMware Virtual SAN (VSAN) needing three or more servers to build a proof of concept or personal lab system?

    Then the TS140 could be a fit for you.

    storage I/O Lenovo TS140
    Image via StorageIOlabs, click to see review

    Why the Lenovo TS140 now?

    Recently I have seen a lot of site traffic on my site with people viewing my reviews of the Lenovo TS140 of which I have a few. In addition have got questions from people via comments section as well as elsewhere about the TS140 and while shopping at Amazon.com for some other things, noticed that there were some good value deals on different TS140 models.

    I tend to buy the TS140 models that are bare bones having power supply, enclosure, CD/DVD, USB ports, power supply and fan, processor and minimal amount of DRAM memory. For processors mine have the Intel E3-1225 v3 which are quad-core and that have various virtualization assist features (e.g. good for VMware and other hypervisors).

    What I saw on Amazon the other day (also elsewhere) were some Intel i3-4130 dual core based systems (these do not have all the virtualization features, just the basics) in a bare configuration (e.g. no Hard Disk Drive (HDD), 4GB DRAM, processor, mother board, power supply and fan, LAN port and USB with a price of around $220 USD (your price may vary depending on timing, venue, prime or other membership and other factors). Not bad for a system that you can tailor to your needs. However what also caught my eye were the TS140 models that have the Intel E3-1225 v3 (e.g. quad core, 3.2Ghz) processor matching the others I have with a price of around $330 USD including shipping (your price will vary depending on venue and other factors).

    What are some things to be aware of?

    Some caveats of this solution approach include:

    • There are probably other similar types of servers, either by price, performance, or similar
    • Compare apples to apples, e.g. same or better processor, memory, OS, PCIe speed and type of slots, LAN ports
    • Not as robust of a solution as those you can find costing tens of thousands of dollars (or more)
    • A DIY system which means you select the other hardware pieces and handle the service and support of them
    • Hardware platform approach where you choose and supply your software of choice
    • For entry-level environments who have floor-space or rack-space to accommodate towers vs. rack-space or other alternatives
    • Software agnostic Based on basically an empty server chassis (with power supplies, motherboard, power supplies, PCIe slots and other things)
    • Possible candidate for smaller SMB (Small Medium Business), ROBO (Remote Office Branch Office), SOHO (Small Office Home Office) or labs that are looking for DIY
    • A starting place and stimulus for thinking about doing different things

    What could you do with this building block (e.g. server)

    Create a single or multi-server based system for

    • Virtual Server Infrastructure (VSI) including KVM, Microsoft Hyper-V, VMware ESXi, Xen among others
    • Object storage
    • Software Defined Storage including Datacore, Microsoft SOFS, Openstack, Starwind, VMware VSAN, various XFS and ZFS among others
    • Private or hybrid cloud including using Openstack among other software tools
    • Create a hadoop big data analytics cluster or grid
    • Establish a video or media server, use for gaming or a backup (data protection) server
    • Update or expand your lab and test environment
    • General purpose SMB, ROBO or SOHO single or clustered server

    VMware VSAN server storageIO example

    What you need to know

    Like some other servers in this class, you need to pay attention to what it is that you are ordering, check out the various reviews, comments and questions as well as verify the make, model along with configuration. For example what is included and what is not included, warranty, return policy among other things. In the case of some of the TS140 models, they do not have a HDD, OS, keyboard, monitor, mouse along with different types of processors and memory. Not all the processors are the same, pay attention, visit the Intel Ark site to look up a specific processor configuration to see if it fits your needs as well as visit the hardware compatibility list (HCL) for the software that you are planning to use. Note that these should be best practices regardless of make, model, type or vendor for server, storage, I/O networking hardware and software.

    What you will need

    This list assumes that you have obtained a model without a HDD, keyboard, video, mouse or operating system (OS) installed

    • Update your BIOS if applicable, check the Lenovo site
    • Enable virtualization and other advanced features via your BIOS
    • Software such as an Operating System (OS), hypervisor or other distribution (load via USB or CD/DVD if present)
    • SSD, SSHD/HHDD, HDD or USB flash drive for installing OS or other software
    • Keyboard, video, mouse (or a KVM switch)

    What you might want to add (have it your way)

    • Keyboard, video mouse or a KVM switch (See gifts for a geek here)
    • Additional memory
    • Graphics card, GPU or PCIe riser
    • Additional SSD, SSHD/HHDD or HDD for storage
    • Extra storage I/O and networking ports

    Extra networking ports

    You can easily add some GbE (or faster ports) including use the PCIe x1 slot, or use one of the other slots for a quad port GbE (or faster), not to mention get some InfiniBand single or dual port cards such as the Mellanox Connectx II or Connect III that support QDR and can run in IBA or 10GbE modes. If you only have two or three servers in a cluster, grid, ring configuration you can run point to point topologies using InfiniBand (and some other network interfaces) without using a switch, however you decide if you need or want switched or non-switched (I have a switch). Note that with VMware (and perhaps other hypervisors or OS) you may need to update the drives for the Realtek GbE LAN on Motherboard port (see links below).

    Extra storage ports

    For extra storage space capacity (and performance) you can easily add PCIe G2 or G3 HBAs (SAS, SATA, FC, FCoE, CNA, UTA, IBA for SRP, etc) or RAID cards among others. Depending on your choice of cards, you can then attach to more internal storage, external storage or some combination with different adapters, cables, interposers and connectivity options. For example I have used TS140s with PCIe Gen 3 12Gbs SAS HBAs attached to 12Gbs SAS SSDs (and HDDs) with the ability to drive performance to see what those devices are capable of doing.

    TS140 Hardware Defined My Way

    As an example of how a TS140 can be configured, using one of the base E3-1224 v3 models with 4GB RAM, no HDD (e.g around $330 USD, your price will vary), add a 4TB Seagate HDD (or two or three) for around $140 USD each (your price will vary), add a 480GB SATA SSD for around $340 USD (your price will vary) with those attached to the internal SATA ports. To bump up network performance, how about a Mellanox Connectx II dual port QDR IBA/10GbE card for around $140 USD (your price will vary), plus around $65 USD for QSFP cable (you your price will vary), and some extra memory (use what you have or shop around) and you have a platform ready to go for around or under $1,000 USD. Add some more internal or external disks, bump up the memory, put in some extra network adapters and your price will go up a bit, however think about what you can have for a robust not so little system. For you VMware vgeeks, think about the proof of concept VSAN that you can put together, granted you will have to do some DIY items.

    Some TS140 resources

    Lenovo TS140 resources include

    • TS140 StorageIOlab review (here and here)
    • TS140 Lenovo ordering website
    • TS140 Data and Spec Sheet (PDF here)
    • Lenovo ThinkServer TS140 Manual (PDF here) and (PDF here)
    • Intel E3-1200 v3 processors capabilities (Web page here)
    • Enabling Virtualization Technology (VT) in TS140 BIOS (Press F1) (Read here)
    • Enabling Intel NIC (82579LM) GbE with VMware (Link to user forum and a blog site here)

    Image via Lenovo.com

    What this all means

    Like many servers in its category (price, capabilities, abilities, packaging) you can do a lot of different things with them, as well as hardware define with accessories, or use your own software. Depending on how you end how hardware defining the TS140 with extra memory, HDDs, SSDs, adapters or other accessories and software your cost will vary. However you can also put together a pretty robust system without breaking your budget while meeting different needs.

    Is this for everybody? Nope

    Is this for more than a lab, experimental, hobbyist, gamer? Sure, with some caveats Is this apples to apples comparison vs. some other solutions including VSANs? Nope, not even close, maybe apples to oranges.

    Do I like the TS140? Yup, starting with a review I did about a year ago, I liked it so much I bought one, then another, then some more.

    Are these the only servers I have, use or like? Nope, I also have systems from HP and Dell as well as test drive and review others

    Why do I like the TS140? It’s a value for some things which means that while affordable (not to be confused with cheap) it has features, salability and ability to be both hardware defined for what I want or need to use them as, along with software define them to be different things. Key for me is the PCIe Gen 3 support with multiple slots (and types of slots), reasonable amount of memory, internal housing for 3.5" and 2.5" drives that can attach to on-board SATA ports, media device (CD/DVD) if needed, or remove to use for more HDDs and SSDs. In other words, it’s a platform that instead of shopping for the motherboard, an enclosure, power supply, processor and related things I get the basics, then configure, and reconfigure as needed.

    Another reason I like the TS140 is that I get to have the server basically my way, in that I do not have to order it with a smallest number of HDDs, or that it comes with an OS, more memory than needed or other things that I may or may not be able to use. Granted I need to supply the extra memory, HDDs, SSDs, PCIe adapters and network ports along with software, however for me that’s not too much of an issue.

    What don’t I like about the TS140? You can read more about my thoughts on the TS140 in my review here, or its bigger sibling the TD340 here, however I would like to see more memory slots for scaling up. Granted for what these cost, it’s just as easy to scale-out and after all, that’s what a lot of software defined storage prefers these days (e.g. scale-out).

    The TS140 is a good platform for many things, granted not for everything, that’s why like storage, networking and other technologies there are different server options for various needs. Exercise caution when doing apples to oranges comparison on price alone, compare what you are getting in terms of processor type (and its functionality), expandable memory, PCIe speed, type and number of slots, LAN connectivity and other features to meet your needs or requirements. Also keep in mind that some systems might be more expensive that include a keyboard, HDD with an OS installed that if you can use those components, then they have value and should be factored into your cost, benefit, return on investment.

    And yes, I just added a few more TS140s that join other recent additions to the server storageIO lab resources…

    Anybody want to guess what I will be playing with among other things during the up coming holiday season?

    Ok, nuff said, for now…

    Cheers gs

    Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
    twitter @storageio

    All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2024 Server StorageIO and UnlimitedIO LLC All Rights Reserved