Here’s two little tips to solve some small hardware issues you might run into with a Compellent SAN. But first, you’re never on your own with CoPilot support. They are just one phone call away so I suggest if you see these to minor issues you give them a call. I speak from experience that CoPilot rocks. They are really good and go the extra mile. Best storage support I have ever experienced.
Always notify CoPilot as they will see the alerts come in and will contact you for sure . Afterwards they’ll almost certainly will do a quick health check for you. But even better during the entire process they keep an eye on things to make sure you SAN is doing just fine. And if you feel you’d like them to tackle this, they will send out an engineer I’m sure.
Note that we’re talking about the SC40 controllers & disk bays here. The newer genuine DELL hardware is better than the super micro ones.
The audible alert without any issues what so ever
We kept getting an audible alert after we had long solved any issues on one of the SANs. The system had been checked a couple of times and everything was in perfect working order. Except for that audible alarm that just didn’t want to quit. A low priority issue I know but every time we walk into the data center we were going “oh oh” for a false alert. That’s not the kind of conditioning you want. Alerts are only to be made when needed and than they do need to be acted upon!
Working on this with CoPilot support we got rid of it by reseating the upper I/O module. You can do this on the fly – without pulling SAS-cables out or so, they are redundant, as long as you do it one by one and the cabling is done right (they can verify that remotely for you if needed).
But we got lucky after the first one. After the “Swap Clear” was requested every warning condition was cleared and we got rid of the audible alert beep! Copilot was on the line with us and made sure all paths are up and running so no bad things could happen. That’s what you have a copilot for.
Front panel display dimming out on a Compellent Disk Bay
We have multiple Compellent SANs and on one of those we had a disk bay with a info panel that didn’t light up anymore. A silly issue but an annoying one as this one also show you the disk bay ID.
Do we really replace the disk bay to solve this one? As that light had come on and of a couple of time it could just be a bad contact so my colleague decided to take a look. First he removed the protective cover and then, using some short & curved screw drivers, he took of the body part. The red arrow indicates the little latch that holds the small ribbon cable in place.
That was standing right open. After locking that down the info appeared again on the panel. The covers was screwed on again and voila. Solved.
This question came up recently, once again, and deserves it a little blog post. If you want to see the benefits of ODX you’ll need to connect your virtual disks to a vSCSI controller or other supported controller option. These are iSCSI, vFC, a SMB 3 File Share or a pass-through disk. But unless you have really good reason to use pass-through disks, don’t. It’s limiting you in to many ways.
Basically in generation 1 virtual machines that boot from a vIDE this rules out the system disk. So the tip here is to store your data that’s moved around in or between virtual machines in vSCSI attached VDH or (preferably) VHDX virtual disks. If you can use generation 2 virtual machines, you’ll be able to leveraged ODX on the system partition as well as it boots from vSCSI .
It goes without saying you need to store any virtual disks involved on ODX capable LUNs via iSCSI, FC, FCoE, SMB 3 File Share or SAS for ODX to be available to the virtual machine.
Also beware that ODX only works on NTFS partitioned disks. The files cannot be compressed or encrypted. Sparse files are not supported either. And finally, the volume cannot be BitLocker protected.
Here’s a screenshot of a copy of 30GB worth of ISO files to a VHDX attached to a vSCSI controller:
Here’s a screenshot of a copy of 30GB worth of ISO files to a VHDX attached to a vIDE controller.
You’ll notice quite a difference. Depending on the load on the controllers/SAN it’s on average 3 times slower than the same action to a VHDX disk on a vSCSI controller.
Recently I was asked to take a look at why UNMAP was not working predictably in a Windows Server 2012 R2 Hyper-V environment. No, this is not a horror story about bugs or bad storage solutions. Fortunately, once the horror option was of the table I had a pretty good idea what might be the cause.
San snapshots are in play
As it turned out everything was indeed working just fine. The unexpected behavior that made it seem that UNMAP wasn’t working well or at least at moments they didn’t expected it was caused by the SAN snapshots. Once you know how this works you’ll find that UNMAP does indeed work predictably.
Snapshots on SANs are used for automatic data tiering, data protection and various other use cases. As long as those snapshots live, and as such the data in them, UNMAP/Trim will not free up space on the SAN with thinly provisioned LUNs. This is logical, as the data is still stored on the SAN for those snapshots, hard deleting it form the VM or host has no impact on the storage the SAN uses until those snapshots are deleted or expire. Only what happens in the active portion is directly impacted.
Take a VM with a dynamically expanding VHDX that’s empty and mapped to drive letter D. Note the file size of the VHDX and the space consumed on the thinly provisioned SAN LUN where it resides.
Create 30GB of data in that dynamically expanding virtual hard disk of the virtual machine
Create a SAN snapshot
Shift + Delete that 30GB of data from the dynamically expanding virtual hard disk in the virtual machine. Watch the dynamically expanding VHDX grow in size, just like the space consumed on the SAN
Run Optimize-Volume D –retrim to force UNMAP and watch the space consumed of the Size of the LUN on the SAN: it remains +/- the same.
Shut down the VM and look at the size of the dynamic VHDX file. It shrinks to the size before you copied the data into it.
Boot the VM again and copy 30GB of data to the dynamically expanding VHDX in the VM again.
See the size of the VHDX grow and notice that the space consumed on the SAN for that LUN goes up as well.
Shift + Delete that 30GB of data from the dynamically expanding virtual hard disk in the virtual machine
Run Optimize-Volume D –retrim to force UNMAP and watch the space consumed of the Size of the LUN on the SAN: It drops, as the data you delete is in the active part of your LUN (the second 30GB you copied), but it will not drop any more than this as the data kept safe in the frozen snapshot of the LUN is remains there (the first 30GB you copied)
When you expire/delete that snapshot on the SAN we’ll see the size on the thinly provisioned SAN LUN drop to the initial size of this exercise.
I hope this example gave you some insights into the behavior
So people who have snapshot based automatic data tiering, data protection etc. active in their Hyper-V environment and don’t see any results at all should check those snapshot schedules & live times. When you take them into consideration you’ll see that UNMAP does work predictably, all be it in a “delayed” fashion .
The same goes for Hyper-V checkpoints (formerly known as snapshots). When you create a checkpoint the VHDX is kept and you are writing to a avhdx (differencing disk) meaning that any UNMAP activity will only reflect on data in the active avhdx file and not in the “frozen” parent file.
A big thank you to Ben and Microsoft for the confidence they have shown in me and the opportunity to do this. A mention to our CEO who has the ability to look beyond the daily needs and facilitates his and encourages his employees to get out of the village to learn, grow and prosper. This is the principle one of my high school teachers lived and worked by, help people be all they can be.
The IT community around the Microsoft ecosystem is both a local and a global one. In this day and age knowledge gets shared and flows freely. People work with people and with organizations. No one gets anywhere in isolation.I’m happy to see so may of my buddies do so well. It’s great to see people succeed, grow, enjoy their work and reap the fruits of their efforts. Look at Benedict Berger who was presenting in the room next to ours or Aidan Finn, a long time community member and experienced speaker who won speaker idol and by doing so secured a speaker slot for next year. This has many reasons and one of them is people believing in you and giving you the chance to grab opportunities. To those I say, thank you very much!
Ben Armstrong, Principal Program Manager on the Hyper-V team, will be showing you the wealth of features that provide capability, scalability, performance, availability and reliability in Windows 2012 R2 Hyper-V that make it THE capable and scalable cloud OS.
I’m honored to be able to show case a few of the technologies in Windows 2012 R2 we are leveraging in production today. So can you, really!
Good times ahead as today I’m making my way over to the USA (Houston Texas) or TechEd 2014 North America. I’m in good company of a few of my colleagues and I have a great number of my buddies & industry relations inbound as well.
Time for some serious education, networking & passionate discussions on the state of the industry with people form all over the globe. I’ll also make good use of my time over there to meet up with the people in my network that are US based.
I’ll be spending time in cloud/hybrid/virtualization tracks and focus on networking and identity. That’s starts off very well with a pre conference track hybrid identity on Sunday by john Craddock, a true scholar!
No need to bring SFP+ or RJ45, don’t worry. Next to sessions & labs don’t forget to connect with others. The ability to network with peers and industry experts is a great benefit of this conference so make the best of it. There are few events with this concentration of expertise & talent, tap into that resource.
Most companies don’t get that body shopping and contracting is not consulting. And today way too many consultants don’t match up to the requirements of being one. So basically these people pay way too much for too little. I have no mercy for fake consultants and companies so messed up in their political crap they fail to recognize quality and value. Just read The do’s and don’ts when engaging consultants Part I and The do’s and don’ts when engaging consultants Part II where I’ve given some rules of fist for optimizing the use of consultants. But here’s some advice to fake consultants. At least try to act professional and be nice. Because one on the main rules of long term happiness while staying employed is the fact that people not want to work with a*holes.
Don’t download or synchronize hacking tools while at your employer/customer.
If you’re an IT Pro or developer and you cannot figure out how to expand or mount an ISO with the tools on a modern PC/OS you should realize they want to fire you on the spot for gross incompetence.
Playing games like a long term frustrated employee with help desk personnel regarding mundane tasks and leveraging policies or rules as excused not to get work done is annoying, immature & frowned upon.
Waiting until you get asked why you didn’t get work done to mention your PC got confiscated due to a security incident an the replacement was missing a tool proves you’re a dead weight.
Churning is nothing short of fraud. You deliver real work or get out.
It’s rather easy to see that the hours you telecommute are not matching your time sheets. It gets noted. At best they’ll pull a Yahoo on you.
When you state that optimizing parallelism in SQL Server and using multiple LUNs and data files is waste of time in an environment that’s on the record for being in the top 5 SQL server operations in the country you might perhaps realize you are being an a* hole. Yes it requires some work, the database is not designed to cater to your laziness, but for performance.
It’s designed and build at a great value point by skilled people who earn mediocre wages but are very good and love what they do. Do you even realize the level of your arrogance and the fact that you are now on the “do not hire list”?
Realize that not all people who’re working as an employee are clueless sheep that only serve to line your pockets. Most of them are good. Many of of them are great, some even excellent to exceptional. Even when not subject matter experts they’ll see through any fake expertise.
Look, when you’re dead, you don’t realize that, so for you there is no drawback, it’s all good. It’s your surroundings that notice it and have to deal with the loss, the pain. It’s the same when you are stupid. It’s the people around you that suffer. Get it?