Windows Server vNext Soft Restart – A way to speed up reboots? Not in Technical Preview 9841


As you all probably know I’m also playing around with and testing Windows Server vNext Tech Preview and one of the nice new features in there I have my eye on is Soft Restart.

image

There is little information on this feature out there right now but from the description “Soft Restart” looks like a way to get faster Windows boot times by cutting down on device firmware initialization. When it’s not needed that would be a great thing to have as with > 10gbps live migration speeds the boot time of our hardware loaded (DRAC, NICs, HBA, BMC, …) servers is what makes it the longest single step per node during cluster aware updating. Interesting if this is indeed what it’s there for.

But let’s find out if this is indeed what we think it is Smile. First of all the installation of this feature requires a restart. Keep this in mind.

There are 2 ways to kick it off that I know of but to me there must be more … it would be a shame not to have this integrated as an option into Cluster Aware Updating for example.

Option 1: via shutdown

image

So let’s try shutdown /r /soft /t 000.  No joy, doesn’t make one bit of difference and nothing logged or so to indicate an issue.

Option 2: PowerShell via Restart-Computer –Soft

No joy here either …

image

What could be the problem?

So I figured I needed enterprise grade server hardware with some FC cards & lots of NIC and memory to notice the difference. On a VM it might do nothing, but I assure you I doesn’t do anything on the PC based home lab either. So I dragged a DELL PowerEdge R730 with exactly that into the game. But still no joy. Then I thought some more and decided it might integrate with the hardware capabilities to do so of I went to install the latest and greatest DELL Server Manager software to see if that make a difference. But again, no joy.

It’s probably not lit up yet in this release of the Technical Preview 9841. For now I’ll be content with the 28-30% improved reboot speeds the DELL R730 UEFI brought us. I’d love to speed things up a bit as time is money and valuable Winking smile but we’ll have to wait for the next code drop to see if and how it works …

Windows 2003? Let it go!


Reflecting on some of the discussions I was in recently I can only say that there is no escaping reality. Here are some reference blogs for you.

You can’t get of Windows 2003 you say? Held hostage by ancient software from a previous century?  Sure I understand your problems and perils. But we do not negotiate with hostage takers. We get rid of them. Be realistic, do you think this is somehow going to get any better with age? What in 24 months? What about 48? You get the drift. What’s bad now will only be horrible in x amount of time.

Look at some issues people run into already:

Issues like this are not going to go away, new ones will pop up. Are you going to keep everything in your infrastructure frozen in time to try an avoid these? That’s not even coping, that’s suffering.

image

What ever it is that’s blocking you, tomorrow is when you start planning to deal with it and execute on that plan. Don’t be paralyzed by fear or indecision. Over 12 years it will have been a supported OS by its end of life. Windows 2003 had a real good run but now it’s over. Let it go before it hurts you. You have no added value from a more recent version of Windows? Really? We need to talk, seriously.

UPDATE: Inspired by Aidan Finn (@joe_elway) who offered a very good picture to get the message across => click the picture to get the soundtrack! LET IT GO!

Embedded image permalink

Failed at dumping XP in a timely fashion? Reassert yourself by doing better with Windows Server 2003!


I could write a blog post that repeats the things I said bout XP here for Windows 2003 with even some more drama attached so I won’t. There’s plenty about that on the internet and you can always read these blogs again:

I also refer you to a old tweet of mine that got picked up by some one and he kind of agreed:

image

Replace “XP” with “Server 2003” and voila. Instant insight into the situation. You are blocking yourself from moving ahead and it getting worse by the day. All IT systems & solutions rot over time. They become an ever bigger problem to manage and maintain, costing you time, effort, money and lost opportunities due to blocking to progress. There comes a day that creative solutions won’t pop up anymore like the one in this blog post  Windows XP Clients Cannot Execute Logon Scripts against a Windows Server 2012 R2 Domain Controller – Workaround and more recently this on where people just waited to long to move AD over from Windows Server 2003 to something more recent It turns out that weird things can happen when you mix Windows Server 2003 and Windows Server 2012 R2 domain controllers. All situations where not moving ahead out of fear to break stuff actually broke the stuff.

In the environments I manage I look at the technology stack and plan the technologies that will be upgraded in the coming 12 months in the context of what needs to happen to support & sustain initiatives. This has the advantage that the delta between versions & technologies can never become to big. It avoids risk because it doesn’t let delta grow for 10 years an blocks introducing “solutions” that only supports old technology stacks. It make sure you never fall behind too much, pay off existing technology debt in a timely fashion and opens up opportunities & possibilities. That’s why our AD is running Windows Server 2012 R2 and our ADFS was moved to 3.0 already. It’s not because a lot of things have become commodities you should hand ‘m over to the janitor in break/fix mode. Oh the simplicity by which some wander this earth …

OODA

Observe, Orient, Decide, Act. Right now in 2014 we’ve given management and  every product/application owner their marching orders. Move away from any Windows 2008 / R2 server that is still in production. Why? They demand a modern capable infrastructure that can deliver what’s needed to grasp opportunities that exits with current technology. In return they cannot allow apps to block this. It’s as easy and simple as that. And we’ll stick to the 80/20 rule to call it successful and up the effort next year for the remainder. Whether it’s an informal group of dedicated IT staff or a full blown ITIL process that delivers that  doesn’t matter. It’s about the result and if I still see Windows 7 or Windows 2008 R2 being rolled out as a standard I look deeper and often find a slew of Windows 2003 or even Windows 2000 servers, hopefully virtualized by now. But what does this mean? That you’re in a very reactive modus & in a bad place. Courage & plans are what’s needed. Combine this with skills to deal with the fact that no plan ever woks out perfectly. Or as Mike Tyson said “Everybody has a plan until they get punched in the mouth. … Then, like a rat, they stop in fear and freeze.”

Organizations that still run XP and Windows Server 2003 are paralyzed by fear & have frozen even before they got hit. Hiding behind whatever process or methodology they can (or the abuse of it) to avoid failure by doing the absolute minimum for the least possible cost. Somehow they define that as success and it became a mission statement. If you messed up with XP, there’s very little time left to redeem yourself and avoid the same shameful situation with Windows Server 2003. What are you waiting for? Observe, Orient, Decide, Act.

NTFS Permissions On A File Server From Hell Saved By SetACL.exe & SetACL Studio


Most IT people don’t have a warm and fuzzy feeling when NTFS permissions & “ACLing” are being discussed. While you can do great & very functional things with it, in reality when dealing with file servers over time “stuff” happens. Some of it technical, most of it is what I’ll call “real life”. When it comes to file servers, real life, especially in a business environment, has very little respect, let alone consideration for NFTS/ACL best practices. So we all end up dealing with the fall out of this phenomena. If you haven’t I could state you’re not a real sys admin but in reality I’m just envious of your avoidance skills Smile.

You don’t want to fight NTFS/ACLs, but if it can’t be avoided you need the best possible knowledge about how it works and the best possible tools to get the job done (in that order).

If you have not heard of SetACL or DelProf2, you might also not have heard of uberAgent for Splunk, let alone of their creator, community rock star Helge Klein. If you new to the business I’ll forgive you but if you been around for a while you have to get to know these tools. His admin tools, both the free or the paying ones, are rock solid and come in extremely handy in day to day work. When the shit hits the fans they are priceless.

Helge is an extremely knowledgeable, experienced, talented and creative IT Professional and developer. I’ve met him a couple of times (E2EVC, where he’s an appreciated speaker) and all I can say is that on top of all that, he’s a great guy, with heart for the community.

Having the free SetACL.exe available for scripting of NTFS permissions is a luxury I cannot do without anymore. On top of that for a very low price you can buy SetACL Studio. This must be the most efficient GUI tool for managing NFTS permissions / ACLs I have ever come across.

Not long ago I was faced with a MBR to GPT LUN migration on a very large file server. It’s the proverbial file server from hell. We’ve all been there too many times and even after 15 years plus we still cannot get people to listen and follow some best practices and above all the KISS principle. So you end up having to deal with the fall out of every political, organizational, process and technical mistake you can imagine when it comes to ACLs & NTFS permissions. So what did I reach for? SetACL.exe and SetACL Studio, these are my go to tools for this.

image

Check out the web page to read up on what this tool can do for you. It very easy to use, intuitive and fast. It can do ACL on file systems, registry, services, printers and even WMI. It helps you deal with granting ownership and rights without messing up the existing NTFS permissions in an easy way. It works on both local and remote systems. Last but not least it has an undo function, how cool is that?!  Yup and admin tool that let you change your mind. Quite unique.

As an MVP I can get a license for free form Helge Klein but I recommend any IT Pro or consultant to buy this tool as it makes a wonderful addition to anyone’s toolkit, saving countless of hours, perhaps even days. It pays itself back within the 15 minutes you use it.

Other useful tools in your toolkit are http://www.editpadlite.com/ as it can handle the large (550-800 MB) log files RoboCopy can produce and some PowerShell scripting skills to parse these files.

Windows XP Clients Cannot Execute Logon Scripts against a Windows Server 2012 R2 Domain Controller – Workaround


The issue

The real issue is that you are still running Windows XP. The secondary issue is that you have Windows XP clients that cannot connect to a file share (NETLOGON) on a Windows Server 2012 R2 Domain Controller. If you try manually via \\domaincontroller\Netlogon it will throw an error like  "The specified network name is no longer available".  Security wise & moral pressure wise I kind of think this drives home the message you need to get off Windows XP. But I realize you’re in a pickle so here’s the workaround/fix.

Root Cause & Fix

Windows XP talks SMB 1.0 and that’s it. If this is not offered by the server (file server or domain controller) we have a problem. Now if you installed new Windows Server 2012 R2 servers they do not deploy the SMB 1.0 feature by default. If you upgraded from Windows 2008 R2 (perhaps even over Windows 2012) to get to Windows 2008 (R2) this feature kept in place. Other wise you’ll need to make sure SMB 1.0 is installed, it often (always?) is. Just check.

image

However there is a big change between Windows Server 2008 R2/Windows 2012. The LanmanServer service has a dependency set to SMB 2.0 and no longer to SMB 1.0

This is what it looks like on a Windows Server 2012 (or lower) domain controller:

image

This is what it look like on a Windows Server 2012 domain controller

image

So we need to change that on Windows 2012 R2 to support Windows XP. We can do this in the registry. Navigate to

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\DependOnService

  1. Change SamSS Srv2 to SamSS Srvimage
  2. Restart the Server (Lanmanserver) service (it will restart the dependent services like netlogon, DFS Namespace, .. as well)

You’re XP clients should be able to authenticate again. You can test this by navigating to \\domaincontroller\Netlogon on a XP client. This should succeed again.

If you have issues with Windows Server 2012 R2 file servers … this is also valid. When you do get rid of Windows XP. Go back to the original settings please Smile.

If you want to read more on SMB read this blog Windows Server 2012 R2: Which version of the SMB protocol (SMB 1.0, SMB 2.0, SMB 2.1, SMB 3.0 or SMB 3.02) are you using? by Jose Barreto (File Server team at Microsoft)

Finally, get off XP!

I think I said it enough on twitter and my blog Legacy Apps Preventing Your Move From Windows XP to Windows 8.1? Are you worried about HeartBleed? Good! Are you worried about still being on XP? No? Well dump SSL and use clear text authentication as XP is a free fire zone  anyway (as of April 8th 2014) and it’s just a matter of time before you’re road kill. Any company who has CIO/CTO/IT managers and other well paid functions and have let their organization be held hostage on XP (I’m not talking about a few PCs or VMs left and right) by legacy apps & ISV should realize they are the one who let this happen. Your watch. Your responsibility. No excuses.

SSL Certs And Achieving “A” Level Security With Older Windows Versions


So a mate of mine pings me. Says they have an problem with their web mail SSL security  (Exchange 2010) running virtualized on Hyper-V.  The security guy states they need to move to a more secure platform that supports “modern SSL standards” and proposes to migrate from Exchange 2010 to Exchange 2013 in an emergency upgrade. Preferably to VMware as “MickeySoft” is insecure. Oh boy! Another profit of disaster who says the ship is lost unless …

You immediately know that the “security guy” is an incompetent fraud who only reads the IT press tabloids, runs some  freely available vulnerability toys (some are quite good) to determine what to check off on his list and shout out some “the sky is falling” rubbish to justify his daily rate and guarantee his paycheck. I’ve said it before, your mother told you not to trust strangers just like that, so why do so many companies do this with “consultants”? Choose your advisers wisely and remember Machiavelli’s notes on the use of mercenaries Winking smile!

  • VMware is not more secure than Hyper-V. That’s so wrong and so loaded with prejudice it immediately invalidates the persons credibility & reputation. If you need proof, do your research but as a recent example the “HeartBleed” issue left VMware scrambling, not Hyper-V. And for what it’s worth. IT security is like crime, statistically we’ll all be victims a couple of times in our life time.
  • Exchange 2010 running on Windows 2008R2 fully patched is just fine. So what was all the drama about? The issue was that the Qualys SSL Labs tool gave their Outlook Web Access a F grade. Why? Well they still allowed SSL 2.0, they didn’t run TLS 1.2 and they don’t have Forward Secrecy support.

My advice to my buddy? First he needs to get better security advice. Secondly, to get an “A” for secure SSL configuration all you need to is some easy tweaking. You don’t want to support any clients that can’t handle the better SSL configurations anyway. No one should be allowed to use these anyway. But what do I use? SSL 3.0? TLS 1.0/1.1/1.2? What to use & do? Here’s some documentation on how to enable/disable protocols: How to restrict the use of certain cryptographic algorithms and protocols in Schannel.dll. This will tell you how to do it? But which SSL versions can you dump today without suffering to many support calls. Server side, drop SSL 2.0 & SSL 3.0, keep TLS 1.0/1.1/1.2. On the client side you’ll need to do the same. That will keep most things working. Not ideal but the trick is to allow / enable the better protocols server side so all clients that can use it, can use it, while you block the really bad ones that just don’t have any use any more. We’ll play a bit with this.

Test 1: Disable SSL 2.0 and Enable SSL 3.0

image

As you can see this gave them an B grade. We need to enforce the current best TLS 1.2 protocol to get that and we might want to get rid of SSL 3.0 as XP &n IE 6.0 have had there time and that’s over.

Test 2: Enable TLS 1.2

There you go. I hope this helps you out if you need to make sure you environment supports only more modern, stronger protocols.

image

There it is. A- Smile Compliance achieved! Now it would best to disable SSL 2.0/3.0, TLS 1.0/1.1 on the server and forget about any browsers, operating systems and software that can’t handle it. But that’s not that easily done you’ll need Outlook 2013 for RPC over HTTP if you want to enforce TLS 1.2. But as far as the auditors go they are all so happy now and effectively you’re now supporting the more modern clients. Now my buddy can get to an A or A+ rating when they make sure to get Forward secrecy support in the future. I really advise the latter as HeartBleed made it obvious the wide use of this is long overdue.

Some Testing Fun

Grab a laptop, WireShark and a number of twitter clients, cloud storage products and take a peak a what version of SSL/TLS those apps use. Some tests you can do:

MetroTwit uses SSLv3, OneDrive uses TLSv1, Yammer seems to be at TLSv1 as well. Try disabling TSL 1.0 on a client and see how it breaks Outlook  2010 RPC over HTTPS and even OneDrive by the way.

image

What you can get away with depends on the roles of the servers and the level security the clients for that role can handle.

Won’t this break functionality?

As you’ve seen above it can but for what matters on the e-mail server, probably not. If it does you’re in need of some major work on your client infrastructure. But in most cases you’ll be fine, especially with web browsers. But I have a underpaid employee who needs food stamp support so she cannot afford to upgrade her PC from Windows XP! Dude, pay a decent living wage, please. That aside, yes you can turn on better protocol support and block the oldest, most insecure ones on your servers. You call the shots on the use of your businesses infrastructure and you are under no obligation to allow your employees to access your services with obsolete clients. You want to be in the green zone, in the right column with TLS 1.2 if possible, but that’s going to be a challenge for a lot of services.

image

Do as I say, don’t do as I do

The funny thing is that I ran the same test against the web (mainly e-mail) servers of 4 governments levels that are enforcing/promoting the (mandatory) use of security officers in an attempt to get to a more secure web for the benefit of all man kind. Not only does this fail because of such fine examples of security officers but 2/3 don’t seem to take their own medicine. The intentions are good I’m sure but the road to hell is paved with those and while compliancy is not the same a being secure, even this is hard to get to it seems.

Federal Government Department

image

Undisclosed State Government

image

Undisclosed Local Government

image

Medium Sized City (they did well compared to the above braches with more resources)

image

Don’t panic

That’s what it says on the cover of “The Official Hitchhiker’s Guide to the Galaxy Companion”. Get some good advise and if you want or read more about how the rating is done (as of 2014) then please read this SSL Labs: Stricter Security Requirements for 2014 which also provide a link to their SSL Server Rating Guide.

Legacy Apps Preventing Your Move From Windows XP to Windows 8.1?


Are old applications holding you back getting rid of Windows XP? It’s A reason we hear a lot and these apps do exist. But often it’s because the effort to make it work isn’t considered worth the cost. Year after year. So some people today are stuck on a Windows Server 2000/2003 & XP infrastructure. How does that cost compare now to the cost of dealing with the application? Was it worth not moving the application & have an out of date infrastructure holding your ENTIRE company down?

image

While some things can’t be fixed, putting in some effort could have prevented you of being in this mess. Yes it would have cost you a decent penny but nothing compared to where you are at now with your infrastructure “challenges”.

Here’s a little example for you. Over a period of 13 years we’ve moved an old application (using a Borland database engine & ISAPI DLLs in IIS). It ran on Windows Server 2000. It was P2V’d to VMware Server. Over the years the data base swapped from Informix to SQL Server 2000, 2005, 2008, 2008 R2. We upgraded the VM to Windows Server 2003(x86), moved to Hyper-V, upgraded to Windows 2008(x86) & final now put on W2K12R2(x64). So what do you mean you can’t get rid of XP? We’ve moved the client app for that VM to x64 with Vista in 2007.  We were not to let that app block our way to the future and Windows 7(x64) and Windows 8 & 8.1(x64). In 2014 you should be able to move to or you need to reconsider your approach to IT as you have totally painted the organization into a corner. We did not have installers for anything. We extracted registry entries & bits form installed systems and build installers ourselves with the free NSIS installer. We used  Windows SysInternals tools to figure out where the application wrote & read, what permissions where needed and add those to the installer to make sure it did not need local admin rights. It gave the business over a decade to get a grip on application live cycle management & replace the app. They failed twice, and while that’s bad and we do not like it, it was not deadly as they haven’t let the rest of the company suffer for it. Never, ever let your infrastructure get stuck in the past. But wait you say, what you did is not supported. That’s right. That’s one app, that works, and it beats being left with an unsupportable infrastructure blocking progress Winking smile

You might need some help and here’s a great place to start helping yourself The App Compat Guy. Read and view (TechEd presentations) anything Chris Jackson is offering on this subject and you’ll be on your way. Need a helping hand? Here’s a good place to start if your in Belgium: Microsoft Extended Experts Team (MEET). Chances are some of them known some one who knows how to get it done or are the person to talk to.