Failed at dumping XP in a timely fashion? Reassert yourself by doing better with Windows Server 2003!

I could write a blog post that repeats the things I said bout XP here for Windows 2003 with even some more drama attached so I won’t. There’s plenty about that on the internet and you can always read these blogs again:

I also refer you to a old tweet of mine that got picked up by some one and he kind of agreed:


Replace “XP” with “Server 2003” and voila. Instant insight into the situation. You are blocking yourself from moving ahead and it getting worse by the day. All IT systems & solutions rot over time. They become an ever bigger problem to manage and maintain, costing you time, effort, money and lost opportunities due to blocking to progress. There comes a day that creative solutions won’t pop up anymore like the one in this blog post  Windows XP Clients Cannot Execute Logon Scripts against a Windows Server 2012 R2 Domain Controller – Workaround and more recently this on where people just waited to long to move AD over from Windows Server 2003 to something more recent It turns out that weird things can happen when you mix Windows Server 2003 and Windows Server 2012 R2 domain controllers. All situations where not moving ahead out of fear to break stuff actually broke the stuff.

In the environments I manage I look at the technology stack and plan the technologies that will be upgraded in the coming 12 months in the context of what needs to happen to support & sustain initiatives. This has the advantage that the delta between versions & technologies can never become to big. It avoids risk because it doesn’t let delta grow for 10 years an blocks introducing “solutions” that only supports old technology stacks. It make sure you never fall behind too much, pay off existing technology debt in a timely fashion and opens up opportunities & possibilities. That’s why our AD is running Windows Server 2012 R2 and our ADFS was moved to 3.0 already. It’s not because a lot of things have become commodities you should hand ‘m over to the janitor in break/fix mode. Oh the simplicity by which some wander this earth …


Observe, Orient, Decide, Act. Right now in 2014 we’ve given management and  every product/application owner their marching orders. Move away from any Windows 2008 / R2 server that is still in production. Why? They demand a modern capable infrastructure that can deliver what’s needed to grasp opportunities that exits with current technology. In return they cannot allow apps to block this. It’s as easy and simple as that. And we’ll stick to the 80/20 rule to call it successful and up the effort next year for the remainder. Whether it’s an informal group of dedicated IT staff or a full blown ITIL process that delivers that  doesn’t matter. It’s about the result and if I still see Windows 7 or Windows 2008 R2 being rolled out as a standard I look deeper and often find a slew of Windows 2003 or even Windows 2000 servers, hopefully virtualized by now. But what does this mean? That you’re in a very reactive modus & in a bad place. Courage & plans are what’s needed. Combine this with skills to deal with the fact that no plan ever woks out perfectly. Or as Mike Tyson said “Everybody has a plan until they get punched in the mouth. … Then, like a rat, they stop in fear and freeze.”

Organizations that still run XP and Windows Server 2003 are paralyzed by fear & have frozen even before they got hit. Hiding behind whatever process or methodology they can (or the abuse of it) to avoid failure by doing the absolute minimum for the least possible cost. Somehow they define that as success and it became a mission statement. If you messed up with XP, there’s very little time left to redeem yourself and avoid the same shameful situation with Windows Server 2003. What are you waiting for? Observe, Orient, Decide, Act.


Saying Goodbye To Old Hardware Responsibly

Last year we renewed our SAN storage and our backup systems. They had been serving us for 5 years and where truly end of life as both technologies uses are functionally obsolete in the current era of virtualization and private clouds. The timing was fortunate as we would have been limited in our Windows 2012, Hyper-V & disaster recovery plans if we had to keep it going for another couple of years.

Now any time you dispose of old hardware it’s a good idea to wipe the data securely to a decent standard such as DoD 5220.22-M. This holds true whether it’s a laptop, a printer or a storage system.

We did the following:

  • Un-initialize the SAN/VLS
  • Reinitialize the SAN/VLS
  • Un-initialize the SAN/VLS
  • Swap a lot of disks around between SAN/VLS and disk bays in a random fashion
  • Un-initialize the SAN/VLS
  • Create new (Mirrored) LUNS, as large as possible.
  • Mounted them to a host or host
  • Run the DoD grade  disk wiping software against them.
  • That process is completely automatic and foes faster than we were led to believe, so it was not really such a pain to do in the end. Just let it run for a week 24/7 and you’ll wipe a whole lot of data. There is no need to sit and watch progress counters.
  • Un-initialize the SAN/VLS
  • Have it removed by a certified company that assures proper disposal

We would have loved to take it to a shooting range and blast the hell of of those things but alas, that’s not very practical Smile nor feasible Sad smile. It would have been very therapeutic for the IT Ops guys who’ve been baby sitting the ever faster failing VLS hardware over the last years.

Here’s some pictures of the decommissioned systems. Below are the two old VLS backup systems, broken down and removed from the data center waiting disposal. It’s cheap commodity hardware with a reliability problem when over 3 years old and way to expensive for what is. Especially for up and out scaling later in the life time cycle, it’s just madness. Not to mention that those thing gave us more issues the the physical tape library (those still have a valid a viable role to play when used for the correct purposes). Anyway I consider this to have been my biggest technology choice mistake ever. If you want to read more about that go to Why I’m No Fan Of Virtual Tape Librariesimageimage

To see what replaced this with great success go to Disk to Disk Backup Solution with Windows Server 2012 & Commodity DELL Hardware – Part II

The old EVA 8000 SANs are awaiting removal in the junk yard area of the data center. They served us well and we’ve been early customers & loyal ones. But the platform was as dead as a dodo long before HP wanted to even admit to that. It took them quite a while to get the 3Par ready for the same market segment and I expect that cost them some sales. They’re ready today, they were not 24-12 months ago. image


So they’ve been replaced with Compellent SANs. You can read some info on this on previous blogs Multi Site SAN Storage & Windows Server 2012 Hyper-V Efforts Under Way and Migration LUNs to your Compellent SAN

The next years the storage wares will rage and the landscape will change a lot. But We’re out of the storm for now. We’ll leverage what we got Smile. One tip for all storage vendors. Start listening to your SME customers a lot more than you do now and getting the features they need into their hands. There are only so many big enterprises so until we’re all 100% cloudified, don’t ignore us, as together we buy a lot of stuff to. Many SMEs are interested in more optimal & richer support for their windows environments if you can deliver that you’ll see your sales rise. Keep commodity components, keep building blocks and from factors but don’t use a cookie cutter to determine our needs or “sell” us needs we don’t have. Time to market & open communication is important here. We really do keep an eye on technologies so it’s bad to come late to the party.

The Zombie ISV®

The Zombie ISV® is the type that should have been extinct based on the current state of technology. Let me give you an idea what that current state of technology means in our neck of the woods. Last week our team started deploying some DELL R720  PowerEdge servers to replace the last W2K8R2 Hyper-V cluster in the company with a Windows Server 2012 one. The older hardware will be recycled. Some will live on as test servers, backup media servers. All running Windows Server 2012 of course. One of them will become our physical (SAN) LUN to VHDX converter server so we can move our large LUNS (2T-15TB) to vhdx. Later this year 10Gbps networking, RDMA Mellanox cards and ODX will provide for fast vhdx movement to their new virtual hosts. Work in progress, but it should give you an idea about what we’re working with.

It may surprise you but even we have 2 Windows Server 2003 physical servers left. One is a DELL NX1950 Storage server that has been serving local workspace to a team that does image parsing (12TB). That one is >6 years old and is slated for retirement. We don’t need this concept anymore. We can build anything we want for such purposes using Windows Server 2012 Storage spaces and if required leverage the in box iSCSI target. To build it we can just draw disk bays, disk, servers from the retired hardware shelf, no sweat. We have plenty of spare parts and it works just fine. If it’s cost efficient and an effective solution, we roll that way.

The other one is a server for the financial software sold by a company (the Zombie ISV®) that does not believe in virtualization. It’s running code that’s over a 12 years old (legacy java run times and even that was a success because it used to be JInitiator until a a few years ago). There is no life cycle planning what so ever and when after 5 years the hardware needed replacing we got nothing but silence form the vendor. After months of asking for a meeting on the what and how (OS upgrade, x64, virtualization) and being ignored we just took a decommissioned server that had two years of warranty left and transplanted the disks. Even if the warranty runs out on that one we have some of the same model in the spare parts cabinet.

The workload itself runs just fine virtualized but they don’t support that. Luckily for the people that have to do it in their environment they run zero change of that Zombie ISV® ever noticing that a server is virtualized anyway. They also don’t get the concept of a dedicated service account in windows. So they end up with the database or BI services running under their remote support credentials that expire and get disabled by the helpdesk. Sigh. They don’t see the need to proactively support operating systems above Windows XP or browsers after IE 6.0. We did a lot of hacks to keep that system working and came to despise the total lack of technological expertise and professionalism of the vendor. Their “consultants” that’s don grasp x64 bit, or they download installers for 4 hours during a paid day of consulting … sickening to the stomach. Meetings with the account managers (they seem to travel in packs) is a lot of vacant bank stares and apathy. They don’t have answers, they don’t look for answers, they simply don’t care. The idea was to replace the package, but it was not to be. But in the end we settled for throwing all responsibility for it so they’ll find a place to host it and our bookkeepers can access over a secure remote connection. At least we have gotten rid of this security risk in our environment.

That people, is the miserable state of some ISVs in the 21st century. But it’s not just them. It’s a testimonial to the degree in which companies get tied up and locked in to mediocre solutions and technology debt. In the infrastructure world (storage, networking, servers, virtualization) people who know what they’re doing do not allow this to happen. As more and more decisions on software and applications are made by business & analyst types we are seeing an increase in technology debt and lack of any life cycle management. So where we have seen infrastructure get more and more bang for the buck we’ve also seen the software & services cost explode and on top of that incur technology debt, expenses and risks on the business. That’s pretty bad. I see a growing divide in a lot of companies between ever more efficient and cost effective infrastructure (combined with cloud solutions) and the slowness of getting custom software into production combined with issues concerning supportability and upgradeability. All this at ever increasing costs and FTEs. That’s not supposed to happen but it is, despite the high investments in * analysts, business consultants, architects, * coaches, project managers, IT managers etc. in the era of the cloud. This is regression.  It all sounds like the result of the feel good EQ approach to business without results but hey, no one feels left behind Confused smile. I believe a mate of mine calls this the race to the bottom. No wonder some companies that I know have done away with all this and just let business units organize themselves organically. They either fail and disappear of thrive and prosper, but a no time to they fall in to the trap of over organized pseudo flat structure (i.e. pass the hot potato and no responsibility) that still manages to create ever more managerial positions (flat?) whilst realizing ever less results. We’ve seen the financial and housing market charades collapse. Guess what’s next? There won’t be a bail out for you or me, beware of that.