Technical Projects, Planning, Skills, Motivation & Psychopaths


When planning a technical project complexity adds ups very fast. Take a virtualization project for example; a lot more things than just the hyper visor installation are coming into play. You’ll need to asses a lot of needs and desires about SANs (snapshots, redundancy, replication, FC, iSCSI, FCoE), network (VLAN, 1/10 Gbps Ethernet, redundancy), disaster recovery/business continuity, hyper visors and there capabilities, management of it all and security. That is a lot of stakes and agendas to take into consideration. And then you haven’t even talked to the business managers, the application owners and developers. Now this isn’t limited to virtualization, but this is just a nice example on how so many stakes come together in one project.

One of the major mistakes, that is made again and again even up until this day in the second decade in the 21st century is the fact that entire important or even critical IT systems are being put into place with a plan that can be paraphrased as follows “We’ll just set it up and sort of see how it evolves and just wing it from there”. I have been forced to do this quite often. This creates many problems some of which I will address below.

The single worst problem is that you create a vacuum. That can be storage space, bandwidth, ample resources for a huge amount of virtual machines or a mixture of all this. The results however are always the same and is one of two possibilities. Either they really don’t want and need it so it will never be used. You can also achieve this by keeping it hidden so they can’t use it. The other option is the most natural one. In nature there is a thing called a “horror vacui”. That means that a vacuum unless protected cannot exist, it has to be filled. Empty LUNSs with data, hyper visor hosts with guest, networks with bandwidth and backup capacity with even more terabytes. You might think the second option is better than the first one as at least the infrastructure is getting used. Unfortunately the reality is that this is creating a very expensive mess to run, support & troubleshoot. The legacy this creates is not a valuable inheritance but a bank breaking, efficiency and effectiveness ruining debt. Stop doing that right now, you are killing your business. You see technology debt is about more than just old hardware and software. It’s about what you build with it or what grows organically with it. Is that a fertile land that sustains the business or a cancer that is killing it?

The way to prevent this is planning done by competent, involved people with experience and context. No plan is perfect, but a plan gives you a framework to achieve the desired result. Even great people make mistakes but they have the skills and attitude to fix them or work around them.

What are some other problems? Wasting money. Take for example a completely oversized server farm. That thing will consume so much money over a three year period in energy and idle capacity that the amount would be sufficient to replace it with new right sized hardware (more bang for the buck, better energy efficiencies in three years) I don’t know about you but those are very disconcerting numbers.

You can also be wasting money and time. And those who know me I loath wasting time. What if the SAN solution you bought doesn’t perform as planned or isn’t the right fit? There goes 500.000 € or you find yourself in the CEO office explaining why you need an extra 400.000 € to get what is really needed. Oh oh! Do you have money and time to do it all over again or will you be living with that expensive mistake until the current solution is end of life? Do you have to wait until the CFO and CEO have recovered enough from the shock to allow a new attempt? Or perhaps you bought a SAN solution that is enough to run NASA’s workload and you’ve invested 4.000.000 € in a rather expensive data room heater.

Getting a virtualization project wrong can wreak havoc on a business and create a sizable financial hemorrhage. You can say that that’s not your problem but I beg to differ. If the project goes south that means you’ll have to find another job. The IT world where I live is rather small so you might even have to switch to another field as you’ll be forever known as the guy that sunk company X with his little “plan”.

The reverse, being rewarded for your hard work and success is not a given. In the end they pay you for getting the job done so results are expected, and to Joe Average manager all ICT is a PC with a software packet to install. So for all you eager beavers who think that with this kind of responsibility and risk management comes big reward when you get it right I suggest you think again. I have witnessed quite the opposite personally. Even when you’re running multiple enterprise SAN’s, networks, infrastructures like SQL Server, Exchange clusters, Hyper-V clusters, geo clusters, load balancers and providing 2nd and 3rd line support for those and taking 24/7 responsibility for the environment the only thing some managers care about is why the PC they never ordered with the software they never ordered can’t be installed tomorrow. “What kind of a chicken shit outfit are you running here” is what they’ll think when you can’t do that. They’ve read the glossy brochure that IT is a commodity and they expect it cheap and always on, much like electricity. In the end some (incompetent) managers act like ungrateful psychopaths. They’ll just abuse you less when you get it right. Don’t expect anything else. Often it’s the ones that are not capable to integrate things they can’t do or don’t understand into their business. They can not value anything that’s beyond their comprehension so they’ll never recognize it. To them, people are, for all practical purposes, resources that are identical, “Full Time Equivalents”. So don’t buy into the hype that there is a skills shortage from that lot and they can’t fill job openings. The volume in which they often waste talent and flush motivation down the drain is shockingly high and indicates that there is no shortage at all or that they can’t recognize skills when they find it and they’ll hire anyone. Surely they didn’t make a mistake so it must be a skills shortage. So you still want to be some hot shot technical architect? Or does a job that only produces open opinions and optional advice on paper sound more attractive. Per hour worked you’ll earn more, run less risk and have a lot less stress. My advice? Don’t switch fields if you enjoy what you’re doing, switch jobs. The best career advice I ever got was “don’t work with or for assholes”.

Well if you don’t agree with your bosses and you dare go against them you’re surely playing with your job, you could get fired! So? Does living in fear of being fired make good employees? Does not being strong and confident enough to tell your managers they are doing certain things totally wrong or that they are mistaken make for good advisors? The worst thing a boss can have are a bunch of “yes men” around him or her. That boss should be smarter than that. It doesn’t work. Having trust in the abilities and loyalty of your employees does not mean you need to agree on everything. As a boss you’ll make the final decisions, yes, but you’d better listen very carefully to your advisors and staff or you might as well have hired some monkeys. You can train them to say yes all the time, all it takes are some bananas. As an employee, don’t let yourself be treated like a monkey and if they fire you for throwing the banana back, good for you!

So you’d better love technology and building solutions because that means you are intrinsically motivated to go the extra miles. When you are, select a small group of people with the same attitude. You’ll be able to drag the devil himself out of hell with such a team at relatively very low cost. Whatever you, do don’t think you can externally motivate or coerce people into achieving this. Charles “Chargin’ Charlie” Beckwith knew that all along when he said “I’d rather go down the river with seven studs than with a hundred shitheads”. And guess what, he wasn’t taught this in some course, by getting a title or by being told this by a manager. He learned it himself by working with the best. These people will keep learning and growing on their own. They don’t need to be told what to do, how to train, what to use, they don’t need nannies & micro management. They need an end state and they’ll get it for you. Frankly that kind of skillset and ability scares the shit out of some bosses as they micro manage actions & items instead of doing their jobs. You can’t use force, treats or authority to make people achievers. In the end you can cut a diamond, but you cannot create it. Trust me. Putting that amount of pressure on someone that isn’t a diamond only turns them into a heap of crushed remains of what used to be a human being or FTE in your typical HR speak.

“Mate you’re not a conformist” my friend said … you’d better believe I’m not Winking smile

Shameless Plug For Mastering Hyper-V Deployment By Aidan Finn


In October 2010 Aidan Finn (MVP) his book “Mastering Hyper-V Deployment” was released and in November three copies of this book landed on my desk. I bought them (pre order) via Amazon. Nope I did not get them as a gift or anything. Why Three? Well that’s the number of people I wanted to get up to speed about Hyper-V and virtualization management and operations in a Microsoft environment.

His book takes you along a journey through a Hyper-V project that will teach you about virtualization in all it’s aspects. It also touches on many supporting technologies and products such as System Center Virtual Machine Manager 2008 R2, System Center Essentials 2010, Data Protection manager 2010 and System Center Operations Manager 2007 R2. No one book can be the only source of knowledge and understanding, but using this book as a start for both new and experienced IT Pros to learn about virtualization with Hyper-V will give you the best possible start. Consider it going to an Ivy league college on a scholarship paid for by Aidan’s experience and hard work. The subsidized tuition fee is the price of the book.

We feel a bit sorry that Aidan only got one copy so we made a group picture of the gang of three on the desk of our newest team member. He got a copy of the book together with 4 recycled PC’s and a TechNet subscription to build a lab.

image

If you know people who want or need to learn about Hyper-V, you’d do well to make sure they get this book and have them set up a lab to play with the technologies. Those efforts will pay off big time when they implement their solutions in the wild. If Ireland is doomed it won’t be because of smart & hardworking Irish IT professionals like Aidan. You see when you design, build and support IT solutions that your customers depend on 24/7 you can not hide behind false promises, you can’t fake away the fact when “stuff” doesn’t work or hide behind vast amounts of papers & documents void of any substance. Nope you are responsible for everything and anything you build. Aidan backed and supported by some very knowledgeable colleagues has made that burden a bit lighter for you to bear with this book. Aidan’s blog lives here: http://www.aidanfinn.com/

Cheap IT Support Requests & The Value of Time


I value my time tremendously. I also accept the fact that you don’t give a rat’s ass about my time. To you every hour I spend not working on your issues is a gigantic waste of time, but to me, it is not. And this is about my time. You cannot get time back, once used it’s gone. You cannot sell memories to get time back. You can’t produce time. You can’t save time. You have what you have and you need to use it when you have it. What does this have to do with IT? The fact than when you’re in IT people almost expect to get advice and support at no cost or on the cheap. This behavior stems from the fact that for some reason they expect that when they buy a server and software all the rest is included for free. With a car they don’t have this mind set. They expect to pay for maintenance, insurance, road taxes and gas. Partially this is the industries fault since they market everything a great, easy, fast, and cheap. Partially it’s the buyers fault for believing commercials and sales men. So how do I deal with the ever returning attempts to get me to work for free and how do I make sure they stop asking. Very simply, I price myself out of “the market”.

One day I discovered this also works outside of IT. Everyone who knows me wouldn’t think of asking me to baby sit but once a female colleague did just that. I guess she was really desperate. Really very desperate I should add. I told her I was not interested. She insisted. I told her again that I was absolutely not interested. She decided to make a case that I should help her out. That’s asking for it. I told her it would be 150 €/hour. I got a speech that babysitting isn’t that hard and worth that much, that it’s unaffordable, that kids, a house mortgage, car payments and life are already expensive enough. All true but not my problem.  You see I do not want to baby sit and my time is very valuable to me. I asked her what day and time she needed a baby sitter, trying to get my point across. She said Saturday night. Oh, on a Saturday to Sunday night in a weekend, after office hours and no retainer for my services. That makes it 300 €/hour and for watching of the most precious and loved human being in your life that’s a bargain! Needless to say I was not hired and luckily never asked again. Mission accomplished.

Think about it, time is the most limited resource the human species has. As I said, you can’t get any more of it. Gone is gone. That makes it more precious to me than anything else.  That means I want to spend it as well as I can. So when it comes to work I try to do things I enjoy and that pay well enough so that I can have enough free time to do other things I also enjoy. This means that when I do work I will not do it at 1 € /hour. Why would I? Even if I can only work 40% of my time at 5 €/hour I’m still way ahead and have more time to myself. With some luck and effort the better paying work is also the type of work I like to do. Cool, two goals achieved in one go.

So why on earth would I baby sit or fix your IT mess (which I dislike) in my spare time (time which is extremely valuable to be)  for some pocket money given the fact that it’s not my job responsibility and I have no financial pressure to do so?  Now I don’t know a thing about babies but IT can get a lot more complicated and involved that the owner of the mess realizes. It takes a lot of time and it just isn’t worth it. So there is your answer. I don’t want to and that’s why I price it so highly. To make sure no one asks or agrees to it. With these of hand support requests, changes are you’re a small shop running a couple of servers & workstations that are mediocre at best. This is probably combined with some older, hopefully legal, operating systems and applications that might suck and have their own issues. The environment was probably not designed, is most likely mismanaged for whatever reason and most of the time you won’t like the recommendations (get Adobe Acrobat and Office of your server and stop surfing on it so you don’t get spyware on the box). You complain about how expensive the hardware is, that the software costs money, that the small business IT shop is expensive and can’t get it right like you want. Perhaps the reason is that they can’t do it for the price your willing to pay, you are asking for things that can’t be done or perhaps they are not very good at their business. Whatever the reason, somehow you think that I should fix all that for a token fee since you already paid all that money to hardware vendors, software vendors, your “IT Guys” and because it won’t take me very long since I good at what I do. Well, it doesn’t work that way. My rate is not determined by how easy it might be for me. It’s determined by my knowledge, expertise and quality of my work. I don’t do the easier work as that won’t get me as much money for the same amount of time and I get bored doing it.

Am I a money hungry capitalist pig? No. I will and do work for free for a good cause, a close friend or a sport club I sympathize with. It’s called voluntarism and you can beat that as a motivation. I will not spend my valuable spare time fixing a mess that I did not create for free or cheaply. Actually I rather have my time to myself even when the money is good. You see, you’re in that mess because you don’t know what you’re doing; you’ve had very bad counseling or services and perhaps want things you can’t afford or are willing to pay for. The effort and cost of fixing all this is probably going to make you shout at me in anger. The impact that will have on your business processes and culture is something you’ll find unacceptable. The cost and needs of a professional IT environment are beyond what you can grasp, are willing or capable to pay. So the best thing for you is use free, cloud based services and make due with what you have or can get from those services. You cannot expect people to feel obligated to fix your problems because you already spent so much money on it. My free time at night and weekends is for studying, reading, hobbies, and friends. Not for fixing other peoples problems. So if you need a good environment hire one or more good IT partners to take care of your infrastructure needs in a professional manner. That’s the only sustainable and workable way of doing it.

The Dilbert® Life Series: Enterprise Architecture Revisited One Year Later


The Dilbert® Life series is a string of post on corporate culture from hell and dysfunctional organizations running wild. This can be quite shocking and sobering. The amount of damage that can be done by "merely" taking solid technology, methodologies, people and organizations, which you then abuse the hell out of, is amazing. A sense of humor will help when reading this. If you need to live in a sugar coated world were all is well and bliss and think all you do is close to godliness, stop reading right now and forget about the blog entries. It’s going to be dark. Pitch black at times actually, with a twist of humor, if you can laugh at yourself that is. And no, there is no light to shine on things, not even when you lite it. You see, pointing a beam in to the vast empty darkness of human nature doesn’t make you see anything. You do realize there is an endless, vast and cold emptiness out there. This is not unlike the cerebral content of way to many people I come across by in this crazy twilight zone called “the workplace”. I believe some US colleagues refer to those bio carbon life forms as “sheeple”.

Last year my very first blog post (http://workinghardinit.wordpress.com/2010/01/16/hello-world/) was about the one and only meeting I ever had with the Enterprise Architecture consultants that came in to help out at place where I do some IT Infrastructure Fu. Now one year, lots of time, money, training and Power Point slide decks after that meeting, the results on the terrain are nowhere to be seen. Sure there were lots of meetings, almost none of which I attended unless they dragged enterprise architecture into an IT related meeting on some other also vague action items like the IT strategy that was never heard of again. They’ve also created some new jobs specifications and lots of lip service and they’ll probably hire some more consultants to help out in 2011. But for now the interaction with and impact of any Enterprise Architecture on their IT infrastructure is nowhere to be found.

We put a good infrastructure plan in place for them. It’s pretty solid for 2011, pretty decent for 2012 and more like a road map for the time span 2013-2014. Meaning it’s flexible as in IT the world can change fast, very fast. But none of all this has come to be due to insights, needs, demands or guidance of any enterprise architecture, IT strategy or business plan.  No, it’s past experience and gut feeling, knowing the culture of the organization etc.  Creating strategies, building architectures is difficult enough in the best of circumstances. Combine this with fact that there is a bunch of higher pay grade roles up for grabs and the politics become very dominant. Higher pay grades baby? What do I need to get one? Skills and expertise in a very critical business area of cause!  Marketing yourself as a trusted business advisor, taught leader and architect becomes extremely important. As you can imagine getting the job done becomes a lot more difficult and not because of technical reasons. My predictions for 2011 are that by the end the year those pay grades will have been assigned. Together with a boatload of freshly minted middle management, who’ll be proud as hell and will need to assert their new found status, they’ll start handing out work to their staff.  Will that extra work materialize into results or only hold them back from making real progress? Well, we’ll need to wait for 2012 to know as 2011 will be about politics.

Basically from the IT infrastructure point of view and experience we have not yet seen an Enterprise Architecture and I don’t think they’ll have one in the next 12 months. Perhaps in 24 to 36 months but by then the game plan in IT infrastructure will be up and running. So realistically, I expect, if it leads anywhere against expectations, the impact of an Enterprise Architecture will be for 2014 and beyond. Which means an entirely new ball game and that will need a revised architecture. The success of the effort will no doubt be that they detected the need to change. This sounds uncomfortable similar to the IT strategy plan they had made. So for now we’ll do for them what’ we’ve always done. We’ll work with one year plans, two to three year roadmaps combined with a vision on how to improve the IT infrastructure. The most important thing is to stay clear of ambition and politics. Too much of that makes for bad technical decisions.

You got to love corporate bull. They don’t lie, no sir, they just sell bull crap. Which is worse, truth or lies don’t even matter, just the personal agendas. Liars at least, by the very fact of lying, acknowledge the value of truth, so much in fact, they’d rather have you not knowing it. Most consultancy firms send out kids that are naïve enough to believe the scripts and don’t even realize they are talking crap. They are told over and over again they are right, the best and they like to believe this so much they really do. It’s a bit like civil servants at the EU. Pay people double their market value, sweet talk their ego’s all day long and they will become prophets for the religion of the day. No, I’m not saying Enterprise Architecture is bull crap. I’m saying that way too many people & companies claiming to do enterprise architecture are turning it into exactly that. IT strategies, architectures that are so empty and void of content that all those binders are thrown in a drawer never to be seen again. A fool with a tool is still but a fool. Agile methodologies or tools don’t make your programmers agile gurus just like owning a race car doesn’t make you a race car pilot. All of this has happened before, and all of this will happen again. Every new, innovative process, methodology or concept falls victim to this. The money grabbing sales crowd gets there paws on it and starts selling it as competitive advantage or even innovation in a bottle to the corporate sheeple & management failures that should know better. They end with less money, loads of wasted time and a shitload of dead trees. As a side node, this whole “* Architect” thing  has runs it’s inflationary course. We need a new professional status currency once more. Take care and keep laughing clip_image001!

Exchange 2010 Public Folder Worries At Customer: No existing ‘PublicFolderProxyInformation’ matches the following Identity


A customers was recently using the EMC GUI in their Exchange 2010 environment, having a look a the public folder properties when they got this error:

—————————
Microsoft Exchange
—————————
Can’t log on to the Exchange Mailbox server ‘DAGMBX.demolab.com’. No existing ‘PublicFolderProxyInformation’ matches the following Identity: ‘\demolab\HeadQuarters\FincanceDepartment\FiscalUnit’. Make sure that you specified the correct ‘PublicFolderProxyInformation’ Identity and that you have the necessary permissions to view ‘PublicFolderProxyInformation’.. It was running the command ‘Get-MailPublicFolder -Identity ”\demolab\HeadQuarters\FincanceDepartment\FiscalUnit” -Server ‘DAGMBX.demolab.com”.
—————————
OK  
—————————

image

Hey … when did this start?  They never complained about this before, but did they ever use it.This probably was actually the first time they tried to look/edit the public folder permissions after doing the following over the past month and in this particular order:

  1. Moving to Exchange 2010 SP1
  2. Removing the last Exchange 2007 servers from the organization.

Now I know about a bug that exist and that was recently blogged about by Dan Rowley in Exchange 2010 get-mailpublicfolder \name returns No existing ‘PublicFolderProxyInformation’. The point is that there should be a mailbox database mounted on the server that has the System Attendant mailbox associated with it.  However, this is not the case here.  The mailbox servers are member of a DAG and all of them host a copy of the PF. The replication runs fine, users can work with them, the remaining Outlook 2003 users report no issues. But there is more in that blog: “Basically the work around is to mount a mailbox store on the server that is generating the error, or if there is a database already mounted – verify the system attendant is properly configured to point to a valid homemdb.” Now that last point is interesting and indeed that was the issue here. On two members of the DAG the homeMDB attribute was not set. Now what could be the root cause of this? I don’t know, certainly not in this case. All things have been done by the book … Ah well, luckily the fix is not very difficult. We need to put a valid entry in the homemdb. In this case we’ll take the value of the DAG member that had it filled in. This seems to be the most recently created database in the DAG. In Exchange 2010 this is done as described below. Note we have a DAG here, so we can work with any database that has a valid copy on the server(s) in question.

How to check the homeMDB attribute value:

  • Start ADSI Edit and navigate to CN=Configuration,DC=,DC=,DC=/Services/Microsoft Exchange//Administrative Groups/Exchange Administrative Group (FYDIBOHF23SPDLT)//Servers/MBXServerWithIssue
  • Right-click Microsoft System Attendant, and then click Properties to display the  Attributes list and find the homeMDB attribute.
  • If the homeMDB attribute has a value make sure  it points to a valid mailbox database. If the value of the homeMDB attribute is empty (not set) or incorrect you need to fix this.

image

How Fix the homeMDB attribute value:

  • In ADSI Edit navigate to Start ADSI Edit and navigate to CN=Configuration,DC=,DC=,DC=/Services/Microsoft Exchange//Administrative Groups/Exchange Administrative Group (FYDIBOHF23SPDLT)/Databases."
  • Right-click a mailbox database that is local (NON DAG) or has a valid copy on the server (DAG) , select Properties and in  the Attributes list, select the distinguishedName, and then click View.
  • Copy the value of the distinguishedName attribute and close the dialogs

image

NOTE in this particular case we can copy the value that was filled in the homeMDB attribute on one of the DAG members. You might not have one set in any.

  • Right-click Microsoft System Attendant, and then click Properties to get to the Attributes list, click homeMDB, and then choose Edit
  • In the Value box, paste the value that you copied form the distinguishedName attribute
  • Close the dialog boxes and exit ADSI Edit

When you’ve don this you’ll find following entry in the application event viewer:

Log Name:      Application

Source:        MSExchangeSA

Date:          11/2/2010 3:25:59 PM

Event ID:      9159

Task Category: General

Level:         Warning

Keywords:      Classic

User:          N/A

Computer:      DAGMBX.demolab.com

Description:

Microsoft Exchange System Attendant has detected that the system attendant object in the DS has been modified. System Attendant needs to restart the Microsoft Exchange Free Busy Publishing Service.

image

After that, I wait 10 minutes to get AD replicated and make sure to close the EMC and start it again and voila, it’s fixed.

No ADSI Edit required to fix “Object is read only because it was created by a future version of Exchange: 0.10 (14.0.100.0). Current supported version is 0.1 (8.0.535.0).”


During the removal of the last Exchange 2007 SP3 Mailbox server after completing the transition of Exchange 2007 to Exchange 2010 SP1 we ran into the following well known error: Object is read only because it was created by a future version of Exchange: 0.10 (14.0.100.0). Current supported version is 0.1 (8.0.535.0).

 

image

The issue is that due to the coexistence of Exchange 2007 & Exchange 2010 we can no longer remove the public folder database with the Exchange 2007 GUI (EMC). But the public folder is not visible in the Exchange 2010 GUI (EMC) as it lives on an Exchange 2007 server. Trying to remove the public folder database manually using the Exchange 2007 GUI confirms this, you’ll get the same error.

This error has been described in some blogs as early as October 2009 on http://www.proexchange.be/blogs/exchange2010/archive/2009/10/28/remove-exchange-2007-mailbox-role-fails-with-error-object-is-read-only-because-it-was-created-by-a-future-version-of-exchange-0-10-14-0-100-0-current-supported-version-is-0-1-8-0-535-0.aspx and later on as recently as October 2010 on http://www.howexchangeworks.com/2010/10/object-is-read-only-because-it-was.html

The described solution/work around in these blogs get the job done perfectly, using ADSI Edit to delete the offending Exchange 2007 public folder database. It wouldn’t be the first time ADSI Edit saves an Exchange Consultants proverbial bacon. But if it can be done without using it I often recommend not to do it.  I’ve seen to many over eager deletions in ADSI Edit get people into trouble (like deleting a public folder database before it could be dumped safely without data loss).

For this problem, it’s not required to use ADSI Edit to get rid of the public folder on the Exchange 2007 Mailbox server. You can just fire up the Exchange Command Shell (EMS) in Exchange 2010 and execute following PowerShell command:


Remove-PublicFolderDatabase "E2K7MBX\SGPublicFolders\StoreSGPublicFolders"

Confirm   4: Are you sure you want to perform this action?5: Removing public folder database "E2K7MBX\SGPublicFolders\StoreSGPublicFolders".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): y
WARNING: The specified database has been removed. You must remove the database file located in K:\E2K7Data\SGPublicFolders\PublicFolderDatabase.edb from your computer manually if it exists. Specified database: PublicFolderDatabase

This works just fine. I have no objections using ADSIEdit when needed but I don’t advise using it to others unless really necessary.In this case it just isn’t needed to fix the problem. For good measure I also deleted the storage group in which the public folder lived. After that the install went well end without any issues.

EMC Does Not Show All Database Copies After Upgrade To Exchange 2010 SP1– Still Investigating


LATEST UPDATE March 9th 2011: I have installed Exchange 2010 SP1 Rollup 3 at customer and this did indeed fix this issue finally.

Updates to this post are being added as we get them below. Last update was October 13th 2010. The have identified the cause of the issue. It’s a case sensitivity bug. The fix is WILL be contained in Exchange 2010 Sp1 Roll Up 3? But they ARE working on a incremental update in between. See below for more details and the link to the Microsoft blog entry.

At a customer we have a 3 node geographically dispersed DAG. This DAG has two nodes in the main data center and one in the recovery site in another city, but it is in the same AD Site. This works but is not ideal as DAC in Exchange 2010 RTM presumes that the node will be in another Active Directory site. As you can imagine at that location we’re very interested in Exchange 2010 SP1 since that adds support for the DAC to be used with a geographically dispersed DAG node in the same Active Directory site.

We did an upgrade to SP1 following the guidelines as published in http://technet.microsoft.com/en-us/library/bb629560.aspx and we made sure all prerequisites where satisfied. We upgraded the backup software to a version that supported Exchange 2010 SP1 and made sure no services that hold a lock on Exchange resources are running. The entire process went extremely well actually. We did have to reconfigure redirection for OWA as the SP1 installation resets the settings on the Default Web Site on the CAS Servers. But apart from that we had no major issues apart from one very annoying GUI problem. Everything was fully functional, which we verified using EMS and by testing failovers. But in the EMC GUI we had the problem under Organization / Mailbox / Database Management we only see the database copies listed on one server and not on all tree.

clip_image002

When you check the properties of the databases shows all three servers that are hosting copies. We used EMS commands to test for problems but it all checks out and works. Failing over a server works, both in the GUI and in PowerShell, just like activating a database.

The same issue can be seen in Server Configuration /Database Copies as demonstrated in the screenshots below. In the first figure you we selected the mailbox server where the database copies are visible.

clip_image004

But on the other two nodes nothing shows up, just “There are no items to show in this view”.

clip_image006

No errors in the vent logs or installation logs. All is working fine. So what gives? We tried all the usual suspects like throwing away any user related MMC cache information and cleaning out the Exchange specific information in the user profile up to deleting the profile etc. But nothing worked.

Running the script below, which is given to you by Microsoft to check your DAG before upgrading to SP1, confirms all is well.

(Get-DatabaseAvailabilityGroup -Identity (Get-MailboxServer -Identity $env:computername).DatabaseAvailabilityGroup).Servers | Test-MapiConnectivity | Sort Database | Format-Table -AutoSize

Get-MailboxDatabase | Sort Name | Get-MailboxDatabaseCopyStatus | Format-Table -AutoSize

function CopyCount

{

$DatabaseList = Get-MailboxDatabase | Sort Name

$DatabaseList | % {

$Results = $_ | Get-MailboxDatabaseCopyStatus

$Good = $Results | where { ($_.Status -eq "Mounted") -or ($_.Status -eq "Healthy") }

$_ | add-member NoteProperty "CopiesTotal" $Results.Count

$_ | add-member NoteProperty "CopiesFailed" ($Results.Count-$Good.Count)

}

$DatabaseList | sort copiesfailed -Descending | ft name,copiesTotal,copiesFailed -AutoSize

}

CopyCount

Searching the internet we find some folks who have the same problem. Also with a 3 node DAG that is geographically distributed. Is this a coincidence or is this related? http://social.technet.microsoft.com/Forums/en-US/exchange2010/thread/37d96c3d-433e-4447-b696-c0c00e257765/#5071f470-13cb-4256-8aa7-ade05bb4d67d. At first I taught it might have been related to the issue described in the following blog post http://blogs.technet.com/b/timmcmic/archive/2010/08/29/exchange-2010-sp1-error-when-adding-or-removing-a-mailbox-database-copy.aspx but in the lab we could not reproduce this. The only thing we managed to confirm is that you can delete the Dumpsterinfo registry key without any problem or nasty side effects. I’m still looking into this, but I’ll need to get Microsoft involved on this one.

Updates:

  • As an other test we created a new mailbox database and by the time we got the copies set up to the 3 nodes that brand new database and its copies showed the same behavior. For that new database the registry key Dumpsterinfo doesn’t even exist (yet?). So  That’s another nail in the coffin of the idea that behavior being related to the Dumpsterinfo key I guess.
  • Next test was that I added two static IP addresses to the DAG. One for each subnet in use. Until now we had a DCHP address and I noticed it was an address for the subnet of the node that is showing the database copies. I might as well give it a try right? But nope, that didn’t make a difference either. Still waiting for that call back from Microsoft Support.
  • Meanwhile I’m thinking, hey this DAG is only showing the database copies with the lowest preference (3). So I change the preference on a test database to 1 and refresh the EMC. No joy. This must really be just a GUI hiccup or bug. Now what would prevent the EMC GUI from displaying that information?
  • Some one on the newsgroup has the same issue with a 2 node DAG in the same subnet. So not related to a 3 Node geographically dispersed DAG.
  • MS Support got in touch. They have heard it before. But unless it was related to net logon errors they don’t have a cause or solutions. There are other cases and they will escalate my support call.

On September 27th 2010:

  • After a call from an MS support engineer to confirm the issue and pass on more feedback last week, we got an update via e-mail. After completing a code review and analysis they believe to have identified the problem.  They have also been able to reproduce the issue. More information is being gathered with reference customers to confirm the findings. More updates will follow hen they have more information on how to proceed. Indeed all is well with Exchange 2010 SP1 and PowerShell is your friend :-) Well progress is being made. That’s good.

On October 4th 2010:

We requested feedback today and tonight we got an e-mail with a link to a blog post confirming the issue and the cause. When the Exchange Management Console draws the database copies pane, it compares the host server name of a database copy to the server name of a database copy status.  This comparison is case sensitive and if they do not match up like in DAG-SERVER-1 <> Dag-Server-1 the database copies are not shown in the GUI. Again in EMS all works just fine. A fix is still in the make. You can find the Microsoft bug here: http://blogs.technet.com/b/timmcmic/archive/2010/10/04/database-copies-fail-to-display-after-upgrading-to-exchange-2010-service-pack-1.aspx

On October 10th:

I received another mail from Microsoft support just now. They expect this issue to be fully resolved in Exchange 2010 Service Pack 1 Rollup Update 3.  At this time they also intend to release an incremental update that corrects the issue. But this has some caveats.

1)  The incremental update would have to be applied to all servers where administrators would be utilizing the Exchange Management Console.  I think this is expected, like with most updates.

2) The incremental update cannot be applied with other incremental updates – for example if later an issue is encountered that is fixed in a different incremental update one would have to be removed prior to installing the second.  This can be a problem for people in that situation, so pick what is most important to you

3) The incremental update would only be valid for a particular Rollup Update.  For example, if the incremental update is installed for Exchange 2010 SP1 RU1, and you desire to go to Exchange 2010 SP1 RU2, you would have to contact Microsoft to have the incremental update built and released for Exchange 2010 SP1 RU2.  This may inadvertently delay the application of a rollup update.  Nothing new here, we’ve seen this before with interim fixes.

The workaround for customers not desiring to install an incremental update would be to continue using the Exchange Management Shell with the Get-MailboxDatabaseCopyStatus command. Nothing new here Smile

They have also updated their blog: http://blogs.technet.com/b/timmcmic/archive/2010/10/04/database-copies-fail-to-display-after-upgrading-to-exchange-2010-service-pack-1.aspx

I’m planning on keeping the case open in order to get my hands on the fix to test in the lab and have it for customers who so desire.

October 13th:

The fix WILL be included in Exchange 2010 SP1 Roll Up 3. They ARE working on the interim updates but this will take several weeks or longer.

 

Microsoft Project 2007-2010 Default Format When Saving Group Policy


For a roll out of Office 2010 we needed to set the default format when saving to Microsoft Project 2007 and not Microsoft Project 2010. This was the prevent any accidental file format issues with people still using Microsoft Project 2007. The rest of the Office 2007 formats such as Word, Excel, Visio, … have remained the same as in 2007 (in the case of Visio even the same as 2003).

We had already loaded the ADMX GPO templates for Office 2010 some and intend to set the default to Microsoft Project 2007. So we navigate to User Configuration => Policies => Administrative Templates: Policy definitions (ADMX files) retrieved from central store => Microsoft Project 2010

MSProject2010-1 (2)

Where we find following options:

MSProject2010-2

Funny we thought we’d find three options: Project 2010, Project 2007 and Project 2000-2003. Now what mpp format would that “Project” setting refer to? Microsoft Project 2007 or 2010? Well we checked the contents of the ADMX file proj14.admx which can be found, when using the central store that is, at:

\\blog.demo\SYSVOL\blog.demo\Policies\PolicyDefinitions\proj14.admx

There we see the following:

MSProject2010-3

Aha it’s the mpp format of Microsoft Project 2007 as indicated by MSProject.MPP.12. An entry for Microsoft Project 2010 doesn’t even exist. We quickly had a look at the proj12.admx (Microsoft Project 2007) and there the entries are exactly the same for that setting for “Save” policy.

It seems that even when the file formats is different between Microsoft Project 2007 and Microsoft Project 2010 they offer no way to easily identify what mpp format you’re setting it to in the GPO. Why they don’t offer Microsoft Project 2010 I don’t know. I guess their take is that you could leave it disabled in the GPO. Which is true but GPO setting are normally very unambiguous and you can explicitly set every option when you enable it, even the default when disabled. Perhaps they just forgot as they did not need this for the other office products in the Office 2010 release. I don’t know.

Anyway we tested this and indeed it set the default format for saving to Microsoft Project 2007. So all is well. Tip of the day: check the content of the ADMX file when in doubt.

Exchange 2007 & 2010 Event ID’s: 2601, 2604, 2501 & Users Can’t Access Mailboxes / Public Folders On My Day Off


I took the day off as I needed some time to deal with government administration. Good thing this is a blog about IT issues because holey crap what a time eating, confusing and rather pointless mess government administration can be. The process to get to the desired outcome is very tedious, prone to misunderstanding & pretty inefficient . What the entire duration of the process and the number of administrative entities involved contribute to the desired result is a mystery. It’s pure show and window dressing. But OK, we took the day of to finally get it all sorted after 5 months of patiently waiting for this day.

So I sleep until 08:00, get up and head for the kitchen for a jar of coffee. With the only Java I truly like in my hand I make my way to the home office. I check mails/alerts from System Center, Support Requests etc. I’m like a responsible guy dude, even when I need a day off. I do monitor the condition of my projects in production and I do step in when needed and document my findings. It keeps me honest when I design and sell my solutions. Beware of some architects that are not the ones having to deal with the crap architectures they design, they are often empty suits. Anyway, I see an issue that could be a warning of more to come. Someone has a problem with Outlook 2007 which reports the following error (translation from Dutch):

“Unable to expand the folder. The Microsoft Exchange Server computer is not available. Either there are network problems or the Microsoft Exchange Server computer is down for maintenance.(/o=<DOMAIN>/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Configuration/cn=servers/cn=<dagmember1>)”

Now I know that user. Smart, diligent and reliable. That user even provides the relevant and necessary information in their support request. Yes they do exist and HRM should hire those exclusively. So in combination with that error we knew we did not have an PEBKAC or ID-10T on our hands but a real issue.

I quickly check that DAG member node Outlook of that user is trying to connect to but I know that due to maintenance their mailboxes currently reside on another member of the DAG. So i could very well be just the public folders. Bingo. A quick test reveals this to be the case. Also the Windows 2008 R2 server and Exchange 2010 itself are running perfectly fine, happy as can be, except on that one node we see the Application Event Log messages:

Log Name:      Application
Source:        MSExchange ADAccess
Date:          8/19/2010 7:12:43 AM
Event ID:      2601
Task Category: General
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      dagmember1.company.blog
Description:
Process MSEXCHANGEADTOPOLOGY (PID=1620). When initializing a remote procedure call (RPC) to the Microsoft Exchange Active Directory Topology service, Exchange could not retrieve the SID for account <WKGUID=XXXXXXXXXXNOREALIDXXXXXXXXXXXXXX,CN=Microsoft Exchange,CN=Services,CN=Configuration,…> – Error code=8007077f. The Microsoft Exchange Active Directory Topology service will continue starting with limited permissions.

Log Name:      Application
Source:        MSExchange ADAccess
Date:          8/19/2010 7:12:43 AM
Event ID:      2604
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      dagmember1.company.demo
Description:
Process MSEXCHANGEADTOPOLOGY (PID=1620). When updating security for a remote procedure call (RPC) access for the Microsoft Exchange Active Directory Topology service, Exchange could not retrieve the security descriptor for Exchange server object DAGMEMBER1 – Error code=8007077f. The Microsoft Exchange Active Directory Topology service will continue starting with limited permissions.

Log Name:      Application
Source:        MSExchange ADAccess
Date:          8/19/2010 7:12:43 AM
Event ID:      2501
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      dagmember1.company.blog
Description:
Process MSEXCHANGEADTOPOLOGY (PID=1620). The site monitor API was unable to verify the site name for this Exchange computer – Call=DsctxGetContext Error code=8007077f. Make sure that Exchange server is correctly registered on the DNS server.

I think I’m OK when I see the possible cause. Why? Because I also know even if that probable cause isn’t the problem, it’s a hiccup I’ve seen before and I know how to fix its one. When you search those errors you can find a TechNet article describing a possible cause: “An inactive network connection is first on the binding list” http://technet.microsoft.com/en-us/library/dd789571(EXCHG.80).aspx. The fix is quite simple. Correct the NIC order and restart the MSExchange ADTopology Service. I had my scare about Active Directory and DNS horrors the first time I ever saw this one. So no gut wrenching panic here :-)

But why do servers ever get in to this state when the NIC ordering is just fine? We did some firmware and upgrade recently after hours but that didn’t affect the NIC binding order. Now I’m pretty weird at times but I still know what I’m doing. Those NIC where OK when I configured those servers. Checking that has become a second nature on multi homed and clustered servers. I also remember happening this to me once before somewhere in February 2010 with another setup of Exchange 2010 on Windows 2008 R2. And in that case the NIC order in the binding list was also OK. I checked back then as well just to make sure. But since I build those Exchange 2010 setups myself I just know they are close to godliness both in design and implementation :-). Back then the issue went away by restarting the server, restarting the MSExchange ADTopology Service will do however, and the problem never came back. For some reason the AD Site information query fails. Now Windows retries and is OK after a while. Exchange, tries to get the AD Site information once, fails and keeps thinking there is an issue. With as a result clients have no connectivity and those errors that initially make you think you could have DNS issues, AD problems etc. But fortunately it’s a lot less serious.

So when the NIC binding order is OK why does this happen? I can’t tell you for sure but I do know that I’m not the only one (not that weird after all) since Microsoft published KB Article “MSExchange ADAccess Event ID’s 2601, 2604, 2501” http://support.microsoft.com/kb/2025528 . This article is a so called FAST PUBLISH from Microsoft Support and states that the issue only occurs on Windows 2008 R2 and that it affect Exchange 2007 and Exchange 2010. The cause? Well this is where they provide only what I already knew:

“During a restart of the server, the operating system queries Active Directory to get its AD Site information.  On a Windows 2008 R2 server, this will sometimes fail.  As the Exchange services are starting, it also will do a query for its AD Site and that too will fail. Windows will continue to try and determine its AD Site name and will eventually succeed.  However, Exchange does not re-try the query and the above errors are logged in the application log every 15 minutes.”

And yes the workaround/fix is also nothing new:

“After the server has been up for a minute or two, run NLTest /DSGetSite to verify that that the proper Active Directory Site is being returned by Windows.  Once that has been verified, restart the MSExchange ADTopology Service.”

Do note that this will also restart a slew of dependant Exchange services so it takes a little while.

  • Microsoft Exchange Transport Log Search
  • Microsoft Exchange Transport Log
  • Microsoft Exchange Service Host
  • Microsoft Exchange Search Indexer
  • Microsoft Exchange Replication Service
  • Microsoft Exchange Mail Submission
  • Microsoft Exchange Mailbox Assistants
  • Microsoft Exchange File Distribution
  • Microsoft Exchange EdgeSync
  • Microsoft Exchange Anti-spam Update

So after some manual intervention we had the users back in business. And all is well for them, as they rise and sleep under the watchful eye of a bunch of good IT Pro’s who’ll protect them form further harm and problems ;-) Now I need to get an auto fix for this I think until Microsoft fixes this one for good. SCOM where are you? No, no, no … It’s my day off for getting that administration done!

IT Strategies from Window NT 3.51 to the Cloud Era – Part 2


How do we get strategic IT?

That’s a hard question to answer. I can only give you my take on the subject. It’s far from complete but it addresses the way I look at it and how I try to do my part in achieving this. This is part 2 of a series on IT Strategies. Just some musings. You can find Part 1 here: http://workinghardinit.wordpress.com/2010/08/02/it-strategies-from-window-nt-3-51-to-the-cloud-era-part-1/

Merely leveling the playing field or value adding instrument?

Knowing what you have learned already, take a good look at the Total Cost of Ownership (TCO) and the Return on Investment (ROI) of the technology you own. You might wonder if any of those expensive IT systems are ever going to live up to all their potential. Can you easily pin point some real and substantial business benefits from your technology investments? We sincerely hope you can. Or do you have the “feeling” that most of that capital computing power is just sitting there idle and is hopelessly underused?

Use IT or lose IT!

Even in this day and age, it never ceases to amaze us how many routine and repetitive tasks are still done manually. IT can do two major things for your business. One is to improve physical operations by making them more efficient and the other is to reduce logical operations by automating as much of it as possible. If IT is not doing that, something is wrong. Why on earth are people reduced to mere (and rather slow and unreliable) biological processing units within the IT systems they work with? This not only wastes productivity but also leads to many avoidable errors that are not only time consuming to clean up but are often untraceable. The point is that you don’t want it to be like this and you need to fix it. This is not a question about wanting your technology to deliver value to your operations but needing it to do so. Not using your IT’s potential is literally squandering vast amounts of money. No business can afford to do that so you’ll either have to deal with that issue or risk losing funding for IT. Which means you’ll be even worse off. Any chance of ever recuperating the money you invested in technology has gone and funds to compete and get ahead of your competitors can only be dreamt of. This is not a good situation to be in and even harder to get out of.

Let’s face the fact that IT very often sucks at delivering value. That’s not technologies fault directly but it’s how we use it, or rather, abuse it. Business doesn’t know a thing about what IT can or can’t do and what’s the real cost of that. IT often is run by IT managers and CIO’s that are busier entrenching themselves in vast layers of expensive and often overly complex hardware and software. The IT industry itself is a willing accomplice. They really like to sell all that stuff and services as they love the color of money. They are partners in crime and hide that painful fact behind change management, methodology religion and an air of “we are keeping the sky from falling down”. But how do they explain it takes 14 days to restore a file server to service, that it costs 3 to 6 times more for enterprise IT to do than the market will bear? I’ll give you a hint, partially because the need to hire consultants to do their jobs. Why does it take over 4 years to consolidate to a new mail system in big government agencies? What happened Look, in corporate life a lot of what’s been doing done is bull shit and exists only to create career opportunities, to inflate ones importance and (they hope) ones wages. They built extremely expensive, large, complex systems that are hell to touch, let alone, change, improve or replace. People all follow their own time limited agenda’s and build up an extensive technology debt that will wreck a business sooner or later. But no one, not in business or in technology wants to acknowledge that. When you do bring it up and try to evangelize better ways you’re a bit of an annoying person rocking the boat. Hey, what we don’t need 10 consultants, a 20 person project team and 2 years to add a sub domain to Active Directory? What do you say? Our home build MIS that took 2 years and a couple of million € could have been bought for 3000,00- € with 80% of the features. That must be some real important report content that only our system can produce I bet! You think I’m joking? I hope you think that because you’re all doing it right and are not in an industry that will collapse by doing what I described above. Fast forward 10 to 15 years. What will “the cloud” have done to your little empire? Ok perhaps you’ll be retired by then, what if you’re 45 and a fake IT professional, developer or heaven forbid a fake consultant? Yeah you’ll call it age discrimination at the unemployment office, I’m sure, but what good will that do you really? But even then the true fakers will thrive. Never, ever forget that if the organization is large enough perception outperforms reality and, guess what fakers experts are at? Right! The most “reliable” consultants seem to work for organized crime these days (Consigliere). Make a mistake in that field of endeavor or don’t deliver and you find yourself standing on a piece of industrial plastic in the bosses office looking down the biggest and blackest hole you’ll ever see. And that shiny sliver of light at the end is most likely a Winchester Silvertip® Jacketed Hollow Point. They don’t buy crap services, only results.

Now What?

Now given all this, how the hell do we make IT a strategic asset? Well let’s first go and make sure we get a functional IT environment and then go for tactical, operational and strategic value. Just focusing on the strategy is not enough and even dangerous. The operational aspect is very important as well and you really do need both. To achieve strategic IT you really need to have an IT that’s functional. In the end you also need context, knowledge, skills, hard work and a “getting things done” attitude. Please do not think you can by this as a product or a service. You can only achieve an IT strategy by hard work and dedication. Trying to use recipes or methodologies without context is doomed to fail. It just doesn’t work. Unfortunately so many organizations really do that. Why? Because to many bad consultancy firms are milking those tools for the cash cows they have become. Look consultancy firms live by cheap personnel (to them, not to you) that get their jobs done by following guide lines, boilerplate documents and such. They couldn’t survive without them, the methodologies and processes are their life line. So when those don’t work they’ll sell you more of the same because that’s all they can handle and know. They don’t really care about your business. Essentially they are what are giving methodologies a bad name. They are used to let mediocre firms with mediocre or even incompetent personnel survive. Stay away from them. They have a hammer and they’ll use that tool everywhere for every job so basically they are frauds. This often continues way to long on because so much money is involved no one even dares to tell the truth anymore. Have you ever wondered why most over certified “BIG” corporations are hated and despised by their customers? They are over organized, to many weird processes. On paper all is well to perfect but in reality it doesn’t work anymore. This is probably the reason why we’re losing so many jobs in the West. We’re too expensive for the limited productivity we have and are burdened by the cost of dysfunctional methodologies. Are the other areas better? Nope they are cheaper, hungry for a better life and willing to make a real and prolonged effort.

Invest In Solutions & People

Basically what we’re saying is to stop buying technology and instead start investing in solutions. Solutions are built with technology. But there is more to it. Used correctly, technology can be a lot more than just a means to an end. The solutions built with technology must become an instrument that provides operational, tactical and strategic advantages to your organization. This means that IT will then realize its true potential.

So here is where your ICT staff (external or internal) can and must really differ from other commercial ICT providers or personnel. That piece of hardware and the operating system running on it is more to us than just a means to sell even more services, hardware and software. To us, it is the very basis of working solutions to real business issues. That’s the attitude we have towards addressing your problems. This requires knowledge, experience and dedication to finding the best approach for your organization.

This means that often you don’t have to buy yet more or new technology and software to solve problems. The best hidden secret in IT seems to be the value and opportunities that are available within most systems. It’s time for you to find them and be pleasantly surprised by what you what you already have but are not using. Use the money saved here to differentiate you in the market and be competitive. To achieve this you need knowledgeable people who care about the business.

Select your vendors and consultants with care. They should have one goal and that is to help you solve existing problems cost effectively and as a result deliver value to your company. They should not have the goal to sell only products or time to fix – perhaps even only perceived – problems. Think about it, you do not want to buy consulting hours, a dual CPU server or a body to fill an empty cubical. You want to buy a solution to an existing need. Of course we do work with vendors and other consultants in order to help you, but they have to agree and comply with our approach. If not, that’s a collaboration terminating event. It’s this attitude that helps us provide outstanding services. Yes we keep up to date on technology advances in our field of expertise, yes some of us are certified but we do not sell you acronyms and titles instead of skills. We are certified because we are experts, not vice versa. Most of all we are interested in learning about your business needs and matching them to satisfying solutions. Good consultants and IT Professionals will always seek the best way to help you. They will do this even when this means hiring others to do the job. If not, they are body shops who have only one goal, renting out people at daily rates. Walk away from these. They will never produce the results needed. Likewise, people just writing strategy documents, enterprise architecture designs and practice methodology religion that doesn’t improve or even relate to the realities in the field is nothing more than wasting money. They who do that are frauds. Why do I call this a “religion”? Because they only promise things later when you’re a good boy and when it never delivers you must have sinned against the methodologies? Oh please … enough already! Typical bullshit form people with no real field experience and knowledge, let alone the context to achieve something with methodologies.

A Vision on Realizing Solutions

Today anyone in the world with the right skills, attitude and motivation can provide services that result into high quality information technology solutions for any small, medium & large sized businesses around the globe. Fast internet connections in combination with secure remote administration tools have made this possible, especially when you realize that many technology implementations are being replaced by “good enough” services on the internet. Well, what good clients, employees and managers have in common is that they take their automation and computing needs seriously. However, they don’t always have the resources or know-how available to make the most of their investments in technology. By providing the necessary advice, assistance and implementation, technologists can add value to a company instead of delivering just another cost factor that merely levels the playing field. Solutions may range from single server systems over organization wide infrastructure roll outs to hybrid cloud implementations but the principle remains the same. We offer the knowledge and experience needed to make technology work for the business. A properly implemented IT solution can provide tactical, operational and strategic advantages in the market place. It makes you more competitive. To achieve these goals we rely on a number of key elements in any successful professional relationship.

Trust

All long term business relationships are based on mutual trust. Trust is not given away for free, it needs to be earned. You will need to establish a trust relationship between business and ICT in order for them to be your counselor and guide for current and future technology needs.

Integrity

Technology is a tool, a means to an end. Good IT offers to help your company use the tools it has or requires to their fullest potential. The business needs are important and IT should care about finding the right solution that the organization. An IT strategy must show commitment to work according to your needs and budget to find the right level of services and technology for your company. This needs to be done at a pace you can sustain. The goal is to add value to your business, not to extract money from it and waste time performing unnecessary tasks. On the opposite side, IT advisors must be honest and be able to walk away from organizations and managers who are not capable of being professional and have no real desire to improve. Only time and money can be wasted in such a situation. The money can be replaced. Our time cannot so it should never go to waste.

Reliability

All technology can and will fail at some point in time. Such events can result in lost data, lost time and lost revenue. Providing a business with a well-designed infrastructure however, will help you avoid such risks. This, in combination with excellent troubleshooting skills, facilitates recovering from such “disasters”. Keeping IT systems running smoothly requires a proactive approach consisting of maintenance and periodic auditing of the systems. The result of this is the reliability needed in order to prevent unscheduled down time. In the cloud era the ones responsible for all that change but the essence remains the same. Be advised however that everything can and will fail. Unbreakable does not exist. The differentiator is how we prevent it and how we deal with it when it occurs.

Attitude

You have a lot of choices when it comes to choosing someone to help you with your computing needs. Experience is one of the distinguishing factors, as is commitment, know-how and the ability to develop a vision about a company’s IT needs. This attitude provides guidance to organizations, so they can enjoy the benefits of well-designed computing systems without draining vast amounts of capital from their financial reserves.

Technology

Understanding technology is paramount to providing a great service. You cannot design and build something you do not understand. The correct mix of hands on experience, knowledge of complementing technologies and design skills, combined with continuous training is what enables IT Pro’s to optimize your technology investment.

Isn’t this going to very or prohibitively expensive?

This remark always provides for an opportunity to convince the person of the real value ICT can add. I have always made a genuine effort to position the IT solutions I design as qualitatively superior in both our approach and in the results we deliver. But that doesn’t mean they are overly expensive. Within our area of expertise, we will always seek to find the best and most cost effective solution that is viable to that particular business need. We will customize that solution to “tailor fit” the environment, but only if and when required. If IT is prohibitively expensive somebody is not doing their job right.

This, in my opinion, is a great way to provide an outstanding service and results; the value of which is reflected in the investment costs. How can you expect strategic IT and its implementation to be cheap? Good value for money, yes, but cheap, no. Now, some companies will never pay the rates and price needed to achieve this. Sometimes, in many ways, they are totally correct in not doing so. The most likely and best reason is that they do not need our services, i.e. it is just not worth it. For them IT is nothing more than a necessary and valuable commodity. The unfortunate reason is that our services can’t help them. Some simplified examples might help to clarify this. A lawyer who uses his laptop for writing reports and printing them out is not going to pay us 150 Euros per hour for setting up his free e-mail in Outlook, and he’s absolutely correct in not doing so. However, an organization that needs an effective, secure and rock solid virtualization solution will appreciate the benefits of good design and is willing to make that investment to achieve results. Until the day arrives the cloud does that better and more cost effective.

Costs for ICT and rates for personnel are not dictated by the complexity or the lack of complexity in an organization’s environment. Rates are established based on knowledge and the value IT brings to your business. An environment’s complexity only dictates how much time we will spend designing, implementing and/or supporting it, not how much time or solutions are worth. In the end you are engaging skills and knowledge, not time. Smart managers and IT architects simply refuse to compete with “low cost, low skill” operators who can only follow generic guidelines (if even that) and are stumped when “it doesn’t work”. But since you are reading this there is most likely a reason why you are looking beyond those players. You are interested in high quality strategic, tactical and operational ICT! So there you have it. It is the combination of skills, aptitudes, work ethics and intellectual capital that enables IT professionals to offer quality services that amount to added value and competitive advantage for your business. In the end, isn’t that what you want for your organization?

Think of any solution like a triangle. The three sides of the triangle have the following characteristics that can all be a quality by itself or combined together, depending on the circumstances.

  • Cheap
  • Fast
  • Good

However be advised that the three sides combined together almost never materialize in real life. So you’ll have to pick two of them and, more or less, sacrifice the third. Which combination is needed depends on the issue at hand. Do not fool yourself into thinking that Information Technology is something that anyone can do by just reading a book, following some courses or owning a PC. It takes a tremendous amount of time, dedication and a sustained effort to become and stay competent and ahead in the world of IT, far beyond a 9 to 5 job. So realistically, if you expect this kind of expertise to be available at low ball rates or for free, perhaps it is time to take another look at the triangle and reevaluate your position on this. This is true in a mainframe, client server, web, Service Oriented Architecture and cloud world. The environment, technologies, tools and solutions change and we have to change with it.

Final Thoughts

It is important for both management and employees alike to realize that choosing a dynamic and scalable IT infrastructure solution in combination with a flexible and integrated development platform is very important. It provides one of the pillars for the successful implementation and support of the business strategy. Today this is no longer an opinion or a choice; it is a fact of doing business.

This means that these choices are as much a part of creating competitive advantage as any other strategic decision. The need to provide long term backward compatibility, support for current and new technologies, facilitate agile development and allow easy deployment and maintenance throughout its life cycle are paramount. All of this has to be done in an effective, efficient and affordable manner. These considerations are valid whether we are looking for a single line of business application or an entire infrastructure solution. Do realize solutions are fluid. There is no permanent solution, just the best solution at that moment for a certain period of time for that particular situation. That situation might be unique or it might be more common than muck. Much care must be taken not to build up an unsustainable technology debt and to keep working on reducing any non-managed debt present.

To end, never make the mistake of thinking that having a strategy is enough. You need the right people, the skills, the attitude and the guts where and when they are needed to execute your strategy. In essence it is never easy and it takes a lot of hard work. Plans and ideas that are not executed are worthless. Beware that acknowledging this must result in more than lip service to this vision. You must act upon it, translate it into plans, and provide leadership and guidance to achieve an IT strategy that will produce results. Money and tools are no substitute for solid skills and motivation. But those are subjects for another discussion. So for now, stop faking, study hard, work hard and build real good solid IT!

This is part 2 of a series on IT Strategies. Just some musings. You can find Part 1 here: http://workinghardinit.wordpress.com/2010/08/02/it-strategies-from-window-nt-3-51-to-the-cloud-era-part-1/