Home › 2013 › May

Monthly Archives: May 2013

Why should “normal” people care about IT?

Posted on by

I did a presales call with a small dentist office a few months ago.  I have some dentist customers, so I’ve come to understand a little bit about how dentist offices operate.  But this office was, well, different.  The “server” was really an old, failing Windows XP PC tucked away in a dusty corner of an unused office.  Other workstations were in similar states of disrepair.  This office had a challenge – the receptionist’s brother maintained all the IT equipment, but he recently moved from Minnesota to Colorado and was no longer available to come onsite and resolve the latest emergency.

I promise – I am not making this up.

Apparently, nobody knew how to boot their “server” and they had to call the brother every morning to get the office up and running.  The process was generally to turn that central PC off and back on again and hope it booted. Once up and running, nobody was to touch it for the rest of the day.  Although PCs in the exam rooms had the ability to store a few patient updates locally, they all depended on this failing central repository to access historical patient data, including X-Ray images.  If that PC died, all the X-Rays and all patient data died with it.

The dentist/business owner said he knew he needed a server and we talked at length about setting one up.  Then I asked my key question:  “What happens if this PC you’re using as a server dies and you can’t access any patient X-Rays?”  His answer left me speechless.  “I don’t need computers to practice dentistry.”  The meeting went downhill after that, and this dentist office never returned another phone call or email.

I use that dentist as an example because, unfortunately, his attitude seems typical of so many business owners and otherwise intelligent executives.  Here is another quick story to drive the point home.  Several years ago, I was in a presales meeting at a bank to talk about IT security.  The banker proudly showed me the shiny new security audit report he undoubtedly paid a small fortune for and asked me to look it over.   I sat across the table from him, looked over the report, and commented it covered the bank’s website pretty well, but where was the section about the bank internal IT operations?  His reply – “Thanks for coming over” and he quickly hustled me out the door.  To this this very day, I don’t know what nerve I touched.  But I have some theories.

I think IT is boring for most “normal” people.  Most people don’t care about what DHCP servers do or the difference between 1 gb and 10 mb.  Some bankers probably never stop to think about the difference between their internal operations and public facing website.  At least one dentist never took the time to think through what would happen if all his patient records disappeared.   And because IT is boring and “technical” and costs money, it must be at best a necessary evil.  For most business decision makers I’ve met, IT is not an asset to be maintained and enhanced, IT is an expense to be minimized.

This is a shame.  Consider:

  • medical and dental clinics, who keep patient data inside a server instead of a large room full of paper files and film X-Rays.
  • transportation companies who can keep images of millions of invoices and other paperwork inside a computer network instead of whole buildings filled with file cabinets
  • email and the world wide web
  • automated airline check-in systems
  • online banking
  • and thousands or maybe millions of other applications we take for granted today.

What would happen if we turned all those off?  Think about a bank branch without access to the central databases.  Think about an airline without the automated ticketing and check-in systems we’ve become used to using.  Think about cutting off access to email and the world wide web.  Think about a dentist trying to run a modern office without access to computers and historical patient data.

If you are a small business owner, here is a challenge.   Turn off your servers and computers for one day and try to run without them.  Observe the chaos that will surely follow.  Try to calculate the lost revenue from all the customer service disasters that will happen.  Try to calculate the increased cost when everyone has to operate manually, with no access to any data.

I dare you to take up these challenges and send me some comments about your experience.  And then, let’s have a conversation about how to protect your critical assets and how you can use IT to at least gain competitive parity and maybe a competitive edge versus your competition.

Computer Whodunit Detective Story – the Conclusion

Posted on by
­

In part one of our computer detective saga, the story opened with a few users unable to access their emails. Similar to a Hollywood detective story, we followed the clues through several unexpected twists and turns, with each clue answering questions and generating new questions.  Continuing in the style of great whodunit detective mysteries, we eventually uncovered the culprit, a rogue DHCP server.  This changed everything.

And now the conclusion.

DHCP – Dynamic Host Control Protocol – is the reason we can connect our laptops and tablets and smartphones to the Internet.  DHCP servers assign all the attributes our devices need to enable communications.  Think of the Internet as similar to the telephone network, but with one important difference.  In the telephone network, your phone number stays the same no matter where your phone travels. On the Internet, an IP Address defines your device.  But unlike phone numbers, IP Addresses change, depending on where your device is located.  That’s why we need DHCP servers, to assign IP Addresses and other attributes to devices when they attach to an office network or the Internet.

Here is how DHCP works.  When you connect your device to a network, your device sends a broadcast to anyone on the local network who will listen.  It’s essentially a cry for help.  (Help!  Load me with what I need so I can talk to the world.)  The DHCP Server listens to the broadcast and downloads an IP Address and other attributes to the requesting device.  This is called an IP Address lease, and the lease expires after a settable amount of time, called a TTL (Time to Live).  Once the device acquires its IP Address lease, it can interact with the world.

DHCP is a thing of beauty when set up properly and works so well, only a few hard-core IT people think about it anymore.  Except when things go wrong.  And one of the worst things that can go wrong is a rogue DHCP Server wreaking havoc on the network.  When this happens, random devices get the wrong attributes and lose all ability to communicate.  Depending on how long the lease TTLs are set, sometimes the passage of a few hours can cure the problem, or sometimes make it worse.  The problem can “hop” from device to device as leases expire and new leases come online.  Sometimes devices can end up with duplicate IP Addresses that come and go and interfere with communications.  This can be maddening to troubleshoot.

The usual culprit in an office network is a wireless router somebody brought in from home.  This happens all the time as end users decide they want to build their own private wireless networks, but don’t think about the consequences to everyone else as their wireless router hands out home IP Addresses to random devices across the company network.

Obviously, the cure for a rogue DHCP server is to find it and get rid of it.   The challenge is how to find it?

Enter structured cabling.  Essentially, a structured cable plant runs network cables from stations all over the building to a central patch panel in the server room.  Each cable is labeled, preferably with the labels on both ends of the same cable matching.   All buildings should have a structured cabling.  Unfortunately, many don’t.  Fortunately, this one did.  And that proved to be a tremendous aid finding my rogue DHCP server.

Instead of walking the entire building and looking for a device that looked out of place, I set up a laptop near the patch panel and assigned the laptop a hard IP Address to fit the rogue DHCP server scheme.  After warning everyone their network connections may be disrupted briefly, I set up the laptop to continuously ping the rogue DHCP server IP address while I disconnected and reconnected each network cable.

The idea – one of those cables had to lead to the rogue DHCP server.  I would find the cable leading to my rogue DHCP server by watching for pings to stop responding when I disconnected that cable.  Once I found the correct cable, I could walk to the other end of that cable with a hammer and put the rogue DHCP Server on the other end out of its misery.

I eventually found it, chased it to the other end of the cable, and disconnected it.  It turned out, my friend James brought in a wireless router over a weekend to help with some work he needed to do.  He forgot to disconnect it and that was why my users started complaining on Monday morning.

The moral of the story?  These things happen and that’s why good troubleshooting techniques are invaluable.