With all the IT security issues in the news lately, suddenly IT security is everyone’s problem. One natural question behind the headlines is, what is the right way to handle IT security vulnerabilities?
Here are some thoughts.
To keep things simple, let’s limit this discussion to three major players. The real world is more complicated, but this is enough to illustrate the concepts. The first player is Bob, leader of an organization. Next is Ingrid who discovers a security vulnerability. And, of course, Trudy, the evil intruder we all love to hate. Trudy spends most of her waking hours probing the Internet, looking for weaknesses she can exploit and secrets she can steal.
Let’s say Bob’s business operates a website and Ingrid finds a security vulnerability that exposes sensitive information about Bob’s customers. How should Ingrid proceed?
Here is a blog post I put together a few months ago with an example of what happens when players proceed the wrong way.
This is what should happen. When Ingrid finds the vulnerability, she realizes Trudy is already trying to exploit the weakness to steal personal information from Bob’s customers. The race is on to fix the problem before Trudy exploits it for her own evil purposes. And Trudy has a head start.
Ingrid has an ethical duty to immediately inform Bob about the problem and make Bob aware of the potential consequences. Bob, always skeptical about gloom and doom warnings, listens to Ingrid because Ingrid makes a coherent and credible presentation about the problem. Bob heeds the warning, fixes the problem, and quickly informs his customers and takes remedial action. A newspaper or popular blog eventually publishes the story, giving credit to Ingrid for her dedication. Evan, an executive from an influential software company, reads the story and offers Ingrid a job as Director of IT Security. Everyone lives happily ever after, except Trudy, who was denied the opportunity to steal from somebody.
That’s how things should work. But it doesn’t always happen that way.
Let’s say Ingrid presents the problem to Bob, but Bob ignores the warnings. Now what? Trudy is out there. When Trudy finds Bob’s vulnerability, she will exploit it and steal from Bob’s customers. Trudy might even drive Bob out of business. How does Ingrid respond if Bob fails to respond?
Let’s say Bob uses software from a company named, say, Orange Computer, and Ingrid finds a security problem with that software. Ingrid contacts the right people at Orange, but Orange sits on the problem and does nothing. Trudy is out there. If Orange fails to address the problem, Trudy will exploit it. What does Ingrid do?
Ingrid’s only course of action in this case is to follow a best practice called responsible disclosure. After trying to warn Bob. After contacting Orange. After taking all reasonable steps to inform the right people, and after waiting a reasonable amount of time for a response, and as a last resort, Ingrid has a duty to disclose the problem publicly. Ingrid must assume Trudy and her friends are already quietly exploiting the problem, and Trudy will hurt too many people if Ingrid fails in her duty.
Ingrid also has a duty to protect herself. She should document her attempts to contact Bob and the people at Orange Computer as appropriate because when the problem becomes public, it will ignite a firestorm of controversy with Ingrid in the middle. This will create an opportunity for Ingrid to educate the public and a threat from people who blame the messenger for creating the problem.
Politicians will weigh in with uninformed opinions and instant experts hungry for publicity will offer canned analysis for gullible press outlets hungry for sensational stories. The noise will be deafening; real information will be scarce.
Amid all the noise, what about customers, the people who use software from Orange Computer and the people who use Bob’s website. How do they respond?
Customers should do independent homework and look for the real story. Security vulnerabilities happen all the time. Is this one just another sensational story or is it real? What are the prudent steps to protect against it? What are the plans from Bob and/or Orange Computer to address the problem? What are the consequences of not addressing the problem? Customers need to find credible answers to these questions and make informed choices on how to respond.
After the initial disclosure shock wears off, some other questions are appropriate. Who is Ingrid? What were her motives? How did she find the problem? Before the problem went public, what steps did Ingrid take to contact the right people?
That scenario assumes Ingrid discloses the vulnerability responsibly. What if Ingrid wants to make a name for herself and she discloses the vulnerability without first informing Bob? In this case, Ingrid is really a bad guy disguised as a good guy and trying to gain notoriety at the expense of Bob’s company.
Bob learns about the problem on the TV news along with the rest of the world and his company phones start ringing a few seconds later as press outlets everywhere look for comments and controversy. What does Bob do?
Bob faces multiple threats. He faces a public relations threat from sensational press stories spawned by Ingrid’s improper disclosure. Bob and his customers also face a material threat from Trudy, quietly exploiting the vulnerability at the expense of Bob and his customers.
To meet the PR threat, Bob needs to get in front of a runaway public relations train and slow it down. This is the time for visible leadership and Bob must get in front of the cameras and take charge. Provide explanations and frequent progress updates, and answer questions honestly and directly to repair credibility with a skeptical public.
Simultaneously and behind the scenes, Bob must also immediately address the actual vulnerability because Trudy wants to steal from Bob’s customers. This might mean bringing in outside experts, it may even mean temporarily suspending business. It will cost money. Probably lots of money. But if Bob handles this crisis properly, it can also be an opportunity for Bob’s company to come out of it with more trust and more credibility than before.
What if Bob himself is a bad guy?
In 2005, Mark Russinovich was Ingrid and multibillion dollar Sony Corporation was both Bob and Trudy when Sony compromised thousands of computers around the world by surreptitiously introducing a rootkit when anyone played a Sony BMG music CD on a Windows PC. A rootkit is illicit software that modifies core system components and is designed to conceal itself from malware countermeasures such as antivirus products. Bruce Schneir summarized the story here. Mark Russinovich’s original blog post with details on his great detective work uncovering the problem here.
Russinovich found the problem and reported it publicly in his blog. This was the right thing to do and Sony eventually paid millions of dollars to settle fines and class action lawsuits.
What if Bob is a government agency and Ingrid discovers a vulnerability or abuse of power? Now the consequences might be global. Scenarios like this have spawned long discussions over the generations about ethics and whistle-blowing. Sometimes, Ingrid is a lonely crusader pursuing justice against powerful forces. Other times, Ingrid is an egomaniac, pursuing her own interests at the expense of everyone else. And Trudy is always out there, ready to strike at every opportunity. Ingrid has a duty to proceed with caution and carefully weigh the consequences of any action.
If you find yourself in a position similar to my hypothetical Ingrid, how do you decide what to do? Who is harmed, who is helped if you disclose the vulnerability? And who is harmed, who is helped if you do not disclose it? If you take action, are you serving justice or your own ego? Confide in a few people you trust and make your choice based on honest answers to those questions. Do it responsibly. Careers and lives may depend on the choices you make.