"I remember buses pulling up to the Microsoft campus to shuttle engineers away from their day jobs to go work the phones down at (product support)," said Thomlinson, who heads Microsoft’s security engineering efforts. "That was just heartbreaking."
The Blaster worm had just hit, swamping Microsoft’s support lines with calls from angry customers.
Andrew Cushman, director of the Microsoft Security Response Center, remembers standing in Muck boots and installing a catch basin in his front yard when he got a call from an account manager. It was just days after September 11, 2001, and one of Microsoft’s largest customers had just been hit with what turned out to be the Nimda worm.
George Stathakopoulos, Cushman’s boss, still hasn’t seen the end of the movie Master and Commander. In spring 2004, he was sitting on his couch watching the film when he got the call that Sasser had hit.
Indeed, much of Microsoft’s current security practices can be traced to painful lessons learned during the past decade by people whose job it is to secure Microsoft’s products.
Because of the experience of Mike Nash, a vice president at Microsoft, the company finally instituted calling trees as a way to quickly reach people in an emergency. When the Slammer worm hit in January 2003, Nash had to work feverishly to track down the vice president of SQL Server, Gordon Mangione, eventually locating him at his sister’s wedding in Canada. (Slammer used Microsoft’s SQL Server database to propagate a denial-of-service attack.) Nash first heard reports of Slammer on the local news radio station at 6 a.m. At first, he thought he was dreaming. But as the report played a second time, he knew it was real and headed into work. "I was the second one there," Nash recalls.
Slammer also taught the company that it was not enough to have a patch; the patch had to be easy enough to deploy so that most customers would do so, lessening the chances that outbreaks would propagate so quickly. And it was Blaster that taught the company that it wasn’t enough to patch a single flaw; it needed a systematic process for catching whole classes of vulnerabilities, a realization that paved the way for Microsoft’s current approach, known as the Security Development Lifecycle, or SDL.
"We’ve put a lot of our best people in these areas," Microsoft Chairman Bill Gates said in an interview with CNET News.com. "Still tons to be done, but you know, we’ve definitely made five years of progress in the last five years."
Much of the reason for that traumatic on-the-job training can be traced to Microsoft’s decade-long evolution in how it and its employees deal with security. Until 1997, security was seen mainly as a set of features that the company bolted onto its software long after product design and development. The idea of securing code as it was being developed had not been considered.
IE flaws send Microsoft scrambling
That all began to change in March 1997, when the first significant flaws were discovered in Internet Explorer. Researchers at Worcester Polytechnic Institute found a vulnerability in browser shortcuts known as .LNK files. Even as Microsoft was scrambling to deal with the problem, word of the flaw hit cable television news. A few hours later, researchers at the University of Maryland found a second problem and reported it to Microsoft.
Simultaneously, the IE team, which Stathakopoulos was part of, was in the process of moving into a new building. The timing couldn’t have been worse: most of their equipment was in boxes. Someone had to run to a store to buy a power supply for one of the team’s laptops--the power cords had been packed away--before the battery went dead. Jason Garms, now a senior director for technical strategy, wrote the company’s first security bulletin in a Windows’ Notepad file and then copied it to a floppy so it could be distributed to customers.
At the time, the company didn’t even have a system in place where outsiders could report security bugs directly to Microsoft engineers. The IE flaw came to light because someone had called Microsoft’s support line and the matter had gradually escalated.
"We said ’This has to stop,’" Stathakopoulos recalls thinking of the disjointed system at the time. "It’s not working for us."
In the aftermath of that bug, Microsoft created the Microsoft Security Response Team as well as a separate Internet Explorer security group. The company also created an e-mail address where outsiders could report potential issues.
The Microsoft Security Response Team was made up of volunteers--employees who had other day jobs, but were interested in helping out when there was a security problem.
No comments:
Post a Comment