better metrics for security

A few readers have asked me to comment on Symantec's recent report. I held off because it didn't make sense for me to jump out in front of the official word.

I'm extremely proud of our approach to security. The Symantec report, when one actually reads it, validates our process and makes it clear that we're getting better all the time.

Posted by asa on September 26, 2006 7:12 PM

reactions, thoughts, comments, etc.

At Mozilla, you can see all of our bugs.
That's not correct, is it? I remember some (security related) bugs that were not public.

Posted by: Dao | September 26, 2006 11:00 PM

Hi Asa,
Would it be possible to get the author name shown clearly on devnews posts? I think it looks nicer when there's a name put to stuff rather than a vague "Mozilla" label, and besides, I can see in the feed that dc:creator=window :).

Posted by: Robert Marshall | September 26, 2006 11:20 PM

"In the first half of this year, Mozilla had a window of exposure of one day."

Sounds great, at least when compared to competing browsers. But then I read:

"In the second half of 2005, Mozilla had a window of exposure of negative two days, meaning that exploit code in that period was generally released after patches were available."

Huh? What does that mean?

Posted by: Jesse Ruderman | September 27, 2006 6:01 AM

Jesse... that means that, on average, an exploit was available 2 days *AFTER* Mozilla patched each issue.

Posted by: John T. Haller | September 27, 2006 7:07 AM

@Dao: Security-related bugs that have been hidden are made public once the fixed version is released.

Posted by: Kelson | September 27, 2006 10:16 AM

A really good analysis would distinguish between security issues properly reported to the vendor, and reports irresponsibly made public first. The average response time is relevant not just for the former, but also for the latter, measured from the time a bug was properly reported. A negative response time is a ridiculous concept IMHO.

BTW, there should also be a clear separation between the truly bad issues (computer compromised by visiting a website), spoofing issues, and other issues.

Posted by: Rijk | September 27, 2006 3:16 PM

The best metric I've seen is "the number of days during which there is a public exploit available for at least one sg:critical bug". http://bcheck.scanit.be/bcheck/page.php?name=STATS2004&page=3 used this for the year 2004. A security hole fixed by the vendor before an exploit is available does not affect this metric either way -- there's no "negative response time".

Posted by: Jesse Ruderman | September 27, 2006 4:24 PM

Robert Marshall-

What's unclear about the author? It clearly says at the bottom:

"- Window Snyder"

Posted by: Eddie | September 28, 2006 6:37 AM