But it does: http://www.eecs.harvard.edu/~stuart/papers/usenix06.pdf

> 1) Patches don’t make risk go down unless every single vulnerable system everywhere in the world is patched.

Calculating risk for “the whole world” is a pointless endeavor. However, calculating risk for, say, a client’s network, is a legitimate excercise.

Case & point: SQL-Slammer. Anybody with an IDS listening to unfiltered Internet traffic is still seeing this worm. However, if they have patched and/or blocked SQL across their border points, then the risk is negligibly low. Patches make risk go down. Period.

> 2) Detection only affects risk if you can prevent it.

Untrue. You can’t look at risk only in terms of whether or not a host has been compromised, but for how long, since business risk isn’t simply whether or not you’ve lost exclusive control of a machine on your network, but what information has been stolen or altered or what services have been disrupted. Time to detect ~= time to respond ~= length of compromise. The smaller those numbers are, the less risk the compromise represents to the business.

> Since there is a finite number
> of vulns on a system at any given time, an increase in the number of attackers who
> know about it also increases risk.

Agreed, though since spam/phishing/worms are all highly-efficient means of delivering an exploit, that I have to wonder how significant the increase is in some cases. A handful of bot-herders with a local 0day is potentially more dangerous than 100K script kids with a published remote exploit.

> 3) You assume that people will patch. They don’t, and more attackers will know about
> the bug. Increased risk. I don’t disagree that software is being developed more securely;
> I do disagree if anyone asserts this was the only way. Neither of us can prove/disprove each
> other.

Funny. That study you cited showed that nearly 40% of systems were patched within 30 days, prior to the presence of a worm that exploited that bug.

> 4) I am saying that if there is no relationship between good bugfinders and bad
> bugfinders, given the total number of vulns in the world, it is highly unlikely
> that there will be many collisions. I use Ozment’s paper as representative – you
> are right that it leaves a lot to be desired in this context, but I think it is
> also likely to be a best case scenario. You would need 100% overlap to succeed.

The problems is that this is very much the chicken and the egg, as you pointed out earlier. We can correlate the overlap between researchers and criminals, but identifying whether or not disclosure causes Russian spamsploits or vise-versa is impossible.

> 5) Are you suggesting that Windows 3.1 and XP (and every other flavor of Windows)
> have the same exact code base?

Of course not. But look at the WMF exploits from New Year’s – pieces of code from the 3.0 days still live on in XP. But my point was that Microsoft has learned from its experience with NT 4.0. We all benefit from that, and it’s not at all far-fetched to say that disclosure of vulnerabilities played a part in that.

Perhaps its a little Ayn-Randian, but network security really is an objective and selfish exercise. If disclosure helps me but hurts you because I read bugtraq while you read Penny Arcade, then maybe that’s just the nature of things and maybe you’ll “get it” eventually, probably after you experience some pain.

I believe that it is more important for an individual organization to be prepared to handle risks, even if that means that the availability of the information used by an organization to protect itself can also be used to the detriment of others. The continuity of my sphere of influence and responsibility has to come first – that’s what they pay me for. They pay me to make sure that they’re part of the 40% that aren’t impacted by the next worm.

@Paul -

“Anecdotal” quantification? Get real. For every instance of short-term “good” that’s been forced upon the unsuspecting Internet user by bugfinders, there have been tens of thousands of “bads”.

Good points on my comments. I will elaborate:

1) Patches don’t make risk go down unless every single vulnerable system everywhere in the world is patched.

2) Detection only affects risk if you can prevent it. Since there is a finite number of vulns on a system at any given time, an increase in the number of attackers who know about it also increases risk. If all systems everywhere get patched, you decrease risk. You simply choose to ignore all of the other risk associated with that target. It is not impossible to protect yourself without knowing about specific vulns.

3) You assume that people will patch. They don’t, and more attackers will know about the bug. Increased risk. I don’t disagree that software is being developed more securely; I do disagree if anyone asserts this was the only way. Neither of us can prove/disprove each other.

4) I am saying that if there is no relationship between good bugfinders and bad bugfinders, given the total number of vulns in the world, it is highly unlikely that there will be many collisions. I use Ozment’s paper as representative – you are right that it leaves a lot to be desired in this context, but I think it is also likely to be a best case scenario. You would need 100% overlap to succeed.

5) Are you suggesting that Windows 3.1 and XP (and every other flavor of Windows) have the same exact code base?

All of the bullet points from Thomas’ post are quantifiable, at least anecdotally. You can dismiss them as purely opinion, but you’d be wrong.

> Btw, I have the evidence on my side:
> 1) risk warnings never go down with the latest disclosure of vulns;

No, they go down with the latest release of a patch. That patch is likely the result of a found and disclosed vuln.

> 2) any single infection of a disclosed vuln is evidence that risk went up;

OK, but if we’re quantifying risk, is the risk to a target greater if a vuln is known to attacker and target and, if not mitigated, at least detected? Or is the risk greater if the vuln is known only to the attacker and the target cannot mitigate or detect the attack and therefore cannot respond to it? This one’s just common sense.

> 3) bugfinding doesn’t make software more secure;

In an atomic instance (OpenSSL v0.9.6 vs. v0.9.7) there’s a clear impact from patching the bug – a remotely exploitable buffer overflow is eliminated. Over time, maybe it doesn’t have a direct impact.

But you cannot discount the indirect impact that disclosure has on software. Look at the evolution of NT 4.0 -> 2003 Server. Had it not been for some global incidents and a PR nightmare for Microsoft, 2003 Server might not have features like DEP or host firewalls that can be configured via Group Policy. Had there been no research and disclosure, this process might have taken longer with different (likely worse, but that’s my opinion) results.

> 4) there may be about 7% overlap in bugfinding;

So is your conclusion that since, within the same community and industry, there is a 7% rediscovery rate of the same vulnerability, that there is only 7% overlap in the bugs found by infosec researchers working openly and those working privately (for the Russian Mob)? I can’t even begin to list all of the flaws in your assertion. Clearly Ozment’s study represents nothing of the kind.

> 5) bugfinding may make software more secure after 7 years (i.e. about 2-4 years after its useful lifetime).

Yes, Microsoft Windows server software has run its course. The technology is almost 15 years old and has run its course. Nobody uses it anymore.

2. When vulnerability disclosure actually creates new vulnerabilities.

3. The fact that clientside vulnerabilities in IE are worth more money than serverside vulnerabilities in .

4. We wait the minimal amount of time possible. Others wait even less time.

5. Nothing.

6. Windows XP.

7. Because their risk ratings are arbitrary and subjective.

8. Because from 1988 to 1995 there wasn’t a single buffer overflow advisory, despite the fact that the Morris worm exploited one.