Freedom to Tinker has an excellent post on journalistic skepticism. I agree wholeheartedly that numbers can take on a life of their own — it concerns me even more because it is a real downside of estimates and measurements, of which I am a huge advoca…

http://www.ojp.gov/bjs/survey/ncss/faq.htm
says that:

“a restricted-use data file will be made available for research purposes only. It will be carefully scrubbed of all identifying information so that the individual companies’ identities and responses will be protected.”

Hopefully, they won’t be jerks about defining “research”. Maybe I still have some letterhead from grad school…

It is an area I knew, deep down, that would be most controversial and most distracting aspect of the book. Unfortunately so. As I state in Geekonomics, “The real cost of something is not always measured in money. The real cost of something is what you have to give up in order to get it.”

Insecure software communicates an unmistakable message of disorder into cyber space: no one is in control of software. Not manufacturers. Not consumers. And certainly not governments. Lack of order imposes a cost. So the real cost is not a dollar amount; it is in the threat to national and economic security. Vulnerabilities in software are being exploited, and rampantly so. To focus primarily on the dollar amount, is helpful, but misses the point. That said, I put the number out knowing the possibility for distraction and you are being more than reasonable to challenge the number’s accuracy.

Somewhere in the muddle of hype and deflation are the “real” numbers. From my conversations with journalists and industry experts, the cost of insecure software is somewhere above $100 billion (US only); this much we know (or at least this number was felt to be “about right.”). So $180 billion is not unreasonable, but it is a reach to state this number, or any number on the subject, with any level of confidence. Talking about, and getting reliable numbers on insecure software is like talking about sexually transmitted diseases: we can see the effects but very few actually admit to, or are even aware of, their contribution to the problem. Similar to the stigma associated with sexually transmitted diseases, there is also a bevy of circumstances in the software market that cloud both the collection and reporting of numbers. This is a shame. And the reality. The greater upset perhaps is not so much that $180B may be soft, but that we have no idea by how much.

But there is a glimmer of hope that only appeared after Geekonomics went to print. The Department of Justice received a report from RAND Corporation last year from an in-depth study on cyber crime across 8,000 companies. This report is not public, but should be made public by DOJ early in 2008. This is good news. RAND promised that the participating companies would get sanitized data about their individual industries outside of what was in the public report (which was the incentive RAND used to promote participation in the first place). This potentially means that anonymity, and the subsequent freedom from stigmatization, might indeed provide more reliable numbers, if only from a limited subset. But given that a 2003/2004 CIO report (just prior to the time period of collection for the RAND data) showed that only 12 percent of surveyed companies (approx. 5,000) had a reliable way of quantifying their losses due to exploitation, hope might need plenty of salt.

Your analysis is a welcome addition to the discussion. Thanks so much.