Cost-Benefit vs. Cost-Effectiveness

Dans Geer and Conway have their new "For Good Measure" column up where they deprecate cost-benefit in favor of cost-effectiveness. It is a great column to learn about cost-effectiveness. The part that leverages true/false positives/negatives is particularly useful to folks trying to work out effectiveness of any controls. They also indirectly show a process that can be used to calculate ROI in security.

The only piece I take issue with is the initial assertion that cost-benefit is worthless. This, of course, is flawed. If you are operating in the interests of your enterprise, you can't opt-out of cost-benefit, you can only obscure it. Luckily, the meaning detective is at work to show you the light ;-) .

The article shows its true colors in the last paragraph of the first page:

The first test, if used alone, would leave you with nearly a million false positives—too many to
fix; the second test, if used alone, would cost you $100,000,000—completely unaffordable; but used together and in that order, you find 90% of the flaws for US$11,233.34 apiece.

Do you see all the value statements in the paragraph? the phrase "too many" and the word "unaffordable" topped with an implicit assumption that the $11,233.34 (note the impact of perceptual contrast) must somehow be "affordable" all suggest that the value is not worth it in the first two cases and clearly worth it in the final one. So we have a rudimentary confidence interval of expected value between the "unaffordable" $100,000,000 and an "affordable" $10,000,000 (the $11k times 900 vulns).

A more detailed analysis would factor in risk to make the full value judgement, and there we have the makings of value which can be used in a cost-benefit analysis.

It is clear that the authors are working from experience to assess what is "worth it" or not. That is what we all do. But we shouldn't ignore the fact that we are making value judgements all the time and we can (should, and in many cases must) translate that with a bit more rigor, lest we end up spending too much in the first place.