There is a good example in the book
“Super Crunchers” that happens to parallel the issues and needs in the
information security community for “proving” the negative, i.e. demonstrating
that your protection methods led to a reduction in compromises. The excerpt in
the book says that the author (Ian Ayres) and Steven Levitt demonstrated the
ROSI of using LoJack, not only to owners of LoJack systems, but also to everyone
in a particular area where LoJack was deployed. I believe this is the actual
academic research, but it is behind a paywall: (http://links.jstor.org/sici?sici=0033-5533%28199802%29113%3A1%3C43%3AMPEFUV%3E2.0.CO%3B2-B&size=LARGE&origin=JSTOR-enlargePage).
This story may also be in Freakonomics.
This doesn’t necessarily “prove” a
negative, but it is an interesting approach to demonstrate that there ARE
methods to help quantify benefits in these areas with reasonable accuracy (and
even that accuracy can be estimated) that is certainly better than qualitative
guesses alone.
Kristof gives more numbers here, for those who can’t get through the paywall but want more specifics:
http://www.nytimes.com/2005/06/28/opinion/28kristof.html?ex=1277611200&en=54885fd31890c085&ei=5090&partner=rssuserland&emc=rss
Hi
I read the paper and I couldn’t find any reference about ROI (or ROSI) on it.
Hi, Gustavo -
They don’t talk about ROSI directly, since the topic is not information security, but they are essentially performing the same calculations that ROSI requires.
The challenge with ROSI is that it is difficult to say how much risk was reduced. With the LoJack scenario, they quantified the risk savings – this is the “prove a negative” problem we have.
The broader point is that you can prove a negative if you have two or more groups to compare – one with the control measure and one without.
Hope this helps.
Pete