One of the crucial aspects of risk management for infosec pros to learn is how to estimate consequences. It can be helpful to review incidents and create a model for thinking about losses. Amazon’s outage for an hour yesterday, is a good, simple example for us to play with – this exact example used to be the one I used when teaching my security metrics class because it is so clean. Or is it?

When estimating losses, it isn’t entirely unreasonable to do the high-level straight-line math like IT World did here:

“Amazon.com’s latest earnings report showed that the company makes about $10.8 billion per quarter, or about $118 million per day and $4.9 million per hour.”

It’s really quick and dirty – and in a general sense legitimate – but can we do better? There are other ways to look at this that might shed some light on impact assessment. First, the assessment above makes no mention of costs. That might be the biggest weakness since costs are more under the control of Amazon and (probably) don’t fluctuate as much as revenue.

Luckily for us, Amazon just released its quarterly earnings report and this report asserts that its operating margin is about 3%. So right off the bat, we could suggest that Amazon lost 97% of $5 million or $4.85 million in costs. A more conservative estimate might try to determine whether the costs were unrecoverable or not, etc. Hopefully, you get the idea. A cost-oriented approach also works well as an example in infosec since that is often a big piece of the losses we face.

It is important to note here that these costs are additive to the lost revenue estimate – not only did we lose the $4.85 million in operating costs, but also (presumably) we lost that initial $4.9 million in revenue, for a total of (let’s say) $10 million.

Now, let’s look again at that lost revenue estimate. As mentioned earlier, coarse numbers like those used in the calculation above are certainly justifiable but we can probably do better. A quick thought exercise can help here – by creating the experience of an “average customer” of Amazon’s we can better assess the impact of the outage. This is harder than it sounds because we’ll have to second guess our own biases, but let’s try anyway. Let’s call him Joe.

Given that the outage was simply a “denial-of-service” of sorts, the big variable we must evaluate is time. More specifically for our scenario, we need to answer the question “How timely does Joe’s interaction with Amazon need to be, or, how likely is Joe to wait an hour to complete his purchase?” At the very least, we know Joe is willing to wait two days (maybe more – not sure what the average delivery time is for Amazon) to receive whatever goods he purchases. Throw in what we might assume (my bias) about Amazon’s low prices and the corresponding brand loyalty that comes with it and it seems reasonable to conclude that Joe will wait an hour to make the purchase, and therefore the lost revenue is actually only deferred revenue to be recognized in the future.

But not everyone is average (usually nobody is), and so once we cover a generic case, it is useful to consider the impact of the outliers. Now, we can imagine scenarios where even though a customer can wait for delivery, she can’t wait to place the order – too many other things going on in life. Or even a case where the customer would actually lose a full day due to delivery cutoff times. These are the types of cases that warrant more attention. Certainly it is reasonable to factor these cases into a loss scenario. Let’s say this is true 10% of the time.

The goal here is to be conservative in our estimates (even though it is sometimes beneficial for companies to be liberal after the fact – can hide other problems) so we should remember that these scenarios are typically useful in identifying some sort of discount factor to apply to the initial $5 million estimate. Though it is possible to come up with scenarios where there is a multiplier – maybe holiday seasons – it is less common.

Our lost revenue evaluation has led us to conclude that 90% of purchases will still be made in the future, so the remaining 10% of cases will discount our $5 million loss down to $500,000. Add that to our lost costs and we are back to the initial $5 million estimate, though from a different perspective. While it might be attractive to decide all was for nought, it is worth considering the situations where the costs are much lower, or the revenue is more likely to be lost to see the value in the exercise.

Now, some might suggest (essentially) that the above analysis is really not worth it because a loss is a loss. Not only that, but Amazon’sown numbers have shown (?) that there is no discernible uptick in sales in the period following the outage. As mentioned earlier, it is easier to see how costs are fairly static and therefore turn into losses. On the revenue side, however, it is not clear at all.

In assessing lost revenue in this case, one must do two things: first distinguish between necessity and convenience and second evaluate the impact of buyer’s capacity. The purported lack of a noticeable uptick in sales in the short term could easily be explained if purchases are more oriented around convenience than necessity. Measures associated with shopping carts might be of assistance here (I sometimes leave items in my shopping cart for days if not weeks). Again, this information can be factored into the estimates if need be.

It is uncommon to consider a “buyer’s capacity” but especially with convenience purchases, one might decide that the rate of purchase is a determining factor and even though the shopper returns, she will be buying other items, etc. This justification is easier to believe in cases where capacity is high – that is, the shopper is buying at a rate where fitting in the “lost” purchases is unlikely (and when it happens is noticeable in the numbers). My assessment is that this scenario is unlikely; people are more casual in their shopping experience and will therefore wait to make their purchases. (A similar capacity limit could have an effect on the Amazon side, but that is even more farfetched).

My conclusion is that $5 million is a reasonable loss estimate for Amazon’s outage, but not for the reasons initially believed.