Engineers are focused on quality, so when they hear about vulnerabilities in software, their immediate reaction is to want to fix them… all of them. Regardless of whose software it is. Regardless of where it’s deployed. In fact, some of them care so much that they go out seeking vulnerabilities simply to fix them. They are the type of people who are great at solving problems, but not at understanding the downstream implications of their actions.

Economists, on the other hand (get it?), look at cause and effect, actions and reactions, and, most importantly, outcomes. The root of the economic problem lay in the ultimate unwanted outcome – the breach.Economics-oriented security pros understand that everything we do is intended to thwart the breach. It is easy to lose track of unwanted outcomes in the face of compliance needs and operational activities, but even those activities are all intended to minimize damages from attacks and exploits.

The engineer correctly believes that fixing vulnerabilities creates high quality (“stronger”) software. If the program starts with 300 vulnerabilities and you fix one, that obviously leaves 299 – one less than when it started. More importantly, if an enterprise has 1,000 systems that all have that same vulnerability and they apply a patch to 500 of them, they have decreased their attack surface by 500 vulnerabilities. From both perspectives, the level of vulnerability is, in fact, reduced.

But the economist knows that fewer vulnerabilities is not the ultimate objective. The ultimate objective is to reduce the likelihood of an incident.

The economist understands that there is a key missing ingredient to the engineer’s scenario – the intelligent adversary, aka the threat. And in pursuit of higher quality software the vulnerability details usually get published, leading to lower attack costs for the adversary. Given the scalability of technology, this typically leads to more attackers connecting to more targets, albeit in a (somewhat) smaller population of targets.

That is the key observation for this discussion – a breach requires both an attacker (threat) and a target (vuln), which manifests itself in the form of a connection between source and destination. Even though the population of targets may be reduced (perhaps even significantly so), if the threat is sufficiently motivated, more connections can be made with the vulnerable targets. The only way to guarantee reduced risk is to bring one of the populations (most likely the vulnerable targets) to zero. History shows us this is not likely with commercial software in enterprises. Interestingly, the increasingly common scenario for cloud-based software (e.g. Software-as-a-Service) may be able to do just that.

And there you have it – given the need for both threats and vulnerabilities, the reduction in one doesn’t force a reduction overall. And if the other element is increased in the process, the marginal difference in each population must be evaluated to truly understand the impact. Historically, this has led to scenarios where the vulnerability is reduced while the risk is simultaneously increased.

For reference:

More Sex is Safer Sex…

I believe the folks at Gartner put a lot of research and effort into their Magic Quadrant analysis. That said, I can’t help but conclude that “vision” and “execution” don’t quite do it for me when it comes to identifying appropriate candidate solutions to address a problem. They just seem to be too much about marketing, which is very important to the companies but only ancillary to an enterprise’s needs. Sure, they want a solution that will be viable for the long-term, but other than that it is pretty insignificant.

To address this issue, I have put together a set of questions in 4+1 evaluation categories that I believe provide more insight into the important attributes of a solution. The first round of categories was introduced at AMP NYC a month ago. Here is my second revision. Opinions and advice are welcome.

1. Company/Product Information: What level of confidence does the company information provide that the company and product will remain viable for your organization?

Consider:
• What year was the company founded?
• What is the background of the management team?
• How many employees does the company have?
• What is the funding status/source of finances?
• What is the product name and version?
• How many customers does the company have for the pertinent product?
• What certifications and tests were done on the product?
• What other 3rd party reviews, awards, or other supporting evidence exists about the product?
• What is the pricing model for the solution?

2. Functional Operation: What level of benefit does the functional operation of the product have?

Consider:
• Primary operation – scan memory state, scan configuration/file system/network state, monitor/record system call activity, monitor/record network traffic, isolate memory, isolate system activity, isolate network communications.
• Trigger action – detect “known good” execution, detect “known good” activity, detect “known bad” execution, detect “known bad” behavior, detect anomalous execution, detect anomalous behavior.
• Response options – allow, deny execution, kill process, kill network connection, reroute network communication, log event, notify user, notify admin (alert), other.
• Recovery options (post-infection) – Restore config to known good state, remove bad files/objects, identify similar issues across network, notify/update other control solutions.

3. Architecture & Administration: How well does the product’s architecture fit in with your organization’s existing security processes? How likely is it to provide benefits? What features does it have to support implementation and administration?

Consider:
• Where/how are any product sensors or agents deployed throughout an enterprise (endpoint, network, cloud, other)? How are they protected?
• Where/how does the product admin/management function work? How is it protected? (endpoint, network, cloud, other)
• Where/how does the product log/data/storage function work? How is it protected? (endpoint, network, cloud, other)
• How is information shared a) with the solution components; and b) with others?
• How does the solution get installed/implemented in the environment?
• How customizable is the configuration and interface?

4. Technical Integration: How well does the solution integrate into the IT ecosystem? How easy will it be to implement and maintain?

Consider:
• How does the solution integrate with other products from the same company?
• How does the solution integrate with 3rd party security solutions?
• How does the solution integrate into an IT architecture?
• What are the prerequisites for user directories, management servers, etc?
• What standards, communication protocols, platforms, languages, frameworks, etc. are supported?
• How robust is the API for third party access?

The final category is actually a rollup of the other four, since the differentiators and value come from the previous specifics being identified.

Key Differentiators / Overall Value Proposition
When looking at the complete picture of the solution, how strong are the overall benefits derived from the individual evaluation categories?

I believe these evaluation categories more properly reflect the needs of the enterprise. What do you think?

1. Application Control / Whitelisting Solutions. Whitelisting solutions change the security approach from one that allows software to install/run unless otherwise specified on a “blacklist” (“default allow”) to one that requires explicit permissions on a “whitelist” for software to be executed (“default deny”).

Clearly, the goal of whitelisting is to reduce the number of malware infections by preventing unidentified software from running thus saving the aforementioned recovery costs. Given the common predisposition for organizations to consider infections separately from incidents, whitelisting solutions also are intended to reduce the likelihood of a bigger incident.

The tradeoff for whitelisting solutions is determining whether costs associated with false positives – legitimate software that is kept from running – will offset these additional benefits. Generally speaking, the more dynamic and decentralized an organization is, the larger the problem. Nowadays, whitelisting solutions have varying ways to deal with this known issue.

2. Sandboxes and Virtual Machines. Perhaps the most varied set of solutions addressing malware these days are the sandboxes and virtual machines. Some sanboxes – primarily on the network – are designed simply to provide an out-of-band (and sometimes near-real-time) environment to execute suspicious software and determine whether it is malware. As with whitelisting, the goal is to identify more malware more quickly, thereby reducing costs.

Other solutions – focused on the endpoint – actually isolate the production operating environment to reduce recovery costs by reducing the downtime associated with re-imaging a solution, and/or reduce the impact by containing malware in an environment separate from other production resources.

There are some tradeoffs in the sandbox/virtual arena depending on the architecture. Network solutions may not see as much traffic in highly mobile environments. Endpoint solutions have performance considerations and/or architectural dependencies to consider.

3. Active Forensics. Recently, a number of solutions have arisen to offer a near-real-time approach to forensics. By recording system calls and/or scanning system state looking for anomalies, their goal is to identify malware infections within shorter time periods than existing methods can.

Active forensics solutions look to reduce the costs of recovery by providing detailed information on changes that were made by malware so a responder can recover more quickly. In addition, the solutions provide comprehensive information so that recovery may be possible without re-imaging. In environments where users can install their own software, this could significantly reduce end-user productivity losses associated with recovery techniques. In addition, active forensics attempt to reduce the time-to-discovery such that further exploit and escalation chances are reduced.

The tradeoff with active forensics is determining whether the detailed information is enough to ensure completeness of recovery so that recovery without re-imaging is a possibility. On the risk side, enterprises must determine whether the new insight provided will lead to a fast enough response time to offset the cost of the solution.

Each of these product categories (as well as others) have a value proposition that may provide benefits to organizations looking to augment their antimalware protection programs. The key is for companies to understand exactly what benefits they provide and decide for themselves which particular type of solution, if any, is likely to have the largest benefit.

Pete Lindstrom is Principal and VP of Research for Spire Security, LLC, a research and advisory firm. Learn more about Advanced Malware Protection by “Drinking from the Firehose” in New York City on 9/17/13. Details at www.regonline.com/AMPFirehoseNYC. Complimentary access for those who qualify. Contact petelind@spiresecurity.com for details.

In addition, every qualified participant will receive their choice of an ARDUINO or RASPBERRY PI “security architect” Kit (while supplies last).

AMP Workshop, In Brief:

A bakeoff of whitelist, sandbox, active forensics vendors using a consistent model for evaluation of their ability to help stop 0days, “advance persistent threats,” and any other “malware that gets through”. Collaborate with peers and determine your own need (or lack thereof) with a custom anti-malware protection economic model.

Benefits of the Workshop:

- Get real, specific clarity about the costs and benefits of key AMP solutions.

- Get key feedback from your peers and colleagues pursuing similar goals.

- Customize your own economic model and vendor scorecard based on your needs.

- Compare and contrast vendors using the same framework.

Logistics:

WHEN: September 17th, 9am-5pm (8am coffee)

WHERE: Times Square, NYC (1601 Broadway in Crowne Plaza building – Executive Conference Center)

REGISTER: http://www.regonline.com/AMPFirehoseNYC

Read about the issues, decision points, and conclusions from an enterprise perspective at http://spiresecurity.com/?cat=18.

Can’t make the workshop, but would like the “Spire AMP Economic Model” to review? Contact Pete Lindstrom petelind@spiresecurity.com.

The first stage of this process is to understand the costs and benefits of your existing program. This is a 4-step process:

1. Determine the probability and (economic) impact of being compromised by malware. This is the overall risk an organization is trying to address with anti-malware solutions.
2. Collect the total cost of ownership of *all* of your anti-malware solutions.
3. Estimate the amount of risk reduced by the current anti-malware solutions in the IT environment.
4. Compare the amount of risk reduced in step 3 to the total cost of ownership in step 2. A simple comparison should show higher benefits than costs (if not, you are doing it wrong). More advanced comparisons (division!) can provide your “risk reduced per unit cost” for your current anti-malware program.

These four steps are notionally simple but TechRisk professionals will recognize that any non-trivial environment will have challenges developing some of the estimates. It may be worth perusing my website or contacting me directly to discuss some useful ways to do this.

Once the assessment of the current situation is complete, it is time to review both the TCO information and the remaining (or “residual”) risk to determine if it is worthwhile to modify the program in any way. Given that there are existing costs to work with, a new solution may provide opportunities to reduce the existing TCO along with the ultimate objective to reduce the residual risk.
To evaluate the value of new solutions, it is beneficial to delve a little deeper into costs and the amount of risk reduced. This is especially true since many solutions shift costs from operating expenses to a capital investment.

Taking a page out of the “activity-based costing” book can help an organization evaluate its cost structure more effectively. To do this, an organization should allocate its costs to a set of identified anti-malware activities. A new solution may help lower these costs by, for example, reducing the number of infections that must be cleaned over time.

On the risk side, new solutions may reduce the likelihood of an infection or incident by identifying more malware prior to infection. They also may provide a means to reduce the impact of an infection by lowering the response and recovery costs or addressing some other aspect of loss.

Understanding risk and costs is a crucial aspect of managing a security program. In addition to recognizing the value provided by an existing anti-malware program, performing this analysis may highlight areas of inefficiency or weakness.

Pete Lindstrom is Principal and VP of Research for Spire Security, LLC, a research and advisory firm. Learn more about Advanced Malware Protection by “Drinking from the Firehose” in New York City on 9/17/13. Details at www.regonline.com/AMPFirehoseNYC. Complimentary access for those who qualify. Contact petelind@spiresecurity.com for details.

1. Need. I choose the word “need” with caution, since, as you will find out below, it does not necessarily mean there is “demand” for a better solution. However, I don’t think techrisk professionals can deny that the malware dropping attack vector is alive and well. It is highlighted as the key to the Aurora attacks that catalyzed the “advanced persistent threat” concern.
2. Varied Solutions. There are a number of vendors that have cropped up through the years with solutions to address the malware problem, and the techniques vary significantly. Whitelisters only allow identified executables to run; sandboxes isolate malware and/or identify actions; and real-time forensics track system calls and/or configured state.
3. Mature Market. Even with an identifiable need and newer interesting solutions, the most powerful security market in the world – antivirus (nee antimalware) – operates in pseudo-commodity mode and dominates in endpoint security.

As an industry analyst, I have had the opportunity to interview over a dozen solution providers and even more enterprise security architects and executives on the state of antimalware in the enterprise. Here are a few of my conclusions:

• Companies are moderately satisfied (and perhaps complacent) with their existing antimalware solutions. They acknowledge that these solutions are not blocking all malware but believe that every solution in the category has similar problems and so are reluctant to switch.
• The only factor that could affect existing signature-base antimalware is price – a lower-cost solution (which many agree is unlikely) could have a strong-enough value proposition. Notably, a few organizations are evaluating Microsoft’s free antimalware solution as one of these alternative options.
• Organizations are looking to gain more benefit from their existing antimalware solutions. Many are still focused on signature-based functionality and are now looking at more advanced capabilities. In addition, organizations are considering and employing new capabilities like Microsoft’s EMET functionality.
• For those times when malware gets through and infects a system, re-imaging is the standard approach, though some organizations are mildly reluctant to do it. Most of these malware infections are not classified as “incidents” per se – there is an ad hoc evaluation process to decide whether any infection should be escalated into being classified as an incident.
• Organizations are looking at architectural changes and not product changes when it comes to endpoint client-side security. This means they are focusing on BYOD and/or VDI (or even dumb terminals) as options in their client security strategies.
• Control over (physical) clients continues to relax, with certain “pockets” of exceptions (kiosks or manufacturing systems). For some, this was after a long period of control strengthening (e.g. finally taking away local administrative rights).

As I mentioned at the start, the market dynamics fascinate me here. I don’t think there is a techrisk professional left that believes signature-based antimalware is “good enough” and yet we see its dampening impact everywhere. At this stage, it has simply become the “checkbox compliant” easiest approach.

As someone extremely interested in cybersecurity economics I am encouraged by the attention being given to the bottom line – organizations should be very careful about cost-benefit in their security programs. While some of the organizations I interviewed had done a comprehensive analysis, it appeared to me that a number of organizations had not undergone a thorough review of their strategies.

I will be addressing these issues at my “Drinking from the AMP Firehose” workshop in New York City in a couple of weeks. The workshop concept was driven by these ideas and aims to break through the logjam brought on by complacency and confusion. Regardless of the conclusions that individual organizations come to, I think the entire field will be better off for it.

Pete Lindstrom is Principal and VP of Research for Spire Security, LLC, a research and advisory firm. Learn more about Advanced Malware Protection by “Drinking from the Firehose” in New York City on 9/17/13. Details at www.regonline.com/AMPFirehoseNYC.

On the protection side of the equation, although antimalware solutions provide a basic (and compliant) level of protection, security professionals are well aware of the limitations of signature-based approaches. Solutions that have been around for a while, such as host intrusion prevention and whitelisting, have gained renewed interest. Other approaches like network or endpoint sandboxes for isolation and/or analysis or active forensics for near-real-time analytics are coming on strong.

The challenge is determining whether the additional cost is worth it, deciding whether a new solution will significantly reduce the problem, and identifying which type of solution(s) are best.

While it is easy for security professionals to claim they will spend “whatever it takes” to address technology-related risk, that assertion is easily deflated through extreme examples (millions? billions? trillions?). While the intentions are valiant (and I get the point), no organization has an unlimited supply of money to spend on security. Therefore, it is crucial to make good decisions about how and where to spend money.

Any business decision is accompanied by some sort of justification, and cybersecurity is no different. In security, we typically evaluate total cost of ownership of the solution and compare it to our notion of how much risk is reduced. At the very least, every purchasing decision is supported by a claim that the spending “is worth it.” At best, a more formal cost-benefit approach should be employed.

Evaluating the cost-benefit of an “advanced malware protection” solution can be extremely challenging. Dropping malware (in the form of viruses and worms) onto systems is one of the oldest methods of attacking and compromising computing environments. Because of this, all enterprises already have controls in place that attempt to protect against malware infection. In addition, there are a number of techniques that can be used to address the problem.

Regardless of the challenge, conducting an economic analysis of newer AMP solutions may lead to some surprising conclusions. Here are six key considerations for conducting your analysis.

1. Ignore the “Advanced” Part of Advanced Malware Protection

The first distinction you should make in reviewing your needs for “advanced” malware protection is that the “advanced” part is extremely nebulous – the bar keeps changing in defining exactly which techniques are advanced and which aren’t advanced. Accordingly, the first takeaway is “evaluate your AMP solutions in concert with all antimalware efforts in your organization.”

This should not be a radical thought.

2. Cover All the Antimalware Bases

Diving a bit deeper into costs, enterprises should consider the costs of all capabilities – the capital investments made on hardware and software, maintenance costs, and personnel costs. The vendor solution (capital investment) side of antimalware protection can include endpoint antimalware, email or gateway-based antimalware, intrusion detection (potentially), and secure web gateways. On the operational expense side, organizations should consider the personnel costs associated with identification, prevention, mitigation, response, and recovery activities associated with malware infections and incidents.

3. Allocate Partial Costs of Broader Solutions

Focusing on the costs associated with one type of threat – in this case, malware – can be challenging. Some solutions, like endpoint antimalware, focus directly on the problem while others provide varying levels of accompanying support. In my research, for example, secure web gateways were cited as a means for detecting malware infections that were undetected by endpoint antimalware solutions, but secure web gateways provide more capability than malware infection detection.

The key in the analysis is to allocate costs based on the proportional value provided by the broader solution. If 10% of the ongoing value of the solution comes from antimalware detection, then 10% of the future costs should be allocated to antimalware.

4. Ignore Sunk Costs

The maturity level of antimalware makes it likely that capital investments to address the problem have already occurred. Any spending that occurred in the past should be excluded from the analysis, though any current and future operational expenses should be included. In contrast, a decision involving a future capital investment should include that amount allocated (either amortized or depreciated) over its lifetime as well as the operational costs.

5. Factor in Employee Productivity

The second economic issue to consider is the productivity of employees. The productivity costs associated with the impacted worker should be considered along with the costs associated with the IT triage person. If it takes four hours to recover an infected system, then four hours of the worker’s lost productivity should be included (nothing fancy here – use a single average number based on salary for all workers).

6. Use a Breakeven Approach

Perhaps a bigger challenge in justifying antimalware spending is in determining the amount of potential losses. That “it is worth it” decision means that the security professional spending $100,000 on a security solution believes the solution will offset at least $100,000 in risk.

While some cringe a bit at the realization that spending reveals the minimum expectation of risk reduction, it also provides an opportunity to conduct a standard financial breakeven analysis. Rather than attempting to figure out the exact amount of financial loss at stake, you need only consider the total amount being spent and determine whether it is less than the potential losses. So as long as you’ve properly accounted for all costs, an appropriate decision can be made.

The AMP solution decision is not an easy one – with a handful of controls at different maturity levels in the organization already and a variety of newer solutions vying for attention. Some organizations may even come to the conclusion they don’t need to augment their existing capabilities. Others will find out they really do. Regardless, enterprises should be conducting the necessary analysis to make the best decision for its needs.

Pete Lindstrom is Principal and VP of Research for Spire Security, LLC, a research and advisory firm. Learn more about Advanced Malware Protection by “Drinking from the Firehose” in New York City on 9/17/13. Details at www.regonline.com/AMPFirehoseNYC.

Key Benefits: 

• Create and use an economic/risk model to justify your need for Advanced Malware Protection (AMP).
• Cut through the confusion of biased vendor presentations to identify the l functional benefits of AMP solutions.
• Evaluate vendors based on an objective model (i.e., your needs) customized to match your requirements.
• Benefit from collaboration and feedback of your peers who face the same challenges at their organizations.

With an economic model in hand, participants will hear from up to 10 vendors (maximum  10 minutes each) as they provide details on how their AMP solutions address current needs and conforms with the requirements in the scorecards. After vendors are excused, participants will discuss and debate capabilities and ultimately assign their own scores. The process is akin to speed-dating, but with group feedback (and no alcohol).

Each participant takes away the proceedings, along with their economic model (a quantitative risk assessment) and vendor scorecard,  that includes their unique values and scores, as well as the group summary scores.

Before I get into this, I should re-acknowledge that I believe there are better methods to measure/evaluate risk, and I fully subscribe to their development. However, I am looking for evolution not revolution – Geoffrey Moore pointed out the challenges of disruptive innovation in “Crossing the Chasm” many years ago and I agree wholeheartedly. Evolution to me means slightly modifying existing approaches in beneficial ways. That is why a few of us are developing the Tech Risk Mgt Maturity Model.

So my goal is, essentially, to be “better than existing practices in techrisk mgt” – I am looking for marginal utility.

I also believe that resources are scarce and that every time infosec/techrisk folks make decisions about allocating them they are revealing preferences that are measurable in very coarse ways. Even though the existing models are seen as “qualitative” we can create control horizons and conduct breakeven analysis in ways to tease out some thresholds at the very least.

Now, to answer the questions:

• Pr(t): Yes, “the probability that a (sufficiently capable) threat actor will attack the system of interest” characterizes my belief well. And since I go out of my way to remind liability-minded folks that the intelligent adversary makes our situation much different from the “acts of god” kinds of hazard, I should acknowledge the non-randomness of the threat… but I am not ready to do that, exactly… for the same reasons as the “random walk down Wall Street” problem – easy to assert non-randomness yet hard to show otherwise.

Here is my thought process:
a) If it isn’t random, it should be predictable; and
b) if it isn’t predictable, then it approximates randomness (especially in the aggregate).
c) Since we can’t predict threat (afaik) then we should be evaluating any model compared to random, so
d) random is a good place to start.

There are many ways to approach how to determine Pr(t) – could be degrees of belief, could be public data (real-time blacklists, etc.), could be based on historical data, could be something else. My favorite application is a simple comparison of two scenarios. I don’t even quantify – just look at the accessibility of the two “systems of interest” and determine which one is higher (compare, say, a bluesnarf attack that requires local proximity to a sql injection that can happen from anywhere; or assess the diff in wi-fi attacks btwn being in the city and in the country). I come up with higher Pr(t) for the latter and the former in my two examples. (It may also be useful to factor in attacker’s costs in the first example).

• Pr(v): This is difficult to characterize, but I think of it more as “the probability that a system of interest will be attacked, and that the attack will succeed [within some time period].” While I agree that any non-trivial system is vulnerable in a theoretical sense, it does not appear that every system is compromised (and I think that “two kinds of orgs – those that are compromised and those that don’t know it yet” *is* closer to nonsense than r=t*v*i). Whether there is an over-abundance of targets, the attacker costs are too high, the control environment is sufficiently strong, or some other reason, not all systems are in a compromised state and so it is worthwhile to measure. It is especially important since the bulk of our defensive efforts revolve around reducing this probability.

Again, estimating Pr(v) can be done in similar ways as Pr(t). In my comparative analysis – I look at things like number of users (as vulns), size of code base, number of open ports, RASQ, etc…

• It is worth discussing why breaking down Pr(event) into Pr(t)*Pr(v) is beneficial. For the most part, I would actually prefer to simply use Pr(event) if we have enough information (historical data). For example, I think we have pretty good data on email-borne attacks and so I wouldn’t be working too hard on assessing ‘t’ and ‘v’ there, though the McColo takedown can show how much of an impact a change in ‘t’ can have.

Maybe the biggest reason is that the respective populations are different and can change drastically. Here are some use cases:

a) One of the better uses is to compare two scenarios/architectures. Banking from smartphone vs. laptop; moving to cloud from internal; determining risk btwn WEP vuln and remote Windows vuln; etc…

b) Acknowledge that if ‘t’ or ‘v’ is 0, then Pr(event) is 0. Though it is hard to conceive of a case where ‘v’ is 0, we can see ‘t’ approaching it in lots of PoCs.

c) Showing the significance of ‘t’ or ‘v’ as its partner approaches 1. I agree that ‘v’ is essentially 1.0 so why do we spend all our time on it? Maybe we should be doing other things… this is also why I think the move towards threat intel is so important.

d) To help folks see how changes in populations of either ‘t’ or ‘v’ might affect each other, and ultimately risk. Like the McColo takedown, bounties (on malware writers and bugs), etc. My favorite use may be pointing out that vuln disclosure does nothing to ‘v’ since it was already there; the impact is on ‘t.’

To round things out, all this is “good enough” at the level of precision we are working at, and “better than” existing practices, IMO.

For our latest debate – Android vs. iOS – there are three sets of numbers that have recently come into play for evaluation:

1. Number of vulnerabilities: A recent blog post on TheVerge.com highlights that iOS and its 238 vulns from 2007-2013 has 8.8x more vulnerabilities than Android’s 27 from 2009-2013.
2. Number of malware samples: In April, a Symantec report [PDF] pointed out that Apple’s 387 vulns in 2012 dwarfs Android’s 13 and yet Android had 103 “mobile threats” (malware) compared with Apple’s 1. Importantly, they also point out that “most mobile threats have not used software vulnerabilities.”
3. Percent of traffic: A paper presented at NDSS ’13 [PDF] monitored actual smartphone traffic and found that a) “The mobile malware found by the research community thus far appears in a minuscule number of devices in the network: 3,492 out of over 380 million (less than 0.0009%)” and b) “users of iOS devices are virtually identically as likely to communicate with known low reputation domains as the owners of other mobile platforms, calling into question the conventional wisdom of one platform demonstrably providing greater security than another“

Now, since we all know that security is the number one priority for IT decisions (heh), the CIO is waiting to hear from us on which platform is more secure. How do you answer?

Here’s my analysis, just using the numbers provided*

First, number of vulnerabilities as a measure is often thought of as a leading indicator of risk even though we all recognize that more vulns found equals fewer vulnerabilities remaining. The perception, however, is that there are actually even more vulns left. Absent of any other information, however, it is worth considering the notion that a higher number here is a measure of stronger security going forward (that is, #vulns is a lagging indicator). It doesn’t help matters that at least one of the sets of numbers inexplicably uses different time periods in its analysis. This measure would be much more useful if we had a way to normalize the numbers across platforms – the two most obvious ways would be with 1) a measure of complexity or size of the code base or 2) a measure of the personhours expended in looking for vulns. While I favor this latter option, it is not very practical.

The second measure, number of malware samples, is interesting because it is closer to the actual compromise. In addition, as Symantec points out many of them don’t exploit software vulnerabilities (this is another knock against using vuln counts). The challenge here is that there is essentially unlimited ability to create more malware samples. Moreover, the notion of a “mobile threat” is fairly broad and not always threatening to the extent that legitimate apps have some similar characteristics. Given the (somewhat) restricted methods for distribution and installation of apps on smartphones, a better measure would be to identify the distribution and accessibility to the population of these malware apps. In this case, getting an understanding of the number of downloads would get significantly closer to understanding the relative risk.

The final measure, compromised smartphones, provides a historical measure of actual infected phones. Aside from the really, really low number, we must decide whether these values are a good reflection of (future) risk or not. Since this number identifies compromised systems, it gets us closest to that which we are trying to prevent, which is useful. Ultimately, I believe this measure is the best of the three in helping us understand “risk” in the mobile world. And right now, it’s a tossup.

A better measure for determining which platform is more secure, in my opinion, would involve a measure of attack surface combined with one of devices sold (as a placeholder for activity volume and popularity).