Patching security governance: an empirical view of emergent governance mechanisms for cybersecurity

3.1 Shifting property rights from users to internet intermediaries

Schneier (2012, 2013) has observed that two recent developments are impacting the authority of owners of nodes: the rise of cloud computing and vendor-controlled platforms. The first trend means that more of our data and computing takes place on the networks of others, rather than on our own node. Obvious examples are Gmail, SalesForce, Amazon Elastic Cloud Compute, Facebook, Uber, Spotify, Office 365, Dropbox, etc. The second trend means that more and more of our devices are closed down, or at least less open than general-purpose computers, and controlled by vendors. Think of Apple’s iOS or Google’s Android operating systems for mobile devices. Both vendors limit what users can do with their devices, i.e. what code they can run, by restricting them to a curated app store with apps that run under limited privileges on the device. Google’s model is a bit more open than Apple’s, as it allows the use of alternate app stores and the side-loading of apps. Apple actively prevents this by legally fighting users who find ways to circumvent these restrictions on the devices that they own. This difference between the two vendors is important, but it is a gradual one.

These two trends are not just technological advances, they are propelled by powerful economic forces. Most notably: cost. Cloud computing offers dramatic efficiency gains compared to consumers and businesses owning and maintaining their own computing infrastructure. In addition to cost, there is also increased reliability, convenience and yes, security. Google is more competent in securing the Gmail platform than most corporations are in securing their mail servers. Closed operating systems are also easier to defend against compromise, as it is more difficult to exploit vulnerabilities or to trick users to install malicious apps, as these are constantly checked and removed from the app stores.

In other words, these developments are not intrinsically wrong. We derive a lot of benefits from them. But they do mean that power has shifted and will continue to shift from the owners of devices to the cloud operators and platform and device vendors. The OECD (2010, 2011) has referred to these companies as internet intermediaries. Their security practices determine to an increasing degree the security of everyone. And, in many cases, we do not really have a good way to evaluate what they are doing. Cloud operators typically do not allow you to audit their systems and service. There is a fundamental information asymmetry that comes into play.

Schneier (2013) has characterized our increasing dependence for security on these intermediaries as a “feudal relationship”. We trade away our some of our sovereignty to these companies and get security in return. Another way to say this is: certain the property rights that we used to associate with the device itself are now tied to the services of cloud and platform operators, and thus are being transferred to those operators. The device itself becoming less important for security governance if my data and computing tasks are no longer performed by that device. In a way, the ownership and associated property rights of data and computing are migrating from the nodes that the user owns to those of the intermediaries.

These intermediary markets are often concentrated in one or a small number of providers. They operate in one- or two-sided markets with positive externalities, which give rise to a “winner takes all” dynamic. These large, concentrated firms offer new control points for governments, as they are much more within the grasp of laws and other instruments of state power than the heavily distributed set of autonomous nodes ever were. We have seen how the interests of corporate and government power have been aligning. A significant number of the so-called internet giants rely on surveillance as much as governments do. Snowden made visible the extent to which NSA was using Google, Facebook, Verizon and others to get access to data it could not otherwise. An example in the other direction is how the entertainment industry is recruiting governments to enforce their business models.

We do not know exactly how our sovereignty is being eroded or what the security and privacy tradeoffs are that we are entering into. It is hard enough to assess what Facebook is doing with user data. What about the many derivative applications that we are not even aware of? Think of the legal use of Facebook data by firms like Cambridge Analytics, which combines Facebook data with other data of users’ online and offline behavior to enable micro-targeted communication for the political campaigns behind Brexit and Trump (Cadwalladr, 2017). Beyond the derivate business models around large intermediaries, there are firms tracking our behavior in ways that are unknown to most of us and whose business models are a mystery (Englehardt and Narayanan, 2016).

In sum, the governance of security has shifted from device owners to large intermediaries, toward a model that Schneier has dubbed feudal security. This shift is generally associated with certain security benefits, most notably more centralized control over the security of devices and services for millions or even billions of users, and economies of scale in terms of concentrating security expertise and resources in a small number of companies that serve many users, rather than users having to fend for themselves. There are also security tradeoffs, however, that are very difficult to evaluate at this moment because of information asymmetries. We currently have no good answer for this problem.

3.2 Limitations on property rights of owners

Next to transfer of property rights from device owners to intermediaries, another trend is the growing number of regulatory constraints on the property rights of device owners. This is mainly happening in sectors that were already strongly institutionalized and regulated, such as health, energy, financial services and transportation.

Information and communication technology has been permeating more and more processes in these regulated sectors, initially without much guidance in terms of standards or regulatory guidelines. Some of the security incidents over the past decade and a half reveal the presence, and failure, of commercial of-the-shelf products in these environments. In 2003, for example, the Slammer worm infected Microsoft Windows systems in the Davis-Besse nuclear power plant in Ohio, disabling a safety monitoring system for nearly five hours. In healthcare, many medical devices have been found to be running outdated or even deprecated operating systems. A ransomware attack in May 2017 crippled operations in several UK hospitals. There has been a range of malware families that target ATMs running some version of Windows.

It has taken a long time for regulatory mechanisms in these sectors to catch up, mostly because of the complexity and speed of the changes. Slowly but surely, though, security standards are being recommended or mandated in these sectors. Many of these standards are process-based (“adopt adequate safeguards”), rather than mandating specific technical security measures. A baseline standard that is being adopted across many sectors is ISO 27001. In the European Union, the directive on security of network and information systems (the NIS Directive) identifies “essential service operators” within the energy, transport, banking, financial market infrastructure, health, drinking water and digital infrastructure sectors and requires them to adopt security precautions and mandatory breach notifications.

Beyond these generic frameworks, there are many sector-specific standards. The US Health Insurance Portability and Accountability Act (HIPAA) of 1996 has a security rule that requires certain controls to be put in place. The North American Electricity Corporation (NERC) has adopted mandatory patching policies for operators. Financial services have had the PCI DSS standard in place for a long time already, but its norms are updated more quickly in light of new threats, and they are also being extended to other players in the value chain, such as third-party service providers.

While these regulations, in principle, constrain the property rights of owners in terms of mandating certain precautions, it is unclear just how much impact they have. These standards are typically articulated quite abstract (e.g. deploy access controls to ensure confidentiality of data) or process-oriented (e.g. allocate responsibility for information security in a Chief Information Security Officer role). Flagrant breaches of payment systems at large US retailers have occurred even though they were PCI DSS compliant, to name one example. On the other hand, even non-binding guidance, such as the US Food and Drug Administration’s (FDA) “postmarket management of cybersecurity for medical devices” might impact security practices, under the threat of law suits by patients whose care was impeded because of a medical device that did not meet best practice standards. Beyond security precautions, there are other regulations in place, such as mandatory breach reporting to various institutions, requirements for sharing data with the government or rules on fraud reimbursement. These regulations, however, do not directly affect the property rights associated with the IT infrastructure that the organizations own.

The importance of sector-specific frameworks is likely to increase over the next years, as the proliferation of the so-called Internet-of-Things (IoT) takes off. IoT, pervasive computing, ubiquitous computing, these terms signal the disappearance of the distinction between devices with and without connectivity and computing capabilities. Without that distinction, it also becomes less meaningful to think about cybersecurity governance as a space with a certain structural coherence. A recent outlook on the governance of IoT suggested that, in many sectors, it will merge with the well-institutionalized framework of safety – e.g. car safety, medical device safety and toy safety. Some of the obligations fall onto the owner, some onto the vendor of the device. (We discuss the latter point in a moment).

3.3 Limitations on property rights of intermediaries

In many democratic countries, legal frameworks for online commerce and services have shielded intermediaries from liability for what happens on their networks and platforms. In the EU, for example, the Electronic Commerce Directive, adopted in 2000, declared that transmission (“mere conduit”), caching and hosting services were not liable for the information they handled, as long as the provider does not have actual knowledge of illegal activity or information and as long as, upon obtaining such knowledge, acts to remove or to disable access to the information (“notice and take down”). The USA has a similar framework with the so-called “safe harbor” provisions in the Communications Decency Act and the Digital Millennium Copyright Act. While these liability exemptions are still in place, intermediaries are regularly under pressure to be more proactive in dealing with hate speech, copyright infringement, anti-terrorism, protection of minors.

In principle, security incidents, such as compromised websites or spam sources, can also be viewed within this framework. While their legal liability is limited, intermediaries have acted more frequently against security threats over time, when they perceived this as being in their interest or part of their responsibility – i.e. to protect their services or users.

As intermediaries have become more dominant, the calls to impose more obligations on them have also intensified, including in the area of security. Some of these calls have helped shape new legislation. The EU NIS directive has articulated security requirements for “digital service providers”, which are defined as legal persons that provide “service normally provided for remuneration, at a distance, by electronic means and at the individual request of a recipient of services” (European Commission, 2017). The directive then specifies this as applying to three types of intermediaries: “online marketplace”, “online search engine” and “cloud computing services”. They are subject to the same requirements as the “essential service operators” discussed above: taking adequate security precautions and mandatory breach notifications. These requirements overlap with the more narrow set of obligations flowing from the EU General Data Protection Regulation, which focuses on the protection of personal data.

Another area of contention and regulation has been around the issue of “net neutrality”, which can be crudely summarized as treating all traffic data as equal. While this issue is not about security, neutrality rules do touch on it, as many security practices of network operators by necessity discriminate good from bad traffic. These practices required exemptions to strict neutrality, to remain legal.

In terms of security, intermediaries’ property rights have also been shaped by softer mechanisms than formal regulation. For example, in many countries, public-private partnerships have emerged in which ISPs agree to block spam from leaving their network or to mitigate botnets by contacting and quarantining customers whose devices are infected with malware. The additional responsibility is sometimes tied to some form of “duty of care”, though the legal basis to subsume security under this duty and what the duty exactly demands of the intermediary are not always clear.

In short: the increasing dominance of intermediaries appears to go hand in hand with more restrictions on their property rights in terms of what they should or should not do. New regulations have required security precautions to be taken. Other security practices, such as botnet mitigation by ISPs, seem the result of softer forms of government pressure and appeals to “duty of care”. That being said, security requirements are limited and liability shields still seem firmly in place. Beyond imposing obligations, where security is aligned with their business interests, intermediaries undertake security measures without external pressure or limitations on their property rights. Facebook, for example, exerts quite a lot of effort to protect its users from malicious content on the platform – e.g. links to phishing or drive-by-download sites.

3.4 Limitations on property rights of vendors

The last shift in security governance to highlight here is changes in the property rights of vendors. Conventionally, software and hardware vendors put their products into the market without requirements in terms of how they were secured. Vendors could offer security features and even compete on them, of course, as part of the freedom of contract. In practice, this freedom has meant that the software industry disclaimed liability for the harm when its products fail. Users, whether corporate or consumer, have to accept End User License Agreements (EULAs) to be able to use the product. This is not without benefits in terms of innovation (“go fast and break things”), but the downside is that time-to-market and other economic incentives have often trumped security. The vendor bears few, if any, of the damages that result from security issues.

In a recent court case, the Dutch consumer union (Consumentenbond) took Samsung to court for failing to release security patches for even recent phones. The unwillingness to release such patches is somewhat startling. Most of Samsung’s phones run on Android. Google diligently patches discovered vulnerabilities and then releases these patches to all manufacturers using Android. So, the cost to Samsung of passing these patches onto their customers is quite limited. It would need to do some quality assurance testing to prevent adverse effects of the patch on Samsung’s own software, which it bundles on top of Android. Still, even this limited effort is apparently too much to ask. The consumer union demanded that patches should be released for at least two years after the phone is purchased, which is in line with the normal period for product warranty. In the first case, the court ruled against the consumer union on procedural grounds. In November 2016, the union initiated a new case that will receive a more in-depth treatment from the court.

An interesting counterpoint to this case is another case that also involves Samsung and a software update – and it points to important changes in security governance. In August 2016, the company released a new flagship phone: the Note 7. Over the next few months, serious problems emerged as some of the batteries overheated, caught fire or exploded. In the end, the company had to issue a total recall. Not all owners of the Note 7 were willing to return their device, however. In response, Samsung rolled out a software update that prevented the phone from charging, rendering it completely unusable.

Note the stark contrast between the lackluster attitude to updating software for security vulnerabilities and the aggressive approach to updating once Samsung was working within the product liability framework. Why is this pointing to a shift in governance? Because the pervasive presence of software in all kinds of products – also known as IoT – will mean that cybersecurity will increasingly become subsumed into product liability. Another clear illustration is Fiat Chrysler’s two recent recalls for software vulnerabilities in its cars. No accident had occurred, nor had the vulnerability been exploited by criminals, as far as we know. Its mere presence was enough to trigger a recall. The most recent one took place in May 2017 and affected 1.3 million cars. Almost at the same time, the FDA sent a scathing letter to St Jude Medical, a manufacturer of pacemakers, faulting it for knowing about grave security issues in its implantable medical devices as early as 2014, but failing to address them with software updates or by replacing those devices.

The product liability regime is much better institutionalized and provides very strong incentives for vendors. It places constraints, sometimes severe ones, on their property rights. One-sided EULAs disclaiming liability for damages, which users have been habituated to click though because they have no meaningful other option, are overruled by the laws and jurisprudence of product liability.

Of course, not all computing devices or software vulnerabilities will fit within the product liability framework. In fact, this framework has been in place all along, but it did not fit the type of damages associated with security flaws: typically small economic damages, rather than fatalities or injuries, which are distributed over many parties, sometimes millions of them, across many jurisdictions. For those types of security issues, other mechanisms are emerging. In 2015, the US Federal Trade Commission filed a complaint against ASUSTeK Computer that alleges that the company did not take reasonable steps to secure the software on its routers. Researchers had uncovered a variety of vulnerabilities in these devices. As per normal industry practice, ASUS did not address these security flaws in a timely manner, nor did it notify consumers about the risks posed by the vulnerable routers. In the end, the case was settled out of court. As part of the settlement, ASUS is required to establish and maintain a comprehensive security program subject to independent audits for the next 20 years.

ASUS is just one example in a sheer endless list of vendors with similar practices. Most of these firms we have never heard of, yet their products enter markets and networks across the world. In September 2016, the Mirai botnet captured over one million of such poorly secured IoT devices; routers, webcams, digital video recorders, IP cameras and many unknown types of “things”. Mirai was used to conduct the largest DDoS attack ever recorded, reaching a peak of over 1 Tb per second during an attack on Dyn, a DNS service provider for Netflix, Twitter, CNN and many other companies, disrupting the services of all these organizations.

Researchers and the security journalist Brian Krebs started calling out vendors whose devices were observed to be compromised and participating in the attacks. Normally, such naming and shaming campaigns have little effect. But the sheer size and headline-grabbing character of the attack made it different this time. In response, a Chinese vendor of network cameras named XiongMai vowed to initiate a product recall. (At the same time, it was threatening legal action against the accusers.) It also said it had changed the default settings of the camera to turn off telnet – a key attack vector – and ensure that users changed the factory-default passwords.

Another development that Mirai triggered is that the European Commission is preparing new legislation to protect IoT devices from security threats. It is looking at the certification and labeling systems for approved secure devices (Krebs, 2016). This could mean that customs will stop devices from entering the EU market that do not have the required certification.

In sum: a number of forces are converging that are reshaping the governance of security and constraining the property rights of vendors. It will require many vendors then to put in place at least basic security practices, such as patching known vulnerabilities, and expose many others to product liability regimes and certification requirements tied to import barriers.

4.1 Mitigating botnets

The global malware outbreaks of the early 2000s, like the ILOVEYOU and CODE RED computer worms, were disruptive and highly visible. Their authors seemed motivated mainly by the quest for notoriety. Then, as more economic transactions moved online and the cost of abusing vulnerabilities decreased, profit-driven criminals entered the scene and rapidly expanded their activities. Their incentives changed malware from visible and disruptive to stealthy code that kept the machine of the victim up and running as part of a criminal infrastructure with a command and control mechanism: a network of robots, a.k.a. botnet. Criminals discovered an expanding array of business models to monetize these infected machines: sending spam, extorting ransom, performing DDoS attacks, harvesting user credentials, selling fake anti-virus software, committing financial fraud, hosting phishing sites, performing click fraud against advertising networks and more.

Most of the attacks are targeting third parties. This undermines the incentives of end users to protect their machines from malware, as they do not bear the full cost of the infections. As end users were not keeping up, the ISPs have faced increasing pressure to become more active in mitigation. A variety of national anti-botnet initiatives have emerged, as well as other voluntary mechanisms, in which ISPs commit to contacting and remediating infected customers.

To evaluate the effectiveness of different mitigation strategies, we developed an approach to generate comparative infection rates per subscriber for ISPs in over 60 countries, drawing on a variety of data sources on infected machines (Asghari et al., 2015a). We found out that the 262 ISPs in our analysis harbor over 70 per cent of all infected machines worldwide. We also observed widely different infection rates among ISPs, as well as among countries, signaling differences in the effectiveness of mitigation. With those metrics in hand, we looked at different governance mechanisms.

4.2 Mitigating Web compromise

Websites and other hosting services are abused for criminal purposes. A wealth of research has identified how hosting infrastructure shows up in various criminal business models. Think of phishing sites, command-and-control servers for botnets, child pornography, malware distribution and spam servers. Nobody contests that hosting providers play a key role in fighting cybercrime. Much of the criminal activity runs on compromised servers of legitimate customers, some on servers rented by the criminals themselves. In either case, hosting providers typically become aware of the problem only after being notified of the abuse.

Somewhat analogous to our work on ISPS, we have mapped the global markets for hosting providers, which consists of around 45,000 firms (Tajalizadehkhoob et al., 2016). We leveraged a variety of data sources on compromised servers in that market to build comparative metrics for compromise rates at providers (Noroozian et al., 2015, 2017). A key problem we needed to solve is how to normalize these rates and correctly account for size and other provider properties. Larger providers, with more customers and servers, will incur more incidents than smaller providers, even with the same security effort. We did indeed find that over 84 per cent of the variance in abuse incidents can be explained by a number of simple size proxies (Tajalizadehkhoob et al., 2017a). Once we control for this difference in exposure to threats, we were able to compute comparative compromise rates.

4.3 IP source address spoofing

Our final empirical case is on the problem of IP source address spoofing. The basic TCP-IP protocol stack allows a device that is sending a packet to basically lie about it source address, the IP address from which it has been sent. The protocols in use do not contain any mechanisms to validate the source address. Despite source IP address spoofing being a known vulnerability for at least 25 years, and despite many efforts to shed light on the problem, spoofing remains a viable attack method for redirection, anonymity and DDoS amplification, as evidenced in February 2014 during a 400 Gbps DDoS attack against Cloudflare. Defeating amplification attacks, and other threats based on IP spoofing, requires providers to filter incoming packets from customer networks with spoofed source IP addresses – in other words, to implement BCP 38, a best current practice also known as source address validation (SAV).

4.4 Comparing the case studies

Summing up, we can observe that a variety of governance mechanisms have emerged around the security externalities that we studied. Remarkably, in all three areas, we find evidence of private actors partially internalizing the externalities. We encountered a surprising amount of voluntary action. This is not meant to say that these problems are solved. Not at all. One could easily argue that the current governance mechanisms are not nearly enough. That being said, it is an important finding that forms of collective action exist in this environment where control is extremely distributed and hierarchical structures are, by and large, absent. Given that environment, it will be difficult to replace or supplement them with something better.

In terms of scale, we have seen governance mechanisms that are geographically concentrated and involve a small number of actors within a specific country, to mechanisms that span the global market, involving thousands of actors. One might speculate that the mechanisms that operate on a smaller scale are more effective, as they can directly engage with relevant actors. However, the cases show mixed results in this respect. Think of the lack of impact of the pressure of Dutch law enforcement on poor performing hosting providers. Furthermore, on the other end of the scale, we find evidence that social norms propagated in the industry community turn out to have measurable impacts.

We can also map the cases onto the four canonical types of governance: market, hierarchy, network and community (Tenbensel, 2005; for a more in-depth discussion in the context of cybersecurity, see Kuerbis and Badiei, 2017). We found no examples of market mechanisms. We did encounter several examples of network governance, all at the level of countries. We also found two examples with a weak form of hierarchy: soft pressure of regulators to improve botnet mitigation seemed to work, while the “shadow of hierarchy” in the form of law enforcement visits made seemingly no impression on the worst Dutch hosters. And, we found two examples where community governance, in the form of social norms (more specifically: best practices around patching vulnerable servers and around anti-spoofing measures), had a measurable impact.

Reputation effects as an incentive could, in theory, work in all four governance types. As we did not study a market mechanism, we did not see it at work there. That said, the hosting providers we talked to were skeptical that customers in their market cared at all how well their provider did in fighting abuse. We did find that reputation effects worked in (national) networks of actors, where even the private sharing of ISP benchmarks had an impact. It is unclear to what extent public reputation – i.e. naming, shaming and praising – can be scale to transnational networks or even global markets. One experimental study found it might (He et al., 2016), but this is a subject of future work.

If nation states were involved, it mostly took the form of civil agencies participating in an industry network – in contrast to the predictions of the “militarization” narrative we discussed in earlier part of the paper. The one example where the state did assert its hierarchy, the law enforcement initiative against the hosters, was also where the state looked weak. The results were disappointing in terms of effectiveness.