Risk

When we talk about security of systems, a lot of focus and attention goes on the external attacker. In fact, we often think of “attacker” as some form of abstract concept, usually “some guy just out causing mischief”. However, the real goal of security is to keep a system running and internal, trusted users, can upset that goal more easily and more completely than any external attacker, especially a systems administrator. These are, after all, the men and women you ask to set up your system and defend it.

So us Security SE members took particular interest when user Toby asked: when the systems admin leaves, what extra precautions should be taken? Toby wanted to know what the best practise was, especially where a sysadmin was entrusted with critical credentials.

A very good question and it did not take long for a reference to Terry Childs to appear, the San Francisco FiberWAN administrator who in response to a disciplinary action, refused to divulge critical passwords to routing equipment. Clearly, no company wants to find themselves locked out of their own network when a sysadmin leaves, whether the split was amicable or not. So what can we do?

Where possible, ensure “super user” privileges can be revoked when the user account is shut down.

From Per Wiklander. The most popular answer by votes so far also makes a lot of sense. The answer itself references the typical Ubuntu setup, where the root password is deliberately unknown to even the installer and users are part of a group of administrators, such that if their account is removed, so is their administrator access. Using this level of granular control you can also downgrade access temporarily, e.g. to allow the systems admin to do non-administrative work prior to leaving.

However, other users soon pointed out a potential flaw in this kind of technical solution – there is no way to prevent someone with administrative access from changing this setup, or adding additional back doors, especially if they set up the box. A further raised point was the need not just to have a corporate policy on installation and maintenance, but also ensure it is followed and regularly checked.

Credential control and review matters – make sure you shut down all the non-centralized accounts too.

Mark Davidson‘s answer focused heavily on the need for properly revoking credentials after an administrator leaves. Specifically, he suggests:

• Do revoke all credentials belonging to that user.
• Change any root password/Administrator user password of which they are aware.
• Access to other services on behalf of the business, like hosting details, should also be altered.
• Ensure any remote access is properly shut down – including incoming IP addresses.
• Logging and monitoring access attempts should happen to ensure that any possible intrusions are caught.

Security SE moderator AviD was particularly pleased with the last suggestion and added that it is not just administrator accounts that need to be monitored – good practice suggests monitoring all accounts.

It is not just about passwords – do not forget the physical/social dimension.

this.josh raises a critical point. So far, all suggestions have been technical, but there are many other issues that might present a window of opportunity to a sysadmin with a mind to cause chaos. From the answer itself:

• Physical access is important too. Don’t just collect badges; ensure that any access to cabinets, server rooms, drawers etc are properly revoked. The company should know who has keys to what and why so that this is easy to check when the sysadmin leaves.
• Ensure the phone/voicemail system is included in the above checks.
• If the sysadmin could buy on behalf of the company, ensure that by the time they have left, they no longer have authority to do this. Similarly, ensure any contracts the employee has signed have been reviewed – ideally, just like logging sysadmin activities you are also checking this as they’re signed too.
• Check critical systems, including update status, to ensure they are still running and haven’t been tampered with.
• My personal favorite: let people know. As this.josh points out, it is very easy as a former employee knowing all the systems to persuade someone to give you access, or give you information that may give you access. Make sure everyone understands that the individual has left and that any suspicious requests are highlighted or queried and not just granted.

Conclusions

So far, that’s it for answers to the question and no answer has yet to be accepted, so if you think anything is missing, please feel free to sign up and present your own answer.

In summary, there appears to be a set of common issues being raised in the discussion. Firstly: technical measures help, but cannot account for every circumstance. Good corporate policy and planning is needed. A company should know what systems a systems administrator has access to, be they virtual or physical, as well as being able to audit access to these systems both during the employee’s tenure and after they have left. Purchasing mechanisms and other critical business functions such as contracts are just other systems and need exactly the same level of oversight.

I did leave one observation out from one of the three answers above: that it depends on whether the moving on employee is leaving amicably or not. That is certainly a factor as to whether there is likely to be a problem, but a proper process is needed anyway, because the risk to your systems and your business is huge. The difference in this scenario is around closing critical access in a timely manner. In practice this includes:

• Escorting the employee off the premises
• Taking their belongings to them once they are off premises
• Removing all logical access rights before they are allowed to leave

These steps reduce the risk that the employee could trigger malicious code or steal confidential information.

One of my worst ever projects was implementing PGP email encryption. Considering I have only worked in large financial services companies, that is saying something. I have reflected on the lessons learnt from this project before, but I felt that PGP was fundamentally flawed. When Apple iOS 5 was unveiled, there was a small feature that no one talked about. It was hidden among the sparkling jewels of notifications, free messaging and just works synchronization. That feature was S/MIME email encryption support. Now S/MIME is not new, however Apple is uniquely positioned with their ecosystem and user-centric design to solve the fundamental problems and bring secure email to the masses.

I have detailed the problems that email encryption solves, and those it does not solve before. To re-iterate and update:

• Attacker eavesdropping or modifying an email in transit. both on public networks such the Internet and unprotected wireless networks as well as private internal networks. Recently I have experienced things that have hammered home the reality of this threat: Heartfelt presentations at security conferences such as Uncon about the difficulties faced by those in oppressive regimes such as Iran, where the government reading your email could result in death or worse for you and your family. Even “liberal” governments such as the US blatantly ignoring the law to perform mass domestic wire tapping in the name of freedom. The right to privacy is a human right. Most critical communication these days is electronic. There is a clear problem to be solved in keeping this communication only to it’s intended participants.

• Attacker reading or modifying emails in storage. The recent attacks on high profile targets such as Sony and low profile like MTGox, who were brought low by simple exploitation of unpatched systems and SQL injection vulnerabilities, have revealed an un-intended consequence. The email addresses and passwords published were often enough to allow attackers access to a users email account as the passwords were re-used. I don’t know about you but these days a lot of my most important information is stored in my gmail account. The awesome search capability, high storage capacity and accessibility everywhere means that it is a natural candidate for scanned documents and notes as well as the traditional highly personal and private communications. There is a problem to be solved of adding a layered defence; an additional wall in your castle if the front gate is breached.

• Attacker is able to send emails impersonating you. It is amazing how much email is trusted today as being actually from the stated sender. In reality normal email provides very little non-repudiation. This means, it is trivial to send an email that appears to come from someone else. This is a major reason why phishing and spear phishing in particular are still so successful. Now spam blockers, especially good ones like Postini have become very good at examining email headers, verified sender domains, MX records etc to reject fraudulent emails (which is effective unless of course you are an RSA employee and dig it out of your quarantine). You can also have more localised proof of concept of this. If your work accepts manager e-mail approvals for things like pay rises and software access, and it also has open SMTP relays you can have some fun by Googling SMTP telnet commands. There is a problem to solve in how to reliably verify the sender of the email and ensure the contents have not been tampered with.

Problems not solved by email encryption and signing:

• Sending the wrong contents to the right person (s)
• Sending the right contents to the wrong person (s)
• Sending the wrong contents to the wrong person (s)

These are important to keep in mind because you need to choose the right tool for the job. Email encryption is not a panacea for all email mis-use cases.

So having established that there is a need for email encryption and signing, what are the fundemental problems with a market leading technology like PGP (now owned by Symantec)?

Fundemental problems in summary:

• Key exchange. Bob Greenpeace wants to send an email to Alice Activist for the first time. He is worried about Evil Government and Greedy OilCorp reading it and killing them both. Bob needs Alice’s public key to be able to encrypt the document. If Bob and Alice both worked at the same company and had PGP universal configured correctly, or worked in organisations that both had PGP universal servers exposed externally, or had the foresight to publish their public keys to the public PGP global key server, all would be well. However this is rarely the case. Also email is a conversation. Alice not only wants to reply but also add in Karl Boatdriver in planning the operation. Now suddenly all three need his public keys and he theirs. Bob, Alice and Karl are all experts in their field but not technical and have no idea how to export and send their public keys and install these keys before sending secure email. They have an operation to plan for Thursday!

• Working transparently and reliably everywhere. Bob is also running Outlook on his Windows phone 7 (chuckles), Alice Pegasus on Ubuntu and Karl Gmail via Safari on his iPad. Email encryption, decryption, signing, signature verification, key exchange all need to just work on all of these and more. It also cannot be like PGP desktop which uses a dirty hack to hook into a rich email client. This gives it extremely good reliability and performance. Intermittent issues like emails going missing and blank emails never occur. Calls to PGP support for any enterprise installation are few and far between. All users in companies love PGP and barely notice it is there. In many organisations email is the highest tier application. It has to have three nines uptime. If Alice and Karl are about to be captured by Greedy Oilcorp and they need to get an email to Bob they cannot afford to get a message blocked by PGP error.

Less fundamental but still annoying:

• Adding storage encryption – there is no simple way with PGP to encrypt a clear text email in storage. You can send it to yourself again or export it to a file system and encrypt it there but that is it. This is a problem in where the email is not sensitive enough to encrypt in transit (or just where you just forgot) but where after the fact, you do care if script kiddies with your email password get that email and publish it on Pastebin.

How Apple could solve these fundamental problems with iOS 5

Key exchange – If you read marknca‘s excellent iOS security guide, you will see that Apple has effectively built a Public Key Infrastructure (PKI). Cryptographically signing things like apps and checking this signature before allowing them to run is key to their security model. This could be extended to users to provide email encryption and specifically transparent key exchange in the following manner:

• Each user with an Apple ID would have a public/private key pair automatically generated
• The public keys would be stored on the Apple servers and available via API and on the web to anyone
• The private keys would be encrypted to the the users Apple account passphrase and be synchronised via iCloud to all iOS and Mac endpoints
• To send a secure email the user would just click a secure button on their email tool
• When sending an email via an iOS device, a Mac or via Mobile me web it would query the Apple servers for the recipients private key and encrypt and sign (optional) the email
• On receipt as long as the device had been unlocked the email would be decrypted. For web access as long as the certificate was stored in the browser the email would be decrypted (could be an addon to Safari)
• To encrypt an email in storage you would just drag it to a folder called private email. You could write rules for what emails were automatically stored there.

This system would make key exchange, email encryption, decryption and signing totally transparent. Because S/MIME uses certificates it is easy to get the works everywhere property. However Apple would first enable this only for .me addresses and iOS devices and Mac’s to enable them to sell more of these and to beef up their enterprise cred. Smart hackers and addon writers would soon make it work everywhere though. There you go, secure email for the masses.

Today we investigate the problem of disposing of hardware storage devices (say, hard disks) which may contain sensitive data. The question which prompted this discussion “Is it enough to only wipe a flash drive once” is about Flash disks (SSD) and received some very good answers; here, we will try to look at the wider picture.

Confidentiality Issues and How to Avoid Them

Storage devices grow old; at some point we want to get rid of them, either because they broke down and need to be replaced, or because they became too small with regards to what we want to do with them. A storage device does not shrink over time, but our needs increase. Quite some time ago, I was sharing some disk space with about one hundred co-students, and the disk offered a hefty 120 megabytes. At that time, a colorful GIF picture was considered to be the top of technology. Today, we take for granted that we can store 2-hour long HD videos on a cellphone. The increase in disk space shows no sign of slowing down, so we have a steady stream of old disks (or USB sticks or SD cards or even ZIP/Jaz cartridges for the old timers — I will not delve into floppy disks) to dispose of. The problem is that all these pieces of hardware have been used quite liberally to store data, possibly confidential data. For the home user, think about the browser cookies, the saved passwords, the cryptographic private keys… In a business context, just about any data element could be of value for competitors.

This is a matter of confidentiality. There are two generic ways to deal with it: encryption, and destruction.

Disk Encryption

Disk encryption is about transforming data in a way which is reversible only with the knowledge of a given secret short-sized element called a key. We are talking about symmetric encryption here; the key is a simple sequence of, say, 128 bits chosen at random. The confidentiality issue is not totally obliterated, only severely reduced: supposedly, it is easier to deal with the secrecy of a sequence of 128 bits than that of 100 GB worth of data. Encryption can be done either by the disk itself, by the operating system, or by the application.

Application-based encryption is limited to what the application can control. For instance, it may have to fight a bit with the OS about virtual memory (that’s pure memory from the point of view of the application, but the OS is prone to write it to the disk anyway). Also, most applications do not have any encryption feature, and modifying all of them is out of the question (not even counting the closed-source ones, that’s just too much work).

OS disk encryption is more thorough, since it can be applied on the complete disk for all files from all applications. It has a few drawbacks, though:

• The computer must still be able to boot up; in particular, to read from the disk the code which is used to decrypt data. So there must be at least an unencrypted area on the disk. This can cause some system administration headaches.
• Performance may suffer, for an internal hard disk. An x86 Core2 CPU at 2.4 GHz can encrypt about 160 MB/s with AES (that’s what my PC does with the well-known OpenSSL crypto library): not only do some SSD go faster than that, I also have other things to do with my CPU (my OS is multitasking).
• For external media (USB sticks, Flash cards…), there can be interoperability issues. There is no well-established standard on disk encryption. You could transport some appropriate software on the media itself but most places will not let you install applications as easily.

Encryption on the hardware itself is easier, but you do not really know if the drive does it properly. Also, the drive must keep the key in some way, and you want it to “forget” that key when the media is decommissioned. As Jesper’s answer describes, good encrypted disks keep the key in NVRAM (i.e. RAM with a battery) and can be instructed to forget the key, but this can prove difficult if the disk is broken: if it does not respond to commands anymore, you cannot really be sure that the NVRAM got blanked.

So while encryption is the theoretically appropriate way to go, it is not complete (you still have to manage the confidentiality of the key) and, in practice, it is not easy. Most of all, it works only if it is applied from day one: encryption can do nothing about data which was written before encryption was envisioned at all. So let’s see what can be done to destroy data.

Data Wiping

Wiping out data is a popular method; but popular does not necessarily mean efficient. The traditional wiping patterns rely on the idea that each data bit will be written exactly over the bit which was at the same logical emplacement on the disk: this was mostly true in 1996, but technology has evolved. Today’s hard disks do not have “track railings” to guide the read/write heads; instead, they use the data itself as guide. The net result is that the new data may be physically off the previous one by a small bit; the old data is still readable “on the edge”.

Also, modern hard disks do not have visibly bad sectors. Bad sectors still exist, but the disk transparently substitutes good sectors instead of bad sectors. This happens dynamically: when a disk writes some data on a sector and detects that the write operation did not go well, then it will allocate a new good sector from its spare area and do the write again there. From the point of view of the operating system, this is invisible; the only consequence is that the write took a few more milliseconds than could have been expected. However, the data has been written to the bad sector (admittedly, one or two bits of it may be wrong, but this leaves more than 4000 genuine bits) and since the sector is now marked as “bad”, it is forever inaccessible from the host computer. No amount of wiping can do anything about that.

On Flash, the same issue arises, multiplied a thousand times. Flash memory works by “blocks” (a few dozen kilobytes) in which two operations can be done:

• changing a bit from value 1 to value 0;
• erasing a whole block: all the bit blocks are set to 1.

A given block will endure only that many “erase” operations, so Flash devices use wear leveling techniques, in which write operations are scattered all about the place. The “Flash Translation Layer” is a standard wear leveling algorithm, designed to operate smoothly with a FAT filesystem. The wear leveling means that if you try to overwrite a file, the wiping pattern will be most of the time written elsewhere. Moreover, some blocks can be declared as “bad” and remapped to spare blocks, in a way similar to what magnetic hard disks do (only more often). So some data blocks just linger, forever unreachable from any software wiping.

The basic conclusion is that wiping does not work against a determined attacker. Simply overwriting the whole partition with zeros is enough to deter an attacker who will simply plug the disk in a computer: logically, a single write is enough to “remove” the data, and by working on the partition instead of files, you avoid any filesystem shenanigans. But if your enemies are so cheap that they will limit themselves to the logical layer, then you are lucky indeed.

@nealmcb’s question, How can I reliably erase all information on a hard drive? sparked some excellent discussion including NIST’s guidelines for Media Sanitisation.

Media Destruction

Without encryption (does not work for data which is already there) and data wiping (does not work reliably, or at all), the remaining solution is physical destruction. It is not that easy, though; a good sledgehammer swing, for instance, though satisfying, is not very effective towards data destruction. After all, this is a hard disk. To destroy its contents, you have to remove the cover (there are often many screws, some of which hidden, and glue, and rivets in some models) and then extract the platters, which are quite rigid disks. A simple office shredder will choke on those (although they would easily munch through older floppy disks). The magnetic layer is not thick, so mechanical abrasion may do the trick: use a sander or a grinder. Otherwise, dipping the platters in concentrated sulfuric acid should work.

For Flash devices, things are simpler: that’s mostly silicon with small bits of copper or aluminium, and a plastic cover. Just burn it.

Bottom-line: media destruction requires resources. In a business environment, this could be a system administrator task, but it will involve extra manpower, safety issues (seriously, a geeky system administrator with access to an acid cauldron or a furnace, isn’t it a bit scary ?) and possibly environmental considerations.

Conclusion

Data security must be thought throughout the complete life cycle of storage devices. Whether you go crypto or physical, you must put some thought and resources into it. Most people will simply store old disks and hope for the problem to get away on its own accord.

Despite the security industry getting ever more professional, with well trained teams, security incidents seem to be increasing. Just look at the news recently:

• Sony – 23 incidents so far?
• UK NHS Laptop – over 8 million patient records
• Citibank – 200,000 customer records lost
• WordPress platform vulnerabilities (see Steve Lord’s presentations on this)
• Stuxnet – targeted at a specific use

From the 2011 Verizon DBIR

• 761 breaches of which 87% were in Hospitality, Retail and Financial Services
• 436 of these were in companies with 11-100 employees – so this is not just a problem for the big targets any more
• 92% of attacks (leading to 99% of records compromised) were external

Some new threat groups:

Anonymous and LulzSec – often punitive, sometimes for the Lulz

• HBGary
• ACS Law
• Spanish Police
• FBI, CIA
• Porn sites
• Took requests

So what do the security professionals do?

• After an audit fix vulnerabilities
• After an attack, fix the issues
• Use encryption and strong authentication
• Patch regularly
• Validate input, encode output
• Assess your 3rd parties
• Use a Secure Development Lifecycle
• Build security into everything you do

No problem, right?  If only it were that easy…

Your CEO wants the business to make money, so wants to minimise bottom line spend but will spend appropriately on risks to the business.

So why doesn’t the CEO see these security issues as business risks?

• Maybe they aren’t significant when compared to other business risks
• Or perhaps the CEO just doesn’t understand the latest security threat

Both of these are your problems. The security community is often called the ‘echo chamber’ for a good reason – we say good things and have good ideas about how to fix security issues, but we pretty much only communicate with other enlightened security professionals. We need to tell others in a way they can understand – but in general we talk technical 1337 speak full of jargon and no one can make head or tail of it.

It is up to you, the security professional to learn how to talk the language of business risk

There are materials out there that will help you:

• IIA GAIT – Guide to the Assessment of IT Risk
• FAIR – Factor Analysis of Information Risk
• ISACA CobIT – Control Objectives in IT
• ISACA – CRISC and CGEIT certifications
• IISP – approved Information Risk Management courses

Begin to understand what is important to a business executive and how they talk about it, and you are already well on the way to being able to articulate security in the same way. If you are a manager of security professionals, make it easy for them to gain exposure to heads of business, C-suite, executive, directors etc. and it is likely to benefit you in the long run.

These are good questions to prepare you for conversations you may have along the way

• What metrics should a CIO rely on to gauge risk
• How would you work with the security paranoiac on your team
• Can you provide loss values on security breaches
• How do you compare risks from website, physical perimeter, staff, etc.
• What are good threats to kick off conversations with others about what worries them