So the purpose of this post is to introduce you, an IT-aware but perhaps not software-engineering person, to the various issues surrounding browser security. In order to do this I’ll go quickly through some theory you need to know first and there is some simple C-like pseudocode and the odd bad joke, but otherwise, this isn’t too much of a technical post and you should be able to read it without having to dive into 50 or so books or google every other word. At least, that’s the idea.

What is a browser?

Before we answer that we really need a crash course on kernel internals. It’s not rocket science though, it’s actually very simple. An application on your system runs and is called a process. Each process, for most systems you’re likely to be running, has it’s own address space, which is where everything lives (memory). A process has some code and the operating system (kernel) gives it a set amount of time to run before giving another process some time. How to do that so everything keeps going really would be a technical blog post, so we’ll just skip it. Suffice to say, it’s being constantly improved.

Of course, sometimes you might want to get more than one thing done at once, in which case threads come in. How threads are implemented across OSes is different – on Windows, each process also contains a single, initial thread. If you want to do more things, you can add threads to the process. On Linux, threads and processes are all the same, but they share memory. That’s the real important common idea behind a thread and is also true of Windows – threads share memory, processes do not.

How does my process get stuff done then?

So now we’ve got that out the way, how does your program get stuff done? Well, you can do one of these three things:

1. If you can do it yourself, as a program, you just do it.
2. If another library can do it, you might use that.
3. Or alternatively, you might ask the operating system to do it.

In reality, what might happen is your program asks a library to do it which then asks the operating system to do it. That’s exactly how the C standard library works, for example. The end result, however, and the one we care about, is whether you end up asking the operating system. So you get this:

Operating system  Program

Later on, I’ll make the program side of things slightly more complicated, but let’s move on.

How do we do plugins generically?

Firstly, a word on “shared objects”. Windows calls these dynamic link libraries and they’re very powerful, but all we’re concerned with here really is how they end up working. On the simplest possible level, you might have a function like this:

void parse_html(char* input)
{
    call_to_os();
    do_something_with(input);
}

A shared library might implement a function like this:

void decode_and_play_swf(char* data)
{
    call_to_os();
    do_some_decoding_and_playing_of(data);
}

When a shared object loads with your code, the result, in your program’s memory, is this:

void decode_and_play_swf(char* data)
{
    call_to_os();
    do_some_decoding_and_playing_of(data);
}</p>

<p>void parse_html(char* input)
{
    call_to_os();
    do_something_with(input);
}

Told you it was straightforward. Now, making a plugin work is a little more complicated than that. The program code needs to be able to load the shared object and must expect a standard set of functions be present. For example, a browser might expect a function like this in a plugin:

int plugin_iscompatible(int version)
{
    if ( version <= min_required_version && version <= max_supported_version )
    {
        return 0;
    }
    else
    {
        return -1;
    }
}

Similarly, the app we’re plugging into might offer certain functions that a plugin can call to do things. An example of this would be exactly a browser plugin for drawing custom content to a page – the browser needs to provide the plugin the functionality to do that.

So can all this code do what it likes then?

Well that depends. Basically, when you do this:

Operating system   Program

the operating system doesn’t just do whatever you asked for. In most modern OSes, that program is also run in the context of a given user. So that program can do exactly what the user can do.

I should also point out that for the purposes of loading shared libraries, they appear to most reporting tools as the program they’re running as. After all, they are literally loaded into that program. So when the plugin is doing something, it does it on behalf of the program.

I should also point out that when you load the plugin, it literally gets joined to the memory space of the program. Yes, this is an attack vector called DLL injection, yes you can do bad things but it is also highly useful too.

I read this blog post to know about browsers! What’s all this?

One last point to make. Clearly, browsers run on lots of different platforms. Browsers have to be compiled on each platform, but forcing plugin writers to do that is a bit hostile really. It means many decent plugins might not get written, so browser makers came up with a solution: embed a programming language.

That sounds a lot crazier than it is. Technically, you already have one quite complicated parser already in the browser; you’ve got to support this weird HTML thing (without using regex) – and CSS, and Javascript… wait a minute, given we’ve stuck a programming language parser for javascript into the browser, why don’t we extend that to support plugins?

This is exactly what Mozilla did. See their introductory guide (and as a side note on how good Stack Overflow really is, here’s a Mozilla developer on SO pointing someone to the docs). Actually in truth it’s a little bit more complicated than I made out. Since Firefox already includes an XML/XHTML/HTML parser, the developers went all out – a lot of the user interface can be extended using XUL, an XML variant for that purpose. Technically, you could write an entire user interface in it.

Just as a quick recap, in order for your XML, XHTML, HTML, Javascript, CSS to get things done, it has to do this:

Operating system   Browser   Javascript or something

So, now the browser makes decisions based on what the Javascript asks before it asks the operating system. This has some security implications, which we’ll get to in exactly one more diagram’s time. It’s worth noting that on a broad concept level, languages like Python and Java also work this way:

Operating system   Interpreter/JVM etc  Script/Java Bytecode

Browsers! Focus, please!

Ok, ok. All of that was to enable you to understand the various levels on which a browser can be attacked – you need to understand the difference in locations and the various concepts for this explanation to work, so hopefully we’re there. So let’s get to it.

So, basically, security issues arise for the most part (ignoring some more complicated ones for now) because of bad input. Bad input gets into a program and hopefully crashes it, because the alternative is that specially crafted input makes it do something it shouldn’t and that’s pretty much always a lot worse. So the real aim of the game is evaluating how bad bad input really is. The browser accepts a lot of input on varying levels, so let’s take them turn by turn:

1. The address bar. You might seriously think, wait, that can’t be a problem can it, but if the browser does not correctly pass the right thing to the “page getting” bit, or the “page getting” bit doesn’t error when it gets bad input, you have a problem, because all the links all over the input are going to break your browser. Believe it or not, in the early days of Internet Explorer, this was actually an issue that could lead you to believe you were on a different site to the one you thought you were on. See here. These days, this particular attack vector is more of a phishing/spoofing style problem than browser manufacturers getting it wrong.

2. Content parsers. For content parsing I mean all the bits that make it up – html, xhtml, javascript, css. If there are bugs in these, the best case scenario is really that the browser crashes. However, you might be able to get the browser to do something it shouldn’t with a specially crafted web page, javascript or CSS file. Technically speaking, the browser should prevent the javascript it is executing from doing anything too malicious to your system. That’s the idea. The browser will also prevent said javascript from doing anything too malicious to other pages too. A full discussion of web vulnerabilities like this is a blog post in itself, really so we won’t cover it here.

3. Extensions in Javascript/whatever the browser uses. Again, this comes down to are the vulnerabilities in the parser/enforcement rules. Otherwise, an extension of this nature will have more “power” than an in page Javascript, but still won’t be able to do too much “bad stuff”.

4. Binary formats in the rendering engine. Clearly, a web page is more than just parseable programming-language like things – it’s also images in a whole variety of complicated formats. A bug in these processors might be exploitable; given they’re written as native functions inside the browser, or as shared libraries the browser uses, the responsibility for access control is at the OS level and so a renderer that is persuaded to do something bad could do anything the user running the process can do.

5. Extensions (Firefox plugins)/File format handlers. These enable the browser to do things it can’t do in and of itself and are often developed externally to the core “browsing” part. Examples of this are embedding adobe reader in web pages and displaying flash applications. Again, these have decoders or parsers, so bugs in these can persuade the plugin to ask the operating system to do what we want it to do, rather than what it should.

You might wonder if 4 and 5 really should be subject to the same protections afforded to the javascript part. Well, the interesting thing is, that is part of the solution, but you can’t really do it in Javascript, because compared to having direct access to memory and the operating system, it would be much slower. Clearly, that’s a problem for video decoding and really heavy duty graphical applications such as flash is often used for, so plugins are and were the solution.

Ok, ok, tell me, what’s the worst the big bad hacker can do?

Depends. There are two really lucrative possible exploits from bugs of this sort; either, arbitrary code execution (do what you want) or drive by downloading (which is basically, send code that downloads something, then run it). In and of itself, the fact that you’re browsing as a restricted user and not an administrator (right?) means the damage the attacker can do is limited to what your user can do. That could be fairly critical, including infecting all your files with malware, deleting stuff, whatever it is they want. You might then unwittingly pass on the infection.

The real problem, however, is that the attacker can use this downloaded code as a launch platform to attack more important parts. A full discussion of botnets, rootkits and persistent malware really ought to also be a separate blog post, but let’s just say the upshot is your computer could be taking part in attacks on other computers orchestrated by criminal gangs. Serious then.

Oh my unicorns, why haven’t you developers fixed this?

First up, no piece of code is entirely perfect. The problem with large software projects is that they become almost too complicated for one person to understand every part and every change that is going on. So the project is split up with maintainers for each part and coding standards, rules, guidelines, tools you must run and that must pass, tests you must run and must pass etc. In my experience, this improves things, but no matter how much you throw at it in terms of management, you’re still dealing with humans writing programs. So mistakes happen.

However, our increasing reliance on the web has got armies of security people of the friendly, helpful sort trying to fix the problem, including people who build browsers. I went at great lengths to discuss threads and processes – there was a good reason for that, because now I can discuss what google chrome tries to do.

Ok…

So as I said, the traditional way you think about building an application when you want to do more than one thing at once is to use threads. You can store your data in memory and any thread can get it (I’m totally ignoring a whole series of problems you can encounter with this for simplicity, but if you’re a budding programmer, read up on concurrency before just trying this) reasonably easily. So when you load a page and run a download, both can happen at once and the operating system makes it look like they’re both happening at the same time.

Any bugs from 2-5, however, can affect this whole process. There’s nothing to stop a plugin waltzing over and modifying the memory of the parser, or totally replacing it, or well, anything. Also, a crash in a single thread in a program tends to bring the whole thing down, so from a stability point of view, it can also be a problem. Here’s a nice piece of ascii art:

|-----------------------------------------------------------------------|
| Process no 3412  Thread/parsing security.blogoverflow.com             |
|  Big pile of     Thread/rendering PNG                                 |
|  shared memory   Thread/running flash plugin - video from youtube     |
|-----------------------------------------------------------------------|

So, all that code is in the same space (memory address) as each other, basically. Simple enough.

The Chrome developers looked at this and decided it was a really bad idea. For starters, there’s the stability problem – plugins do crash fairly frequently, especially if you’ve ever used the 64-bit flash plugin for linux. That might take the whole process down, or it might cause something else to happen, who knows.

The Chrome model executes sub-processes for each site to be rendered (displayed) and for every plugin in use, like this:

   |--------------------------|       |-----------------------------------|
   | Master chrome            |-------| Rendering process for security....|
   | process. User interface  |       |-----------------------------------|
   | May also use threads     |
   |--------------------------|-------|-----------------------------------|
                |                     | Rendering process for somesite... |
   |--------------------------|       |-----------------------------------|
   | Flash plugin process     |
   |--------------------------|

So now what happens is the master process becomes only the user interface. Child processes actually do the work, but if they get bad input, they die and don’t bring the whole browser down.

Now, this also helps solve the security problem to some extent. Those sub-processes cannot alter each other’s memory easily at all, so there’s not much for them to exploit. In order to get anything done, they have to use IPC – interprocess communication, to talk to the master process and ask for certain things to happen. So now the master process can do some filtering based on a policy of who should be able to access what. Child processes are then denied access to things they can’t have, by the master process.

But, can’t sub processes just ask the operating system directly? Well, not on Windows for certain. When the master process creates the child process, it also asks Windows for a new security “level” if you like (token is the technical term) and creates the process in that level, which is highly restricted. So the child process will actually be denied certain things by the operating system unless it needs them, which is harder to do on a single process where parts of it do need unrestricted access.

In defence to the Windows engineers, this functionality (security tokens) has been part of the Windows API for a while; it’s just developers haven’t really used it. Also, it applies to threads too, although there remains a problem with access to the process’s memory.

Great! I can switch to Chrome and all will be well!

Hold up a second… yes, this is an excellent concept and hopefully other browser writers will start looking at integrating OS security into their browsers too. However, any solution like this will also under certain conditions be circumventable, it’s just a matter of finding a scenario. Also, chrome and any other browser cannot defend you from yourself – some basic rules still apply. They are, briefly, do not open dodgy content and do keep your system/antivirus/firewall up to date.

Right, so what’s the best browser?

I can’t answer that, nor will I attempt to try. All browser manufacturers know that “security” bugs happen and all of them release fixes, as do third parties. A discussion on which has the best architecture is not a simple one – we’ve outlined the theory here, but how it is implemented will really determine whether it works – the devil is in the detail, as they say.

So that concludes our high level discussion of the various security issues with browsers, plugins and drive by downloads. However, I would really like to stress there is a whole other area of security which you might broadly call “web application” security, which concerns in part what the browser allows to happen, but also encompasses servers and databases and many other things. We’ve covered a small corner of the various problems today; hopefully some of our resident web people will over time cover some of the interesting things that happen in building complex web sites. Here is a list of all the questions here tagged web-browser.

This week’s question, asked by @AviD, turned out to have subtle implications that even experienced security professionals may not have been aware of.

A quick bit of background:

With all the furore about passwords, most companies and individuals know they should have strong passwords in place, and many use system enforced password complexity rules (eg 8 characters, with at least 1 number and 1 special character) but how could a company actually audit password strength.

John the Ripper was a pretty good tool for this – it would brute force or use a dictionary attack on password hashes, and if it broke them quickly they were weak. If they lasted longer they were stronger (broadly speaking)

So far so good, but what if you are a security professional emulating an attacker to assess controls? You could run the brute forcer for a while, but this isn’t what an attacker will do – maths has provided much faster ways to get passwords:

Hash Tables and Rainbow Tables

Hash Tables are exactly as the name sounds – tables of hashes generated from every possible password in the space you want, for example a table of all DES crypt hashes for unsalted alphanumeric passwords 8 characters or less, along with the password. If you manage to get hold of the password hashes from the target you simply match them with the hashes in this table, and if the passwords are in the table you win – the password is there (excluding the relatively small possibility of hash collisions – which for most security purposes is irrelevant as you can still use the wrong password if its hash matches the correct one). The main problem with Hash tables is that they get very big very quickly, which means you need a lot of storage space, and an efficient table lookup over this space.

Which is where Rainbow Tables come in. @Crunge‘s answer provides excellent detail in relatively simple language to describe the combination of hashing function, reduction function and the mechanism by which chains of these can lead to an efficient way to search for passwords that are longer or more complex than those that lend themselves well to a hash table.

In fact @Crunge’s conclusion is:

Hash tables are good for common passwords, Rainbow Tables are good for tough passwords. The best approach would be to recover as many passwords as possible using hash tables and/or conventional cracking with a dictionary of the top N passwords. For those that remain, use Rainbow Tables.

@Mark Davidson points us in the direction of resources. You can either generate the rainbow tables yourself using an application like RainbowCrack or you can download them from sources like The Shmoo Group, Free Rainbow Tables project website, Ophcrackproject and many other places depending on what type of hashes you need tables for.

Now from a defence perspective, what do you need to know? 

Two things:

Longer passwords are still stronger against attack, but be aware that if they are too long then users may not be able to remember them. (Correct Horse Battery Staple!)

Salt and Pepper. @Rory McCune describes salt and pepper in this answer:

A simple and effective defence that most password hashing functions provide now is salting – the addition of a per user “salt” value stored in the database along with the hashed password. It isn’t intended to be secret but is used to slow down the brute force process and to make rainbow tables impractical to use.  Another add-on is what is called a “pepper” value. This was just another random string but was the same for all users and stored with the application code as opposed to in the database. the theory here is that in some circumstances the database may be compromised but the application code is not, and in those cases this could improve the security. It does, however, introduce problems if there are multiple applications using the same password database.

This question comes up on Stack Overflow and IT Security relatively regularly, and goes along one of these lines:

1. I have a symmetric encryption key I would like to store in my application so attackers can’t find it.
2. I have an asymmetric encryption key I would like to store in my application so attackers can’t find it.
3. I would like to store authentication details of some kind in my application so attackers can’t find them.
4. I have developed an algorithm. How do I make it so attackers can never find it.
5. I am selling commercial music. I need to make it so The Nasty Pirates can’t decode it.

Firstly, a review of what cryptography is at heart: encryption and decryption are all about sending data over untrusted networks or storing data in untrusted places such that only the intended recipient can read that information. This gives you confidentiality; cryptography as a whole also aims to provide integrity checks through signatures and message digests. Confusingly, cryptography is often confused with access control, in which it so often plays a part. Cryptography ceases to be able to protect you when you decide to put the key and the encrypted data together – at this point, the data has reached its destination, the trusted place. The expectation that cryptography can protect data once it is decrypted is similar to the expectation that a locked door will protect your house if you leave the key under the mat. The act of accessing the key and decrypting the data (and even encrypting it) is a weak link in the chain: it assumes the system you’re performing these actions on guarantees your confidentiality and integrity – it assumes that system is trusted. Reading the argument presented here, our resident cryptographer provided the following explanation:  “Encryption does not create confidentiality, it just concentrates confidentiality into the key. Presumably, it is easier to keep confidential a small key of fixed size, and the key uniform structure allows for the key confidentiality to be measured. Yet you have to start confidentiality at something. Once the key is known, confidentiality has left.”

All of the above questions are really forms of the same thing: how do I on an untrusted system safely decode some encrypted data without interception? In other words, you’re now asking for cryptography to guarantee the security of that information even after you’ve decrypted it. This isn’t possible. The problem then becomes one of how do you ensure that the system in question will maintain confidentiality and integrity for you.

The answer, then, is to create a system which acts as the recipient such that data can be decoded in it and never needs to be transferred outside of it. I’ll call it the black box. The rest of this blog post will be about looking for the black box setup.

1. We will begin with the idea we want to write a program that stores something securely whilst preventing the user from accessing that information. So the first place people want to do this is in their source code. Which is fine, except source code can be disassembled. Yes, there are ways to make this more difficult but it is not possible to prevent. The same goes for hiding or stashing files around the system, since a cursory analysis with the right tools will tell you exactly where to look. I should add that disassembly prevention probably makes your software less stable and/or portable.
2. The next option is to coerce the system into helping you hide your information. In any form, this will essentially look like a rootkit. This is dangerous: you may well have your application categorised as malware, for starters, but more importantly at this stage it is very easy to introduce extra vulnerability into the system you have just hooked. You could also crash your customer’s system, which is “not cool” whichever way you look at it. Finally, whilst rootkits are difficult to remove, taking a live listing of files and then an offline one is not. Rootkits can be found and their installation prevented. In fact, a cunning reverse engineer might replace your rootkit with their own equivalent, stealing the information you send it. Handy.
3. Stage 3 is one for the slashdot home page: the operating system vendor is complicit in helping you. Unless you are a music giant I suspect this probably is not an option for you. A complicit operating system is harder to circumvent, but entirely possible. All you need is access to ring 0 (for people not familiar with the term, ring 0 is the mode where the processor will not stop you lifting any restrictions placed upon memory or code. You can ignore read only page checks, rewrite chunks of kernel memory, whatever you want to do). Depending on the system, this might be difficult to achieve, but it certainly is not impossible. Another route to this stage is to compromise the boot process. You can pass the OS the correct validation codes if it checks anything. Etc. Clearly, this stage is difficult to pull off and requires time/effort, but it can be done.
4. So, OS security not enough? Hardware then. At this stage you actually have a real black box (or chip) somewhere on the computer. Of course, if the OS has responses sent to it, see stages 1-3. So now your black box needs to talk to all your other hardware, like your monitor or speaker system. Oh and they need to be intercept-proof too – maybe they can’t be trusted either and are really decoding the data straight back to the hard disk. This might sound far fetched, but High-bandwidth Digital Content Protection (wikipedia) aims to provide exactly this kind of protection.

The move/counter-move sequence carries on – so what is to stop a hardware engineer taking your box apart? Self-destructing hardware? Ok, how do you prevent the fact that you’ll display it on a monitor, sending light-waves and electronic signals out?

What does this mean for software on the PC?

1. You cannot store passwords, encryption keys etc in your code, or anywhere on the system. The only way around that is to allow for user input and aim to prevent interception, which could in itself be difficult.
2. License keys do not work either, for the same reason.
3. Nor does obfuscated code. It has to be decrypted to execute, at some point.

However, there is a case where hardware complicit defences are perfectly possible and may well be encouraged, where the hardware and software combination come together. There are many scenarios in which this happens, the most obvious being the mobile phone. A mobile phone is, relatively speaking, hard to take apart and put back together again whole (unless you’re a mobile phone engineer) so it is possible to have hardware-based security work reasonably effectively. Smart cards with on-board cryptographic function again ensure the keys are exceptionally hard to steal. In the case of smart cards, the boundary problem still exists in terms of transferring the decrypted data back to the untrusted system, but on a mobile phone-like device it would be entirely feasible not to route that information through the OS itself and instead play it directly on the screen, isolating those buffers from the “untrusted” sections of the OS. However, let’s leave that idea here. Trusted platforms is a blog post for another day.

Is there a case for devices capable of such secure display? Absolutely. Want to securely read data on a financial transaction, or have a trusted communication channel with your bank? Upping the bar like this definitely helps protect against malware and other interception threats. However, the most common use case appears to be “how do I defend my asset in an untrusted environment in order to enforce my desired price model?” Which brings me full circle – that is not the problem set cryptography solves.

When people say secure, they often mean “impossible for bad guy, possible for me”. That is almost never the case. Clearly, looking at the above, the number of people skilled enough to counter 3 is actually pretty small. You will, therefore, achieve part of this aim: “hard for bad guy, easyish for me”. When people say secure, they also frequently mean technical security measures. Security is more than just technical security – that’s why we have policies, community, law, education, awareness etc. Going as far as points 2/3 reduces the number of people capable of subverting your protection system to the point where legal action is feasible.

However, I think we still have a narrow definition of security in the first place. If you are talking about delivering content or software, the level of issues a customer is likely to come up against increases exponentially as you move from 1 through to point 4. Does your customer really want to be told that you do not support Windows 8 yet? Or that they cannot use their favourite media player for your content? Or that they need to upgrade their BigMediaCorpSatelliteTVBox because you altered the algorithms? Or that they’ve been locked out of that Abba Specials subscription channel they paid for because of a hardware fault, or… Security here is not just about protecting your content, it is about protecting your business. Will the cost of securing the content using any of 1-4 will make up for the potential revenue you could have made if everyone brought the copy legally? Or will the number of legitimate sales simply go down as consumers react to all those technical barriers shattering their plug-and-play expectation? I do not have any numbers on that one, but I am willing to bet the net result of adding this extra “security” is a loss.

Security needs to be appropriate to the risk of the situation presented, having been fully evaluated from all angles. The general consensus is that DRM is an excessive protection measure given the risks involved – indeed, a very simple solution is to make software/music/whatever at the right price point and value such that the vast majority of users buy it, and allow for the fact that some people will pirate/steal/make unauthorised use of it. In certain situations, providing a value add around the product can make the difference (some companies selling open source software generate their entire revenue using this model), and even persuade users of illegal copies to buy a licence in order to gain access to these services (e.g. support, upgrades).

 

Our question of the week this week was asked by Iszi, who wanted to know how exactly we should determine what to trust when employing NoScript. In the question itself Iszi raises some valid points: how does somebody know, other than by trial and error, whether scripts from a given site or third party site are trustworthy? How does a user determine which parts of the javascript are responsible for which bits of functionality? How do we do this without exposing ourselves to the risks of such scripts.

Richard Gadsden suggests one way to approach a solution is for each site to declare what Javascript it directly controls. He notes that such a mechanism could trivially be subverted if the responding page lists any and all javascript as “owned” by the site in question.

Such resource-based mechanisms have been tried and implemented before, albeit for a different problem domain (preventing XSS attacks). Cross-Origin Resource Sharing (W3C, Wikipedia) attempts to do just that by vetting incoming third party requests. However, like HTML-based lists, it does not work well when the trusted end users are “everyone”, i.e. a public web service.

Zuly Gonzalez discusses a potential solution her startup has been working on – running scripts on a disposable vm. Zuly makes some good points – even with a whitelisted domain, you cannot necessarily trust each and every script that is added to the domain; moreover, after you have made your trust decision, a simple whitelist is not enough without re-vetting the script.

Zuly’s company – if you’re interested, check out her answer – runs scripts on a disposable virtual machine rather than on your computer. Disclaimer: we haven’t tested it, but the premise sounds good.

Clearly, however, such a solution is not available to everyone. Karrax suggested that the best option might be to install plugins such as McAfee SiteAdvisor to help inform users as to what domains they should be trusting. He notes that the NoScript team are beginning to integrate such functionality into the user interface of NoScript itself. This is a feature I did not know I had, so I tried it. According to the trial page, at the time of writing the service is experimental, but all of the linked to sites provide a lot of information about the domain name and whether to trust it.

This is an area with no single solution yet, and these various solutions are in continuous development. Let’s see what the future holds.

Bogus SSL certificates are not a new problem and incidents can be found going back over a decade. With a fake certificate for Google.com in the wild, they’re in the news again today. My last two posts have touched on the SSL topic and Moxie Marlinspike’s Convergence.io software is being mentioned in these news articles. At the same time, Dan Kaminski has been pushing for DNSSEC as a replacement for the SSL CA system. Last night, Moxie and Dan had it out 140 characters at a time on Twitter over whether DNSSEC for key distribution was wise.

I’m going to do two things with the discussion: I’m going to cover the discussion I said should be had about replacing the CA system, and I’m going to try and show a risk-based assessment to justify that.

The Risks of the Current System

Risk: Any valid CA chain can create a valid certificate. Currently, there are a number of root certificate authorities. My laptop lists 175 root certificates, and these include names like Comodo, DigiNotar, GoDaddy and Verisign. Any of those 175 root certificates can sign a trusted chain for any certificate and my system will accept it, even if it has previously seen a different unexpired certificate before and even if that certificate has been issued by a different CA. A greater exposure for this risk comes from intermediate certificates. These exist where the CA has signed a certificate for another party that has flags authorizing it to sign other keys and maintain a trusted path. While Google’s real root CA is Equifax, another CA signed a valid certificate that would be accepted by users.

Risk mitigation: DNSSEC. DNSSEC limits the exposure we see above. In the DNSSEC system, *.google.com can only be valid if the signature chain is the DNS root, then the “com.” domain. If Google’s SSL key is distributed through DNSSEC, then we know that the key can only be inappropriate if the DNS root or top-level domain was compromised or is mis-behaving. Just as we trust the properties of an SSL key to secure data, we trust it to maintain a chain, and from that we can say that there is no other risk of a spoofed certificate path if the only certificate innately trusted is the one belonging to the DNS root. Courtesy of offline domain signing, this also means that host a root server does not provide the opportunity to provide malicious responses even by modifying the zone files (they would become invalid).

Risks After Moving to DNSSEC

Risk: We distrust that chain. We presume from China’s actions that China has a direct interest in being able to track all information flowing over the Internet for its users. While the Chinese government has no control over the “com.” domain, they do control the “cn.” domain. Visiting google.cn. does not mean that one aims to let the Chinese government view their data. Many countries have enough resources to perform attacks of collusion where they can alter both the name resolution under their control and the data path.

Risk mitigation: Multiple views of a single site. Convergence.io is a tool that allows one to view the encryption information as seen from different sites all over the world. Without Convergence.io, an attacker needs to compromise the DNS chain and a point between you and your intended site. With Convergence.io, the bar is raised again so that the attacker must compromise both the DNS chain and the point between your target website and the rest of the world.

Weighing the Risks

The two threats we have identified require gaining a valid certificate and compromising the information path to the server. To be a successful attack, both must occur together. In the current system, the bar for getting a validly signed yet inappropriately issued certificate is too low. The bar for performing a Man-in-the-Middle attack on the client side is also relatively low, especially over public wireless networks. DNSSEC raises the certificate bar, and Convergence raises the network bar (your attack will be detected unless you compromise the server side).

That seems ideal for the risks we identified, so what are we weighing? Well, we have to weigh the complexity of Convergence. A new layer is being added, and at the moment it requires an active participation by the user. This isn’t weighing a risk so much as weighing how much we think we can benefit. We also must remember that Convergence doesn’t raise the bar for one who can perform a MITM attack that is between the target server and the whole world.

Weighing DNSSEC, Moxie drives the following risk home for it in key distribution: “DNSSEC provides reduced trust agility. We have to trust VeriSign forever.” I argue that we must already trust the DNS system to point us to the right place, however. If we distrust that system, be it for name resolution or for key distribution, the action is still the same — the actors must be replaced, or the resolution done outside of it. The counter Moxie’s statement in implementing DNSSEC for SSL key distribution is we’ve made the problem an entirely social one. We now know that if we see a fake certificate in the wild, it is because somebody in the authorized chain has lost control of their systems or acted maliciously. We’ve reduced the exposure to only include those in the path — a failure with the “us.” domain doesn’t affect the “com.” domain.

So we’re left with the social risk that we have a bad actor who is now clearly identified. Convergence.io can help to detect that issue even if it only enjoys limited deployment. We are presently ham-strung by the fact that any of a broad number of CAs and ever-broader number of delegates can be responsible for issuing “certified lies,” and that still needs to be reduced.

Making a Decision

Convergence.io is a bolt-on. It does not require anybody else to change except the user. It does add complexity that all users may not be comfortable with, and that may limit its utility. I see no additional exposure risk (in the realm of SSL discussion, not in the realm of somebody changing the source code illicitly) from using it. To that end, Moxie has released a great tool and I see no reason to not be a proponent of the concept.

Moxie still argues that DNSSEC has a potential downside. The two-part question is thus: is reducing the number of valid certification paths for a site to one an improvement? When we remove the risk of an unwanted certification of the site, is the new risk that we can’t drop a certifier a credible increase in risk? Because we must already trust the would-be DSNSEC certifier to direct us to the correct site, the technical differences in my mind are moot.

To put into a practical example: China as a country can already issue a “valid” certificate for Google. They can control any resolution of google.cn. Whether the control the sole key channel for google.cn. or any valid key channel for google.cn., you can’t reach the “true” server and certificate / key combination without trusting China. The solution for that, whether it be the IP address or the keypair is to hold them accountable or find another channel. DNSSEC does not present an additional risk in that regard. It also removes a lot of ugliness related to parsing X.509 certificates and codes (which includes such storied history as the “*” certificate that worked on anything before browser vendors hard-coded it out). Looking at the risks presented and arguments from both sides, I think it’s time to start putting secure channel keys in the DNS stream.

This question on Security.SE made me think in a rather devious way. At first, I found it to be rather poorly worded, imprecise, and potentially not worth salvaging. After a couple of days I started to realize exactly how many times I’ve really been asked this question by well intentioned, and often, knowledgeable people. The real question should be, “Is there a recommended set of firewall rules that can be used as a standard config?” Or more plainly, I have a bunch of systems, so what rules should they all have no matter what services they provide. Now that is a question that can reasonably be answered, and what I’ve typically given is this:

-A INPUT -i lo -j ACCEPT
-A INPUT -p icmp --icmp-type any -j ACCEPT
# Force SYN checks
-A INPUT -p tcp ! --syn -m state --state NEW -j DROP
# Drop all fragments
-A INPUT -f -j DROP
# Drop XMAS packets
-A INPUT -p tcp --tcp-flags ALL ALL -j DROP
# Drop NULL packets
-A INPUT -p tcp --tcp-flags ALL NONE -j DROP
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

The first and last lines should be pretty obvious so I won’t go into too much detail. Enough services use loopback that, except in very restrictive environments, attempting to firewall them will almost definitely be more work than is useful. Similarly, the ESTABLISHED keyword allows return traffic for outgoing connections. The RELATED keyword takes care of things like FTP that use multiple ports and may trigger multiple flows that are not part of the same connection, but are none-the-less dependent on each other. In a perfect world these would be at the top for performance reasons. Since rules are processed in order we really want the fewest number of rules processed as possible, however in order to get the full benefit from the above rule set, we want to run as many packets as possible by them.

The allow all on ICMP is probably the most controversial part of this set. While there are reconnaissance concerns with ICMP, the network infrastructure is designed with the assumption that ICMP is passed. You’re better off allowing it (at least within your organization’s network space), or grokking all of the ICMP messages and determining your own balance. Look at this question for some worthwhile discussion on the matter.

Now, down to the brass tacks. Let’s look at each of the wonky rules in order.

Forcing SYN Checks

-A INPUT -p tcp ! --syn -m state --state NEW -j DROP

This rule performs two checks:

1. Is the SYN bit NOT set, and
2. Is this packet NOT part of a connection in the state table

If both conditions match, then the packet gets dropped. This is a bit of a low-hanging fruit kind of rule. If these conditions match, then we’re looking at a packet that we just downright shouldn’t be interested in. This could indicate a situation where the connection has been pruned from the conntrack state table, or perhaps a malicious replay event. Either way, there isn’t any typical benefit to allowing this traffic, so let’s explicitly block it.

Fragments Be Damned

-A INPUT -f -j DROP

This is an easy one. Drop any packet with the fragment bit set. I fully realize this sounds pretty severe. Networks were designed with the notion of fragmentation, in fact the the IPv4 header specifically contains a flag that indicates whether or not that packet should or should not be fragmented. Considering that this is a core feature of IPv4, fragmentation is still a bit of a touchy subject. If the DF bit is set, then MTU path discovery should just work. However, not all devices respond back with the correct ICMP message. The above rule is one of them, however that’s because ICMP type 3 code 4 (wikipedia) isn’t a reject option in iptables. As a result of this one can’t really know whether or not your packets will get fragmented along the network. Nowadays, on internal networks at least, this usually isn’t a problem. You may run into problems, however, when dealing with VPNs and similar where your 1500 byte ethernet segment suddenly needs to make space for an extra header.

So now that we’ve talked about all the reasons to not drop fragments, here’s the reason to. By default, standard iptables rules are only applied to the packet marked as the first fragment. Meaning, any packet marked as a fragment with an offset of 2 or greater is passed through, the assumption being that if we receive an invalid packet then reassembly will fail and the packets will get dropped anyway. In my experience, fragmentation is a small enough problem that I don’t want to deal with risk and block it anyway. This should only get better with IPv6 as path MTU discovery is placed more firmly on the client and is considered less “Magic.”

Christmas in July

-A INPUT -p tcp --tcp-flags ALL ALL -j DROP

Network reconnaissance is a big deal. It allows us to get a good feel for what’s out there so that when doing our work we have some indications of what might exist instead of just blindly stabbing in the dark. So called ‘Christmas Tree Packets’ are one of those reconnaissance techniques used by most network scanners. The idea is that for whatever protocol we use, whether TCP/UDP/ICMP/etc, every flag is set and every option is enabled. The notion is that just like a old style indicator board the packet is “lit up like a Christmas tree”. By using a packet like this we can look at the behavior of the responses and make some guesses about what operating system and version the remote host is running. For a White Hat, we can use that information to build out a distribution graph of what types of systems we have, what versions they’re running, where we might want to focus our protections, or who we might need to visit for an upgrade/remediation. For a Black Hat, this information can be used to find areas of the network to focus their attacks on or particularly vulnerable looking systems that they can attempt to exploit. By design some flags are incompatible with each-other and as a result any Christmas Tree Packet is at best a protocol anomaly, and at worst a precursor to malicious activity. In either case, there is normally no compelling reason to accept such packets, so we should drop them just to be safe.

I have read instances of Christmas Tree Packets resulting in Denial of Service situations, particularly with networking gear. The idea being that since so many flags and options are set, the processing complexity, and thus time, is increased. Flood a network with these and watch the router stop processing normal packets. In truth, I do not have experience with this failure scenario.

Nothing to See Here

-A INPUT -p tcp --tcp-flags ALL NONE -j DROP

When we see a packet where none of the flags or options are set we use the term Null Packet. Just like with the above Christmas Tree Packet, one should not see this on a normal, well behaved network. Also, just as above, they can be used for reconnaissance purposes to try and determine the OS of the remote host.

A key aspect of IT and Information Security is the acceptance piece. People aren’t naturally good at this kind of security and generally see it as an annoyance and something they would rather not have to deal with.

Because of this, it is typical in organisations to send out regular security update emails – to help remind users of risks, threats, activities etc.

However it is often the case that these are deleted without even being read. This week’s question: How to write an email regarding IT Security that will be read, and not ignored by the end user? was asked by @makerofthings and generated quite a number of interesting responses, including:

• Provide a one line summary first, followed by a longer explanation for those who need it
• Provide a series of options to answer – those who fail to answer at all can be chased
• Tie in reading and responding to disciplinary procedures – a little bit confrontational here, but can work for items such as mandatory annual updates (I know various organisations do this for Money Laundering awareness etc)
• Using colours – either for the severity of the notice, or to indicate required action by the user
• Vary the communications method – email, corporate website, meeting invite etc
• Don’t send daily update messages – only send important, actionable notices
• Choose whose mailbox should send these messages – if it is critical everyone reads it, perhaps it should come from the FD or CEO, or the individual’s line manager
• Be personal, where relevant – users who receive a few hundred emails a day will happily filter or delete boring ones, or those too full of corporate-speak. Impress upon users how it is relevant to them
• Add “Action Required” in the subject

It is generally agreed that if it isn’t interesting it will be deleted. If there are too many ‘security’ emails, they’ll get deleted. In general, unless you can grab the audience’s attention, the message will fail.

Having said that, another take on it is – do you need to send all these security emails to your users? For example, should they be getting antivirus updates or patch downtime info in an email every week?

Can users do anything about antivirus, or should that be entirely the responsibility of IT or Ops? And wouldn’t  a fixed patch downtime each week that is listed on the internal website make less of an impact on users?

Thinking around the common weak points in security – such as users – for most actions can make much more impact when you actually do need the users to carry out an action.

Associated questions and answers you should probably read:

What are good ways to educate about IT Security in a company?

What policies maximize employee buy-in to security?

Internet connectivity issues kept me from being timely in updating, and a need for sleep upon my return led me to soak up all of the rest of BSides and DEFCON. That means just a few talks are going to be brought up.

First, there’s Moxie Marlinspike‘s talk about SSL. In my last post, I had mentioned that I thought SSL has reached the point where it is due to be replaced. In the time between writing that and seeing his talk, I talked with a few other security folk. We all agreed that DNSSEC made for a better distribution model than the current SSL system, and wondered before seeing Moxie’s presentation why he would add so much more complexity beyond that. The guy next to me (I’ll skip name drops, but he’s got a security.stackexchange.com shirt now and was all over the news in the last year) and I talked before his presentation, and at the end agreed that Moxie’s point about trusting the DNS registry and operators to not change keys could be a mistake.

Thus, we’re still in the world of certificates and complicated x.509 parsing that has a lot of loopholes, and we’ve moved added something that the user needs to be aware of. However, we have one solid bonus: many independent and distributed sources must now collaborate to verify a secure connection. If one of them squawks, you at least have an opportunity to be aware. An equally large entry exists in the negative column: it is likely that many security professionals themselves won’t enjoy the added complexity. There’s still a lot of research work to be done, however the discussion is needed now.

PCI came up in discussion a little bit last year, and a lot more this year. In relation, the upcoming Penetration Testing Execution Standard was discussed. Charlie Vedaa gave a talk at BSides titled “Fuck the Penetration Testing Execution Standard”. It was a frank and open talk with a quick vote at the end: the room as a whole felt that despite the downsides we see structures like PCI and the PTES, we were better off with them than without.

The line for DEFCON badges took most people hours and the conference was out of the hard badges in the first day. Organizers say it wasn’t an issue of under-ordering, but rather that they had exhausted the entire commercial supply of “commercially pure” titanium to make the badges. Then the madness started…

AT&T’s network had its back broken under the strain of DEFCON resulting in tethering being useless and text messages showing up in batches sometimes more than 30 minutes late. The Rio lost its ability to check people into their hotel rooms or process credit cards. Some power issues affected the neighboring hotel at a minimum — Gold Coast had a respectable chunk of casino floor and restaurant space in the dark last evening. The audio system for the Rio’s conference area was apparently taken control of and the technicians locked out of their own system. Rumors of MITM cellular attacks at the conference, and now days later in the press abound. Given talks last year including a demonstration and talks this week at the Chaos Computer Camp, the rumors are credible. We’ll wait to see evidence, though.

DEFCON this year likely had more than 15,000 attendees, and they hit the hotel with an unexpected force. Restaurants were running out of food. Talks were sometimes packed beyond capacity. The Penn and Teller theater was completely filled for at least three talks I had interest in, locking me outside for one of them. The DEFCON WiFi network (the “most hostile in the world”) suffered some odd connectivity issues and a slow-or-dead DHCP server.

Besides a few articles in the press that have provided interesting public opinion, one enterprising person asked a few random non-attendees at the hotel what they thought of the event. The results are… enlightening.

When we talk about security of systems, a lot of focus and attention goes on the external attacker. In fact, we often think of “attacker” as some form of abstract concept, usually “some guy just out causing mischief”. However, the real goal of security is to keep a system running and internal, trusted users, can upset that goal more easily and more completely than any external attacker, especially a systems administrator. These are, after all, the men and women you ask to set up your system and defend it.

So us Security SE members took particular interest when user Toby asked: when the systems admin leaves, what extra precautions should be taken? Toby wanted to know what the best practise was, especially where a sysadmin was entrusted with critical credentials.

A very good question and it did not take long for a reference to Terry Childs to appear, the San Francisco FiberWAN administrator who in response to a disciplinary action, refused to divulge critical passwords to routing equipment. Clearly, no company wants to find themselves locked out of their own network when a sysadmin leaves, whether the split was amicable or not. So what can we do?

Where possible, ensure “super user” privileges can be revoked when the user account is shut down.

From Per Wiklander. The most popular answer by votes so far also makes a lot of sense. The answer itself references the typical Ubuntu setup, where the root password is deliberately unknown to even the installer and users are part of a group of administrators, such that if their account is removed, so is their administrator access. Using this level of granular control you can also downgrade access temporarily, e.g. to allow the systems admin to do non-administrative work prior to leaving.

However, other users soon pointed out a potential flaw in this kind of technical solution – there is no way to prevent someone with administrative access from changing this setup, or adding additional back doors, especially if they set up the box. A further raised point was the need not just to have a corporate policy on installation and maintenance, but also ensure it is followed and regularly checked.

Credential control and review matters – make sure you shut down all the non-centralized accounts too.

Mark Davidson‘s answer focused heavily on the need for properly revoking credentials after an administrator leaves. Specifically, he suggests:

• Do revoke all credentials belonging to that user.
• Change any root password/Administrator user password of which they are aware.
• Access to other services on behalf of the business, like hosting details, should also be altered.
• Ensure any remote access is properly shut down – including incoming IP addresses.
• Logging and monitoring access attempts should happen to ensure that any possible intrusions are caught.

Security SE moderator AviD was particularly pleased with the last suggestion and added that it is not just administrator accounts that need to be monitored – good practice suggests monitoring all accounts.

It is not just about passwords – do not forget the physical/social dimension.

this.josh raises a critical point. So far, all suggestions have been technical, but there are many other issues that might present a window of opportunity to a sysadmin with a mind to cause chaos. From the answer itself:

• Physical access is important too. Don’t just collect badges; ensure that any access to cabinets, server rooms, drawers etc are properly revoked. The company should know who has keys to what and why so that this is easy to check when the sysadmin leaves.
• Ensure the phone/voicemail system is included in the above checks.
• If the sysadmin could buy on behalf of the company, ensure that by the time they have left, they no longer have authority to do this. Similarly, ensure any contracts the employee has signed have been reviewed – ideally, just like logging sysadmin activities you are also checking this as they’re signed too.
• Check critical systems, including update status, to ensure they are still running and haven’t been tampered with.
• My personal favorite: let people know. As this.josh points out, it is very easy as a former employee knowing all the systems to persuade someone to give you access, or give you information that may give you access. Make sure everyone understands that the individual has left and that any suspicious requests are highlighted or queried and not just granted.

Conclusions

So far, that’s it for answers to the question and no answer has yet to be accepted, so if you think anything is missing, please feel free to sign up and present your own answer.

In summary, there appears to be a set of common issues being raised in the discussion. Firstly: technical measures help, but cannot account for every circumstance. Good corporate policy and planning is needed. A company should know what systems a systems administrator has access to, be they virtual or physical, as well as being able to audit access to these systems both during the employee’s tenure and after they have left. Purchasing mechanisms and other critical business functions such as contracts are just other systems and need exactly the same level of oversight.

I did leave one observation out from one of the three answers above: that it depends on whether the moving on employee is leaving amicably or not. That is certainly a factor as to whether there is likely to be a problem, but a proper process is needed anyway, because the risk to your systems and your business is huge. The difference in this scenario is around closing critical access in a timely manner. In practice this includes:

• Escorting the employee off the premises
• Taking their belongings to them once they are off premises
• Removing all logical access rights before they are allowed to leave

These steps reduce the risk that the employee could trigger malicious code or steal confidential information.

This week’s post came from this question: I just discovered major security flaws in my web store!, but covers a wider scope and includes some resources on Security Stack Exchange on defending your website.

Any application you connect to the Internet will be attacked within minutes of plugging it in, so you need to look very carefully at protecting it appropriately. Guess what – many application owners and developers go about this in entirely the wrong way, building the application and then thinking about security at the very end, where it is expensive and difficult to do well.

The recommended approach is to build security in from the start – have a look at this post for a discussion on Secure Development Lifecycle. This means having security as a core pre-requisite, just like performance and functionality, and understanding the risks your new application will bring. Read How do you compare risks from your websites, physical perimeter, staff etc

To do this you will need developers to write the application securely. Studies show that this is the most cost effective in the long run, as securely developed applications require very little remediation work (as they don’t suffer from anywhere near as many security flaws as traditionally developed applications)

So, educating developers is key – even basic awareness can make the difference: What security resources should a white-hat developer follow these days?

Assuming your application has been developed using secure practices and you are ready to go live, What are the most important security checks for new web applications? and Testing your web application to gain some confidence that the security controls you have implemented are correctly working and effective is an essential step before allowing connection to the Internet. Penetration testing and associated security assessments are essential checks.

What tools are available to assess the security of a web application?

The globally recognised open standard on web application security is the Open Web Application Security Project, (OWASP) who provide guidance on testing, sample vulnerable applications, and a list of the top ten vulnerabilities. The answers to Is there a typical step-by-step A-Z process for testing a Web site for possible exploits? provide more information on approaches, as do the answers to Books about Penetration Testing.

Once your application is live, you can’t just sit back and relax, as new attacks are developed all the time – being aware of what is being attempted is important.  Can I detect web app attacks by viewing my Apache log file is Apache specific, but the answers are broadly relevant for all web servers.

This is obviously a very quick run-through – but reading the linked questions, and following the related questions links on them will give you a lot of background on this topic.