Published by O’Reilly, Liz Rice’s Container Security book provides a security checklist covering the need-to-know when deciding how to protect deployments running on containers.

Liz gave us an outline of the checklist in her GOTOpia Europe 2020 presentation and took a deep dive into the specifics of certain likely vulnerabilities that you need to prevent. Her talk is divided into six parts answering the following six questions:

Are your builds running separately from your production cluster?

Is all executable code added to a container image at build time?

Are you avoiding –privileged?

Are you keeping hosts up to date with the latest security releases?

Are your secrets encrypted at rest and in transit?

Can you prevent container drift?

In this post, we take a quick look at the main takeaways Liz had to share on each question.

Build machines

Are your builds running separately from your production cluster?

At the beginning of your container lifecycle, you’re going to be building container images using some kind of CI/CD pipeline — you might be using a hosting service, or, more commonly in an enterprise scenario, we see self-managed CI/CD solutions.

Although it’s perfectly possible to run your builds in the same machines where your application code is running. 

Your build processes are sharing the same kernel as you application code if they’re running on the same host — unless you’re running them on separate virtual machines, they’re sharing the same kernel, and although kernel escapes are rare, it’s pretty scary to think about the consequences of builds and machines getting compromised and then being able to access your applications.

When you run a docker file you get to run any arbitrary code you like — docker files can contain any instructions,  so anyone that can make code changes to your docker files can instruct your docker files to run arbitrary code.

Hopefully only your team and trusted members can edit your docker files, but you must remember that you containers can run arbitrary code

Build container images

Is all executable code added to a container image at build time?

The second part focuses on building container images. During this step you are taking your docker file and then Build Step converts that into an image that you then store in a registry. At some point you will deploy that and run it.

Treat containers as immutable (you should be scanning your containers).

Take your application container that you want to scan for vulnerabilities — there are plenty of scanners out there. You can run your scanner as part of a build step in much the same way as you would run integration tests within your build pipeline. Assuming your scan comes back without any vulnerabilities you will eventually deploy that application onto the cloud

The point about treating containers as immutable is to make sure you add all the code during that build step to make sure the vulnerability scanner gets to scan everything that’s going to be a part of that application.

Sometimes people don’t treat containers as immutable and add new code to that container and application that hasn’t been scanned, adding in potential vulnerabilities.

Deploy container config

Are you avoiding –privileged?

This is about how you configure your container to run. One of the things you can configure is the –privileged flag.

We tend to think that root has all the permissions and unprivileged users don’t have all the permissions. While that is kind of true, the root permissions are broken down into a much more granular set of permissions called linux capabilities.

When you start a container you get a set of these capabilities which you can change. A lot of those capabilities are unnecessary for most applications.

For example, most containers don’t need to:

• Install kernel modules 
• Change the system time
• Trace / modify arbitrary processes 

People like to think that –privileged means ‘I’m running as root’ because they believe that you’re not root if you’re not running with –privileged — that’s not true — you’re root by default. You don’t need –privileged in order to be root. 

Host machines

Are you keeping hosts up to date with the latest security releases?

You need to check that your host’s software is up to date. For example, Kubernetes releases three to four times a year and has support for the last four versions, so you want to make sure you’re keeping up with releases.

There are tools that help you check for this and alert you if your hosts are running out-of-date software.

If you’re on a public cloud and you’re running a managed service., it’s going to take care of the updates for you. That’s one of the advantages of public clouds and managed services.

Secrets

Are your secrets encrypted at rest and in transit?

With regards to secrets, Liz refers to the credentials that your application code needs in order to run effectively: you might have app code that needs database credentials, or you might need tokens to communicate with some other service within your deployment.

If we’re talking about containers, we’ll often mean we want secrets available to only one kind of container running a particular microservice or application.

You want them to be encrypted and inaccessible to people and applications both in storage and as they are moved from wherever they’re held into your container.

Unless you have taken steps to encrypt your secrets, by default, they’re unencrypted, even when using Kubernetes with native implementation of secrets.

Runtime and network security

Can you prevent container drift?

Container drift is the idea that your running container should only contain executable code that was in the container image when you built it.

There are lots of reasons why seeing executable code appear in your container at runtime is a bad idea, the main one being that anything that wasn’t in your container image when it was built couldn’t be scanned and hasn’t been through any testing that you might do on your container images.

Whatever scanning or verification you might do on your container image, if you subsequently have different code in your runtime container, it hasn’t been through that testing and that change from what’s in your image to what’s running is what we call container drift — you want to avoid container drift to ensure your not building in vulnerabilities.