[Docker] Include Dockerfile in another Dockerfile

Docker does not support including a Dockerfile in another Dockerfile, luckily the Dockerfile plus project comes to the rescue.

Simply setup your Dockerfile with:

# syntax = edrevo/dockerfile-plus:0.1.0

INCLUDE+ someDockerfile

FROM someimage

You might need to enable Docker buildkit in order to use this extension, simply set the environment variable:

DOCKER_BUILDKIT=1

[Docker] Redirect app log file to container output

When running applications in containers, we might have log files that are relevant to monitor but are not automatically printed with the container output.

A workaround for this is to force the redirect in our Dockerfile by adding a symbolic link:

RUN ln -sf /dev/stdout /path/to/logfile.log 

Can also be applied to /dev/stderr of course

[Docker] Set Java UTF-8 encoding

Sometimes a Java based app running in a container requires UTF8 encoding. Not all available images enable that by default, this can be fixed by adding one line to your Dockerfile setting the JAVA_TOOL_OPTIONS environment variable:

ENV JAVA_TOOL_OPTIONS -Dfile.encoding=UTF8

This also works outside containers of course.

[Docker] Multi stage builds

I've been recently introduced to a nice Docker feature: multi-stage builds.

The idea is simple, if the build is containerized as well, the build itself is a developer responsability as well and the operations team need only provide a build server with docker installed on all worker nodes.

To achieve the result, we use a simple Dockerfile where we specify multiple FROM statements and tag each layer as necessary. The last layer will be the one responsible to run the application, while the previous layers are only used for the build. A sample file for a SpringBoot app looks like this:

 # syntax=docker/dockerfile:1  
 # build layer  
 FROM adoptopenjdk/openjdk11:latest as build  
   
 WORKDIR /app  
   
 # copy project files into container workdir  
 COPY . .  
   
 # build jar, skip tests, avoid daemon  
 RUN ./gradlew build -x test --no-daemon  
   
 # run layer  
 FROM adoptopenjdk/openjdk11:latest as prod  
   
 WORKDIR /app  
   
 #copy fat jar from previous layer into current workdir and rename it  
 COPY --from=build /app/build/libs/*.jar ./myApp.jar  
   
 # not mandatory, must use -p 8080:8080 later anyway  
 EXPOSE 8080  
   
 # start the spring boot app  
 CMD ["java", "-jar", "myApp.jar"]  
   

Then it can be placed in the project directory and we can trigger the build with:

docker build -t TAG .

Finally run it with (binding for example port 8080 and executing it in background):

docker run -p 8080:8080 -d TAG

We can also see the container output with (find container name with docker ps first):

docker logs -f CONTAINER_NAME

[Java] Kerberos login and retrieve GSSCredentials from KerberosTicket

A scenario that popped up recently was to login a user via Java code to Kerberos and retrieve a GSSCredential object containing the Kerberos ticket. I used Java 8, but this works since Java 7 onwards.

Java offers a Krb5LoginModule class which can be used in conjuction with a LoginContext to achieve this.

The flow is quite simple (once you have read all the Kerberos documentation):

• on the machine where the code runs, place a correct krb5.conf file in the default location for your OS (Windows uses krb5.ini) OR set the java.security.krb5.conf system property pointing to the file
• define a PasswordCallback handler class
  
• create a LoginContext with a configuration using Krb5LoginModule and provide the password callback handler. The configuration must force the login to request a user input, which will then be routed to the callback handler. It is possible to use a keytab or cache credentials, but it's not shown here
  
• login the user and get its KerberosTicket
• create a GSSCredentials object using the ticket

This procedure allows handling multiple login mechanisms in the application and even multiple Kerberos realms.

[Docker] Move docker directory to different location on NTFS filesystem

The default location of docker files (images, volumes, networks, etc) on most distributions is /var/lib/docker

This can be easily changed with the daemon.json configuration file.

First stop the docker service and make sure it is completely stopped.

You can now edit or create the configuration file under /etc/docker folder. In there, you can specify a new location for the docker files with the graph attribute:

{
   "graph": "/path/to/new/docker_folder"
}


You can then copy ALL the content to the new location or let docker recreate everything from scratch there.

WARNING: if the new location is on a different filesystem you might need to change the storage driver AND existing data might become inaccessible (it will remain on the filesystem though)

To set a new storage driver, set the storage-driver attribute as well (example for NTFS):

{
   "graph": "/path/to/new/docker_folder",   "storage-driver": "vfs"
}

[Java] Testcontainers copy file to container

Once you have your container up and running, you might want to upload some files to it. The outcome of this action is largely dependent on the container type and configuration, but in general you want to use the copyFileToContainer API paying attention to a couple things:

1. If the target location in the container does not exist, it will not be automatically created for you, you can work around this by issuing a command to do so:

container.execInContainer("mkdir", "-p", "TARGET_LOCATION");

2. The uploaded files will likely be owned by root, but you won't be running apps in it as root, so you want to set good enough permissions (at least 775) to allow the actual application user to access them:

container.copyFileToContainer(MountableFile.forHostPath(FILE_ON_HOST, 0777, "TARGET_LOCATION");

where FILE_ON_HOST is a File object representing the resource you are trying to copy to the container.

[Java] Testcontainers set container name

When you create a new container with testcontainers, you might want to assign it a name so that it's easier to spot when you list all of them from docker. This is as simple as:

 GenericContainer container = new GenericContainer<>("IMAGE_NAME")  
                   .withCreateContainerCmdModifier(cmd -> cmd.withName("YOUR_NAME"));  

[Java] Testcontainers start container with fixed port binding

If you're working in Java, you will likely be using docker in your testing process and quite likely you'll be using docker-compose or the amazing testcontainers project so configure and start a bunch of necessary dependencies.

While in a CI/CD environment for obvious reasons it is NOT recommended to fix a port binding for your containers, there might be instances (Kafka anyone?) where it might be preferable to grab a random port BEFORE the container is started AND force the binding to that port. This allows for easier configuration setup while your test suite boots up.

Once you have your random port, you need to configure your FixedHostPortGenericContainer to bind to it:

 int port = findFreePort();  
   
 GenericContainer container = new FixedhostPortGenericContainer<>("IMAGE_NAME")  
                   .withNetworkMode("host")  
                   .withFixedExposedPort(port, port);  
   
 container.setPortBindings(Collections.singletonList(port + ":" + port));  

which will configure the container to run in host network mode and will fix the host and container port to expose and bind to. In the Kafka example you could then set the KAFKA_ADVERTISED_LISTENERS environment variable directly in the container specification, since your port is now fixed.