Create "layered" Docker build from thin launcher

[ascheman]

Being inspired from the discussion in spring-projects/spring-boot#1813 I would like to see the following based on thin launcher: Create a Docker build with the following properties

• Add all dependencies as Docker layers (by separate Docker ADD commands)
• Add the application jar as the latest Docker layer

As long as only the application changes this should result in

• a very fast Docker build since all other layers would be already cached
• an improved shipment of the Docker image (ship dependency layers once, then only ship final application layer on changes)

The idea was also discussed here: https://github.com/JCrete/jcrete2017/tree/master/Day3/Session2/Java-Docker-Alignment and a PR is prepared and on it's way.

[dsyer]

Kind of a nice idea, but the PR #26 has a lot of code for such a simple thing it seems to me (and there are ominous comments about needing more). Would it be better to make this a Maven/Gradle feature rather than a jar launcher feature? That way the name of the jar will be easy to find, and there will already be a build directory.

My other concern is that a Dockerfile can have a lot of features, and we only can provide defaults for a handful of them. It would be better to think of the role of the thin launcher as providing a default, or a starting point for a Dockerfile, that we then expect users to edit and personalize.

[ascheman]

My PR was only meant as a proof-of-concept. In particular I do not like the current implementation by providing a hard coded Docker file (with source image etc.). Probably there is a simpler solution or streamlining of the code if it were integrated better with the rest of the code?

The nice thing about the thin launcher here is that

1. it provides a boot loader which adds a very small overhead to the original jar,
2. it already computes all dependencies,
3. it computes/provides the dependencies in M2 repository layout under the thin.root; from my point of view this is convenient
   • since one easily sees the full GAV coordinates of the used dependencies,
   • you can easily change versions, eg. SNAPSHOT artifacts,
4. it computes a classpath with all the dependencies with respect to thin.root.

On the other hand, if the full set of dependencies is known and a classpath is provided it would probably be possible to drop the wrapped original jar at all and use the original jar instead?

Additionally I would like to see a better mechanism to create the Dockerfile. Hand crafting or maintaining the file manually is not an option for me since the Dockerfile mainly contains the large number of dependencies and the computed classpath which would be a major advantage of the solution. A template would be nice since it would allow to provide source, maintainer, ports etc. with some application background knowledge. If the template engine could iterate over the dependencies it could create suitable ADD statements and a CLASSPATH environment (or JAVA call parameter) in the Dockerfile.

[dsyer]

There is no reason to use a computed classpath in a Dockerfile because other tools are available, principally the standard Spring Boot jar layout is already nicely segregated into BOOT-INF/lib and BOOT-INF/classes. The getting started guide uses this approach (https://spring.io/guides/gs/spring-boot-docker/). Jib would make the same calculation, and could be used with a thin jar I think.

The only reason I haven’t closed this issue is because of the “change the classpath” options that thin launcher provides. I haven’t really heard much demand for it in containers though, even though I use it all the time in benchmarks. If no-one has anything else to say, I will close this issue and leave the discussion for posterity.

[plamentotev]

I think the profile feature (changing the runtime dependencies) plays nice with Docker tags. Consider an application that may work with either Kafka or RabbitMQ. Then for each version of the application we can have two tags - one for Kafka and one for RabbitMQ. This would help keep the Docker images minimal - the one for Kafka would not include the RabbitMQ dependencies and vice versa. Here is an example of Docker layers:

• Base layer that contains the common dependencies
• Kafka layers containing the Kafka related dependencies build on top of the base layer and tagged as appVersion-kafka
• RabbitMQ layers containing the RabbitMQ related dependencies build on top of the base layer and tagged as appVersion-rabbitmq

I'm not sure if currently there is an easy way to achieve this. You basically need to separate the dependencies into different groups. One for the common dependencies and one for each profile. Spring Boot now provides layering feature to help with building optimized Docker images, but don't think it understands the thin launcher profiles. Still, it looks like the way to go forward. Or not?

I don't know how much demand is there for this feature, but I can give a real world example with Spring Cloud Contract Stub Runner Docker image. Spring Cloud Contract Stub Runner Boot application uses thin launcher and has three profiles - default (base) one and two additional profiles if you want to use messaging stubs - one for Kafka and one for RabbitMQ. Currently there is a single Docker image contains all dependencies no matter if you need the messaging stubs or if you are going to use Kafka or RabbitMQ. A better approach would be to have three images instead of one - 3.0.1, 3.0.1-kafka, 3.0.1-rabbitmq, each containing only the dependencies for the respective profile. This is considered a good practice from both performance (smaller image to download) and security point of view.

I would imagine there are other projects like Spring Cloud Contract that publish single image with all dependencies just because there is no easy way to separate them into profiles and layers. Many of them probably even still don't use the thin launcher and its profiles (I myself learned about the thin launcher from the Spring Cloud Contract sources) which is a shame.

As a side note Gradle dependencies configurations are probably an easy way to group dependencies into profiles, but Maven is missing similar functionality (unless I'm missing something).

[dsyer]

I think you might have a point. There's nothing stopping you from creating multiple images with different labels (via the thin launcher or other build time tools). But it's not that easy to set up and we could make it more of a feature. The Spring Boot team have expressed some interest in extending the layering options and making them more aware of the dependency graph. I think that would also be a good place to add value here. Cc @wilkinsona and @philwebb.

Maven is missing similar functionality

You could use Maven profiles to build apps (hence images) with different dependencies.