Investigate layered docker support

[philwebb]

Somewhat related to #1813 and also raised in https://github.com/dsyer/spring-boot-thin-launcher/issues/25. It would be nice if it were easy to create layered JAR files to reduce the amount of space needed.

[psoares]

It's definitely an interesting idea. However, most of my springboot apps require different dependencies (mostly different spring-boot-starters). How would we create a reusable base image without cramming all the spring boot starters and their dependencies in it? There are so many possible combinations.

As far as I'm aware, we can only derive from a single base layer with the FROM instruction of a Dockerfile.

[wilkinsona]

The idea is that the base image would be specific to a particular application. It would contain the things that rarely change (typically the application's third-party dependencies). The application's own code (that changes more frequently) would then be layered on top.

Right now, I think the most likely outcome of this issue is some build tooling that helps to create that separation.

[gclayburg]

Hey everyone, I ran into this issue as well a while back and created a dockerprepare gradle plugin that does this.

plugins {
  id "com.garyclayburg.dockerprepare" version "1.3.2"
}

When you just run the plugin like this with no configuration, it splits the spring boot app into 2 docker layers as suggested by @wilkinsona .

It also can create a 3rd layer for cases where you know you will be creating many microservices applications that share similar dependencies. A typical configuration might look like this in build.gradle:

dockerprepare {
    commonService = ['org.springframework.boot:spring-boot-starter-web']
}

So in this case the docker image will have:

1. common service layer, composed of spring-boot-starter-web and all its transitive dependencies
2. application dependencies, composed of all other dependencies listed in build.gradle
3. application code built by gradle

What do you think?

[philwebb]

@gclayburg Thanks for the link. This looks very similar to what we had in mind except we were thinking we might be able to change our fat jar support to directly support a layered setup.

[gclayburg]

Right, that makes sense @philwebb . I originally started working on that project to learn more about the gradle dsl and just ended up making an entire plugin. As I got farther along I was thinking this stuff probably should be in the spring boot plugin itself. I used groovy to create it, but if I were to do it over I'd probably just stick with Java and its strong typing. There are times where groovy just feels messier than it should be.

[stefanocke]

Google's jib might also be of interest.
Currently it creates layers for:

• dependencies
• snapshot dependencies
• application resources
• application classes

Even with just one Spring Boot app, that saves a good amount of megabytes to up- and download from the registry, since dependencies don't change that often.
If they implement this feature it might become even more flexible and one can split the dependency layer further to allow layer reuse across multiple Spring Boot apps / microservices (like the common-service-layer mentioned above or maybe something like a spring boot + spring cloud layer).

[dsyer]

Please can everyone just remember that spring boot fat jars are already (since 1.3?) optimized for layering - BOOT-INF/lib and BOOT-INF/classes are the only sensible choices and the same in practice to what jib would choose. The getting started guide even does it that way: https://spring.io/guides/gs/spring-boot-docker/. It’s hard to see how we could improve on that.

[mdiskin]

Hi Dave @dsyer , I agree work has been done to optimize the run-time but this is also a play to optimize the build, artifact push and deploy where similar containers (as in our case use the same base jar rails).
I like the idea of leveraging jib now or when it's feature complete enough for spring boot layers.
I'd also like to see memory and performance nods to graals and AOT to be considered in the future since containers are already opinionated on target OS might as well get native platform benefits offered.

[dsyer]

@mdiskin your first sentence looks like maybe you didn't finish it, so I'm not sure what you were saying. Can you expand a bit on what the actual problem is you are trying to solve?

It sounds like Jib might be what you need. What did you mean about Jib not being feature complete?

I don't see what AOT has to do with this issue. If you have an app that can be AOTed, you can containerize it, but there are no layering strategies that will help with that.

[mdiskin]

@dsyer the main benefits I see is the ability to reuse docker image layers across multiple containers. So in our micro services work we use the same set of jars over and over again that adds 30-50mb. If this can be created as a shared layer the actual final image creates just the small final layer of the unique assets for each. This helps speed up our CI builds, lessens the push time/network and storage on the nexus artifact and at deploy time speeds up the container pull to the k8s nodes and startup times (cached layers).

Yes the AOT is separate but something similar that either jib or spring boot fat jars could offer to reduce down that plumbing donkey work. Did you want me to add an issue for this?

To jib feature complete the post earlier referenced a open feature GoogleContainerTools/jib#403 and other 0.10 ones such as this GoogleContainerTools/jib#431

[dsyer]

Wouldn't the 30-50mb all go in BOOT-INF/lib? So they would be pretty efficiently cached most of the time anyway. I can see that if you had more control over the layers the BOOT-INF/lib went to (i.e. more than one layer) you could potentially segregate them by transitive dependency sets. The Thin Launcher is probably a better place to discuss that. I'm pretty sure it would get quite complicated, quite quickly, and simple might be better, but we can analyse it a bit more if you want.

I'm not sure if opening a single issue about AOT is going to help, or what the fat jar format would have to do with it. If you can explain, maybe I could comment.

[mdiskin]

Agreed. Maybe what this is turning into is a hybrid of your thin launcher along with the tooling work from jib side.

The AOT can be seen abstractly like another layering/separation approach in this case using native shared libraries over fat jars. Again the benefit would be to seek re-use and optimization of resources.

[dsyer]

A fat jar is more like a native image than a shared library. The idea of extracting some features of a library jar out into a shared library is definitely interesting, but way off topic for this issue.

[manderson23]

It would be nice if the Spring Boot Maven Plugin would natively support creating the layout detailed in https://docs.spring.io/spring-boot/docs/2.2.1.RELEASE/reference/html/deployment.html#containers-deployment ready for use in a Dockerfile.

[dsyer]

You mean, in case the user doesn't know how to run jar? If so, there's always this (which I see was merged): #18932. Maybe there is no need for any further action?

[manderson23]

Obviously most users know how to run jar. I was thinking of convenience out of the box. Probably along the lines of #16197 (comment)

It isn't a big deal but presumably since both these issues are still open changes in this area are still under consideration.

[philwebb]

This issue has now been superseded by #19697 and #19698. We'll add more notes to those in due course.

[sansnom]

My 2 cents: at the moment, I use the maven assembly plugin to create two layers. Using the default directories (BOOT-INF/lib, BOOT-INF/classes) generated by the spring boot maven plugin is unsuitable when you have a maven multi-module project. Simply because other modules jar would be inside BOOT-INF/lib. :)

[wilkinsona]

@sansnom There are four layers in the support that's just shipped in 2.3.0.M1:

• application code
• static resources
• snapshot dependencies
• dependencies

In a multi-module project with a -SNAPSHOT version, they'll go in the snapshot dependencies layer. We're also considering something more sophisticated that will place modules from the same project in a separate layer even when they do not have a -SNAPSHOT version.

The documentation is rather sparse at the moment. We'll improve that in M2. There's also a blog post in the works.

[sansnom]

Cool ! Snapshot layer is nice in development mode. But if I read you well for now, when the project is released other modules from the project will still be in the "dependencies" layer. I think the more sophisticated option could be nice to avoid overhead between version when external dependencies don't change. :)

In my assembly setup, I just use the groupId to discriminate between the external dependencies and the application code layer.

EDIT: link to the blog https://spring.io/blog/2020/01/27/creating-docker-images-with-spring-boot-2-3-0-m1