Application properties outside of jar should take precedence over profile-specific variants inside the jar

[bjfr]

In reference to #2404 which has been closed...

Spring Booot, being an opinionated framework, should try as hard as possible to adhere to established standards in the industry.

Any application can have three levels of property values: Global default values, Environment default values and Instance specific values.

Profiles, by definition, are Environment default values. They should always be overridden by any instance specific property.

As I see it, you might want to ship default values both for the global settings and for environment specific settings (ie PROD, TEST etc) embeded in the application.

But any instance specific setting should always override the two latter. And, usually, instance specific settings are deployed together with the application on the server.

So to force people to use application-PROD.properties when deploying a production instance and application-TEST on a test instance, is not only wrong, but also a cause of many headaches. I have just spent an hour trying to explain to an infrastructure guy why he must add some text to the properties file based on which profile is used. He did not get it!

So instead of

1. Command line arguments.
2. JNDI attributes from java:comp/env.
3. Java System properties (System.getProperties()).
4. OS environment variables.
5. A RandomValuePropertySource that only has properties in random.*.
6. Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
7. Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
8. Application properties outside of your packaged jar (application.properties and YAML variants).
9. Application properties packaged inside your jar (application.properties and YAML variants).
10. @propertysource annotations on your @configuration classes.
11. Default properties (specified using SpringApplication.setDefaultProperties).

we should have

1. Command line arguments.
2. JNDI attributes from java:comp/env.
3. Java System properties (System.getProperties()).
4. OS environment variables.
5. A RandomValuePropertySource that only has properties in random.*.
6. Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
7. Application properties outside of your packaged jar (application.properties and YAML variants).
8. Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
9. Application properties packaged inside your jar (application.properties and YAML variants).
10. @propertysource annotations on your @configuration classes.
11. Default properties (specified using SpringApplication.setDefaultProperties).

[philwebb]

I do see the logic here but I'm still reluctant to change the default order for 1.3 as people may depend on it.

@dsyer @wilkinsona Perhaps we could consider this change for Boot 2.0?

[philwebb]

Another uglier solution that might work for 1.2 would be to offer something like application-overrides.properties that always trumps the package application.properties.

[snicoll]

not in that format please (considering someone having an overrides profile somewhere)

[wilkinsona]

@philwebb Agreed on this being something we consider for 2.0

[bjfr]

Why not make the load order configurable, for instance by adding a property spring.config.external-properties-overrides-embedded which can be set in an embedded property-file.

Or, if it is too late to control load order once the properties are loaded, add a method on SpringApplication setExternalPropertiesOverridesInternal(boolean) which does the same thing?

That way you can leave the default as is, but still give users the option to use the alternate load order if they prefer?

[philwebb]

As much as I'd like to include this in 1.3 it appears that it's not going to be a trivial change. I've made a start in https://github.com/philwebb/spring-boot/tree/gh-3845 by changing the load order in ConfigFileEnvironmentPostProcessor but this alone won't fix the problem.

The problem is that regardless of the actual file load order, PropertySourcesLoader will still add new groups at the front. We'd need to revisit that logic as well in order to fix this issue.

[mbhave]

This would be a breaking change and something we can do in 3.0

[mbhave]

The recent work in 2.4.x related to config file processing and profiles processes file such that application properties outside of the jar take precedence over profile-specific variants inside the jar.

[membersound]

I'm quite disappointed that this feature has been introduced as a breaking change. As well I cannot follow the arguments here.

Imagine a commons use case: you have a commons.jar library with global configuration properties for email relay, ldap access, metrics config and other stuff that every of your applications in your company should inherit from.

Prior 2.4, you could just throw that library into your microservices, and individually override specific properties at will.

Now, you simply cannot override the common properties anymore in any way. Because application.properties from outside will always override my microservice properties. What's the sense in all of this??

[philwebb]

@membersound Can you provide some more details about what you're trying to do? Perhaps a sample application would help?

Are you saying that you have properties defined in commons.jar and you don't want them to be overridden by an application.properties outside of the jar? Were you previously using profiles for this?

[membersound]

I have a commons.jar with /src/main/resources/application.properties. I want those properties to be taken into account as long as they are not overridden by the implementing microservice. Be it by application.properties of the microservice or any application-{profile}.properties.

As far as I understood, it won't be possible to override the application.properties from the commons.jar by the microservices after this change, as the commons will be seen as properties outside of the microservice.jar, and thus always win?

[philwebb]

@membersound The scenario you describe should still work. The only thing that's a little unusual is putting application.properties in commons.jar. We'd usually recommend using an EnvironmentPostProcessor loaded via spring.factories if you want a jar to contribute properties from a dependency.

If you're finding things aren't working as expected can you please open a new issue and provide a sample application that we can run and debug?

[philwebb]

I've opened #24688 to see if we can make it easier for common jars to contribute properties.

[membersound]

The way I handled it so far was naming them eg application-commons.properties, defining a CommonsConfig class in the commons package, importing them with @PropertySource(classpath:application-commons.properties), and wrapping the config in a @CommonsConfig annotation.

The microservice can then load the properties (and also other defaults) by using @CommonsConfig on their @SpringBootApplication configuration class.

That prevents having to use factories and post processors. And as far as I could see now, things are still working as before with spring 2.4.x.

Anyways it would be great if a commons jar could simply throw in a application.properties that is loaded before the implementation project, which can then override specific properties at will only. Thanks for opening the issue :)