Improve "profile" reference documentation with additional admonitions

[GrahamHannington]

From the Spring Boot 3.4.5 docs topic "Externalized Configuration", heading "Merging Complex Types":

When lists are configured in more than one place, overriding works by replacing the entire list.

I interpret this to mean that, if I set the environment variable SPRING_PROFILES_INCLUDE_0, then that value will replace all values (all list elements) of any spring.profiles.include property in a configuration file.

However, in practice, that's not true. In practice, the value of the environment variable SPRING_PROFILES_INCLUDE_0 adds to the elements of the spring.profiles.include property in a configuration file.

Is this a bug, or working by design?

If it's working by design, then the docs are incorrect; or at least, misleading.

Test case

Using a "minimal" Spring Boot 3.4.5 app (e.g. the reproducer in #45370, but without setting an environment prefix).

Current working directory:

application.yaml
application-alpha.yaml
application-bravo.yaml
application-charlie.yaml

Contents of application.yaml:

spring:
  profiles:
    include:
      - alpha
      - bravo

(I don't think the contents of the profile-specific files are significant here.)

Java command line to start the app:

SPRING_PROFILES_INCLUDE_0=charlie java -jar /path/to/app.jar

Log excerpt:

The following 3 profiles are active: "charlie", "alpha", "bravo"

Based on the docs, I expected only "charlie" to be active.

[GrahamHannington]

From a comment on an answer to the Stack Overflow question "Environment variables for list in spring boot configuration":

I want to add one caution: as soon as you override some property of object-item in the list this way, you'll actually override all the list, not just a one property of one list item. So, the all list will be created from scratch, ignoring any bindings in original yaml configuration file. So, you can't just override one small property. You are forced to set up all list items in environment variables.

That comment matches the behavior that I expected. But it doesn't match the behavior I've tested in practice, in Spring Boot 3.4.5.

[philwebb]

This is by design and documented in https://docs.spring.io/spring-boot/reference/features/profiles.html#features.profiles.adding-active-profiles.

The spring.profiles.include property is a little special, but the "When lists are configured in more than one place, overriding works by replacing the entire list." statement is generally true for other properties.

[GrahamHannington]

@philwebb wrote:

This is by design

Thanks for the clarification, much appreciated. 👍

and documented in ["Adding Active Profiles" in the Spring Boot docs]

No, it's not.

Or rather: I don't see where "Adding Active Profiles" documents this.

I'd already read that "Adding Active Profiles" section before creating this issue.

From that section:

Sometimes, it is useful to have properties that add to the active profiles rather than replace them. The spring.profiles.include property can be used to add active profiles on top of those activated by the spring.profiles.active property.

Yes. I've tested this. More specifically, in the context of this issue, I've tested this with environment variables:

• SPRING_PROFILES_INCLUDE_n environment variables (where n is the element index, starting from 0) can be used to add active profiles on top of those activated by the spring.profiles.active property in a configuration file.

and, also:

• The SPRING_PROFILES_ACTIVE environment variable can be used to add active profiles on top of those activated by the spring.profiles.include property in a configuration file.

No arguments there.

However, I don't see anything in that section to indicate that:

• spring.profiles.include is "a little special".
• The following statement elsewhere in the Spring Boot docs does not apply to the spring.profiles.include list property:

  When lists are configured in more than one place, overriding works by replacing the entire list.

• As I've tested (and reported in this issue), and as you've stated is by design:
  • SPRING_PROFILES_INCLUDE_n environment variables can be used to add active profiles on top of those activated by the spring.profiles.include property in a configuration file.

I'm sincerely grateful for your time, and I don't mean to try your patience: could you please quote the specific text in that section that supports your statement "and documented in"?

[philwebb]

I see your point, I guess we should add a note to the "Adding Active Profiles" section to make things a little clearer.

[GrahamHannington]

As discussed in earlier comments, spring.profiles.include is an exception to the following general rule under the heading "Merging Complex Types":

When lists are configured in more than one place, overriding works by replacing the entire list.

With apologies to any readers who feel that, given the previous statement, the following statement is the bleeding obvious:
spring.profiles.include is also an exception to the following general rule in the Spring Boot 3.4.5 docs topic "Externalized Configuration":

Spring Boot uses a very particular PropertySource order that is designed to allow sensible overriding of values. Later property sources can override the values defined in earlier ones. Sources are considered in the following order:

in the sense that there is no overriding of values of spring.profiles.include (setting aside for now the overriding of property values in different profiles; more on that later). All list elements of all instances of spring.profiles.include are used to activate profiles. Correct?

Some questions:

• Should the "Merging Complex Types" section mention that spring.profiles.include is an exception?
• Should the "override order" list in the "Externalized Configuration" topic be followed by a note mentioning that spring.profiles.include is an exception?
• Are there other undocumented exceptions? That is, other Spring Boot list properties that behave in the same way as spring.profiles.include?

[GrahamHannington]

How does "last wins" apply to profiles activated by multiple property sources?

Perhaps this comment should be in a separate issue (reworded as a docs enhancement, e.g. "Clarify last-wins strategy for profiles activated by multiple property sources") or relegated to a question in Stack Overflow. If so, please let me know. I want to do the right thing; to not annoy anyone.

From the Spring Boot docs topic "Externalized Configuration", heading "Profile Specific Files":

If several profiles are specified, a last-wins strategy applies. For example, if profiles prod,live are specified by the spring.profiles.active property, values in application-prod.properties can be overridden by those in application-live.properties.

I've been wondering what "last wins" means when the user specifies profiles in multiple property sources; by which I mean, in the context of this issue, spring.profiles.include and/or spring.profiles.active, possibly specified in multiple configuration files, and also specified in the corresponding environment variables.

In brief

I've done some testing. The answer: it depends!

Is it worth mentioning something about this in the docs?

In detail

I could describe an unlikely "monster" scenario with many combinations, but let's start with some relatively simple examples that I think are realistic:

Example A: spring.profiles.include and SPRING_PROFILES_ACTIVE

The user has specified:

• In application.yaml in a default location:

  spring:
    profiles:
      include:
        - alpha
        - bravo

• The environment variable SPRING_PROFILES_ACTIVE=charlie

Log excerpt:

The following 3 profiles are active: "alpha", "bravo", "charlie"

Log shows that the app is using server port 9073 set by charlie.

That is: the value of the environment variable SPRING_PROFILES_ACTIVE is appended to the list specified in the configuration file by spring.profiles.include.

Example B: spring.profiles.include and SPRING_PROFILES_INCLUDE_0

The user has specified:

• In application.yaml in a default location:

  spring:
    profiles:
      include:
        - alpha
        - bravo

• The environment variable SPRING_PROFILES_INCLUDE_0=charlie

Log excerpt:

The following 3 profiles are active: "charlie", "alpha", "bravo"

Log shows that the app is using server port 9072 set by bravo.

That is: the value of the environment variable SPRING_PROFILES_INCLUDE_0 is prepended to the list specified in the configuration file by spring.profiles.include.

Example C: spring.profiles.active and SPRING_PROFILES_INCLUDE_0

The user has specified:

• In application.yaml in a default location:

  spring:
    profiles:
      active: alpha,bravo

• The environment variable SPRING_PROFILES_INCLUDE_0=charlie

Log excerpt:

The following 3 profiles are active: "charlie", "alpha", "bravo"

Log shows that the app is using server port 9072 set by bravo. (Same as example B.)

That is: the value of the environment variable SPRING_PROFILES_INCLUDE_0 is prepended to the list specified in the configuration file by spring.profiles.active.

Other examples?

I haven't tested:

• Multiple instances of spring.profiles.include in different application.yaml files in different default locations (e.g. one in the current working directory, another in the config subdirectory of the current working directory).

Some thoughts on these results

I'm bothering asking this question because it occurs to me that other users might ask this question, too, and so, perhaps: is it worth mentioning something about this in the docs?

That is, for profiles, the "last" in "last wins" depends on which combination of property (spring.profiles.active or spring.profiles.include) and property source (e.g. configuration file vs environment variable) you use.

Can the vague "it depends" be described more clearly, but still concisely, matching the results in these examples, and all other potential cases? I confess that, after performing these tests, my own thinking isn't clear enough to distill even just these three results into a single brief sentence. I need to sleep on it.

Or, should the docs even delve into this level of detail; should this be up to the user to figure out for themselves, as I've done here in these examples?

Let sleeping dogs lie?

[GrahamHannington]

Perhaps there is no way to concisely encapsulate the order of all of the possible combinations.

In which case: does the following Spring Boot log message reliably reflect the profile order:

The following n profiles are active: ...

?

The testing that I've done so far indicates "Yes", but I don't want to just assume that.

[philwebb]

Perhaps there is no way to concisely encapsulate the order of all of the possible combinations.

It's a bit of a balancing act with the documentation, we want to give enough information but also not totally overwhelm the users. I've added a few more hints that I hope will help.

[GrahamHannington]

@philwebb, thanks for the docs updates. Yes, I think those hints will help. 👍

Just tying up two loose ends (I acknowledge this issue is closed)...

---

I asked:

does the [...] Spring Boot log message ["n profiles are active: ..."] reliably reflect the profile order?

That question might have come across as rhetorical. It wasn't meant to be. I think the answer is "Yes". @philwebb, could you please confirm? Please let me know if you'd prefer me to ask this is as a separate question in Stack Overflow, no problem.

---

Regarding the examples I've described, I wrote:

my own thinking isn't clear enough to distill even just these three results [...] I need to sleep on it.

FYI (no action or response requested or expected; feel free to stop reading now 🙂), a fresh thought...

Examples B and C break my general expectation* that, in Spring Boot, you can use environment variables to override values in configuration files (e.g. application.yaml). In those examples, the profile specified by an environment variable is prepended to the "last wins" profile order; hence, the profile specified by the environment variable doesn't override the profiles activated by configuration files.

I acknowledge that this behavior is covered by the following new note:

NOTE: Included profiles are added before any configprop:spring.profiles.active[] profiles.

* Re: "my general expectation", I anticipate that some readers might accuse me of conflating the "very particular" property source order with the "last-wins" profile order. I'm not. I'm aware that these orders are distinct. Here, I'm referring to the more abstract concept of user expectation of consistency across different aspects of a system's behavior. That is, in the context of "property source order", the Spring Boot docs set an expectation that environment variables override configuration files. I suspect that users will bring that expectation, as I did, to the distinct-but-related context of "profile order": that a profile activated by an environment variable should override profiles activated by configuration files. I acknowledge that the new note usefully clarifies this; resets that expectation.

[GrahamHannington]

With apologies if I'm flogging a dead horse: in my opinion, example B demonstrates undesirable behavior. I think that the environment variable SPRING_PROFILES_INCLUDE_0 should append to the spring.profiles.include list specified in application.yaml. The environment variable should "win". Just me?

The remainder of this comment is draft content for user docs that I'm writing for a Spring Boot app (I've changed the example profile names and base config file name). I have mixed feelings about including the "Alternatively,..." part of the example. I'm tending toward including it because, if I were the user and I tried that, on seeing the resulting profile order, I'd probably think, "that looks like a bug to me".

Thoughts?

---

Example: Using an environment variable to add to the profiles activated by a configuration file

Given the following properties in an application.yaml file:

spring:
  profiles:
    include:
      - alpha
      - bravo

you can add to those active profiles by setting the following environment variable:

export SPRING_PROFILES_ACTIVE=charlie,delta

The resulting profile order contains the profiles activated by SPRING_PROFILES_ACTIVE after the profiles activated by spring.profiles.include:

1. alpha
2. bravo
3. charlie
4. delta

Alternatively, you can set a SPRING_PROFILES_INCLUDE_n environment variable for each profile that you want to add, where n is an element index starting at 0:

export SPRING_PROFILES_INCLUDE_0=charlie
export SPRING_PROFILES_INCLUDE_1=delta

In this case, the resulting profile order is different: it shows the profiles activated by the SPRING_PROFILES_INCLUDE_n environment variables before the profiles activated by spring.profiles.include in the configuration file:

1. charlie
2. delta
3. alpha
4. bravo

[philwebb]

does the [...] Spring Boot log message ["n profiles are active: ..."] reliably reflect the profile order?

Yes, See this code which calls context.getEnvironment().getActiveProfiles(). Spring Frameworks AbstractEnvironment uses a LinkedHashSet so order is preserved.

With apologies if I'm flogging a dead horse: in my opinion, example B demonstrates undesirable behavior. I think that the environment variable SPRING_PROFILES_INCLUDE_0 should append to the spring.profiles.include list specified in application.yaml. The environment variable should "win". Just me?

This is for back compatibility reasons. See #26189

Please let me know if you'd prefer me to ask this is as a separate question in Stack Overflow, no problem.

We prefer to use GitHub issues only for bugs and enhancements so questions are best asked on stackoverflow.com. I'm sorry, but I don't personally have a lot of time to answer these type of questions. You may find someone from the community on stackoverflow.com who can help.