Strategy for releasing experimental logs (and metrics in future)

[srikanthccv]

Starting this new thread to find a consensus on how do we want to release the experimental logs sdk. This equally applies to the metrics prototype when it becomes ready. I looked at how other SIGs have approached this and at this point only Java SIG is actively working on both metrics and logs implementation. They have signal specific packages. Metrics and Logs sdk are released as 1.x.0-alpha separately.

The idea of releasing signal specific packages may not be reasonable choice for Python SIG now. We probably have two (please comment if any alternative approaches) options to pick from

1. Continue to use feature branches to do non-stable development work and make ad hoc releases.
2. Move the completed work to main under private (opentelemetry.(_logs/_metrics).{...}) and develop on top it. Once people start using it we will receive a feedback, bug fixes, enhancements. All of it can happen in parallel without any problem. It also makes maintainer life little easier.

I think the latter has many advantages since it avoids all the pitfalls of feature branches with a caveat that experimental work will be part of stable release packages. I think it is fine as long as we explicitly call out that There are no guarantees provided and everything will be private until it is marked as stable and warn users they are at their own risk using the non-stable parts in their code. I don't think we will have any drastic changes but still no compatibility guarantees between versions. Our rationale document has different approach to this https://github.com/open-telemetry/opentelemetry-python/blob/main/rationale.md#immature-or-experimental-signals. Please share your thoughts and suggestion. Please vote your preferred approach using emojis 🚀 -> _logs 🎉 -> feature branch.

[srikanthccv]

@overmeulen @plajjan Both of you have expressed interest in using the experimental work in this issue #1890. Please share your feedback as well.

[owais]

No emoji voting? I'll start with 🚀 for _logs module in main.

[overmeulen]

Done, +1 for a private module in main

[ocelotl]

🚀

[lzchen]

For (2), this means users would probably have to change their code (import paths) when we move from experimental -> stable right? Also what exactly should we do in terms of informing users of the experimental signals? Include this in the README for api/sdk packages?

[codeboten]

For (2), this means users would probably have to change their code (import paths) when we move from experimental -> stable right? Also what exactly should we do in terms of informing users of the experimental signals? Include this in the README for api/sdk packages?

Might be worth adding a table to the main readme in the repo and maybe even adding a section for experimental modules in each release notes?

[owais]

@lzchen Correct but we can always deprecate _logs module and keep it as an alias for the eventual stable import path for a few releases (or even forever).

[owais]

We could also have _logs emit a warning on import to let users know they are using an experimental feature and let users silence it with OTEL_PYTHON_EXPERIMENTAL_API_WARN=0 or something like that.

[lzchen]

@owais
I think we have a majority vote. Might be worth to bring this up (the decision) during the SIG meeting.

[srikanthccv]

@aabmass what are your thoughts on this? you usually play devil's advocate and put forward alternative approaches.

[ocelotl]

Hey, I just realized something: I do have an objection, the winning approach messes up the git history.

To be considered. Having a separate long-standing branch is much less of a problem if it is rebased on top of main every day.

[plajjan]

I'm a little confused. The issue title is about how to release, which in my head translates to pip packages etc, but then goes on to talk about branches, which is how to deal with source code.

I'm really a user of this library, so my focus is on how I can consume it and as long as I can install like the latest version of tracing together with experimental versions of logs + metrics, I am happy. Extra points for being able to pin versions of tracing separate from (logs + metrics).

I know how difficult long lived branches can be to maintain, so if I were developing on this repo, I guess I'd argue to avoid separate branches if possible. I dunno if it has to translate to _log in the source dir, I mean, can't that stuff be handled purely on the release end, like you bake separate packages for tracing vs (logs+metrics)?

I don't feel entirely up to speed on how build + release works here, so maybe my confusion detracts from the conversation. Apologies if so.

[srikanthccv]

Hey, I just realized something: I do have an objection, the winning approach messes up the git history.

To be considered. Having a separate long-standing branch is much less of a problem if it is rebased on top of main every day.

Not sure I understand this completely but this looks fine to me https://github.com/open-telemetry/opentelemetry-python/compare/logs.

[codeboten]

Hey, I just realized something: I do have an objection, the winning approach messes up the git history.

To be considered. Having a separate long-standing branch is much less of a problem if it is rebased on top of main every day.

@ocelotl I would recommend using a rebase commit rather than a squash merge for committing the changes from a long standing branch. Otherwise you're right the git history would be lost.

[lzchen]

@codeboten

I would recommend using a rebase commit rather than a squash merge for committing the changes from a long standing branch. Otherwise you're right the git history would be lost.

I've mistakenly done the latter before so we probably should add this to RELEASING.MD.

[overmeulen]

Now that both logs and metrics have been merged in main as private packages is it possible to do a new release of opentelemetry-python ?

[codeboten]

@overmeulen the discussion from the last SIG meeting was that the next release would happen sometime this week.

[lzchen]

@lonewolf3739
Can this issue be closed?