🚀 Pre-release master -> staging_Baklava3

[matusdrobuliak66]

What kind of pre-release?

master branch

Sprint Name

Baklava

Pre-release version

3

Commit SHA

3e947577b7deb5f43f82f5722e92e64367f0a97b

Did the commit CI suceeded?

• The commit CI succeeded.

Motivation

• staging release (we want to observer some changes in AWS staging)

What Changed

• 🐛Handle case where metadata in comp_runs are NULL #4646 by @sanderegg
• ♻️📝 API-server files collection cleanup and doc #4645 by @pcrespov
• ✨ Frontend: wait for backend's confirmation before removing a node #4639 by @odeimaiz
• 🐛 Fixes schema error in web-server OAS #4647 by @pcrespov
• ✨ new API entrypoint web-server:clone_project and drafts api-server:clone_study #4650 by @pcrespov
• ✨ Frontend: open Wallets view after log in #4649 by @odeimaiz
• 🎨 e2e: update Kember test #4611 by @odeimaiz
• ♻️ Removes assert w/ validator and fixes code redundancy #4653 by @pcrespov
• 🐛 Sort in multipart upload #4626 by @bisgaard-itis
• 🐛 Add missing email env variables to webserver (:warning: DEVOPS) #4648 by @YuryHrytsuk
• 🐛Director-v2: Fix UNKNOWN state not correctly converted #4660 by @sanderegg
• ✨ new clone_study in api-server #4663 by @pcrespov
• 🐛 Fix: Managers invite new Members #4662 by @odeimaiz
• 🐛 Bugfix: list item #4671 by @odeimaiz
• 🐛Fixed issue where a large number of node progresses were slightly above 1.0 #4672 by @sanderegg
• ✨Tracking messages get a discriminator #4677 by @sanderegg
• ✨ Feature: Payment Gateway in a new window #4658 by @odeimaiz
• ✨ modify rabbit resource tracking started message #4679 by @matusdrobuliak66
• ✨ New payments service (⚠️ devops) #4676 by @pcrespov
• 🐛 storage fixed paginated file listing #4675 by @GitHK
• 🔨 Fixes deploy push payment service #4681 by @pcrespov
• ✨ 🗃️ Is922/resource usage tracker process rabbit messages #4666 by @matusdrobuliak66
• 🐛 storage better logging #4684 by @GitHK
• ✨RabbitMQ: Add RPC decorator for server side #4682 by @sanderegg
• 🎨 Wallets: minor Cosmetics and some fixes #4680 by @odeimaiz
• ♻️ adding logs to project copy in storage #4689 by @GitHK
• ♻️ Cleanup in preparation for new payment plugin #4686 by @pcrespov
• 🐛 fix bool in query parameter #4690 by @matusdrobuliak66
• ✨ Usage overview: request per wallet #4688 by @odeimaiz
• Rabbitmq/separate clients #4685 by @sanderegg
• 🎨 Cosmetics: Description below #4691 by @odeimaiz
• 🐛 removing unsubscribe from rabbitmq + ♻️ comp resource tracking id change #4687 by @matusdrobuliak66
• ⚗️Storage: reduce pressure while copying and fix logs #4694 by @sanderegg
• 🐛Storage: Improve documentation and fix number of files shown to be copied #4695 by @sanderegg
• 🎨 Enhancements: Account, Preferences and Resource Selection #4693 by @odeimaiz
• ⚗️ storage remove task concurrency locking while duplicating a project #4698 by @GitHK
• 🎨 Cosmetics: User center #4699 by @odeimaiz
• ⬆️Storage: dependencies upgrade #4696 by @sanderegg
• ✨🗃️ Is922/resource usage tracker introduce credits #4692 by @matusdrobuliak66
• Comp backend/clusters creation functionalities #4602 by @sanderegg
• ♻️Storage: Reduce noisy loggers impact #4705 by @sanderegg
• Bump actions/checkout from 3 to 4 #4706 by @dependabot
• ♻️ Maintenance: removes unused modules #4710 by @pcrespov
• 🐛 🗃️ fix migration #4712 by @matusdrobuliak66
  No response

Devops check ⚠️ devops

• Ensure SMTP (Email) envs are in repo.config
• Catch up with @pcrespov regarding payments service and any required OPS changes

e2e testing check 🧪

No response

Summary 📝

• make release-staging name=Baklava version=3 git_sha=3e947577b7deb5f43f82f5722e92e64367f0a97b
  • https://github.com/ITISFoundation/osparc-simcore/releases/new?prerelease=1&target=<commit_sha>&tag=staging_<sprint_name><version>&title=Staging%20<sprint_name><version>
• Draft pre-release
• Announce (add redis key maintenance in every concerned deployment)

  {"start": "2023-09-06T12:30:00.000Z", "end": "2023-09-06T13:00:00.000Z", "reason": "Release Baklava 3 "}

• Announce release in Mattermost

  :loud_sound:  Maintenance scheduled for **NAMED_DAY DD. MM from START_TIME - END_TIME**.
  =========================================================================
  
  @all Be aware that you will automatically be logged out and your projects stopped and saved during the maintenance time. Affected:
  *   [https://staging.osparc.io](https://staging.osparc.io/)
  *   [https://https://staging.s4l-lite.io/](https://https://staging.s4l-lite.io//)
  
  and on premises:
  *   [https://osparc-staging.speag.com](https://osparc-staging.speag.com/)
  *   [https://tip-staging.speag.com](https://tip-staging.speag.com/)
  *   [https://s4l-staging.speag.com](https://s4l-staging.speag.com/)
  *   [https://s4l-lite-staging.speag.com](https://s4l-lite-staging.speag.com/)
  
  
  Reason: Scheduled staging-release of STAGING_NAME_AND_VERSION.
  
  Thanks for your understanding and sorry for the inconveniences,
  
  Your friendly oSparc Team
  
  

Releasing

• Check hanging sidecars. Helper command to run in director-v2 CLI simcore-service-director-v2 close-and-save-service <uuid>
• Release (release draft)
• Check Release CI
• Check deployed
  • aws deploy
  • dalco deploy
• Delete announcement
• Check e2e runs
• Announce

https://github.com/ITISFoundation/osparc-simcore/releases/tag/staging_<sprint_name><version>

[matusdrobuliak66]

@YuryHrytsuk @mrnicegyu11 please turn on the resource usage tracker service in the stagging with next release (we can run it by default in 3 replicas) thanks!

[mrnicegyu11]

@YuryHrytsuk @mrnicegyu11 please turn on the resource usage tracker service in the stagging with next release (we can run it by default in 3 replicas) thanks!

Done in ITISFoundation/osparc-ops-environments#340

[YuryHrytsuk]

Secrets update 📢

During the release the following aws staging secrets were updated:

• portainer password
• s3 access key
• postgres password

It caused the following issues:

• Cumbersome portainer password update
  • The proper solution is documented here
  • Puppeter was broken (it couldn't handle the special character in the new password) and fixed by @sanderegg
    • Fix introduced in this PR
• Some hiccups with S3 Access Key update
  • The proper solution is documented here. Mostly, it requires the update of registry since it is also uses S3 Access Key

[YuryHrytsuk]

🐛 Release issues 🐛

AWS Staging:

• registry issues (domain + new S3 Access Key. Both are ops configuration problem) -- fixed & merged
• autoscaling nodes problem (wrong labels + key pair switch) -- fixed & merged
• puppeter password problem (cannot process special characters) -- fixed & merged
• resource usage tracker is misconfigured (@matusdrobuliak66)
  • Service can not start, because there are issues with Prometheus not collecting data/misconfiguration, an investigation is needed (@mrnicegyu11 @YuryHrytsuk )

Dalco:

• resource usage tracker is misconfigured (@matusdrobuliak66)
  • Service can not start, because there are issues with Prometheus not collecting data/misconfiguration, an investigation is needed (@mrnicegyu11 @YuryHrytsuk )