FREEZE_XieXie #1669
Merged
FREEZE_XieXie #1669
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
odeimaiz
commented
Aug 4, 2020
odeimaiz
commented
- UI/UX improvements (UI/UX improvements #1657)
- Bump yarl from 1.4.2 to 1.5.1 in /packages/postgres-database (Bump yarl from 1.4.2 to 1.5.1 in /packages/postgres-database #1665)
- Bump ujson from 3.0.0 to 3.1.0 in /packages/service-library (Bump ujson from 3.0.0 to 3.1.0 in /packages/service-library #1664)
- Bump pytest-docker from 0.7.2 to 0.8.0 in /packages/service-library (Bump pytest-docker from 0.7.2 to 0.8.0 in /packages/service-library #1647)
- Improving storage performance (Improving storage performance #1659)
- Bump aiozipkin from 0.6.0 to 0.7.0 in /packages/service-library (Bump aiozipkin from 0.6.0 to 0.7.0 in /packages/service-library #1642)
- Theming (Theming #1656)
- Platform stability: (Platform stability: #1645)
- is1594 fix and re-activate e2e testing (is1594 fix and re-activate e2e testing #1620)
- 2 bugs fixed + Some improvements (2 bugs fixed + Some improvements #1634)
- Fixes default (Hotfix copy in storage service #1640)
- Bump lodash from 4.17.15 to 4.17.19 (Bump lodash from 4.17.15 to 4.17.19 #1639)
- Is1585/cleanup storage (Is1585/cleanup storage #1586)
- Fixes on publish studies handling (Fixes on publish studies handling #1632)
- Some enhancements and bug fixes (Some enhancements and bug fixes #1608)
- Improve e2e (Improve e2e #1631)
- filter studies by name before deleting them ([fix e2e] Filter studies by name before deleting them #1629)
- Maintenance/upgrades test tools (Maintenance/upgrades test tools #1628)
- Bugfix/concurent opening projects (Bugfix/concurent opening projects #1598)
- Bugfix/allow reading groups anonymous user (Bugfix/allow reading groups anonymous user #1615)
- Bump docker from 4.2.1 to 4.2.2 in /packages/postgres-database (Bump docker from 4.2.1 to 4.2.2 in /packages/postgres-database #1605)
- fix testing if node has gpu support (fix testing if node has gpu support #1604)
- [bugfix] Invalidate cache before starting a study ([bugfix] Invalidate cache before starting a study #1602)
- Feature/fix e2e 2 (Feature/fix e2e 2 #1600)
- fix deploy not needing e2e testing since it is disabled
- reduce cardinality of metrics (reduce cardinality of metrics #1593)
- Excudes e2e stage from include until fixed (Excludes e2e job from travis CI until fixed #1595)
- Shared project concurrency (frontend) (Shared project concurrency (frontend) #1591)
- Homogenize studies and services (Homogenize studies and services #1569)
- [feature] UI Fine grained access - project locking and notification
- Bugfix/apiserver does not need sslheaders (Bugfix/apiserver does not need sslheaders #1564)
- Cleanup catalog service (Cleanup catalog service #1582)
- Maintenance/cleanup api server (Maintenance/cleanup api server #1578)
- Adds support for GPU scheduling of computational services (Adds support for GPU scheduling of computational services #1553)
- Maintenance/upgrades and tooling (Maintenance/upgrades and tooling #1546)
- Is1570/study fails 500 (Is1570/study fails 500 #1572)
- Bump faker from 4.1.0 to 4.1.1 in /packages/postgres-database (Bump faker from 4.1.0 to 4.1.1 in /packages/postgres-database #1573)
- maintenance fix codecov reports (maintenance fix codecov reports #1568)
- Manage groups, Share studies (Manage groups, Share studies #1512)
- Is/add notebook migration script (Is/add notebook migration script #1565)
- Is1269/api-server upgrade (Is1269/api-server upgrade #1475)
- added simcore_webserver_service in pytest simcore package ([maintenance] improve stability of deploy tests #1563)
- add traefik endpoint to api-gateway (add traefik endpoint to api-gateway #1555)
added mockserver to simulate apiserver service fix e2e test: check for api-gateway
ensure the database is filled up
- new api-server refactored - api-server connected to simcore-stack
[enhancement] add way to import database from a folder [maintenance] script to upgrade projects that uses a notebook 2.13.0 to use a 2.13.1 notebook
Backend:
- adds groups endpoint to list, create, get, modify, delete groups through /groups endpoint
- a user in a group has read/write/delete access rights (e.g. member,manager,administrator)
- add/list/modify/delete users in groups through /groups/{gid}/users endpoint
- allow sharing of projects using access_rights defining for each group id the specific read/write/delete rights
- migrates postgres DB accordingly (migrates old access_rights syntax for projects, deprecates user_to_projects table, adds groups thumbnails, )
Frontend:
- New page in Preferences for managing/displaying Organizations and its members
- Testers can create Organizations
- Organization Managers can invite (actually add) osparc existing members by providing their - emails
- Organization Managers can remove members
- Organization Managers can promote members to Manager
- Organization SuperManagers can edit organization details
- Study owners can share the same instance of a study with Organizations and/or Organization Members (Collaborators)
- Study owners can make other collaborators Owner
- Study owners can remove collaborators
Bonus
Backend:
- adds a decorator for checking role permissions
- docs of how to use sc-pg migration capabilities
Frontend:
- Can edit study details also from inside study
- Allow dropping files with no extension
- Show hand cursor over dragable NodeUI ports
- Show move cursor over NodeUI capationbar
- Fix multiFilePicker bug
- Switch button for changing theme
* add codecov.yml file * set up codecov to report for api/packages/services
Bumps [faker](https://github.com/joke2k/faker) from 4.1.0 to 4.1.1. - [Release notes](https://github.com/joke2k/faker/releases) - [Changelog](https://github.com/joke2k/faker/blob/master/CHANGELOG.rst) - [Commits](joke2k/faker@v4.1.0...v4.1.1) Signed-off-by: dependabot-preview[bot] <[email protected]> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Fixes #1570 * increases verbose in gc errors * Turned gather errors into warnings * Fixing issue when valid token with invalid user * Authz cache cleared when any user is deleted * Tests fix emulating a valid cookie invalid user * Enhances error handling
* script to create invitations * ujson 2.0.3 -> 3.0.0 in ALL packages * Upgrades webserver * Upgrades packages * Autoformats web-server
* added sidecar_gpu debug profile to launch.json * added a new API to the director it is now possible to get service extras containing additional information which hidden from the user * sidecar now supports GPU services - it can now schedule a container requesting GPU resources (VRAM) - sidecard has 2 start modes CPU and GPU - for development a GPU sidecar was added * updating comp_task now adds resoruce requirements - uses the director API to determine if node requires GPU - the requires_gpu field is used by the sidecar * added call to get_service_extras * fixes some pylint warnings * this should redirect errors to strout * disabaled warning * added some logging to detect why CI fails * added more debugging for CI * refactored the way cgroups is parsed * fixed pylint and message * removed sidecar_gpu container used for development * sidecar_gpu removed from the wrong list * node_extras can be None an it is now accounted for * correeclty removes the sidecar_gpu now * sidecar_gpu is not present in all deployments removing it from both lists as it is only required in development * added tests for the gpu and non gpu modes fixed an issue where the process would crash * fixed pylint * refactored tests - names are more readable - removed some typos * this is a dictionary * added comment to rememeber how to run in dev * updated service description and openapi spec * moved generate_service_extras * moved env var resolution to an appropriate place * added annotations and fixed typos * added annotations and less broad exception * cleanedup implementation * added more specific exception * corrected wrong import path * the sidecar_gpu is now properly started locally tests where reverted * fixed pylint caught error * changing import schema refactored tests to reflect changes * removed unused import * moved sidecar_gpu to docker-compose.devel.yml * keeping track of these exludes * replaced with cleaner implementation * cleanedup and fixed boot modes for sidecar if will now properly start as expected on all deployments * fixing import and moved comment * fixed an issue causing issues starting in dev mode * api server ninja'd 3006 port moving to 3007 Co-authored-by: Andrei Neagu <[email protected]>
During the design of this first version, there were many variants of the design that remained in the code. This PR cleans up all the deprecated code before extending it with new features. Some relevant changes: - Pruned and upgraded requirements - Removed trials sandbox folder - Removed unused models, db repos and routes - Left only basic authentication - Added client wrapper for webserver sessions (see osparc-simcore/services/api-server/src/simcore_service_api_server/services/webserver.py) - Reverted logger from loguru to default python logger - documented dev: services/api-server/README.md
Cleanup of catalog service before adding more functionality * Major refactoring - api/dependencies: dependencies injected in handlers - api/routes: endpoint handlers and routing - api/root.py: join all openapi-specs into a single router - models: domain (orm and business logic models) and schema (i/o schemas for openapi) models - db: database tables (i.e. sa schemas for tables) and repositories (crud layer between handlers and db calls) - core: init application and settings (parses values by args, environs, .env or default, in this order) - services: modules/plugins with logic for the app - __main__,__version__ : main entrypoint and version
* no need for sslheader in apiserver * sslheader middleware does not like hyphens * declare swarm stack name in e2e testing * wait for services now checks the latest task * discover name of postgres container
* write permissions needed to remove a user not delete permission * added test for opening a shared project 2 times * access_rights renamed to accessRights *added test for groups access rights * adding state endpoint * mypy * now check project is locked * user is automatically enters room upon successful login * project state now returns the value + owner of the lock if any
- "Services" tab renamed to "Discover". Under this tab, all the available resources (templates and services) are listed. - To start a new study: - If a templates is selected, a study containing that pipeline will be initialized (same as before) - If a service is selected, a study containing that single node will be initialized (new feature). Since this will be a single node study, the service/app will be automatically maximized. Providing a more app-like user experience. In addition, the content of the Discover tab is also shipped by the frontend compiler with its own customized navigation bar as standalone application that could be served as the Discover/Explore website. - Plotly removed is not used - Removed unused images
- Study cards show lock and tooltip when study is being used somewhere else - Optimized backend calls for getting Orgs and orgMembers
* remove node uuid and project uuid from metrics labels to reduce memory footprint
* templates are now in a different tab * starting a template bug fixed
* Do not allow negative node positions * Before starting a study, make sure the latest version is fetched
* fix testing if node has gpu support
Bumps [docker](https://github.com/docker/docker-py) from 4.2.1 to 4.2.2. - [Release notes](https://github.com/docker/docker-py/releases) - [Commits](docker/docker-py@4.2.1...4.2.2) Signed-off-by: dependabot-preview[bot] <[email protected]> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Co-authored-by: Pedro Crespo <[email protected]>
* add groups.read secutiry role * test that groups right are correctly setup
adds usage of redis lock to prevent opening the same study at the same time
* updates testing libs * Updated packages test reqs (not base.in reqs) * Updates web-server test reqs * Updates storage test reqs * Upgrades api-server tests reqs * Upgrades catalog tests reqs * Updates director test reqs * Updates sidecar tests reqs * Moved templates under .vscode with extension
Improve filtering for pre-deletion
* Fixed service submission email subject * Some window styles made uniform. * Changed some layouts. * Added standard osparc window component with clickAway closing support. * Added faded context for modal windows * Worked arround edit study bug
* Increases pool size in storage and webserver services * Fixes typo * Fixes typo in webclient * renamed variables
- upgrade docker to slim - upgrade requirements - upgraded makefile
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19. - [Release notes](https://github.com/lodash/lodash/releases) - [Commits](lodash/lodash@4.17.15...4.17.19) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Fixes error while copying data in storage service.
* Responsive Toolbar * Pin icon instead of "This node" * logLevel converted to a SelectBox * 25% max for the sidepanel * osparc.ui.window.Window close with "Esc" * workaround to make the splitter usable from the beginning * remove template access rights when creating new instance * NodeStatus -> NodeStatusUI * NodeStatus tracks progress and interactiveStatus * limit the accepted strings in interactiveStatus
Solves #1594 and part of #1367 - e2e runs in host ( partially implements #1367). installs/uninstall insecure registry (see e2e.bash ) - e2e flow is reviewed and updated - e2e is reactivated in github actions and removed from travis. Fixes bugs in postgres migration scripts related to unhandled exceptions
* increased number of postgres connections * openNode returns nodeId * wait a bit longer for fetching the files
Bumps [aiozipkin](https://github.com/aio-libs/aiozipkin) from 0.6.0 to 0.7.0. - [Release notes](https://github.com/aio-libs/aiozipkin/releases) - [Changelog](https://github.com/aio-libs/aiozipkin/blob/master/CHANGES.rst) - [Commits](aio-libs/aiozipkin@v0.6.0...v0.7.0) Signed-off-by: dependabot-preview[bot] <[email protected]> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
* added expo generator for more fine grained sleep * deque is more memory and time efficient * removing time wasters * updating file metadata after upload the metadata will be synced by a background worker spawned for for each file when a new upload url is generated * trying to fix codeclimate complaints * - added missing annotations - using logging ensure_future - freformatted code Co-authored-by: Andrei Neagu <[email protected]>
…1647) Bumps [pytest-docker](https://github.com/avast/pytest-docker) from 0.7.2 to 0.8.0. - [Release notes](https://github.com/avast/pytest-docker/releases) - [Changelog](https://github.com/avast/pytest-docker/blob/master/CHANGELOG.md) - [Commits](avast/pytest-docker@v0.7.2...v0.8.0) Signed-off-by: dependabot-preview[bot] <[email protected]> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Bumps [ujson](https://github.com/ultrajson/ultrajson) from 3.0.0 to 3.1.0. - [Release notes](https://github.com/ultrajson/ultrajson/releases) - [Commits](ultrajson/ultrajson@3.0.0...3.1.0) Signed-off-by: dependabot-preview[bot] <[email protected]> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Bumps [yarl](https://github.com/aio-libs/yarl) from 1.4.2 to 1.5.1. - [Release notes](https://github.com/aio-libs/yarl/releases) - [Changelog](https://github.com/aio-libs/yarl/blob/master/CHANGES.rst) - [Commits](aio-libs/yarl@v1.4.2...v1.5.1) Signed-off-by: dependabot-preview[bot] <[email protected]> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
- Allow Study multi-selection pressing Shift - Dashed links if there is no port connected
Codecov Report
@@ Coverage Diff @@
## staging #1669 +/- ##
=========================================
+ Coverage 71.3% 73.6% +2.3%
=========================================
Files 249 278 +29
Lines 10029 10918 +889
Branches 1106 1179 +73
=========================================
+ Hits 7154 8046 +892
+ Misses 2558 2529 -29
- Partials 317 343 +26
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.