Fixes on catalog/services section of the web-API

[pcrespov]

What do these changes do?

Requests in the catalog/services section of the web-API are validated in the webserver and forwarded to the catalog. The responses of the catalog are then again processed in the webserver (adds convenience values for the units) and retransmitted backt o the front-end

• @odeimaiz: list_services return the same inputs as get_service_input via the web-API
  • webAPI operations: list_services, update_service and get_service now have explicit handlers
  • json serialization done with orjson
• activated /catalog/services ... web API (no need for WEBSERVER_DEV_FEATURES_ENABLED=1)
• prunes invalid field defaultValue from output ports

catalog service

• upgrades dependencies
• added model examples and extended test coverage
• reduces size of responses bodies by trimming all null-ed attributes (search for RESPONSE_MODEL_POLICY)

Related issue/s

• Processing of metadata in pipelines: units #2141: Processing of metadata in pipelines: units

How to test

make build
make up-prod

• open http://127.0.0.1:9081 and login
• in a new tab, open http://172.16.8.68:9081/dev/doc#/catalog
• try /catalog/services/* entrypoints

Checklist

• Moves some functionality from api-server+web-server to the catalog service: partially moved models, rest -> Next PR
• Alleviates database connection load -> Next PR
• Simplify catalog API models -> Next PR
• Improve error handling: partially, next -> Next PR
• prune & upgrade all requirements
• create service-fastapi-library -> Next PR
• Did you change any service's API? Then make sure to bundle document and upgrade version (make openapi-specs, git commit ... and then make version-*)
• Unit tests for the changes exist
• Runs in the swarm
• Documentation reflects the changes
• New module? Add your github username to .github/CODEOWNERS

[codecov]

Codecov Report

Merging #2151 (91dce39) into master (d6750a3) will increase coverage by 3.9%.
The diff coverage is 35.1%.

@@           Coverage Diff            @@
##           master   #2151     +/-   ##
========================================
+ Coverage    73.1%   77.0%   +3.9%     
========================================
  Files         464     249    -215     
  Lines       17709   10326   -7383     
  Branches     1741    1019    -722     
========================================
- Hits        12946    7957   -4989     
+ Misses       4313    2053   -2260     
+ Partials      450     316    -134     

Flag	Coverage Δ
integrationtests	65.5% <35.1%> (-0.3%)	⬇️
unittests	69.2% <35.1%> (+2.6%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...er/src/simcore_service_webserver/catalog_client.py	52.3% <16.6%> (-3.8%)	⬇️
.../simcore_service_webserver/catalog_api_handlers.py	35.6% <31.4%> (+0.2%)	⬆️
.../src/simcore_service_webserver/diagnostics_core.py	50.8% <33.3%> (-38.7%)	⬇️
...rc/simcore_service_webserver/catalog_api_models.py	55.5% <40.9%> (-20.7%)	⬇️
...eb/server/src/simcore_service_webserver/catalog.py	95.6% <100.0%> (-0.3%)	⬇️
.../simcore-sdk/src/simcore_sdk/node_data/__init__.py	0.0% <0.0%> (-100.0%)	⬇️
...core-sdk/src/simcore_sdk/node_data/data_manager.py	0.0% <0.0%> (-95.3%)	⬇️
...erver/src/simcore_service_webserver/rest_models.py	0.0% <0.0%> (-91.2%)	⬇️
...src/simcore_service_sidecar/celery_configurator.py	0.0% <0.0%> (-91.2%)	⬇️
...src/simcore_service_webserver/activity/handlers.py	27.1% <0.0%> (-62.9%)	⬇️
... and 260 more

[GitHK]

👍

[sanderegg]

We partially discussed together and I thought we agreed that exlude_none is not good.

if you have a field that can be nullable but has a default of 2 for example. say I set None, then on export this field will disappear and when it comes back it will set as 2 by default.