[maintenance] black-formatted packages/services and new common.Makefile

[pcrespov]

What do these changes do?

• Black-format:
  • added auto-formatter in project makefiles
  • formatted every relevant python package or service
• scripts/common.Makefile collects common recipes: see how services and packages Makefiles are now simplified
• Fixed noisy pprint in the logs
• Added extensions to bash scripts

HINTS on Unhandled exceptions in tasks

Added some hints and logs on possible source of this problem:

and engineio/async_drivers/aiohttp.py:114 shows that
IOError = OSError is raised when received data is corrupted!!
And see web-sockets memory allocation!

"top_tracemalloc": [
"<frozen importlib._bootstrap>:219: size=28.9 MiB, count=343861, average=88 B",
"/usr/local/lib/python3.6/site-packages/aiohttp/http_websocket.py:584: size=3144 KiB, count=85, average=37.0 KiB",
"/usr/local/lib/python3.6/site-packages/prometheus_client/values.py:16: size=3074 KiB, count=87434, average=36 B",
"/usr/local/lib/python3.6/site-packages/amqp/connection.py:262: size=2608 KiB, count=40, average=65.2 KiB",
"/usr/local/lib/python3.6/site-packages/amqp/method_framing.py:97: size=2561 KiB, count=40, average=64.0 KiB",
"/usr/local/lib/python3.6/pathlib.py:66: size=2560 KiB, count=8, average=320 KiB",
"/usr/local/lib/python3.6/site-packages/prometheus_client/metrics.py:526: size=2286 KiB, count=38689, average=61 B",
"/usr/local/lib/python3.6/site-packages/prometheus_client/values.py:15: size=1883 KiB, count=46217, average=42 B",
"/usr/local/lib/python3.6/site-packages/prometheus_client/metrics.py:90: size=1425 KiB, count=4931, average=296 B",
"/usr/local/lib/python3.6/abc.py:133: size=1320 KiB, count=4836, average=280 B"
]

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@ac51c73). Click here to learn what that means.
The diff coverage is 77.5%.

@@            Coverage Diff            @@
##             master    #1342   +/-   ##
=========================================
  Coverage          ?   72.75%           
=========================================
  Files             ?      201           
  Lines             ?     8356           
  Branches          ?      937           
=========================================
  Hits              ?     6079           
  Misses            ?     2005           
  Partials          ?      272

Flag	Coverage Δ
#integrationtests	57.55% <44.46%> (?)
#unittests	67.17% <76.34%> (?)

Impacted Files	Coverage Δ
...ce-library/src/servicelib/jsonschema_validation.py	0% <ø> (ø)
...ackages/service-library/src/servicelib/observer.py	90% <ø> (ø)
..._service_webserver/director/director_exceptions.py	40% <ø> (ø)
...storage/src/simcore_service_storage/rest_models.py	100% <ø> (ø)
...rc/simcore_service_webserver/computation_models.py	77.77% <ø> (ø)
.../service-library/src/servicelib/openapi_servers.py	0% <ø> (ø)
...src/simcore_service_webserver/reverse_proxy/abc.py	100% <ø> (ø)
...er/src/simcore_service_webserver/security_roles.py	100% <ø> (ø)
...es/storage/src/simcore_service_storage/__init__.py	100% <ø> (ø)
...r/src/simcore_service_webserver/computation_api.py	65.3% <0%> (ø)
... and 130 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ac51c73...14cd212. Read the comment docs.

[sanderegg]

I think some things need to be reviewed in pair

[pcrespov]

@sanderegg did you see the FIXME: PC->SAN?? comments. Perhaps we can discuss them tomorrow?

[pcrespov]

@mguidon could you please help fixing the issues reported in storage, please? https://codeclimate.com/github/ITISFoundation/osparc-simcore/pull/1342

[sanderegg]

very pretty!

[mguidon]

Oh boy. Beware of the robots. Go ahead and merge.

[pcrespov]

@mguidon I was expecting whether you could expect some time helping me fixing https://codeclimate.com/github/ITISFoundation/osparc-simcore/pull/1342
The issues remaining are all in storage. I already fixed many of them but it would be good if we go together through the remaining ones.
thx

[mguidon]

I see, sorry missed that one.

[mguidon]

Let me know when you have time

[pcrespov]

Failure in CI while building images is fixed in PR #1353. Once that PR is in, this shall follow