WIP: Fix/integration

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,26 +1,32 @@
 ---
-name: Feature request
-about: Suggest an idea to implement in the simcore plaform
+name: <!-- Feature request -->
+about: <!-- Suggest an idea to implement in the simcore plaform -->
 
 ---
 
 **USER STORY:**
 
-A clear and concise description of how the feature works and looks like from the user's perspective.
+<!--A clear and concise description of how the feature works and looks like from the user's perspective.
 
-Ex. I want to be able the stop the running pipeline by pressing a stop button. If the pipeline is stopped, I see a info-level message confirming it in the logger, if it fails the message should be displayed in red (error). Also, all the progress bars in the nodes must be set to 0.
+Ex. I want to be able the stop the running pipeline by pressing a stop button. If the pipeline is stopped, I see a info-level message confirming it in the logger, if it fails the message should be displayed in red (error). Also, all the progress bars in the nodes must be set to 0. -->
 
 **DEFINITION OF DONE:**
 
-A clear and concise description of what the feature requires.
+<!--A clear and concise description of what the feature requires.
 
 0. Play button turns into stop button when pipeline is running
 1. Stop button turns into play button when pipeline is finished
 2. Stop button turns into play button when pipeline is successfully stopped
 3. Logger displays messages everytime the play/stop button is pressed
 4. Progress bars are set to 0 when stopping pipeline
 5. Stop button has a Python interface
+-->
 
 **EXAMPLE:**
 
-Any file/screenshot/photomontage/video/website is provided for a better understanding of the request
+<!-- Any file/screenshot/photomontage/video/website is provided for a better understanding of the request 
+
+Recommended tools for screen captures:
+	- https://getgreenshot.org/       : very complete screen capture with annotation capabilities [windows only]
+	- https://www.cockos.com/licecap/ : produces lighweight animated screen captures
+-->

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,14 +1,4 @@
-<!-- Thank you for your contribution!
-
- - Add **WIP-** prefix in title if it is still work in progress (i.e. should not be merged but is ready to start reviewing it)
- - Use [waffle.io] keywords in title/descriptions to trigger bot actions:
-   - close, closes, close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved #[issueNumber]
-   - connect to, connects to, connected to, connect, connects, connected #[issueNumber]
-   - parent of, parent to, parent #[issueNumber]
-   - child of, child to, child #[issueNumber]
-   - blocks, block, required by, needed by, dependency of #[issueNumber]
-   - blocked by, required, require, needs, need, depends on #[issueNumber]
--->
+<!--  **WIP-** prefix in title if still work in progress -->
 
 ## What do these changes do?
 
@@ -17,17 +7,20 @@
 
 ## Related issue number
 
-<!-- Are there any issues opened that will be resolved by merging this change?
+<!--
+ Use [waffle.io] keywords in title/descriptions to trigger bot actions:
 
- - add mentions of the user or team responsible for reviewing proposed changes
- - and user/team to .github/CODEOWNERS to automatically assign reviewers
+   - close, closes, close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved #[issueNumber]
+   - connect to, connects to, connected to, connect, connects, connected #[issueNumber]
 -->
 
+
 ## Checklist
 
 - [ ] I think the code is well written
 - [ ] Unit tests for the changes exist
 - [ ] Documentation reflects the changes
+- [ ] **Runs in the swarm**
 - [ ] If you design a new module, add your user to .github/CODEOWNERS
 
 [waffle.io]:https://waffle.io/marketing-assets/documents/waffleio_cheatsheet_v1.pdf?utm_source=blog&utm_medium=cheatsheet-ctabutton&utm_campaign=cheatsheet

.travis.yml

@@ -1,9 +1,11 @@
 language: python
+dist: xenial
 addons:
   apt:
     packages:
-      - "python3"
+      - python3
       - "python3-pip"
+      - docker-ce
 python:
   - "3.6"
 install:

.vscode-template/install-dev-tools.sh

@@ -0,0 +1,3 @@
+
+# uname -s == Darwin
+brew install gnu-sed

README.md

@@ -19,12 +19,28 @@ To generate a new cookiecutter template layout just type
 ```
 and answer the questions.
 
+
+## Backed project
+
+Infrastructure:
+  - [osparc-simcore]-compatible project skeleton for python services
+  - service-level makefile (type ``make help``)
+  - [folder]({{cookiecutter.project_slug}}/extra_osparc-simcore) with stub code to integrate service in [osparc-simcore] framework
+  - Multi-stage [Dockerfile]({{cookiecutter.project_slug}}/Dockerfile) with different boot modes and healt-check
+  -
+  - ...
+
+Service Features:
+  - [aiohttp] server
+  - [OpenAPI](https://www.openapis.org/) compatible RESTful API
+  - ...
+
 Then, to work in the backed project, this is the standard wordflow
 ``` console
   $ cd my project
   $ make help
 
-  # Create and activate virtual envi
+  # Create and activate virtual environment
   $ make venv
   $ source .venv/bin/activate
 
@@ -52,26 +68,6 @@ clean – cleans projects directory
 clean-force – cleans & removes also venv folder
 ```
 
-#### Features of the backed project
-  - [aiohttp] server
-  - [OpenAPI](https://www.openapis.org/) compatible RESTful API
-  -
-  - Predefined project skeleton
-  - ``.vscode-template`` are recommended settings for vscode
-  - ``{{cookiecutter.project_slug}}/extra`` contains code modules within a tree folder structure equivalent to [osparc-simcore] repo
-  - makefile
-  ```console
-  $ make help
-  simcore_service_dummy_service:0.1.0
-
-  venv – Create the virtual environment
-  install – Install project sources in "development mode"
-  uninstall – Uninstall project sources
-  test – Runs unit tests [w/ fail fast]
-  help – Display all callable targets
-  clean – Clean the project's directory
-  ```
-
 ---
 
 ## Acknoledgements
@@ -81,6 +77,6 @@ This template was built upon ideas/snippets borrowed from already existing great
 This project is licensed under the terms of the [MIT License](/LICENSE)
 
 
-[aiohptt]:https://aiohttp.readthedocs.io/en/stable/
+[aiohttp]:https://aiohttp.readthedocs.io/en/stable/
 [cookiecutter]:https://cookiecutter.readthedocs.io
 [osparc-simcore]:https://github.com/ITISFoundation/osparc-simcore

cookiecutter.json

@@ -1,18 +1,23 @@
 {
     "full_name": "Your name",
     "github_username": "Your github username",
+
     "project_name": "Name of the project",
     "project_short_description": "A short description of the project",
     "project_slug": "{{ cookiecutter.project_name.lower().replace(' ', '-') }}",
-    "dockercompose_service_name": "{{ cookiecutter.project_slug }}",
-    "dockercompose_service_api_port": "8080",
+
     "package_name": "simcore_service_{{ cookiecutter.project_slug|replace('-','_') }}",
     "distribution_name": "simcore-service-{{ cookiecutter.project_slug }}",
     "command_line_interface_bin_name": "{{ cookiecutter.distribution_name }}",
-    "simcore_install_root": ["../../", "git+https://github.com/ITISFoundation/osparc-simcore.git@master"],
-    "release_date": "{% now 'local' %}",
+
     "version": "0.1.0",
     "openapi_specs_version": "v{{ cookiecutter.version.split('.')[0] }}",
-    "sdk_package_name": "simcore_{{ cookiecutter.project_slug|replace('-','_') }}_sdk",
+
+    "dockercompose_service_name": "{{ cookiecutter.project_slug }}",
+    "dockercompose_service_api_port": "8080",
+
+    "enable_aiohttp_swagger": ["false", "true"],
+
+    "year": "{% now 'utc', '%Y' %}",
     "_extensions": ["jinja2_time.TimeExtension"]
 }

{{cookiecutter.project_slug}}/client-sdk/codegen.sh → keep/{# client-sdk #}/codegen.sh

@@ -1,5 +1,5 @@
 #/bin/bash
-# TODO: unify scripts
+# FIXME: activate when ITIFoundation/openapi-tools:latest available
 exec {{ cookiecutter.simcore_install_root }}../scripts/openapi/openapi_codegen.sh \
     -i ../src/{{ cookiecutter.package_name }}/oas3/{{cookiecutter.openapi_specs_version}}/openapi.yaml \
     -o . \

keep/{# integration #}/conftest.py

@@ -0,0 +1,9 @@
+
+
+# TODO: docker-compose -f services/docker-compose.yml up --exit-code-from={{ cookiecutter.project_name }}  {{ cookiecutter.project_name }}
+
+# TODO: docker-compose -f services/docker-compose.yml up --detach {{ cookiecutter.project_name }}
+# test if responsive ??
+
+# ls home
+# ls /devel

tests/test_bake_project.py

@@ -1,14 +1,19 @@
-#pylint: disable=W0621
-# W0621:Redefining name 'here' from outer scope (line 12)
+# pylint:disable=wildcard-import
+# pylint:disable=unused-import
+# pylint:disable=unused-variable
+# pylint:disable=unused-argument
+# pylint:disable=redefined-outer-name
+
+import logging
 import os
-import sys
-import pytest
+import shutil
 import subprocess
-import logging
+import sys
 from contextlib import contextmanager
-
 from pathlib import Path
 
+import pytest
+
 logger = logging.getLogger(__name__)
 
 @pytest.fixture
@@ -38,6 +43,8 @@ def inside_dir(dirpath):
         os.chdir(old_path)
 
 
+
+
 def test_project_tree(cookies):
     result = cookies.bake(extra_context={'project_slug': 'test_project'})
     assert result.exit_code == 0
@@ -51,13 +58,28 @@ def test_project_tree(cookies):
 #    with inside_dir(str(result.project)):
 #        assert subprocess.check_call(['flake8']) == 0
 
+# TODO: use pylint via package instead of application entrypoint
 def test_run_pylint(cookies, pylintrc):
     result = cookies.bake(extra_context={'project_slug': 'pylint_compat', 'package_name': 'package_folder'})
     with inside_dir(str(result.project)):
         cmd = 'pylint --rcfile {} -v src/package_folder/'.format(pylintrc.absolute()).split()
         assert subprocess.check_call(cmd) == 0
 
 
+def test_no_tags(cookies):
+    exclude = ['.pylintrc']
+    result = cookies.bake(extra_context={
+                          'project_slug': 'myproject', 'package_name': 'package_folder'})
+    for root, dirs, files in os.walk(result.project):
+        for fname in files:
+            if fname not in exclude:
+                fpath = os.path.join(root, fname)
+                with open(fpath) as fh:
+                    for lineno, line in enumerate(fh):
+                        assert "TODO" not in line, "{}:{}".format(fpath, lineno)
+        # skips
+        dirs[:] = [n for n in dirs if not n.startswith('.')]
+
 def test_run_tests(cookies):
     result = cookies.bake(extra_context={'project_slug': 'dummy-project'})
     working_dir = str(result.project)
@@ -73,3 +95,39 @@ def test_run_tests(cookies):
             logger.info("Running '%s' ...", cmd)
             assert subprocess.check_call(cmd.split()) == 0
             logger.info("Done '%s' .", cmd)
+
+
+
+
+@pytest.mark.skip("TODO: Under development")
+def test_build_docker(cookies, tmpdir):
+    # TODO: check build target base, build, cache, prod and devel
+
+    # bakes cookie within osparc-simcore tree structure
+    result = cookies.bake(extra_context={'project_slug': 'dummy-project'})
+    working_dir = str(result.project)
+
+    tmpdir.mkdir("packages").join("dummy.py").write("import os")
+    new_working_dir = tmpdir.mkdir("services") / os.path.basename(working_dir)
+    shutil.move(working_dir, new_working_dir)
+
+    # ----
+    commands = (
+        "ls -la .",
+        "pip install pip-tools",
+        "make requirements",
+        "docker build -f Dockerfile -t dummy-project:prod --target production ../../"
+    )
+    with inside_dir(new_working_dir):
+        for cmd in commands:
+            logger.info("Running '%s' ...", cmd)
+            assert subprocess.check_call(cmd.split()) == 0
+            logger.info("Done '%s' .", cmd)
+
+
+@pytest.mark.skip("TODO: Under development")
+def test_run_docker(cookies, tmpdir):
+    # check state after boot
+    # check run permissions `simcore-service-storage --help`
+    # check load config `simcore-service-storage -c `
+    pass

tools/cleanup-settings.bat

@@ -0,0 +1,16 @@
+:: cleanup settings
+::
+:: UNDER development 
+::
+@echo OFF
+setlocal
+
+set HERE=%~dp0
+set REPODIR=%HERE%..\
+
+set SETTINGSPATH="%REPODIR%\.vscode\settings.json"
+type %SETTINGSPATH% | python -m json.tool --sort-keys > %HERE%workspace-settings.json
+:: TODO this should update %SETTINGSPATH%
+
+set SETTINGSPATH="%APPDATA%\Code\User\settings.json"
+type %SETTINGSPATH% | python -m json.tool --sort-keys > %HERE%user-settings.json

tools/user-settings.json

@@ -0,0 +1,29 @@
+{
+    "atomKeymap.promptV3Features": true,
+    "editor.formatOnPaste": true,
+    "editor.insertSpaces": false,
+    "editor.multiCursorModifier": "ctrlCmd",
+    "editor.renderWhitespace": "all",
+    "editor.tabSize": 2,
+    "explorer.confirmDelete": false,
+    "files.exclude": {
+        "**/*.pyc": true,
+        "**/.DS_Store": true,
+        "**/.git": true,
+        "**/.hg": true,
+        "**/.svn": true,
+        "**/CVS": true
+    },
+    "git.enabled": true,
+    "git.path": "%ConEmuDir%\\vendor\\git-for-windows\\cmd\\git.exe",
+    "latex-workshop.view.pdf.viewer": "tab",
+    "python.linting.enabled": true,
+    "python.pythonPath": "C:\\Users\\crespo\\Miniconda3\\envs\\py3\\python.exe",
+    "python.unitTest.unittestEnabled": true,
+    "terminal.integrated.shell.windows": "cmd.exe",
+    "terminal.integrated.shellArgs.windows": [
+        "/k",
+        "%ConEmuDir%\\vendor\\init.bat"
+    ],
+    "workbench.iconTheme": "vscode-icons"
+}