Skip to content

Users subsystem in webserver #355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 140 commits into from
Nov 23, 2018
Merged
Show file tree
Hide file tree
Changes from 139 commits
Commits
Show all changes
140 commits
Select commit Hold shift + click to select a range
2741834
Fixes typos in client after bad merge
Jul 6, 2018
9646035
Merge remote-tracking branch 'upstream/master'
Jul 6, 2018
c3ad81c
Merge remote-tracking branch 'upstream/master'
Jul 6, 2018
d7b91f7
Merge remote-tracking branch 'upstream/master'
Jul 10, 2018
a65aa8e
Merge remote-tracking branch 'upstream/master'
Jul 11, 2018
4edc12a
Merge remote-tracking branch 'upstream/master'
Jul 16, 2018
460bdea
Merge remote-tracking branch 'upstream/master'
Jul 16, 2018
408990d
Merge remote-tracking branch 'upstream/master'
Jul 18, 2018
ee1b91b
Merge remote-tracking branch 'upstream/master'
Jul 20, 2018
cf7b3f4
Merge remote-tracking branch 'upstream/master'
Aug 9, 2018
3f4eed8
Merge remote-tracking branch 'upstream/master'
Aug 22, 2018
14cbec2
Merge remote-tracking branch 'upstream/master'
Aug 23, 2018
b57188b
Merge remote-tracking branch 'upstream/master'
Sep 3, 2018
66e5cdc
Merge remote-tracking branch 'upstream/master'
Sep 4, 2018
31aeb8f
Merge remote-tracking branch 'upstream/master'
Sep 13, 2018
c0dbe36
Merge remote-tracking branch 'upstream/master'
Sep 18, 2018
81111e0
Merge remote-tracking branch 'upstream/master'
Sep 19, 2018
6b4c1bf
Merge remote-tracking branch 'upstream/master'
Sep 24, 2018
bbe7558
Merge remote-tracking branch 'upstream/master'
Sep 25, 2018
fd2befe
Merge remote-tracking branch 'upstream/master'
Oct 10, 2018
6b0dd8f
Merge remote-tracking branch 'upstream/master'
Oct 12, 2018
b706243
Merge remote-tracking branch 'upstream/master'
Oct 17, 2018
0bf32b8
Merge remote-tracking branch 'upstream/master'
Oct 17, 2018
d0664c9
Merge remote-tracking branch 'upstream/master'
Oct 19, 2018
5e895e0
Merge remote-tracking branch 'upstream/master'
Oct 19, 2018
90eb8f4
Merge remote-tracking branch 'upstream/master'
Oct 19, 2018
dc90291
Merge remote-tracking branch 'upstream/master'
Oct 19, 2018
152d2cc
Merge remote-tracking branch 'upstream/master'
pcrespov Oct 22, 2018
4628a6e
Merge remote-tracking branch 'upstream/master'
pcrespov Oct 23, 2018
9c49147
Merge remote-tracking branch 'upstream/master'
Oct 24, 2018
21aeab8
Merge branch 'master' of github.com:pcrespov/osparc-simcore
Oct 24, 2018
62eb5c5
Merge remote-tracking branch 'upstream/master'
Oct 26, 2018
65f510c
Merge remote-tracking branch 'upstream/master'
Oct 29, 2018
9e40d37
Merge remote-tracking branch 'upstream/master'
Oct 30, 2018
77d6ac1
Merge remote-tracking branch 'upstream/master'
pcrespov Oct 31, 2018
e733846
Merge branch 'master' of github.com:pcrespov/osparc-simcore
Oct 31, 2018
6ca6aad
Merge remote-tracking branch 'upstream/master'
Oct 31, 2018
d423dcc
Merge remote-tracking branch 'upstream/master'
Nov 2, 2018
0fd2975
Merge remote-tracking branch 'upstream/master'
Nov 5, 2018
4a8c093
Merge remote-tracking branch 'upstream/master'
Nov 7, 2018
cf6c63a
Merge remote-tracking branch 'upstream/master'
Nov 9, 2018
69f047d
Updated api/specs/webserver
Nov 9, 2018
5320cef
Merge remote-tracking branch 'upstream/master'
Nov 9, 2018
e71ed59
Merge remote-tracking branch 'upstream/master'
Nov 13, 2018
0156ec8
Merge remote-tracking branch 'upstream/master'
Nov 13, 2018
ed07bfd
Merge remote-tracking branch 'upstream/master'
Nov 14, 2018
d3aaaea
Squashed Is309/webserver uses apihub - PR #324
Nov 14, 2018
ba4c92c
Added api specs in webserver:
Nov 14, 2018
aeada15
Created users subsystem: handlers, setup, tests
Nov 14, 2018
8ad1ee6
Enhances auto routing in servicelib
Nov 14, 2018
d0ab977
changes in users subsystem:
Nov 14, 2018
f5a7f50
Merge remote-tracking branch 'upstream/master'. Includes all storage PR
Nov 15, 2018
760a171
Merge remote-tracking branch 'upstream/master'
Nov 15, 2018
d9de342
Merge branch 'master' into is239/tokens-db (after PR #324 websever us…
Nov 15, 2018
8b1a1bc
Fixes all js linter errors
Nov 15, 2018
7a2052f
Fixes wrong merge of oa-specs
Nov 15, 2018
2a14e73
Removed unneceary tags and descriptions. Keeping oa-specs to a minimum.
Nov 15, 2018
1bb236a
After merge, login_required was moved to login.decorators module
Nov 15, 2018
78f1645
webclient: removed unused in fake.Data
Nov 16, 2018
68836c4
WIP cleanup
Nov 16, 2018
06ff32e
WIP: started webserver.project subsystem
Nov 16, 2018
ff98b74
Merge remote-tracking branch 'upstream/master'
Nov 16, 2018
0d5a36f
Merge branch 'master' into is239/tokens-db
Nov 16, 2018
629d89c
created schema for project and its converted-openapi-compatible version
Nov 16, 2018
82f79b5
Taken webclient fake/Data.js into webserver data resources
Nov 16, 2018
9b21d91
Bad merge. These files are now under oas3 folder
Nov 17, 2018
46fb7a2
split webserver's api/specs openapi.yaml file into diagnostics, auth,…
Nov 17, 2018
1495fae
servicelib: testing multi-document oas
Nov 17, 2018
baeffea
webserver project management subsystem:
Nov 17, 2018
c3381fc
WIP: searching for api/specs issue
Nov 17, 2018
443c249
Fixes failing references in api/specs
Nov 19, 2018
7004782
Creates tests to check openapi references issue
Nov 19, 2018
d5d756c
Fixes middleware error and formatting fake data
Nov 19, 2018
ee6248a
WIP: Fixes first tests
Nov 19, 2018
ff2232b
WIP: drafted project rest resources in client
Nov 19, 2018
2e482d2
Minor
Nov 19, 2018
13e2c7d
projects api:
Nov 20, 2018
6322355
projects api:
Nov 20, 2018
3b861ce
projects subsystem:
Nov 20, 2018
b16b74f
front-end: started connecting project resource
Nov 20, 2018
47a2037
Merged schemas
Nov 20, 2018
2702d01
By default removed flower from dc
Nov 20, 2018
457a144
project subsystem: added an option to disable login required decorator
Nov 20, 2018
1a66cdc
Removed users subsystem in this branch
Nov 20, 2018
96f1e46
Merge branch 'is239+304+188/project-tokens-api' of github.com:pcrespo…
Nov 20, 2018
46700a7
Merge branch 'master' into is239+304+188/project-tokens-api
pcrespov Nov 20, 2018
2b43200
node-output-list-icon was merged into node-output-list
Nov 20, 2018
94f9e16
minor
Nov 20, 2018
591d45c
make eslint happy
Nov 20, 2018
e20426d
Fixes node linter
Nov 20, 2018
7b0b7db
Added data/ folder in resource
Nov 20, 2018
860e999
Merge branch 'is239+304+188/project-tokens-api' of github.com:pcrespo…
Nov 20, 2018
f117b46
Corrected subject of email. using "Welcome to oSPARC" as the email's …
Nov 20, 2018
b41fa80
Merge branch 'is239+304+188/project-tokens-api' into pcrespov-is239+3…
pcrespov Nov 20, 2018
cdfe251
Merge pull request #7 from odeimaiz/pcrespov-is239+304+188/project-to…
pcrespov Nov 20, 2018
4162928
Fixed linter error
Nov 20, 2018
04264e1
Merge branch 'master' into is239+304+188/project-tokens-api
odeimaiz Nov 20, 2018
d0326b3
Merge branch 'master' into is239+304+188/project-tokens-api
pcrespov Nov 20, 2018
0185ac9
WIP: adding user manager subsystem.
Nov 20, 2018
c484443
Fixes docker-compose after @sanderegg review
Nov 20, 2018
25d32d7
Simplified openapi specs after sandereck review
Nov 20, 2018
4d11352
Merge branch 'is239+304+188/project-tokens-api' of github.com:pcrespo…
Nov 20, 2018
868f7e0
user subsystem does not belong to ths branch
Nov 20, 2018
7863880
WIP: adding users
Nov 20, 2018
8840923
Merge remote-tracking branch 'upstream/master'
Nov 21, 2018
e345754
Merge branch 'master' into is239+304+188/users
Nov 21, 2018
9447215
Merge remote-tracking branch 'upstream/master'
Nov 21, 2018
8c7e388
Implemented get_user_profile
Nov 21, 2018
187792f
oas webserver: creating a token returns uuid
Nov 21, 2018
2125ef4
creation of tokens
Nov 21, 2018
3d44730
webserver.users: can list and read user tokens
Nov 21, 2018
b84bd1c
webserver.users: can delete tokens
Nov 21, 2018
bc2f30f
Fixes deprecated function in passlib
Nov 21, 2018
b7a5b8c
webserver.users subsystem:
Nov 21, 2018
3e8ecb9
Merge branch 'master' into is239+304+188/users
pcrespov Nov 21, 2018
4268be3
Merge remote-tracking branch 'upstream/master'
Nov 21, 2018
35ed334
createTokenResources
Nov 22, 2018
8daa0ab
Resource fixed
Nov 22, 2018
e91e590
Front-end for the tokens
Nov 22, 2018
9e54a8d
Show "service" and "token_key" from existing tokens
Nov 22, 2018
1d2fc4e
Merge pull request #8 from odeimaiz/pcrespov-is239+304+188/users
pcrespov Nov 22, 2018
5fe4d96
Merge branch 'master' into is239+304+188/users
odeimaiz Nov 22, 2018
b9b7962
Fixing failing test_users.py due to different running loops.
Nov 22, 2018
ad55d3d
Merge branch 'is239+304+188/users' of github.com:pcrespov/osparc-simc…
Nov 22, 2018
0029030
Fixes test failure in director by keeping same loop
Nov 22, 2018
ff814d9
Fixes pytest-asyncio loop issue in storage
Nov 22, 2018
9d1d9d5
Fixes small linter error
Nov 22, 2018
2e8fd15
Merge remote-tracking branch 'upstream/master'
Nov 22, 2018
16aa241
Use fontawesome icons
Nov 22, 2018
61e8e80
username and email in a form (with headers)
Nov 22, 2018
396bf21
front end ready
Nov 22, 2018
ce22396
minor
Nov 22, 2018
91da29b
last fixes
Nov 22, 2018
2854023
Merge pull request #9 from odeimaiz/pcrespov-is239+304+188/users
pcrespov Nov 22, 2018
a322fb8
Merge branch 'master' into is239+304+188/users
odeimaiz Nov 23, 2018
b760d6c
Merge branch 'master' into is239+304+188/users
pcrespov Nov 23, 2018
88b7c5f
Merge remote-tracking branch 'upstream/master'
Nov 23, 2018
623d532
Merge branch 'master' into is239+304+188/users
Nov 23, 2018
79f13bb
Fixes and changes in front-end to add tokens (pair-programming with o…
Nov 23, 2018
a4cd23c
Merge branch 'master' into is239+304+188/users
odeimaiz Nov 23, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions api/specs/webserver/v0/components/schemas/my.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,13 @@ Token:
service: 'github-api-v1'
token_key: N1BP5ZSpB


TokenId:
description: toke identifier
type: string
# format: uuid


# enveloped and array versions --------------------------

ProfileEnveloped:
Expand Down Expand Up @@ -70,3 +77,14 @@ TokensArrayEnveloped:
error:
nullable: true
default: null

TokenIdEnveloped:
type: object
required:
- data
properties:
data:
$ref: '#/TokenId'
error:
nullable: true
default: null
5 changes: 5 additions & 0 deletions api/specs/webserver/v0/openapi-user.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,11 @@ paths:
responses:
'201':
description: token created
content:
application/json:
schema:
$ref: './components/schemas/my.yaml#/TokenIdEnveloped'

default:
$ref: './openapi.yaml#/components/responses/DefaultErrorResponse'

Expand Down
4 changes: 3 additions & 1 deletion packages/simcore-sdk/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,14 @@
'mock~=2.0',
'pylint~=2.0',
'pytest~=3.6',
'pytest-asyncio~=0.9',
'pytest-cov~=2.5',
'pytest-docker~=0.6',
'requests~=2.19',
'docker~=3.5'
]
# NOTE: pytest-aiohttp and pytest-asyncio incompatible
# https://github.com/pytest-dev/pytest-asyncio/issues/76


setup(
name='simcore-sdk',
Expand Down
4 changes: 3 additions & 1 deletion services/director/tests/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,6 @@ pytest-aiohttp
pytest~=3.6
pytest-cov~=2.5
pytest-docker~=0.6
pytest-asyncio~=0.9.0

# NOTE: pytest-aiohttp and pytest-asyncio incompatible
# https://github.com/pytest-dev/pytest-asyncio/issues/76
30 changes: 15 additions & 15 deletions services/director/tests/test_handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@

API_VERSIONS = resources.listdir(resources.RESOURCE_OPENAPI_ROOT)

@pytest.mark.asyncio

async def test_root_get():
fake_request = "fake request"
web_response = await rest.handlers.root_get(fake_request)
assert web_response.content_type == "application/json"
assert web_response.content_type == "application/json"
assert web_response.status == 200
healthcheck_enveloped = json.loads(web_response.text)
assert "data" in healthcheck_enveloped
Expand All @@ -30,29 +30,29 @@ def _check_services(created_services, services, schema_version="v1"):
assert len(created_services) == len(services)

created_service_descriptions = [x["service_description"] for x in created_services]

json_schema_path = resources.get_path(resources.RESOURCE_NODE_SCHEMA)
assert json_schema_path.exists() == True
with json_schema_path.open() as file_pt:
service_schema = json.load(file_pt)

for service in services:
for service in services:
if schema_version == "v1":
assert created_service_descriptions.count(service) == 1
json_schema_validator.validate_instance_object(service, service_schema)

@pytest.mark.asyncio

async def test_services_get(docker_registry, push_services):
fake_request = "fake request"
# no registry defined
with pytest.raises(web_exceptions.HTTPInternalServerError, message="Expecting HTTP Internal Error as no registry URL is defined"):
services_enveloped = await rest.handlers.services_get(fake_request)

# wrong registry defined
config.REGISTRY_URL = "blahblah"
config.REGISTRY_URL = "blahblah"
with pytest.raises(web_exceptions.HTTPInternalServerError, message="Expecting HTTP Internal Error as SSL is enabled by default"):
services_enveloped = await rest.handlers.services_get(fake_request)

# right registry defined
config.REGISTRY_URL = docker_registry
with pytest.raises(web_exceptions.HTTPInternalServerError, message="Expecting HTTP Internal Error as SSL is enabled by default"):
Expand Down Expand Up @@ -103,7 +103,7 @@ async def test_services_get(docker_registry, push_services):
services = services_enveloped["data"]
assert len(services) == 2

@pytest.mark.asyncio

async def test_v0_services_conversion_to_new(configure_registry_access, push_v0_schema_services): #pylint: disable=W0613, W0621
fake_request = "fake request"
created_services = push_v0_schema_services(3,2)
Expand All @@ -117,7 +117,7 @@ async def test_v0_services_conversion_to_new(configure_registry_access, push_v0_
# ensure old style services are not retrieved
assert len(services) == 0

@pytest.mark.asyncio

async def test_services_by_key_version_get(configure_registry_access, push_services): #pylint: disable=W0613, W0621
fake_request = "fake request"

Expand All @@ -131,7 +131,7 @@ async def test_services_by_key_version_get(configure_registry_access, push_servi
web_response = await rest.handlers.services_by_key_version_get(fake_request, "whatever", "ofwhateverversion")

created_services = push_services(3,2)
assert len(created_services) == 5
assert len(created_services) == 5

retrieved_services = []
for created_service in created_services:
Expand All @@ -151,19 +151,19 @@ async def _start_get_stop_services(push_services, user_id):

with pytest.raises(web_exceptions.HTTPInternalServerError, message="Expecting internal server error"):
web_response = await rest.handlers.running_interactive_services_post(fake_request, None, None, None, None)

with pytest.raises(web_exceptions.HTTPInternalServerError, message="Expecting internal server error"):
web_response = await rest.handlers.running_interactive_services_post(fake_request, "None", None, None, None)

with pytest.raises(web_exceptions.HTTPInternalServerError, message="Expecting internal server error"):
web_response = await rest.handlers.running_interactive_services_post(fake_request, "None", "None", None, None)

with pytest.raises(web_exceptions.HTTPNotFound, message="Expecting not found error"):
web_response = await rest.handlers.running_interactive_services_post(fake_request, "None", "None", "None", None)

with pytest.raises(web_exceptions.HTTPNotFound, message="Expecting not found error"):
web_response = await rest.handlers.running_interactive_services_post(fake_request, "None", "None", "None", "ablah")

with pytest.raises(web_exceptions.HTTPInternalServerError, message="Expecting internal server error"):
web_response = await rest.handlers.running_interactive_services_get(fake_request, None)

Expand Down Expand Up @@ -201,11 +201,11 @@ async def _start_get_stop_services(push_services, user_id):
assert web_response.content_type == "application/json"
assert web_response.text is None

@pytest.mark.asyncio

async def test_running_services_post_and_delete_no_swarm(configure_registry_access, push_services, user_id): #pylint: disable=W0613, W0621
with pytest.raises(web_exceptions.HTTPInternalServerError, message="Expecting internal error as there is no docker swarm"):
await _start_get_stop_services(push_services, user_id)

@pytest.mark.asyncio

async def test_running_services_post_and_delete(configure_registry_access, push_services, docker_swarm, user_id): #pylint: disable=W0613, W0621
await _start_get_stop_services(push_services, user_id)
17 changes: 9 additions & 8 deletions services/director/tests/test_producer.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@
)

@pytest.fixture
async def run_services(configure_registry_access, push_services, docker_swarm, user_id): #pylint: disable=W0613, W0621
async def run_services(loop, configure_registry_access, push_services, docker_swarm, user_id): #pylint: disable=W0613, W0621
started_services = []
async def push_start_services(number_comp, number_dyn):
pushed_services = push_services(number_comp,number_dyn, 60)
assert len(pushed_services) == (number_comp + number_dyn)
for pushed_service in pushed_services:
for pushed_service in pushed_services:
service_description = pushed_service["service_description"]

service_key = service_description["key"]
Expand All @@ -25,26 +25,27 @@ async def push_start_services(number_comp, number_dyn):
started_service = await producer.start_service(user_id, service_key, service_version, service_uuid)
assert "published_port" in started_service
assert "entry_point" in started_service
assert "service_uuid" in started_service
assert "service_uuid" in started_service
# should not throw
await producer.get_service_details(service_uuid)
started_services.append(started_service)
return started_services

yield push_start_services

#teardown stop the services
for service in started_services:
for service in started_services:
service_uuid = service["service_uuid"]
await producer.stop_service(service_uuid)
with pytest.raises(exceptions.ServiceUUIDNotFoundError, message="expecting service uuid not found error"):
await producer.get_service_details(service_uuid)

@pytest.mark.asyncio

async def test_start_stop_service(run_services): #pylint: disable=W0613, W0621
# standard test
await run_services(1,1)

@pytest.mark.asyncio

async def test_service_assigned_env_variables(run_services, user_id): #pylint: disable=W0621
started_services = await run_services(1,1)
client = docker.from_env()
Expand All @@ -70,7 +71,7 @@ async def test_service_assigned_env_variables(run_services, user_id): #pylint: d
assert "SIMCORE_USER_ID" in envs_dict
assert envs_dict["SIMCORE_USER_ID"] == user_id

@pytest.mark.asyncio

async def test_interactive_service_published_port(run_services): #pylint: disable=W0621
running_dynamic_services = await run_services(0,1)
assert len(running_dynamic_services) == 1
Expand All @@ -88,4 +89,4 @@ async def test_interactive_service_published_port(run_services): #pylint: disabl
low_level_client = docker.APIClient()
service_information = low_level_client.inspect_service(docker_service.id)
service_published_port = service_information["Endpoint"]["Ports"][0]["PublishedPort"]
assert service_published_port == service_port
assert service_published_port == service_port
3 changes: 2 additions & 1 deletion services/sidecar/tests/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@ aiopg
coveralls
pytest
pytest-aiohttp
pytest-asyncio
pytest-cov
pytest-docker
openapi_spec_validator
pyyaml
virtualenv
# NOTE: pytest-aiohttp and pytest-asyncio incompatible
# https://github.com/pytest-dev/pytest-asyncio/issues/76
Loading