Skip to content

🐛 Generate API base URL by active product ⚠️ #7619

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
95a88d1
fix api-key creation response
giancarloromeo Apr 30, 2025
1575ff7
continue
giancarloromeo Apr 30, 2025
32f109e
fix api url
giancarloromeo Apr 30, 2025
8c6c2da
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo Apr 30, 2025
a1a5b2c
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 5, 2025
221b5c0
Merge remote-tracking branch 'upstream/master' into fix-api-base-url-…
giancarloromeo May 5, 2025
710159e
fix host
giancarloromeo May 5, 2025
0ca6baf
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 5, 2025
1e751ac
add test
giancarloromeo May 5, 2025
7dda096
fix host
giancarloromeo May 5, 2025
2f04ab3
fix expected api
giancarloromeo May 5, 2025
2131eb8
typehint
giancarloromeo May 5, 2025
de3ecf6
fix port
giancarloromeo May 5, 2025
4c89d78
fix helper
giancarloromeo May 5, 2025
30f59e0
typecheck
giancarloromeo May 5, 2025
67d23fa
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 5, 2025
26dd896
use generator
giancarloromeo May 5, 2025
ca6d976
move to common
giancarloromeo May 5, 2025
fcd1abb
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 5, 2025
eb24d6b
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 5, 2025
2dc9a92
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 6, 2025
e711dea
move to iterator
giancarloromeo May 6, 2025
90de65e
fix iter
giancarloromeo May 6, 2025
6744d71
make more robust
giancarloromeo May 6, 2025
e7c0234
add product_api_base_url param
giancarloromeo May 8, 2025
c8db246
add product_api_base_url param
giancarloromeo May 8, 2025
73e7fe1
add missing param
giancarloromeo May 8, 2025
becbde2
add param
giancarloromeo May 8, 2025
e3f341b
add product_api_base_url param
giancarloromeo May 8, 2025
3acbfc5
add param
giancarloromeo May 8, 2025
6879782
fix test
giancarloromeo May 8, 2025
0a0d705
add nullable field
giancarloromeo May 8, 2025
b7bbe70
fix tests
giancarloromeo May 8, 2025
b8b8cfd
fix examples
giancarloromeo May 8, 2025
55dafb2
fix
giancarloromeo May 8, 2025
ddf542e
fix
giancarloromeo May 8, 2025
8fe7c52
fix
giancarloromeo May 8, 2025
1aa57ed
fix tests
giancarloromeo May 9, 2025
08f32ab
fix test
giancarloromeo May 9, 2025
d620252
fix typecheck
giancarloromeo May 9, 2025
7c89663
typecheck
giancarloromeo May 9, 2025
b9d66b7
typecheck
giancarloromeo May 9, 2025
817f7d7
add metadata
giancarloromeo May 9, 2025
caaa89b
typecheck
giancarloromeo May 9, 2025
b9e1d73
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 9, 2025
6ec1239
Merge remote-tracking branch 'upstream/master' into fix-api-base-url-…
giancarloromeo May 9, 2025
0e89788
Merge branch 'fix-api-base-url-generation' of github.com:giancarlorom…
giancarloromeo May 9, 2025
10e459f
typecheck
giancarloromeo May 9, 2025
856e7fc
fix iter_origins
giancarloromeo May 9, 2025
2d64c6a
continue
giancarloromeo May 12, 2025
5927feb
Merge remote-tracking branch 'upstream/master' into fix-api-base-url-…
giancarloromeo May 12, 2025
1a0b1df
fix api_base_url generation
giancarloromeo May 12, 2025
b169c6e
add missing param
giancarloromeo May 12, 2025
2e6d2ea
update
giancarloromeo May 12, 2025
03bab7e
remove
giancarloromeo May 12, 2025
5b8c9f8
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 12, 2025
e33c8e2
fix test
giancarloromeo May 12, 2025
d35e22c
Merge remote-tracking branch 'upstream/master' into fix-api-base-url-…
giancarloromeo May 12, 2025
7699e7a
Merge branch 'fix-api-base-url-generation' of github.com:giancarlorom…
giancarloromeo May 12, 2025
40699c0
fix test
giancarloromeo May 12, 2025
20ec910
fix mock
giancarloromeo May 12, 2025
5c9e2c2
fix tests
giancarloromeo May 13, 2025
ee08426
fix test
giancarloromeo May 13, 2025
c97eed0
Merge remote-tracking branch 'upstream/master' into fix-api-base-url-…
giancarloromeo May 13, 2025
416492e
fix tests
giancarloromeo May 13, 2025
d525a27
fix int tests
giancarloromeo May 13, 2025
6cf6dd7
fix
giancarloromeo May 13, 2025
26a01e1
fix
giancarloromeo May 13, 2025
9f77907
fix
giancarloromeo May 13, 2025
b5fd47f
fix
giancarloromeo May 13, 2025
9cec281
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 13, 2025
20094cb
fix param
giancarloromeo May 13, 2025
6d14a2d
fix api_base_url
giancarloromeo May 13, 2025
33765d5
add missing field
giancarloromeo May 13, 2025
891685d
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 13, 2025
c64c8ae
add api_base_url
giancarloromeo May 13, 2025
ed65503
fix param
giancarloromeo May 13, 2025
a574100
typecheck
giancarloromeo May 13, 2025
e8174a2
deprecate env var
giancarloromeo May 13, 2025
e7081b9
update spec
giancarloromeo May 13, 2025
405a480
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 13, 2025
d8b63bd
Merge remote-tracking branch 'upstream/master' into fix-api-base-url-…
giancarloromeo May 14, 2025
d10f236
remove env
giancarloromeo May 14, 2025
b9f6c47
remove env
giancarloromeo May 14, 2025
26e1892
not nullable
giancarloromeo May 14, 2025
38d7853
remove assert
giancarloromeo May 14, 2025
fefd466
add var
giancarloromeo May 14, 2025
866bde0
mocks
giancarloromeo May 14, 2025
394f48e
nullable
giancarloromeo May 14, 2025
64ea381
add assert
giancarloromeo May 14, 2025
1dd8043
add test
giancarloromeo May 14, 2025
8a319e6
tests
giancarloromeo May 14, 2025
c2d9da2
nullable field
giancarloromeo May 14, 2025
fffb8f2
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 14, 2025
1c116ee
remove msg
giancarloromeo May 14, 2025
bf2d4d6
flip
giancarloromeo May 14, 2025
4f89dda
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 14, 2025
56c718b
update spec
giancarloromeo May 14, 2025
e739fe9
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 14, 2025
c4ac547
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 14, 2025
52088bf
Merge branch 'master' into fix-api-base-url-generation
giancarloromeo May 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .env-devel
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,6 @@ DIRECTOR_V2_LOGLEVEL=INFO
DIRECTOR_V2_NODE_PORTS_STORAGE_AUTH=null
DIRECTOR_V2_PORT=8000
DIRECTOR_V2_PROFILING=1
DIRECTOR_V2_PUBLIC_API_BASE_URL=http://127.0.0.1:8006
DIRECTOR_V2_SERVICES_CUSTOM_CONSTRAINTS=[]
DIRECTOR_V2_DOCKER_HUB_REGISTRY=null
DYNAMIC_SIDECAR_ENABLE_VOLUME_LIMITS=False
Expand Down
9 changes: 9 additions & 0 deletions packages/common-library/src/common_library/network.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import ipaddress


def is_ip_address(host: str) -> bool:
try:
ipaddress.ip_address(host)
return True
except ValueError:
return False
19 changes: 19 additions & 0 deletions packages/common-library/src/common_library/test_network.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import pytest
from common_library.network import is_ip_address


@pytest.mark.parametrize(
"host, expected",
[
("127.0.0.1", True),
("::1", True),
("192.168.1.1", True),
("2001:0db8:85a3:0000:0000:8a2e:0370:7334", True),
("256.256.256.256", False),
("invalid_host", False),
("", False),
("1234:5678:9abc:def0:1234:5678:9abc:defg", False),
],
)
def test_is_ip_address(host: str, expected: bool):
assert is_ip_address(host) == expected
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ class ComputationCreate(BaseModel):
Field(description="if True the computation pipeline will start right away"),
] = False
product_name: Annotated[str, Field()]
product_api_base_url: Annotated[
AnyHttpUrl,
Field(description="Base url of the product"),
]
subgraph: Annotated[
list[NodeID] | None,
Field(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from typing import TypeAlias
from typing import Annotated, TypeAlias

from pydantic import BaseModel, ByteSize, ConfigDict, Field
from pydantic import AnyHttpUrl, BaseModel, BeforeValidator, ByteSize, ConfigDict, Field
from pydantic.config import JsonDict

from ..resource_tracker import HardwareInfo, PricingInfo
from ..services import ServicePortKey
Expand Down Expand Up @@ -38,39 +39,61 @@ def from_transferred_bytes(
class DynamicServiceCreate(ServiceDetails):
service_resources: ServiceResourcesDict

product_name: str = Field(..., description="Current product name")
can_save: bool = Field(
..., description="the service data must be saved when closing"
)
wallet_info: WalletInfo | None = Field(
default=None,
description="contains information about the wallet used to bill the running service",
)
pricing_info: PricingInfo | None = Field(
default=None,
description="contains pricing information (ex. pricing plan and unit ids)",
)
hardware_info: HardwareInfo | None = Field(
default=None,
description="contains harware information (ex. aws_ec2_instances)",
)
model_config = ConfigDict(
json_schema_extra={
"example": {
"key": "simcore/services/dynamic/3dviewer",
"version": "2.4.5",
"user_id": 234,
"project_id": "dd1d04d9-d704-4f7e-8f0f-1ca60cc771fe",
"node_uuid": "75c7f3f4-18f9-4678-8610-54a2ade78eaa",
"basepath": "/x/75c7f3f4-18f9-4678-8610-54a2ade78eaa",
"product_name": "osparc",
"can_save": True,
"service_resources": ServiceResourcesDictHelpers.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"wallet_info": WalletInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"pricing_info": PricingInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"hardware_info": HardwareInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
product_name: Annotated[str, Field(..., description="Current product name")]
product_api_base_url: Annotated[
str,
BeforeValidator(lambda v: f"{AnyHttpUrl(v)}"),
Field(..., description="Current product API base URL"),
]
can_save: Annotated[
bool, Field(..., description="the service data must be saved when closing")
]
wallet_info: Annotated[
WalletInfo | None,
Field(
default=None,
description="contains information about the wallet used to bill the running service",
),
]
pricing_info: Annotated[
PricingInfo | None,
Field(
default=None,
description="contains pricing information (ex. pricing plan and unit ids)",
),
]
hardware_info: Annotated[
HardwareInfo | None,
Field(
default=None,
description="contains hardware information (ex. aws_ec2_instances)",
),
]

@staticmethod
def _update_json_schema_extra(schema: JsonDict) -> None:
schema.update(
{
"example": {
"key": "simcore/services/dynamic/3dviewer",
"version": "2.4.5",
"user_id": 234,
"project_id": "dd1d04d9-d704-4f7e-8f0f-1ca60cc771fe",
"node_uuid": "75c7f3f4-18f9-4678-8610-54a2ade78eaa",
"basepath": "/x/75c7f3f4-18f9-4678-8610-54a2ade78eaa",
"product_name": "osparc",
"product_api_base_url": "https://api.local/",
"can_save": True,
"service_resources": ServiceResourcesDictHelpers.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"wallet_info": WalletInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"pricing_info": PricingInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"hardware_info": HardwareInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
}
}
}
)

model_config = ConfigDict(
json_schema_extra=_update_json_schema_extra,
)


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,33 +6,39 @@
from models_library.users import UserID
from models_library.wallets import WalletInfo
from pydantic import BaseModel, ConfigDict
from pydantic.config import JsonDict


class DynamicServiceStart(DynamicServiceCreate):
request_dns: str
request_scheme: str
simcore_user_agent: str

model_config = ConfigDict(
json_schema_extra={
"example": {
"product_name": "osparc",
"can_save": True,
"user_id": 234,
"project_id": "dd1d04d9-d704-4f7e-8f0f-1ca60cc771fe",
"service_key": "simcore/services/dynamic/3dviewer",
"service_version": "2.4.5",
"service_uuid": "75c7f3f4-18f9-4678-8610-54a2ade78eaa",
"request_dns": "some.local",
"request_scheme": "http",
"simcore_user_agent": "",
"service_resources": ServiceResourcesDictHelpers.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"wallet_info": WalletInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"pricing_info": PricingInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"hardware_info": HardwareInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
@staticmethod
def _update_json_schema_extra(schema: JsonDict) -> None:
schema.update(
{
"example": {
"product_name": "osparc",
"product_api_base_url": "https://api.local",
"can_save": True,
"user_id": 234,
"project_id": "dd1d04d9-d704-4f7e-8f0f-1ca60cc771fe",
"service_key": "simcore/services/dynamic/3dviewer",
"service_version": "2.4.5",
"service_uuid": "75c7f3f4-18f9-4678-8610-54a2ade78eaa",
"request_dns": "some.local",
"request_scheme": "http",
"simcore_user_agent": "",
"service_resources": ServiceResourcesDictHelpers.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"wallet_info": WalletInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"pricing_info": PricingInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
"hardware_info": HardwareInfo.model_config["json_schema_extra"]["examples"][0], # type: ignore [index]
}
}
}
)
)

model_config = ConfigDict(json_schema_extra=_update_json_schema_extra)


class DynamicServiceStop(BaseModel):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,13 @@ class UnregisterCheck(InputSchema):

class ApiKeyCreateRequest(InputSchema):
display_name: Annotated[str, Field(..., min_length=3)]
expiration: timedelta | None = Field(
None,
description="Time delta from creation time to expiration. If None, then it does not expire.",
)
expiration: Annotated[
timedelta | None,
Field(
None,
description="Time delta from creation time to expiration. If None, then it does not expire.",
),
]

model_config = ConfigDict(
alias_generator=AliasGenerator(
Expand Down Expand Up @@ -86,11 +89,14 @@ class ApiKeyCreateRequest(InputSchema):
class ApiKeyCreateResponse(OutputSchema):
id: IDStr
display_name: Annotated[str, Field(..., min_length=3)]
expiration: timedelta | None = Field(
None,
description="Time delta from creation time to expiration. If None, then it does not expire.",
)
api_base_url: HttpUrl
expiration: Annotated[
timedelta | None,
Field(
None,
description="Time delta from creation time to expiration. If None, then it does not expire.",
),
]
api_base_url: HttpUrl | None = None
api_key: str
api_secret: str

Expand Down
2 changes: 0 additions & 2 deletions services/director-v2/.env-devel
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,6 @@ DIRECTOR_V2_SELF_SIGNED_SSL_SECRET_ID=1234
DIRECTOR_V2_SELF_SIGNED_SSL_SECRET_NAME=1234
DIRECTOR_V2_SELF_SIGNED_SSL_FILENAME=filename

DIRECTOR_V2_PUBLIC_API_BASE_URL=http://127.0.0.1:8006

DIRECTOR_V2_GENERIC_RESOURCE_PLACEMENT_CONSTRAINTS_SUBSTITUTIONS='{}'

LOG_LEVEL=DEBUG
Expand Down
31 changes: 29 additions & 2 deletions services/director-v2/openapi.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{

Check failure on line 1 in services/director-v2/openapi.json

View workflow job for this annotation

GitHub Actions / OAS backwards compatibility

Error when checking services/director-v2/openapi.json
"openapi": "3.1.0",
"info": {
"title": "simcore-service-director-v2",
Expand Down Expand Up @@ -1291,6 +1291,13 @@
"type": "string",
"title": "Product Name"
},
"product_api_base_url": {
"type": "string",
"minLength": 1,
"format": "uri",
"title": "Product Api Base Url",
"description": "Base url of the product"
},
"subgraph": {
"anyOf": [
{
Expand Down Expand Up @@ -1347,7 +1354,8 @@
"required": [
"user_id",
"project_id",
"product_name"
"product_name",
"product_api_base_url"
],
"title": "ComputationCreate"
},
Expand Down Expand Up @@ -1789,6 +1797,11 @@
"title": "Product Name",
"description": "Current product name"
},
"product_api_base_url": {
"type": "string",
"title": "Product Api Base Url",
"description": "Current product API base URL"
},
"can_save": {
"type": "boolean",
"title": "Can Save",
Expand Down Expand Up @@ -1825,7 +1838,7 @@
"type": "null"
}
],
"description": "contains harware information (ex. aws_ec2_instances)"
"description": "contains hardware information (ex. aws_ec2_instances)"
}
},
"type": "object",
Expand All @@ -1837,6 +1850,7 @@
"service_uuid",
"service_resources",
"product_name",
"product_api_base_url",
"can_save"
],
"title": "DynamicServiceCreate",
Expand All @@ -1855,6 +1869,7 @@
"pricing_unit_cost_id": 1,
"pricing_unit_id": 1
},
"product_api_base_url": "https://api.local/",
"product_name": "osparc",
"project_id": "dd1d04d9-d704-4f7e-8f0f-1ca60cc771fe",
"service_resources": {
Expand Down Expand Up @@ -3002,6 +3017,18 @@
],
"title": "Product Name",
"description": "Current product upon which this service is scheduledIf set to None, the current product is undefined. Mostly for backwards compatibility"
},
"product_api_base_url": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"title": "Product Api Base Url",
"description": "Base URL for the current product's API."
}
},
"additionalProperties": true,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,4 @@

UNDEFINED_STR_METADATA = "undefined-metadata"
UNDEFINED_DOCKER_LABEL = "undefined-label"
UNDEFINED_API_BASE_URL = "https://api.local"
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,7 @@
NoAuthentication,
)
from pydantic import (
AfterValidator,
AliasChoices,
AnyHttpUrl,
AnyUrl,
Field,
NonNegativeInt,
Expand Down Expand Up @@ -233,14 +231,6 @@ class AppSettings(BaseApplicationSettings, MixinLoggingSettings):
description="resource usage tracker service client's plugin",
)

DIRECTOR_V2_PUBLIC_API_BASE_URL: Annotated[
AnyHttpUrl | str,
AfterValidator(lambda v: f"{v}".rstrip("/")),
Field(
...,
description="Base URL used to access the public api e.g. http://127.0.0.1:6000 for development or https://api.osparc.io",
),
]
DIRECTOR_V2_TRACING: TracingSettings | None = Field(
json_schema_extra={"auto_default_from_env": True},
description="settings for opentelemetry tracing",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ class RunMetadataDict(TypedDict, total=False):
node_id_names_map: dict[NodeID, str]
project_name: str
product_name: str
product_api_base_url: str
simcore_user_agent: str
user_email: str
wallet_id: int | None
Expand Down
Loading
Loading