Skip to content

Commit 4ad2973

Browse files
ChristianZaccariaChristianZaccaria
committed
Adjust demo notebooks with new ray image defaults
1 parent 305f01a commit 4ad2973

16 files changed

+130
-35
lines changed

Diff for: ‎demo-notebooks/additional-demos/hf_interactive.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -68,8 +68,12 @@
6868
"source": [
6969
"Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding Ray Cluster).\n",
7070
"\n",
71-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
72-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
71+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
72+
"\n",
73+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
74+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
75+
"\n",
76+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
7377
]
7478
},
7579
{

Diff for: ‎demo-notebooks/additional-demos/local_interactive.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,12 @@
3535
"metadata": {},
3636
"source": [
3737
"\n",
38-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
39-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
38+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
39+
"\n",
40+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
41+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
42+
"\n",
43+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
4044
]
4145
},
4246
{

Diff for: ‎demo-notebooks/additional-demos/ray_job_client.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,12 @@
4141
"metadata": {},
4242
"source": [
4343
"\n",
44-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
45-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
44+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
45+
"\n",
46+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
47+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
48+
"\n",
49+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
4650
]
4751
},
4852
{

Diff for: ‎demo-notebooks/guided-demos/0_basic_ray.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,12 @@
4747
"source": [
4848
"Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n",
4949
"\n",
50-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
51-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
50+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
51+
"\n",
52+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
53+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
54+
"\n",
55+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
5256
]
5357
},
5458
{

Diff for: ‎demo-notebooks/guided-demos/1_cluster_job_client.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,12 @@
4141
"source": [
4242
"Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n",
4343
"\n",
44-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
45-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
44+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
45+
"\n",
46+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
47+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
48+
"\n",
49+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
4650
]
4751
},
4852
{

Diff for: ‎demo-notebooks/guided-demos/2_basic_interactive.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,12 @@
4444
"source": [
4545
"Once again, let's start by running through the same cluster setup as before:\n",
4646
"\n",
47-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
48-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
47+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
48+
"\n",
49+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
50+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
51+
"\n",
52+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
4953
]
5054
},
5155
{

Diff for: ‎demo-notebooks/guided-demos/3_widget_example.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,12 @@
4747
"source": [
4848
"Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n",
4949
"\n",
50-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
51-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
50+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
51+
"\n",
52+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
53+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
54+
"\n",
55+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
5256
]
5357
},
5458
{

Diff for: ‎demo-notebooks/guided-demos/notebook-ex-outputs/0_basic_ray.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,12 @@
4747
"source": [
4848
"Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n",
4949
"\n",
50-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
51-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
50+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
51+
"\n",
52+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
53+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
54+
"\n",
55+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
5256
]
5357
},
5458
{

Diff for: ‎demo-notebooks/guided-demos/notebook-ex-outputs/1_cluster_job_client.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,12 @@
4141
"source": [
4242
"Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n",
4343
"\n",
44-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
45-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
44+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
45+
"\n",
46+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
47+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
48+
"\n",
49+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
4650
]
4751
},
4852
{

Diff for: ‎demo-notebooks/guided-demos/notebook-ex-outputs/2_basic_interactive.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,12 @@
4444
"source": [
4545
"Once again, let's start by running through the same cluster setup as before:\n",
4646
"\n",
47-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
48-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
47+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
48+
"\n",
49+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
50+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
51+
"\n",
52+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
4953
]
5054
},
5155
{

Diff for: ‎demo-notebooks/guided-demos/preview_nbs/0_basic_ray.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,12 @@
4747
"source": [
4848
"Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n",
4949
"\n",
50-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
51-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
50+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
51+
"\n",
52+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
53+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
54+
"\n",
55+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
5256
]
5357
},
5458
{

Diff for: ‎demo-notebooks/guided-demos/preview_nbs/1_cluster_job_client.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,12 @@
4141
"source": [
4242
"Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n",
4343
"\n",
44-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
45-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
44+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
45+
"\n",
46+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
47+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
48+
"\n",
49+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
4650
]
4751
},
4852
{

Diff for: ‎demo-notebooks/guided-demos/preview_nbs/2_basic_interactive.ipynb

+6-2
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,12 @@
4444
"source": [
4545
"Once again, let's start by running through the same cluster setup as before:\n",
4646
"\n",
47-
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
48-
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
47+
"NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n",
48+
"\n",
49+
"- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'\n",
50+
"- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'\n",
51+
"\n",
52+
"If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default."
4953
]
5054
},
5155
{

Diff for: ‎docs/cluster-configuration.md

+8-1
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,14 @@ cluster = Cluster(ClusterConfiguration(
2525
labels={"exampleLabel": "example", "secondLabel": "example"},
2626
))
2727
```
28-
Note: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. If you have your own Ray image which suits your purposes, specify it in image field to override the default image. If you are using ROCm compatible GPUs you can use 'quay.io/modh/ray:2.35.0-py39-rocm61'. You can also find documentation on building a custom image [here](https://github.com/opendatahub-io/distributed-workloads/tree/main/images/runtime/examples).
28+
NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:
29+
30+
- For Python 3.9: 'quay.io/modh/ray:2.35.0-py39-cu121'
31+
- For Python 3.11: 'quay.io/modh/ray:2.35.0-py311-cu121'
32+
33+
If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default.
34+
35+
If you are using ROCm compatible GPUs you can use 'quay.io/modh/ray:2.35.0-py39-rocm61'. You can also find documentation on building a custom image [here](https://github.com/opendatahub-io/distributed-workloads/tree/main/images/runtime/examples).
2936

3037
The `labels={"exampleLabel": "example"}` parameter can be used to apply additional labels to the RayCluster resource.
3138

Diff for: ‎src/codeflare_sdk/ray/cluster/generate_yaml.py

+7-7
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020
import json
2121
import sys
2222
import typing
23+
import warnings
2324
import yaml
2425
import os
2526
import uuid
@@ -34,7 +35,7 @@
3435

3536
SUPPORTED_PYTHON_VERSIONS = {
3637
"3.9": "quay.io/modh/ray@sha256:0d715f92570a2997381b7cafc0e224cfa25323f18b9545acfd23bc2b71576d06",
37-
"3.11": "quay.io/modh/ray:2.35.0-py311-cu121",
38+
"3.11": "quay.io/modh/ray@sha256:db667df1bc437a7b0965e8031e905d3ab04b86390d764d120e05ea5a5c18d1b4",
3839
}
3940

4041

@@ -96,12 +97,11 @@ def update_image(spec, image):
9697
containers = spec.get("containers")
9798
if not image:
9899
python_version = f"{sys.version_info.major}.{sys.version_info.minor}"
99-
try:
100-
if python_version in SUPPORTED_PYTHON_VERSIONS:
101-
image = SUPPORTED_PYTHON_VERSIONS[python_version]
102-
except Exception: # pragma: no cover
103-
print(
104-
f"Python version '{python_version}' is not supported. Only {', '.join(SUPPORTED_PYTHON_VERSIONS.keys())} are supported."
100+
if python_version in SUPPORTED_PYTHON_VERSIONS:
101+
image = SUPPORTED_PYTHON_VERSIONS[python_version]
102+
else:
103+
warnings.warn(
104+
f"No default Ray image defined for {python_version}. Please provide your own image or use one of the following python versions: {', '.join(SUPPORTED_PYTHON_VERSIONS.keys())}."
105105
)
106106
for container in containers:
107107
container["image"] = image

Diff for: ‎src/codeflare_sdk/ray/cluster/test_generate_yaml.py

+37-1
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,9 @@
1111
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1212
# See the License for the specific language governing permissions and
1313
# limitations under the License.
14-
from codeflare_sdk.ray.cluster.generate_yaml import gen_names
14+
from collections import namedtuple
15+
import sys
16+
from .generate_yaml import gen_names, update_image
1517
import uuid
1618

1719

@@ -32,3 +34,37 @@ def test_gen_names_without_name(mocker):
3234
appwrapper_name, cluster_name = gen_names(None)
3335
assert appwrapper_name.startswith("appwrapper-")
3436
assert cluster_name.startswith("cluster-")
37+
38+
39+
def test_update_image_without_supported_python_version(mocker):
40+
# Mock SUPPORTED_PYTHON_VERSIONS
41+
mocker.patch.dict(
42+
"codeflare_sdk.ray.cluster.generate_yaml.SUPPORTED_PYTHON_VERSIONS",
43+
{
44+
"3.9": "ray-py3.9",
45+
"3.11": "ray-py3.11",
46+
},
47+
)
48+
49+
# Create a namedtuple to mock sys.version_info
50+
VersionInfo = namedtuple(
51+
"version_info", ["major", "minor", "micro", "releaselevel", "serial"]
52+
)
53+
mocker.patch.object(sys, "version_info", VersionInfo(3, 8, 0, "final", 0))
54+
55+
# Mock warnings.warn to check if it gets called
56+
warn_mock = mocker.patch("warnings.warn")
57+
58+
# Create a sample spec
59+
spec = {"containers": [{"image": None}]}
60+
61+
# Call the update_image function with no image provided
62+
update_image(spec, None)
63+
64+
# Assert that the warning was called with the expected message
65+
warn_mock.assert_called_once_with(
66+
"No default Ray image defined for 3.8. Please provide your own image or use one of the following python versions: 3.9, 3.11."
67+
)
68+
69+
# Assert that no image was set in the containers since the Python version is not supported
70+
assert spec["containers"][0]["image"] is None

0 commit comments

Comments
 (0)