Developer Guide
The following sections are aimed to provide a comprehensive guide for developers, enabling them to understand the project's architecture and seamlessly contribute to its development.
This project utilizes three branches for the development: the main branch, which hosts the latest development, and two additional branches for each release. These release branches follow a specific naming format: YYYYx, where "YYYY" represents the year, and "x" is an increasing letter. Thus, they help to keep working on minor updates and bug fixes on the supported versions (N & N-1) of each workbench.
The structure of the notebook's build chain is derived from the parent image. To better comprehend this concept, refer to the following graph.
Each notebook inherits the properties of its parent. For instance, the TrustyAI notebook inherits all the installed packages from the Standard Data Science notebook, which in turn inherits the characteristics from its parent, the Minimal notebook.
Detailed instructions on how developers can contribute to this project can be found in the contribution.md file.
This repository has been added to the Openshift CI to build the different notebooks using the flow described in the Container Image Layering section. Every notebook will use a previous notebook as the base image:
images:
- context_dir: ${NOTEBOOK_DIR}
dockerfile_path: Dockerfile
from: ${NOTEBOOK_BASE_IMAGE_NAME}
to: ${NOTEBOOK_IMAGE_NAME}
The opendatahub-io-ci-image-mirror job will be used to mirror the images from the Openshift CI internal registry to the ODH Quay repository.
tests:
- as: ${NOTEBOOK_IMAGE_NAME}-image-mirror
steps:
dependencies:
SOURCE_IMAGE_REF: ${NOTEBOOK_IMAGE_NAME}
env:
IMAGE_REPO: notebooks
workflow: opendatahub-io-ci-image-mirror
The images mirrored under 2 different scenarios:
- A new PR is opened.
- A PR is merged.
The Openshift CI is also configured to run the unit and integration tests:
tests:
- as: notebooks-e2e-tests
steps:
test:
- as: ${NOTEBOOK_IMAGE_NAME}-e2e-tests
commands: |
make test
from: src
This section provides an overview of the automation functionalities.
Piplock Renewal [Link]
This GitHub action is configured to be triggered on a weekly basis, specifically every Monday at 22:00 PM UTC. Its main objective is to automatically update the Pipfile.lock files by fetching the most recent minor versions available. Additionally, it also updates the hashes for the downloaded files of Python dependencies, including any sub-dependencies. Once the updated files are pushed, the CI pipeline is triggered to generate new updated images based on these changes.
Sync the downstream release branch with the upstream [Link]
This GitHub action is configured to be triggered on a weekly basis, specifically every Tuesday at 08:00 AM UTC. Its main objective is to automatically update the downstream release branch with the upstream branch.
Digest Updater workflow on the manifests [Link]
This GitHub action is designed to be triggered on a weekly basis, specifically every Friday at 12:00 AM UTC. Its primary purpose is to automate the process of updating the SHA digest of the notebooks. It achieves this by fetching the new SHA values from the quay.io registry and updating the param.env file, which is hosted on the odh-manifest repository. By automatically updating the SHA digest, this action ensures that the notebooks remain synchronized with the latest changes.
Digest Updater workflow on the live-builder [Link]
This GitHub action works with the same logic as the above and is designed to be triggered on a weekly basis, specifically every Friday. It is also update the SHA digest of the images into the CSV file on the live-builder repo.