Skip to content

[Cm-1631]: Implement Artifact Uploading #42

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 59 commits into from
Dec 21, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
f3a79cd
CM-1631: Implemented Artifact and ArtifactBuilder interfaces.
yaricom Dec 8, 2021
17cbcf9
CM-1631: Fixed AssetType to have shorter type enums
yaricom Dec 9, 2021
f45f97e
CM-1631: Implemented utilities to create assets from file and from fi…
yaricom Dec 9, 2021
eb90c04
CM-1631: Implemented add local and remote assets to the ArtifactImpl
yaricom Dec 9, 2021
fdf977e
CM-1631: Implemented support for adding assets folder to the Artifact…
yaricom Dec 9, 2021
996e98f
CM-1631: Renamed test case for better clarity.
yaricom Dec 9, 2021
1b39ef2
CM-1631: Added missing Override annotation.
yaricom Dec 10, 2021
e8d057e
CM-1631: Fixed JSON request data encoding by explicit use of the Stan…
yaricom Dec 10, 2021
6b2b9df
CM-1631: Fixed typos in JavaDoc.
yaricom Dec 10, 2021
04ebcb0
CM-1631: Fixed OnlineExperimentExample by removing obsolete catch wit…
yaricom Dec 10, 2021
cd136df
CM-1631: Added support for version and version TAGs to the Artifact.
yaricom Dec 10, 2021
c12e91f
CM-1631: Implemented test cases for Artifact version and version tags.
yaricom Dec 10, 2021
3a5bb3e
CM-1631: Removed obsolete import
yaricom Dec 10, 2021
bead80d
CM-1631: Moved REST API data model classes into implementation detail…
yaricom Dec 10, 2021
9f54b4d
CM-1631: Implemented Project model object in the public API. Fixed re…
yaricom Dec 10, 2021
12012b0
CM-1631: Implemented GitMetaData model object in the public API. Fixe…
yaricom Dec 10, 2021
81732d3
CM-1631: Introduced BaseExperimentObject to hold experiment specific …
yaricom Dec 10, 2021
fc724e1
CM-1631: Introduced Value model object to hold values of metrics, par…
yaricom Dec 10, 2021
3c837a3
CM-1631: Introduced ExperimentAsset data model object to hold values …
yaricom Dec 10, 2021
785cc2a
CM-1631: Fixed Experiment to return correct Gig meta-data object from…
yaricom Dec 10, 2021
3073a7d
CM-1631: Implemented null-safe conversion from Long to Instant and Du…
yaricom Dec 13, 2021
db7ebc5
CM-1631: Fixed missing context.
yaricom Dec 13, 2021
d0cbd41
CM-1631: Added @SuppressWarnings("unused") for all data model classes…
yaricom Dec 13, 2021
1ea4ad1
CM-1631: Updated JavaDoc to provide samples of Artifact types.
yaricom Dec 13, 2021
68b0f41
CM-1631: Implemented conversion from JSON metadata string into Map ob…
yaricom Dec 13, 2021
90d9eb7
CM-1631: Implemented LoggedArtifact interface
yaricom Dec 13, 2021
ec41d09
CM-1631: Implemented upsert and update artifact version
yaricom Dec 13, 2021
180258d
CM-1631: Started logArtifact implementation.
yaricom Dec 13, 2021
d8259dd
CM-1631: Implemented safe conversion from list to set
yaricom Dec 14, 2021
e57acff
CM-1631: Implemented GetArtifactOptions to holds options to be used t…
yaricom Dec 14, 2021
4d05ffb
CM-1631: Implemented utility to convert GetArtifactOptions into GET q…
yaricom Dec 14, 2021
7f3cb8e
CM-1631: Implemented getArtifactVersionDetail in the RestApiClient.
yaricom Dec 14, 2021
ae5e777
CM-1631: Implemented synchronous GET in the Connection which is able …
yaricom Dec 14, 2021
d9d4f2b
CM-1631: Implemented support for Comet web errors by all Connection m…
yaricom Dec 14, 2021
2589f44
CM-1631: Fixed ConflictingArtifactAssetNameException name to include …
yaricom Dec 14, 2021
474eda8
CM-1631: Added toString support to the GetArtifactOptions
yaricom Dec 14, 2021
139af32
CM-1631: Implemented exceptions related to the getArtifactVersionDeta…
yaricom Dec 14, 2021
ea9dbc1
CM-1631: Fixed ArtifactVersionState to have value as final. Fixed Art…
yaricom Dec 15, 2021
e3782b2
CM-1631: Implemented GetArtifactOptionsBuilder factory to create prop…
yaricom Dec 15, 2021
b5e3e85
CM-1631: Implemented getArtifactVersionDetail, introduced ArtifactExc…
yaricom Dec 15, 2021
8a643ba
CM-1631: Implemented logArtifact without uploading artifact assets.
yaricom Dec 15, 2021
54762e5
CM-1631: Added support for logging of the artifact assets.
yaricom Dec 15, 2021
4d27a05
CM-1631: Implemented logArtifact with asynchronous upload of artifact…
yaricom Dec 15, 2021
dca9e98
CM-1631: Fixed ArtifactImpl to have only ArtifactAsset and ArtifactRe…
yaricom Dec 15, 2021
0114ccb
CM-1631: Updated upload message to include number of assets to be upl…
yaricom Dec 15, 2021
97abe7d
CM-1631: Fixed RestApiClient to properly process empty body response …
yaricom Dec 16, 2021
782de84
CM-1631: Added inventory tracking for Assets and Artifacts to guarant…
yaricom Dec 16, 2021
66f59ee
CM-1631: Implemented waiting for experiment inventory to be cleaned u…
yaricom Dec 17, 2021
3fb1283
CM-1631: Fixed Connection.waitAndClose to use Awaitility for waiting …
yaricom Dec 17, 2021
94dd512
CM-1631: Implemented support for throwing exception when some asset(s…
yaricom Dec 17, 2021
e1ef65f
CM-1631: Implemented LoggedArtifact.getArtifactId().
yaricom Dec 17, 2021
5f72252
CM-1631: Implemented test case to test artifact logging.
yaricom Dec 17, 2021
ebe8a00
CM-1631: Implemented ArtifactExample to demonstrate basics of artifac…
yaricom Dec 17, 2021
b398153
CM-1631: Removed obsolete import
yaricom Dec 17, 2021
cd95c00
CM-1631: Removed double brace initialization
yaricom Dec 17, 2021
f129bbf
CM-1631: Fixed to avoid linter complains.
yaricom Dec 17, 2021
f7933bf
CM-1631: Refactored Asset data types to be more consistent.
yaricom Dec 20, 2021
4e73583
CM-1631: Removed obsolete imports
yaricom Dec 20, 2021
2dea450
CM-1631: Fixed missing artifactVersionId in the assets of the artifac…
yaricom Dec 20, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
package ml.comet.examples;

import ml.comet.experiment.OnlineExperiment;
import ml.comet.experiment.artifact.Artifact;
import ml.comet.experiment.artifact.LoggedArtifact;

import java.net.URI;
import java.nio.charset.StandardCharsets;
import java.nio.file.Path;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CompletableFuture;

import static java.util.concurrent.TimeUnit.SECONDS;
import static ml.comet.examples.Utils.getResourceFile;
import static ml.comet.experiment.ExperimentBuilder.OnlineExperiment;

/**
* Provides examples of working with Comet artifact.
*
* <p>To run from command line execute the following at the root of this module:
* <pre>
* COMET_API_KEY=your_api_key \
* COMET_WORKSPACE_NAME=your_workspace \
* COMET_PROJECT_NAME=your_project_name \
* mvn exec:java -Dexec.mainClass="ml.comet.examples.ArtifactExample"
* </pre>
* Make sure to provide correct values above.
*/
public class ArtifactExample implements BaseExample {
private static final Map<String, Object> SOME_METADATA = new HashMap<>();

static {
SOME_METADATA.put("someInt", 10);
SOME_METADATA.put("someString", "test string");
SOME_METADATA.put("someBoolean", true);
}

/**
* The main entry point to the example.
*
* @param args the command line arguments if any.
*/
public static void main(String[] args) throws Exception {
try (OnlineExperiment experiment = OnlineExperiment().interceptStdout().build()) {
ArtifactExample.run(experiment);
}
}

private static void run(OnlineExperiment experiment) throws Exception {
experiment.setExperimentName("Artifact Example");

List<String> aliases = Arrays.asList("alias1", "alias2");
List<String> tags = Arrays.asList("tag1", "tag2");
String artifactName = "someArtifact";
String artifactType = "someType";
Artifact artifact = Artifact
.newArtifact(artifactName, artifactType)
.withAliases(aliases)
.withVersionTags(tags)
.withMetadata(SOME_METADATA)
.build();

// add remote assets
//
URI firstAssetLink = new URI("s3://bucket/folder/firstAssetFile.extension");
String firstAssetFileName = "firstAssetFileName";
artifact.addRemoteAsset(firstAssetLink, firstAssetFileName);

String secondAssetExpectedFileName = "secondAssetFile.extension";
URI secondAssetLink = new URI("s3://bucket/folder/" + secondAssetExpectedFileName);
artifact.addRemoteAsset(secondAssetLink, secondAssetExpectedFileName);

// add local assets
//
artifact.addAsset(getResourceFile(CHART_IMAGE_FILE), "amazing chart.png", false, SOME_METADATA);
artifact.addAsset(getResourceFile(MODEL_FILE), false);
byte[] someData = "some data".getBytes(StandardCharsets.UTF_8);
String someDataName = "someDataName";
artifact.addAsset(someData, someDataName);

// add assets folder
//
Path assetDir = BaseExample.copyResourcesToTmpDir();
artifact.addAssetFolder(assetDir.toFile(), true, true);

// log artifact
//
CompletableFuture<LoggedArtifact> futureArtifact = experiment.logArtifact(artifact);
LoggedArtifact loggedArtifact = futureArtifact.get(60, SECONDS);

System.out.printf("\n\nArtifact upload complete, new artifact version: %s\n\n", loggedArtifact.getVersion());

System.out.println("===== Experiment completed ====");
}
}
64 changes: 64 additions & 0 deletions comet-examples/src/main/java/ml/comet/examples/BaseExample.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
package ml.comet.examples;

import ml.comet.experiment.OnlineExperiment;
import org.apache.commons.io.file.PathUtils;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Objects;

import static ml.comet.examples.Utils.getResourceFile;
import static ml.comet.examples.Utils.readResourceToString;

/**
* The definition of common methods and constants to be used by examples.
*/
interface BaseExample {
String CHART_IMAGE_FILE = "chart.png";
String MODEL_FILE = "model.hd5";
String HTML_REPORT_FILE = "report.html";
String GRAPH_JSON_FILE = "graph.json";
String CODE_FILE = "code_sample.py";

static void generateCharts(OnlineExperiment experiment) {
long currentStep = experiment.getStep();

for (int i = 1; i < 15; i++) {
experiment.logMetric("numMetric", 123 + i, currentStep + i, getUpdatedEpochValue(experiment));
}

for (int i = 1; i < 15; i++) {
experiment.logMetric("strMetric", "123" + i, currentStep + i, getUpdatedEpochValue(experiment));
}

for (int i = 1; i < 15; i++) {
experiment.logMetric("doubleMetric", 123.12d + i, currentStep + i, getUpdatedEpochValue(experiment));
}
}

static String generateCustomHtmlReport() throws IOException {
return readResourceToString("report.html");
}

static long getUpdatedEpochValue(OnlineExperiment experiment) {
return experiment.getEpoch() + experiment.getStep() / 5;
}

static Path copyResourcesToTmpDir() throws IOException {
Path root = Files.createTempDirectory("onlineExperimentExample");
PathUtils.copyFileToDirectory(
Objects.requireNonNull(getResourceFile(CHART_IMAGE_FILE)).toPath(), root);
PathUtils.copyFileToDirectory(
Objects.requireNonNull(getResourceFile(MODEL_FILE)).toPath(), root);
Files.createTempFile(root, "empty_file", ".txt");

Path subDir = Files.createTempDirectory(root, "subDir");
PathUtils.copyFileToDirectory(
Objects.requireNonNull(getResourceFile(HTML_REPORT_FILE)).toPath(), subDir);
PathUtils.copyFileToDirectory(
Objects.requireNonNull(getResourceFile(GRAPH_JSON_FILE)).toPath(), subDir);

return root;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,14 @@
import ml.comet.experiment.context.ExperimentContext;
import org.apache.commons.io.file.PathUtils;

import java.io.IOException;
import java.net.URI;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Objects;

import static ml.comet.examples.Utils.getResourceFile;
import static ml.comet.examples.Utils.readResourceToString;

/**
* Provides variety of example logging using OnlineExperiment.
* Provides variety of examples of data logging using OnlineExperiment.
*
* <p>To run from command line execute the following at the root of this module:
* <pre>
Expand All @@ -26,20 +23,14 @@
* </pre>
* Make sure to provide correct values above.
*/
public class OnlineExperimentExample {

private static final String CHART_IMAGE_FILE = "chart.png";
private static final String MODEL_FILE = "model.hd5";
private static final String HTML_REPORT_FILE = "report.html";
private static final String GRAPH_JSON_FILE = "graph.json";
private static final String CODE_FILE = "code_sample.py";
public class OnlineExperimentExample implements BaseExample {

/**
* The main entry point to the example.
*
* @param args the command line arguments if any.
*/
public static void main(String[] args) {
public static void main(String[] args) throws Exception {

//this will take configs from /comet-java-sdk/comet-examples/src/main/resources/application.conf
//be sure you have set up apiKey, project, workspace in defaults.conf before you start!
Expand All @@ -54,8 +45,6 @@ public static void main(String[] args) {

try {
OnlineExperimentExample.run(experiment);
} catch (Throwable e) {
e.printStackTrace();
} finally {
experiment.end();
}
Expand All @@ -73,11 +62,11 @@ private static void run(OnlineExperiment experiment) throws Exception {

experiment.setEpoch(3);

generateCharts(experiment);
BaseExample.generateCharts(experiment);

experiment.setStep(1234);

experiment.logHtml(generateCustomHtmlReport(), false);
experiment.logHtml(BaseExample.generateCustomHtmlReport(), false);

experiment.logParameter("batch_size", "500");
experiment.logParameter("learning_rate", 12);
Expand All @@ -92,7 +81,7 @@ private static void run(OnlineExperiment experiment) throws Exception {

// upload asset files from folder
//
Path assetDir = copyResourcesToTmpDir();
Path assetDir = BaseExample.copyResourcesToTmpDir();
experiment.logAssetFolder(assetDir.toFile(), true, true);

// log remote assets
Expand All @@ -117,45 +106,4 @@ private static void run(OnlineExperiment experiment) throws Exception {
// remove tmp directory
PathUtils.deleteDirectory(assetDir);
}

private static void generateCharts(OnlineExperiment experiment) {
long currentStep = experiment.getStep();

for (int i = 1; i < 15; i++) {
experiment.logMetric("numMetric", 123 + i, currentStep + i, getUpdatedEpochValue(experiment));
}

for (int i = 1; i < 15; i++) {
experiment.logMetric("strMetric", "123" + i, currentStep + i, getUpdatedEpochValue(experiment));
}

for (int i = 1; i < 15; i++) {
experiment.logMetric("doubleMetric", 123.12d + i, currentStep + i, getUpdatedEpochValue(experiment));
}
}

private static String generateCustomHtmlReport() throws IOException {
return readResourceToString("report.html");
}

private static long getUpdatedEpochValue(OnlineExperiment experiment) {
return experiment.getEpoch() + experiment.getStep() / 5;
}

private static Path copyResourcesToTmpDir() throws IOException {
Path root = Files.createTempDirectory("onlineExperimentExample");
PathUtils.copyFileToDirectory(
Objects.requireNonNull(getResourceFile(CHART_IMAGE_FILE)).toPath(), root);
PathUtils.copyFileToDirectory(
Objects.requireNonNull(getResourceFile(MODEL_FILE)).toPath(), root);
Files.createTempFile(root, "empty_file", ".txt");

Path subDir = Files.createTempDirectory(root, "subDir");
PathUtils.copyFileToDirectory(
Objects.requireNonNull(getResourceFile(HTML_REPORT_FILE)).toPath(), subDir);
PathUtils.copyFileToDirectory(
Objects.requireNonNull(getResourceFile(GRAPH_JSON_FILE)).toPath(), subDir);

return root;
}
}
2 changes: 1 addition & 1 deletion comet-examples/src/main/java/ml/comet/examples/Utils.java
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import java.util.Scanner;

/**
* Provides common utilitites used bby examples.
* Provides common utilities used by examples.
*/
public class Utils {

Expand Down
16 changes: 10 additions & 6 deletions comet-java-client/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,16 @@
<artifactId>jackson-databind</artifactId>
<version>2.13.0</version>
</dependency>
<dependency>
<groupId>com.vdurmont</groupId>
<artifactId>semver4j</artifactId>
<version>3.1.0</version>
</dependency>
<dependency>
<groupId>org.awaitility</groupId>
<artifactId>awaitility</artifactId>
<version>4.1.1</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
Expand All @@ -81,12 +91,6 @@
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.awaitility</groupId>
<artifactId>awaitility</artifactId>
<version>4.1.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-inline</artifactId>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package ml.comet.experiment;

import ml.comet.experiment.model.ExperimentMetadataRest;
import ml.comet.experiment.model.RestProject;
import ml.comet.experiment.model.ExperimentMetadata;
import ml.comet.experiment.model.Project;

import java.io.Closeable;
import java.util.List;
Expand All @@ -26,13 +26,13 @@ public interface CometApi extends Closeable {
* @param workspaceName workspace name
* @return List of project DTOs
*/
List<RestProject> getAllProjects(String workspaceName);
List<Project> getAllProjects(String workspaceName);

/**
* Gets all experiment DTOs under specified project id.
*
* @param projectId Project id
* @return List of experiment DTOs
*/
List<ExperimentMetadataRest> getAllExperiments(String projectId);
List<ExperimentMetadata> getAllExperiments(String projectId);
}
Loading