Create, save, load and run models using a new functional API #112

karllessard · 2020-09-11T20:14:16Z

Introducing a new ConcreteFunction class, which allows you to create graphs, load/save models and run inference using the same functional API. See documentation in this PR for some examples.

Exporting SavedModels only supports session-centric models, i.e. models that has a single main graph. This means that if multiple functions are saved with the model, they should all share the same graph for now. That limitation will be addressed later.

CC\ @Shajan

googlebot · 2020-09-11T20:14:19Z

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

karllessard · 2020-09-11T20:15:25Z

@Shajan , I think you need to consent to the CLA, see message above, thank you

Shajan · 2020-09-11T21:05:25Z

@googlebot I consent.

googlebot · 2020-09-11T21:05:29Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

Craigacp

I've noted a few small things, but overall it looks good.

Craigacp · 2020-09-11T21:02:05Z

tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/ConcreteFunction.java

+      runner.feed(t.getName(), tensor);
+    });
+
+    Map<String, TensorInfo> outputToNode = signatureDef.getOutputsMap();


Are we guaranteed that the iteration order of the signature's outputs map is consistent?

Good observation, the ordering of signature's output map need not be consistent.

Here is why:

We depend on the ordering of output returned in the call List<Tensor<?>> resultTensors = runner.run(). Which depends on the order we pass in at runner.fetch(t.getName()), that happens to be the iteration order of signatureDef.getOutputsMap().

In runHelper, See creation of outputOpIndices based on order of outputs.add(output), which gets passed into Session.run(..)

Session.run uses outputOpIndices as the order of output, which is implemented in resolveOutputs and TF_Output

So it will be consistent within an execution of call but could change across different VM executions if the signatureDef.getOutputsMap() is backed by a regular HashMap rather than a LinkedHashMap?

I ask because we have a tendency to iterate the value set of maps in demo code, and I don't want users to think those will be consistent if they aren't. Usually it's because there is only a single output, but if we're not maintaining ordering then people might be confused if they try to apply what we do in the demos/tests to larger problems.

The order we fetch the tensors does not really matter, as we don't return the outputs as a list but as a map, forcing the user to retrieve them by name instead of by ordinal position, like Session.Run does. And since this is not demo code neither, I think it is safe enough to leave it as is.

Is this really correct? One line does outputToNode.values().forEach(t -> runner.fetch(t.getName())); and the other for (String nodeName: outputToNode.keySet()) { - is there any guarantee of consistent ordering between values() and keySet()!?

The code very much depends on this and there's no test to confirm (ConcreteFunctionTest.java only uses the simpler call()-method).

Protobuf uses LinkedHashMap internally, so yeah it's fine. Wouldn't be a bad idea to convert it to one manually though.

Maybe use .values() in both to raise fewer red flags? Though relying on a Map producing the same values() order is perhaps also questionable?

Not if it's a LinkedHashMap, that's the point of the class, it provides stable iteration over a Map with minimal overhead.

tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/ConcreteFunction.java

tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/SavedModelBundle.java

tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Signature.java

tensorflow-core/tensorflow-core-api/src/test/java/org/tensorflow/ConcreteFunctionTest.java

Craigacp · 2020-09-11T22:15:03Z

tensorflow-core/tensorflow-core-api/src/test/resources/saved_model_using_python/readme.txt

@@ -0,0 +1,45 @@
+Saved model created using Python @tf.function, tensorflow version 2.3.0.
+
+Python code used to create saved model below:


Should we just check this in as a python file?

@karllessard any guidance on having python files in the java repo?

@Shajan , what do you think? I think it could be a Python file, as long as we add a notice on top of that the file is only there for reference and is not being used anywhere in the code base.

I've moved forward and changed the file type to Python, please let me know if you think of any reason I should revert it to a text file.

Python models that contain tf.function is inconvenient to be consumed by Java clients. This proposal provides an API to (a) Invoke a tf.function, given the signature name (b) Retrieve the node name in the graph corresponding to a tf.function Co-authored-by: Shajan Dasan <[email protected]> Save models as functions (#103) * Draft: Java API to use tf.function available on SavedModel. (#89) Python models that contain tf.function is inconvenient to be consumed by Java clients. This proposal provides an API to (a) Invoke a tf.function, given the signature name (b) Retrieve the node name in the graph corresponding to a tf.function Co-authored-by: Shajan Dasan <[email protected]> * Change API for creating concrete functions and exporting them to a saved model Co-authored-by: Karl Lessard <[email protected]> Rename signature name to key Print function signature when converting to String Add method that returns the signature of all functions in a saved model Add unit tests for python created SavedModel with tf.function

Craigacp · 2020-09-17T19:03:23Z

tensorflow-core/tensorflow-core-api/src/test/resources/saved_model_using_python/source_model.py

@@ -0,0 +1,48 @@
+# Saved model created using Python @tf.function, tensorflow version 2.3.0.


This should have a copyright statement at the top.

Craigacp

LGTM

Shajan · 2020-09-15T05:17:11Z

tensorflow-core/tensorflow-core-api/src/test/resources/saved_model_using_python/readme.txt

@@ -0,0 +1,45 @@
+Saved model created using Python @tf.function, tensorflow version 2.3.0.
+
+Python code used to create saved model below:


@karllessard any guidance on having python files in the java repo?

Craigacp reviewed Sep 11, 2020

View reviewed changes

Shajan and others added 2 commits September 15, 2020 20:40

Add validations on signatures and saved models

8cd74fc

karllessard force-pushed the shared-saved-model branch from 09c136e to 8cd74fc Compare September 16, 2020 02:35

Convert text file to Python

2bb4622

Craigacp reviewed Sep 17, 2020

View reviewed changes

Add copyright on Python sample

fc68d3a

Craigacp approved these changes Sep 17, 2020

View reviewed changes

Shajan approved these changes Sep 17, 2020

View reviewed changes

Craigacp merged commit 1d35c17 into master Sep 17, 2020

karllessard deleted the shared-saved-model branch March 13, 2021 15:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create, save, load and run models using a new functional API #112

Create, save, load and run models using a new functional API #112

karllessard commented Sep 11, 2020

googlebot commented Sep 11, 2020

karllessard commented Sep 11, 2020

Shajan commented Sep 11, 2020

googlebot commented Sep 11, 2020

Craigacp left a comment

Craigacp Sep 11, 2020

Shajan Sep 13, 2020 •

edited

Loading

Craigacp Sep 14, 2020

karllessard Sep 16, 2020

jxtps Feb 2, 2021

rnett Feb 3, 2021

jxtps Feb 3, 2021

rnett Feb 3, 2021

Craigacp Sep 11, 2020

Shajan Sep 15, 2020

karllessard Sep 16, 2020

karllessard Sep 16, 2020

Craigacp Sep 17, 2020

Craigacp left a comment

Shajan Sep 15, 2020

		@@ -0,0 +1,45 @@
		Saved model created using Python @tf.function, tensorflow version 2.3.0.

		Python code used to create saved model below:

		@@ -0,0 +1,48 @@
		# Saved model created using Python @tf.function, tensorflow version 2.3.0.

Create, save, load and run models using a new functional API #112

Create, save, load and run models using a new functional API #112

Conversation

karllessard commented Sep 11, 2020

googlebot commented Sep 11, 2020

karllessard commented Sep 11, 2020

Shajan commented Sep 11, 2020

googlebot commented Sep 11, 2020

Craigacp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Shajan Sep 13, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Craigacp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Shajan Sep 13, 2020 •

edited

Loading