Add a DNN Image Featurizer Transform Estimator #1447

vaeksare · 2018-10-30T00:37:49Z

Adds a new estimator for doing pretrained DNN model image featurization as a transform. This estimator calls 2 ONNX transforms in a chain in order to achieve the desired result. The first ONNX model does the preprocessing while the second one is the actual pretrained DNN. A total of 4 DNN models are available:

AlexNet
ResNet18
ResNet50
ResNet101

Each model is available through its own project and a related extension method. Building that project automatically downloads the model from a CDN. The model is included with the NuGet package produced by that project when the code is distributed. Fixes #1232

justinormont · 2018-11-01T04:35:53Z

In addition to the estimator, would it be possible make the transform callable from the command-line and GUIs? Perhaps the shell of code could be copied from the internal repo and your new implementation replaces its innards?

vaeksare · 2018-11-01T17:04:10Z

I believe command-line is expected to be changed to use the estimators as a base in the near future. And ML.NET itself currently does not have a GUI. So adding an actual transform is certainly possible, but I don't think it is actually very useful.

In reply to: 434928031 [](ancestors = 434928031)

test/Microsoft.ML.OnnxTransformTest/DnnImageFeaturizerTest.cs

wschin · 2018-11-06T19:13:42Z

test/Microsoft.ML.OnnxTransformTest/DnnImageFeaturizerTest.cs

+                    while (cursor.MoveNext())
+                    {
+                        getter(ref buffer);
+                        Assert.Equal(512, buffer.Length);


Need more strict asserts like checking the prediction values. #Resolved

See the new OldSavingAndLoading test which contains some exact output value checks.

In reply to: 231254842 [](ancestors = 231254842)

src/Microsoft.ML.OnnxTransform/DnnImageFeaturizerTransform.cs

test/Microsoft.ML.OnnxTransformTest/DnnImageFeaturizerTest.cs

…-feat" This reverts commit 22e385b.

eerhardt · 2018-11-26T19:07:56Z

src/Microsoft.ML.DnnImageFeaturizer.AlexNet/AlexNet.cs

+    /// This is an extension method to be used with the <see cref="DnnImageFeaturizerEstimator"/> in order to use a pretrained AlexNet model.
+    /// The NuGet containing this extension is also guaranteed to include the binary model file.
+    /// </summary>
+    public static class AlexNetExtension


AlexNetExtension [](start = 24, length = 16)

(nit) it is nice to keep the class name and the containing file name in sync.

AlexNetExtension vs AlexNet.cs #Resolved

eerhardt · 2018-11-26T19:14:20Z

...Microsoft.ML.DnnImageFeaturizer.ModelRedist/Microsoft.ML.DnnImageFeaturizer.ModelRedist.proj

+  <UsingTask TaskName="DownloadFilesFromUrl" AssemblyFile="$(ToolsDir)Microsoft.DotNet.Build.Tasks.dll"/>
+
+  <PropertyGroup>
+    <ModelPlacementDir>$(RepoRoot)bin\obj\DnnImageModels</ModelPlacementDir>


Use $(ObjDir) instead of $(RepoRoot)bin\obj #Closed

eerhardt · 2018-11-26T19:16:35Z

src/Microsoft.ML.OnnxTransform/DnnImageFeaturizerTransform.cs

+    /// </summary>
+    public sealed class DnnImageFeaturizerInput
+    {
+        public IHostEnvironment Env { get; }


Usually we don't use contractions in public API - Env.

see https://docs.microsoft.com/en-us/dotnet/standard/design-guidelines/general-naming-conventions#using-abbreviations-and-acronyms #Closed

Thanks, renamed it!

In reply to: 236390424 [](ancestors = 236390424)

eerhardt · 2018-11-27T17:08:33Z

pkg/common/DnnImageFeaturizer.targets

+    <Content Include="$(MSBuildThisFileDirectory)..\..\tools\DnnImageFeaturizer\**\*.*">
+      <Link>DnnImageModels\%(RecursiveDir)%(Filename)%(Extension)</Link>
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+      <CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>


You don't need CopyToPublishDirectory. It gets defaulted to CopyToOutputDirectory when it isn't set. #Resolved

eerhardt · 2018-11-27T17:08:46Z

pkg/common/DnnImageFeaturizer.targets

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="utf-8"?>


I think this should be a .props file instead.

The difference is that a .props file gets imported BEFORE an importing .csproj gets executed. Which means that if a user wanted to modify the Content item below, they can in their .csproj without writing a <Target> #Resolved

Fair enough, changed it, thanks!

In reply to: 236760593 [](ancestors = 236760593)

eerhardt · 2018-11-27T17:09:23Z

pkg/common/DnnImageFeaturizer.targets

+<?xml version="1.0" encoding="utf-8"?>
+<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
+  <ItemGroup>
+    <Content Include="$(MSBuildThisFileDirectory)..\..\tools\DnnImageFeaturizer\**\*.*">


Why is this folder named DnnImageFeaturizer, but the Link uses DnnImageModels? I think they should be consistent. #Resolved

Renamed for consistency.

In reply to: 236760874 [](ancestors = 236760874)

eerhardt · 2018-11-27T17:15:24Z

...Microsoft.ML.DnnImageFeaturizer.ModelRedist/Microsoft.ML.DnnImageFeaturizer.ModelRedist.proj

+    DestinationDir="$(ModelPlacementDir)\ResNet101Onnx" />
+  </ItemGroup>
+
+  <Target Name="DownloadDnnModelFiles" 


This looks much cleaner. Nice work.

eerhardt · 2018-11-27T18:05:01Z

pkg/common/DnnImageFeaturizer.props

@@ -0,0 +1,9 @@
+<?xml version="1.0" encoding="utf-8"?>
+<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">


(nit) you can drop DefaultTargets="Build" it doesn't make sense here. (and doesn't do anything)

eerhardt

This looks good. Thanks for the great work!

vaeksare mentioned this pull request Oct 30, 2018

Pretrained DNN Image Featurization #1232

Closed

vaeksare self-assigned this Oct 30, 2018

vaeksare requested review from wschin, Ivanidzo4ka, Zruty0 and justinormont November 2, 2018 19:52

wschin reviewed Nov 6, 2018

View reviewed changes

test/Microsoft.ML.OnnxTransformTest/DnnImageFeaturizerTest.cs Show resolved Hide resolved

wschin reviewed Nov 6, 2018

View reviewed changes

src/Microsoft.ML.OnnxTransform/DnnImageFeaturizerTransform.cs Outdated Show resolved Hide resolved

wschin reviewed Nov 6, 2018

View reviewed changes

src/Microsoft.ML.OnnxTransform/DnnImageFeaturizerTransform.cs Outdated Show resolved Hide resolved

wschin reviewed Nov 7, 2018

View reviewed changes

test/Microsoft.ML.OnnxTransformTest/DnnImageFeaturizerTest.cs Outdated Show resolved Hide resolved

vaeksare added 17 commits November 8, 2018 13:46

Skeleton for implementing with a new tranformer

bc71575

Approach using a chain of transformers

32dc90c

Reworked to use estimator chain

38fecff

Add basic testing

5e1faa5

Reworked to use extension methods

3c3aa2a

Create more extensive tests

66b621e

Remove changes to onnx transform tests

87a5d51

Rework to use CDN approach

5bec174

Add static extensions

8876f02

Added test for static pipeline

2630f7b

Updated to use correct CDN location

4a38bf9

Added some minor comments

75b1b0d

Use extension methods and set up the model download/nuget placement

271ae03

Change the proj to try to fix download

2e5ef2d

Fix the download in csproj

85ca884

Models successfully download into correct folder

b76deda

Include the onnx models in the nuget package

259ac05

vaeksare added 8 commits November 21, 2018 10:30

Address some comments

1bebc64

Merge remote-tracking branch 'upstream/master' into dnn-image-feat

22e385b

Revert "Merge remote-tracking branch 'upstream/master' into dnn-image…

8f9e808

…-feat" This reverts commit 22e385b.

Merge remote-tracking branch 'upstream/master' into dnn-image-feat

a1e571d

updated libmf ref

55b5b0f

Updated to fix issues related to OnnxTransform changes

7405fce

Merge remote-tracking branch 'upstream/master' into dnn-image-feat

ce93709

Update to new namespace

02e8ebc

eerhardt reviewed Nov 26, 2018

View reviewed changes

vaeksare added 6 commits November 26, 2018 11:49

Address more comments

c519a19

Reorganize model downloading

a687ccb

Fix test downloading

52a3e6a

Removed model duplication in NuGets

9f566a2

Updated targets location

d33eb06

Fixed issues with target content inclusion

3e4db44

eerhardt reviewed Nov 27, 2018

View reviewed changes

Change targets file to be consistent

5fbd41d

eerhardt reviewed Nov 27, 2018

View reviewed changes

vaeksare added 2 commits November 27, 2018 09:21

Fix bug due to targets

bdc9f60

Simplify model copy in tests

8effaf9

eerhardt reviewed Nov 27, 2018

View reviewed changes

eerhardt approved these changes Nov 27, 2018

View reviewed changes

Address minor comment

b27ebe2

vaeksare merged commit f92c90d into dotnet:master Nov 27, 2018

vaeksare deleted the dnn-image-feat branch November 27, 2018 18:36

ghost locked as resolved and limited conversation to collaborators Mar 27, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a DNN Image Featurizer Transform Estimator #1447

Add a DNN Image Featurizer Transform Estimator #1447

vaeksare commented Oct 30, 2018 •

edited

Loading

justinormont commented Nov 1, 2018

vaeksare commented Nov 1, 2018

wschin Nov 6, 2018 •

edited by vaeksare

Loading

vaeksare Nov 9, 2018

eerhardt Nov 26, 2018 •

edited by vaeksare

Loading

eerhardt Nov 26, 2018 •

edited

Loading

eerhardt Nov 26, 2018 •

edited

Loading

vaeksare Nov 26, 2018

eerhardt Nov 27, 2018 •

edited by vaeksare

Loading

eerhardt Nov 27, 2018 •

edited by vaeksare

Loading

vaeksare Nov 27, 2018

eerhardt Nov 27, 2018 •

edited by vaeksare

Loading

vaeksare Nov 27, 2018

eerhardt Nov 27, 2018

eerhardt Nov 27, 2018

eerhardt left a comment

		@@ -0,0 +1,9 @@
		<?xml version="1.0" encoding="utf-8"?>
		<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

Add a DNN Image Featurizer Transform Estimator #1447

Add a DNN Image Featurizer Transform Estimator #1447

Conversation

vaeksare commented Oct 30, 2018 • edited Loading

justinormont commented Nov 1, 2018

vaeksare commented Nov 1, 2018

wschin Nov 6, 2018 • edited by vaeksare Loading

Choose a reason for hiding this comment

vaeksare Nov 9, 2018

Choose a reason for hiding this comment

eerhardt Nov 26, 2018 • edited by vaeksare Loading

Choose a reason for hiding this comment

eerhardt Nov 26, 2018 • edited Loading

Choose a reason for hiding this comment

eerhardt Nov 26, 2018 • edited Loading

Choose a reason for hiding this comment

vaeksare Nov 26, 2018

Choose a reason for hiding this comment

eerhardt Nov 27, 2018 • edited by vaeksare Loading

Choose a reason for hiding this comment

eerhardt Nov 27, 2018 • edited by vaeksare Loading

Choose a reason for hiding this comment

vaeksare Nov 27, 2018

Choose a reason for hiding this comment

eerhardt Nov 27, 2018 • edited by vaeksare Loading

Choose a reason for hiding this comment

vaeksare Nov 27, 2018

Choose a reason for hiding this comment

eerhardt Nov 27, 2018

Choose a reason for hiding this comment

eerhardt Nov 27, 2018

Choose a reason for hiding this comment

eerhardt left a comment

Choose a reason for hiding this comment

vaeksare commented Oct 30, 2018 •

edited

Loading

wschin Nov 6, 2018 •

edited by vaeksare

Loading

eerhardt Nov 26, 2018 •

edited by vaeksare

Loading

eerhardt Nov 26, 2018 •

edited

Loading

eerhardt Nov 26, 2018 •

edited

Loading

eerhardt Nov 27, 2018 •

edited by vaeksare

Loading

eerhardt Nov 27, 2018 •

edited by vaeksare

Loading

eerhardt Nov 27, 2018 •

edited by vaeksare

Loading