Add a DNN Image Featurizer Transform Estimator #1447
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk" DefaultTargets="Pack"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>netstandard2.0</TargetFramework> | ||
| <PackageDescription>ML.NET component for pretrained AlexNet image featurization</PackageDescription> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\Microsoft.ML.OnnxTransform\Microsoft.ML.OnnxTransform.nupkgproj" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <Content Include="$(RepoRoot)\bin\obj\DnnImageModels\AlexNetOnnx\AlexNet.onnx" Pack="true" PackagePath="contentFiles\any\any\DnnImageModels\AlexNetOnnx;content\DnnImageModels\AlexNetOnnx" /> | ||
|
There was a problem hiding this comment. Don't use Instead, these files should be placed to See how we are downloading and extracting the Tensorflow binaries: https://github.com/dotnet/machinelearning/blob/master/src/Redist/Microsoft.ML.TensorFlow.Redist/Microsoft.ML.TensorFlow.Redist.proj#L105 There was a problem hiding this comment.
We shouldn't be duplicating the files in the .nupkg. This will lead to larger disk usages, since the file will be duplicated. #Resolved There was a problem hiding this comment. From what I was reading in the documentation, it seemed like depending on how the person uses/includes the NuGet, it might look for the files either in the content directory or in the contentFiles directory. I noticed, for example, that creating a simple test app through VS it did not seem to find the files if they were only in contentFiles. And searching online made it seem like there is no easy solution to not replicate them? In reply to: 234754082 [](ancestors = 234754082) |
||
| <Content Include="$(RepoRoot)\bin\obj\DnnImageModels\AlexNetPrepOnnx\AlexNetPreprocess.onnx" Pack="true" PackagePath="contentFiles\any\any\DnnImageModels\AlexNetPrepOnnx;content\DnnImageModels\AlexNetPrepOnnx" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| <Project DefaultTargets="Pack"> | ||
|
|
||
| <Import Project="Microsoft.ML.DnnImageFeaturizer.AlexNet.nupkgproj" /> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk" DefaultTargets="Pack"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>netstandard2.0</TargetFramework> | ||
| <PackageDescription>ML.NET component for pretrained ResNet101 image featurization</PackageDescription> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\Microsoft.ML.OnnxTransform\Microsoft.ML.OnnxTransform.nupkgproj" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <Content Include="$(RepoRoot)\bin\obj\DnnImageModels\ResNet101Onnx\ResNet101.onnx" Pack="true" PackagePath="contentFiles\any\any\DnnImageModels\ResNet101Onnx;content\DnnImageModels\ResNet101Onnx" /> | ||
| <Content Include="$(RepoRoot)\bin\obj\DnnImageModels\ResNetPrepOnnx\ResNetPreprocess.onnx" Pack="true" PackagePath="contentFiles\any\any\DnnImageModels\ResNetPrepOnnx;content\DnnImageModels\ResNetPrepOnnx" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| <Project DefaultTargets="Pack"> | ||
|
|
||
| <Import Project="Microsoft.ML.DnnImageFeaturizer.ResNet101.nupkgproj" /> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk" DefaultTargets="Pack"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>netstandard2.0</TargetFramework> | ||
| <PackageDescription>ML.NET component for pretrained ResNet18 image featurization</PackageDescription> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\Microsoft.ML.OnnxTransform\Microsoft.ML.OnnxTransform.nupkgproj" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <Content Include="$(RepoRoot)\bin\obj\DnnImageModels\ResNet18Onnx\ResNet18.onnx" Pack="true" PackagePath="contentFiles\any\any\DnnImageModels\ResNet18Onnx;content\DnnImageModels\ResNet18Onnx" /> | ||
| <Content Include="$(RepoRoot)\bin\obj\DnnImageModels\ResNetPrepOnnx\ResNetPreprocess.onnx" Pack="true" PackagePath="contentFiles\any\any\DnnImageModels\ResNetPrepOnnx;content\DnnImageModels\ResNetPrepOnnx" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| <Project DefaultTargets="Pack"> | ||
|
|
||
| <Import Project="Microsoft.ML.DnnImageFeaturizer.ResNet18.nupkgproj" /> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk" DefaultTargets="Pack"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>netstandard2.0</TargetFramework> | ||
| <PackageDescription>ML.NET component for pretrained ResNet50 image featurization</PackageDescription> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\Microsoft.ML.OnnxTransform\Microsoft.ML.OnnxTransform.nupkgproj" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <Content Include="$(RepoRoot)\bin\obj\DnnImageModels\ResNet50Onnx\ResNet50.onnx" Pack="true" PackagePath="contentFiles\any\any\DnnImageModels\ResNet50Onnx;content\DnnImageModels\ResNet50Onnx" /> | ||
| <Content Include="$(RepoRoot)\bin\obj\DnnImageModels\ResNetPrepOnnx\ResNetPreprocess.onnx" Pack="true" PackagePath="contentFiles\any\any\DnnImageModels\ResNetPrepOnnx;content\DnnImageModels\ResNetPrepOnnx" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| <Project DefaultTargets="Pack"> | ||
|
|
||
| <Import Project="Microsoft.ML.DnnImageFeaturizer.ResNet50.nupkgproj" /> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
| // See the LICENSE file in the project root for more information. | ||
|
|
||
| using Microsoft.ML.Runtime; | ||
| using Microsoft.ML.Runtime.Data; | ||
| using System.IO; | ||
| using System.Reflection; | ||
|
|
||
| namespace Microsoft.ML.Transforms | ||
| { | ||
| /// <summary> | ||
| /// This is an extension method to be used with the <see cref="DnnImageFeaturizerEstimator"/> in order to use a pretrained AlexNet model. | ||
| /// The NuGet containing this extension is also guaranteed to include the binary model file. | ||
| /// </summary> | ||
| public static class AlexNetExtension | ||
|
There was a problem hiding this comment.
(nit) it is nice to keep the class name and the containing file name in sync.
|
||
| { | ||
| /// <summary> | ||
| /// Returns an estimator chain with the two corresponding models (a preprocessing one and a main one) required for the AlexNet pipeline. | ||
| /// Also includes the renaming ColumnsCopyingTransforms required to be able to use arbitrary input and output column names. | ||
| /// This assumes both of the models are in the same location as the file containing this method, which they will be if used through the NuGet. | ||
| /// This should be the default way to use AlexNet if importing the model from a NuGet. | ||
| /// </summary> | ||
| public static EstimatorChain<ColumnsCopyingTransformer> AlexNet(this DnnImageModelSelector dnnModelContext, IHostEnvironment env, string inputColumn, string outputColumn) | ||
| { | ||
| return AlexNet(dnnModelContext, env, inputColumn, outputColumn, Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location), "DnnImageModels")); | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// This allows a custom model location to be specified. This is useful is a custom model is specified, | ||
| /// or if the model is desired to be placed or shipped separately in a different folder from the main application. Note that because Onnx models | ||
| /// must be in a directory all by themsleves for the OnnxTransform to work, this method appends a AlexNetOnnx/AlexNetPrepOnnx subdirectory | ||
| /// to the passed in directory to prevent having to make that directory manually each time. | ||
| /// </summary> | ||
| public static EstimatorChain<ColumnsCopyingTransformer> AlexNet(this DnnImageModelSelector dnnModelContext, IHostEnvironment env, string inputColumn, string outputColumn, string modelDir) | ||
| { | ||
| var modelChain = new EstimatorChain<ColumnsCopyingTransformer>(); | ||
|
|
||
| var inputRename = new ColumnsCopyingEstimator(env, new[] { (inputColumn, "OriginalInput") }); | ||
| var midRename = new ColumnsCopyingEstimator(env, new[] { ("PreprocessedInput", "Input140") }); | ||
| var endRename = new ColumnsCopyingEstimator(env, new[] { ("Dropout234_Output_0", outputColumn) }); | ||
|
|
||
| // There are two estimators created below. The first one is for image preprocessing and the second one is the actual DNN model. | ||
| var prepEstimator = new OnnxScoringEstimator(env, Path.Combine(modelDir, "AlexNetPrepOnnx", "AlexNetPreprocess.onnx"), new[] { "OriginalInput" }, new[] { "PreprocessedInput" }); | ||
| var mainEstimator = new OnnxScoringEstimator(env, Path.Combine(modelDir, "AlexNetOnnx", "AlexNet.onnx"), new[] { "Input140" }, new[] { "Dropout234_Output_0" }); | ||
| modelChain = modelChain.Append(inputRename); | ||
| var modelChain2 = modelChain.Append(prepEstimator); | ||
| modelChain = modelChain2.Append(midRename); | ||
| modelChain2 = modelChain.Append(mainEstimator); | ||
| modelChain = modelChain2.Append(endRename); | ||
| return modelChain; | ||
| } | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>netstandard2.0</TargetFramework> | ||
| <IncludeInPackage>Microsoft.ML.DnnImageFeaturizer.AlexNet</IncludeInPackage> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\Microsoft.ML.OnnxTransform\Microsoft.ML.OnnxTransform.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>netstandard2.0</TargetFramework> | ||
| <IncludeInPackage>Microsoft.ML.DnnImageFeaturizer.ResNet101</IncludeInPackage> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\Microsoft.ML.OnnxTransform\Microsoft.ML.OnnxTransform.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
| // See the LICENSE file in the project root for more information. | ||
|
|
||
| using Microsoft.ML.Runtime; | ||
| using Microsoft.ML.Runtime.Data; | ||
| using System.IO; | ||
| using System.Reflection; | ||
|
|
||
| namespace Microsoft.ML.Transforms | ||
| { | ||
| /// <summary> | ||
| /// This is an extension method to be used with the <see cref="DnnImageFeaturizerEstimator"/> in order to use a pretrained ResNet101 model. | ||
| /// The NuGet containing this extension is also guaranteed to include the binary model file. | ||
| /// </summary> | ||
| public static class ResNet101Extension | ||
| { | ||
| /// <summary> | ||
| /// Returns an estimator chain with the two corresponding models (a preprocessing one and a main one) required for the ResNet pipeline. | ||
| /// Also includes the renaming ColumnsCopyingTransforms required to be able to use arbitrary input and output column names. | ||
| /// This assumes both of the models are in the same location as the file containing this method, which they will be if used through the NuGet. | ||
| /// This should be the default way to use ResNet101 if importing the model from a NuGet. | ||
| /// </summary> | ||
| public static EstimatorChain<ColumnsCopyingTransformer> ResNet101(this DnnImageModelSelector dnnModelContext, IHostEnvironment env, string inputColumn, string outputColumn) | ||
| { | ||
| return ResNet101(dnnModelContext, env, inputColumn, outputColumn, Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location), "DnnImageModels")); | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// This allows a custom model location to be specified. This is useful is a custom model is specified, | ||
| /// or if the model is desired to be placed or shipped separately in a different folder from the main application. Note that because Onnx models | ||
| /// must be in a directory all by themsleves for the OnnxTransform to work, this method appends a ResNet101Onnx/ResNetPrepOnnx subdirectory | ||
| /// to the passed in directory to prevent having to make that directory manually each time. | ||
| /// </summary> | ||
| public static EstimatorChain<ColumnsCopyingTransformer> ResNet101(this DnnImageModelSelector dnnModelContext, IHostEnvironment env, string inputColumn, string outputColumn, string modelDir) | ||
| { | ||
| var modelChain = new EstimatorChain<ColumnsCopyingTransformer>(); | ||
|
|
||
| var inputRename = new ColumnsCopyingEstimator(env, new[] { (inputColumn, "OriginalInput") }); | ||
| var midRename = new ColumnsCopyingEstimator(env, new[] { ("PreprocessedInput", "Input1600") }); | ||
| var endRename = new ColumnsCopyingEstimator(env, new[] { ("Pooling2286_Output_0", outputColumn) }); | ||
|
|
||
| // There are two estimators created below. The first one is for image preprocessing and the second one is the actual DNN model. | ||
| var prepEstimator = new OnnxScoringEstimator(env, Path.Combine(modelDir, "ResNetPrepOnnx", "ResNetPreprocess.onnx"), new[] { "OriginalInput" }, new[] { "PreprocessedInput" }); | ||
| var mainEstimator = new OnnxScoringEstimator(env, Path.Combine(modelDir, "ResNet101Onnx", "ResNet101.onnx"), new[] { "Input1600" }, new[] { "Pooling2286_Output_0" }); | ||
| modelChain = modelChain.Append(inputRename); | ||
| var modelChain2 = modelChain.Append(prepEstimator); | ||
| modelChain = modelChain2.Append(midRename); | ||
| modelChain2 = modelChain.Append(mainEstimator); | ||
| modelChain = modelChain2.Append(endRename); | ||
| return modelChain; | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>netstandard2.0</TargetFramework> | ||
| <IncludeInPackage>Microsoft.ML.DnnImageFeaturizer.ResNet18</IncludeInPackage> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\Microsoft.ML.OnnxTransform\Microsoft.ML.OnnxTransform.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not aligned? #Resolved