Skip to content

Image analytics documentation, samples, internalization #2372

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Feb 5, 2019
Merged

Image analytics documentation, samples, internalization #2372

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
976df4c
samples and documentation
sfilipi Feb 1, 2019
9804817
adressing issue #1798 for the Image Analytics project.
sfilipi Feb 1, 2019
9de2af2
rename removng the word 'Transform' from the name of the file, since …
sfilipi Feb 1, 2019
c93dfa1
PR review feedback.
sfilipi Feb 1, 2019
cf8fb0e
PR review comments
sfilipi Feb 4, 2019
1fb1482
post rebase fixes.
sfilipi Feb 4, 2019
256c5b3
PR review comments.
sfilipi Feb 5, 2019
70fa0db
nit
sfilipi Feb 5, 2019
7f0ab4f
regenerating the catalog.
sfilipi Feb 5, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions docs/samples/Microsoft.ML.Samples/Dynamic/ImageAnalytics/ConvertToGrayScale.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
using System.IO;
using Microsoft.ML.Data;

namespace Microsoft.ML.Samples.Dynamic
{
public class ConvertToGrayscaleExample
Copy link
Member

@singlis singlis Feb 4, 2019 •
edited by sfilipi
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to have a description about this example. #Resolved

{
public static void ConvertToGrayscale()
{
var mlContext = new MLContext();

// Downloading a few images, and an images.tsv file, that contains a list of the files, from the dotnet/machinelearning/test/data/images/
// if you inspect the fileSystem, after running this line, the
Copy link
Member

@singlis singlis Feb 4, 2019 •
edited by sfilipi
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the [](start = 71, length = 3)

  • the sentence looks like its not complete.
  • I think some commas can be removed (Downloading a few images and an images.tsv, which contains a list of the files from the dotnet/machinelearning... )
    #Resolved

var imagesDataFile = SamplesUtils.DatasetUtils.DownloadImages();

// Preview of the content of the images.tsv file
//
// imagePath imageType
// tomato.bmp tomato
// banana.jpg banana
// hotdog.jpg hotdog
// tomato.jpg tomato

var data = mlContext.Data.CreateTextLoader(new TextLoader.Arguments()
{
Columns = new[]
{
new TextLoader.Column("ImagePath", DataKind.TX, 0),
new TextLoader.Column("Name", DataKind.TX, 1),
}
}).Read(imagesDataFile);

var imagesFolder = Path.GetDirectoryName(imagesDataFile);
// Image loading pipeline.
var pipeline = mlContext.Transforms.LoadImages(imagesFolder, ("ImageObject", "ImagePath"))
.Append(mlContext.Transforms.ConvertToGrayscale(("GrayScale", "ImageObject")));

var transformedData = pipeline.Fit(data).Transform(data);

// The transformedData IDataView contains the loaded images now
//Preview of the transformedData
var transformedDataPreview = transformedData.Preview();

// Preview of the content of the images.tsv file
Copy link
Contributor

@zeahmed zeahmed Feb 4, 2019 •
edited by sfilipi
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preview of the content of the images.tsv file [](start = 15, length = 45)

The output shown below is not the contents of images.tsv file instead its transformedData. #Resolved

// The actual images, in the ImageReal column are of type System.Drawing.Bitmap.
Copy link
Contributor

@zeahmed zeahmed Feb 4, 2019 •
edited by sfilipi
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ImageReal [](start = 41, length = 9)

typo #Resolved

//
// ImagePath Name ImageObject GrayScale
// tomato.bmp tomato {System.Drawing.Bitmap} {System.Drawing.Bitmap}
// banana.jpg banana {System.Drawing.Bitmap} {System.Drawing.Bitmap}
// hotdog.jpg hotdog {System.Drawing.Bitmap} {System.Drawing.Bitmap}
// tomato.jpg tomato {System.Drawing.Bitmap} {System.Drawing.Bitmap}

}
}
}
57 changes: 57 additions & 0 deletions docs/samples/Microsoft.ML.Samples/Dynamic/ImageAnalytics/ExtractPixels.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
using System.IO;
using Microsoft.ML.Data;

namespace Microsoft.ML.Samples.Dynamic
{
public class ExtractPixelsExample
{
public static void ExtractPixels()
Copy link
Member

@singlis singlis Feb 4, 2019 •
edited by sfilipi
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ExtractPixels [](start = 27, length = 13)

What is this demo doing? It would be great to have a description about this example. #Resolved

Copy link
Member Author

@sfilipi sfilipi Feb 4, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added it; it will show up on the next iteration. Personally, i don' think that is very important, because those samples will display in the website that documents the components, so in context, hopefully there is enough explanation.

In reply to: 253609946 [](ancestors = 253609946)

{
var mlContext = new MLContext();

// Downloading a few images, and an images.tsv file, that contains a list of the files, from the dotnet/machinelearning/test/data/images/
Copy link
Member

@singlis singlis Feb 4, 2019 •
edited by sfilipi
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comments here (this looks to be copy-pasted). #Resolved

// if you inspect the fileSystem, after running this line, the
var imagesDataFile = SamplesUtils.DatasetUtils.DownloadImages();

// Preview of the content of the images.tsv file
//
// imagePath imageType
// tomato.bmp tomato
// banana.jpg banana
// hotdog.jpg hotdog
// tomato.jpg tomato

var data = mlContext.Data.CreateTextLoader(new TextLoader.Arguments()
{
Columns = new[]
{
new TextLoader.Column("ImagePath", DataKind.TX, 0),
new TextLoader.Column("Name", DataKind.TX, 1),
}
}).Read(imagesDataFile);

var imagesFolder = Path.GetDirectoryName(imagesDataFile);
// Image loading pipeline.
var pipeline = mlContext.Transforms.LoadImages(imagesFolder, ("ImageObject", "ImagePath"))
.Append(mlContext.Transforms.Resize("ImageObject",imageWidth: 100 , imageHeight: 100 ))
.Append(mlContext.Transforms.ExtractPixels("Pixels", "ImageObject"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Transforms [](start = 44, length = 10)

Considering specificity of domain, can we have Image subcatalog, like we have ml.Transforms.Text, can we have ml.Transforms.Images ?

Copy link
Member Author

@sfilipi sfilipi Feb 1, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i might misremember, but Pete disliked the idea; i believe becuase of too many catalogs makes things hard to find; but @artidoro and @rogancarr have expressed the same thing: issue #2361

If i get another thumbs up from @TomFinley, i am up for adding the catalog and moving the extensions there.

I am on the fence, personally. If we have more than 5 xtensions, i'd make a catalog for them. Less than that feels unecessary.

In reply to: 253166949 [](ancestors = 253166949)

Copy link
Contributor

@justinormont justinormont Feb 1, 2019 •
edited by sfilipi
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Anipik: We could make a good benchmark for the image processing pipeline.
I'd recommend using the Dog Breeds vs. Fruits dataset which we used in NimbusML for its image examples. We currently host this dataset in our CDN for NimbusML.

In Python, the dataset / image loader looks like:

# Load image summary data from github
url = "https://express-tlcresources.azureedge.net/datasets/DogBreedsVsFruits/DogFruitWiki.SHUF.117KB.735-rows.tsv"
df_train = pd.read_csv(url, sep = "\t", nrows = 100)
df_train['ImagePath_full'] = "https://express-tlcresources.azureedge.net/datasets/DogBreedsVsFruits/" + \
                         df_train['ImagePath']
... load images

Purpose of the dataset is for example code & includes ~775 images of dogs & fruit:
image
image

The Dog Breeds vs. Fruits would also be nice for our samples repo.

/cc @CESARDELATORRE #Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jormont, maybe log an issue about it. It might get forgotten here as a comment.

In reply to: 253212682 [](ancestors = 253212682)

Copy link
Contributor

@justinormont justinormont Feb 2, 2019 •
edited by sfilipi
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sfilipi -- good idea:



var transformedData = pipeline.Fit(data).Transform(data);

// The transformedData IDataView contains the loaded images now
//Preview of the transformedData
var transformedDataPreview = transformedData.Preview();

// Preview of the content of the images.tsv file
// The actual images, in the ImageReal column are of type System.Drawing.Bitmap.
Copy link
Contributor

@zeahmed zeahmed Feb 4, 2019 •
edited by sfilipi
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ImageReal [](start = 41, length = 9)

typo #Resolved

//
// ImagePath Name ImageObject "Pixels"
// tomato.bmp tomato {System.Drawing.Bitmap} [ 255, 255, 255, ..... 232, 243, 226, ...
// banana.jpg banana {System.Drawing.Bitmap} [ 255, 255, 255, ..... 90, 54, 43, ...
// hotdog.jpg hotdog {System.Drawing.Bitmap} [ 255, 255, 255, ..... 132, 143, 126, ...
// tomato.jpg tomato {System.Drawing.Bitmap} [ 255, 255, 255, ..... 16, 21, 23, ...

}
}
}
53 changes: 53 additions & 0 deletions docs/samples/Microsoft.ML.Samples/Dynamic/ImageAnalytics/LoadImage.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
using System.IO;
using Microsoft.ML.Data;

namespace Microsoft.ML.Samples.Dynamic
{
public class LoadImageExample
{
public static void LoadImage()
Copy link
Member

@singlis singlis Feb 4, 2019 •
edited by sfilipi
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoadImage [](start = 27, length = 9)

What does this example do? This looks to only demonstrate how to load images into the pipeline - it would be good to mention that. #Resolved

{
var mlContext = new MLContext();

// Downloading a few images, and an images.tsv file, that contains a list of the files, from the dotnet/machinelearning/test/data/images/
// if you inspect the fileSystem, after running this line, the
Copy link
Member

@singlis singlis Feb 4, 2019 •
edited by sfilipi
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here #Resolved

var imagesDataFile = SamplesUtils.DatasetUtils.DownloadImages();

// Preview of the content of the images.tsv file
//
// imagePath imageType
// tomato.bmp tomato
// banana.jpg banana
// hotdog.jpg hotdog
// tomato.jpg tomato

var data = mlContext.Data.CreateTextLoader(new TextLoader.Arguments()
{
Columns = new[]
{
new TextLoader.Column("ImagePath", DataKind.TX, 0),
new TextLoader.Column("Name", DataKind.TX, 1),
}
}).Read(imagesDataFile);

var imagesFolder = Path.GetDirectoryName(imagesDataFile);
// Image loading pipeline.
var pipeline = mlContext.Transforms.LoadImages(imagesFolder, ("ImageReal", "ImagePath"));
var transformedData = pipeline.Fit(data).Transform(data);

// The transformedData IDataView contains the loaded images now
//Preview of the transformedData
var transformedDataPreview = transformedData.Preview();

// Preview of the content of the images.tsv file
// The actual images, in the ImageReal column are of type System.Drawing.Bitmap.
//
// ImagePath Name ImageReal
// tomato.bmp tomato {System.Drawing.Bitmap}
// banana.jpg banana {System.Drawing.Bitmap}
// hotdog.jpg hotdog {System.Drawing.Bitmap}
// tomato.jpg tomato {System.Drawing.Bitmap}

}
}
}
56 changes: 56 additions & 0 deletions docs/samples/Microsoft.ML.Samples/Dynamic/ImageAnalytics/ResizeImage.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
using System.IO;
using Microsoft.ML.Data;

namespace Microsoft.ML.Samples.Dynamic
{
public class ResizeImageExample
{
public static void ResizeImage()
{
var mlContext = new MLContext();

// Downloading a few images, and an images.tsv file, that contains a list of the files, from the dotnet/machinelearning/test/data/images/
// if you inspect the fileSystem, after running this line, the
var imagesDataFile = SamplesUtils.DatasetUtils.DownloadImages();

// Preview of the content of the images.tsv file
//
// imagePath imageType
// tomato.bmp tomato
// banana.jpg banana
// hotdog.jpg hotdog
// tomato.jpg tomato

var data = mlContext.Data.CreateTextLoader(new TextLoader.Arguments()
{
Columns = new[]
{
new TextLoader.Column("ImagePath", DataKind.TX, 0),
new TextLoader.Column("Name", DataKind.TX, 1),
}
}).Read(imagesDataFile);

var imagesFolder = Path.GetDirectoryName(imagesDataFile);
// Image loading pipeline.
var pipeline = mlContext.Transforms.LoadImages(imagesFolder, ("ImageReal", "ImagePath"))
.Append(mlContext.Transforms.Resize("ImageReal", imageWidth: 100, imageHeight: 100));


var transformedData = pipeline.Fit(data).Transform(data);

// The transformedData IDataView contains the loaded images now
//Preview of the transformedData
var transformedDataPreview = transformedData.Preview();

// Preview of the content of the images.tsv file
// The actual images, in the ImageReal column are of type System.Drawing.Bitmap.
//
// ImagePath Name ImageReal
// tomato.bmp tomato {System.Drawing.Bitmap}
// banana.jpg banana {System.Drawing.Bitmap}
// hotdog.jpg hotdog {System.Drawing.Bitmap}
// tomato.jpg tomato {System.Drawing.Bitmap}

}
}
}
2 changes: 1 addition & 1 deletion docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Concatenate.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ public static void ConcatTransform()
var mlContext = new MLContext();

// Get a small dataset as an IEnumerable and them read it as ML.NET's data type.
IEnumerable<SamplesUtils.DatasetUtils.SampleInfertData> data = SamplesUtils.DatasetUtils.GetInfertData();
var data = SamplesUtils.DatasetUtils.GetInfertData();
var trainData = mlContext.Data.ReadFromEnumerable(data);

// Preview of the data.
Expand Down
2 changes: 1 addition & 1 deletion docs/samples/Microsoft.ML.Samples/Program.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ internal static class Program
{
static void Main(string[] args)
{
WordEmbeddingTransform.ExtractEmbeddings();
ResizeImageExample.ResizeImage();
}
}
}
20 changes: 10 additions & 10 deletions src/Microsoft.ML.ImageAnalytics/EntryPoints/ImageAnalytics.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ internal static class ImageAnalytics
{
[TlcModule.EntryPoint(Name = "Transforms.ImageLoader", Desc = ImageLoaderTransformer.Summary,
UserName = ImageLoaderTransformer.UserName, ShortName = ImageLoaderTransformer.LoaderSignature)]
public static CommonOutputs.TransformOutput ImageLoader(IHostEnvironment env, ImageLoaderTransformer.Arguments input)
public static CommonOutputs.TransformOutput ImageLoader(IHostEnvironment env, ImageLoaderTransformer.Options input)
{
var h = EntryPointUtils.CheckArgsAndCreateHost(env, "ImageLoaderTransform", input);
var xf = ImageLoaderTransformer.Create(h, input, input.Data);
Expand All @@ -24,25 +24,25 @@ public static CommonOutputs.TransformOutput ImageLoader(IHostEnvironment env, Im
};
}

[TlcModule.EntryPoint(Name = "Transforms.ImageResizer", Desc = ImageResizerTransformer.Summary,
UserName = ImageResizerTransformer.UserName, ShortName = ImageResizerTransformer.LoaderSignature)]
public static CommonOutputs.TransformOutput ImageResizer(IHostEnvironment env, ImageResizerTransformer.Arguments input)
[TlcModule.EntryPoint(Name = "Transforms.ImageResizer", Desc = ImageResizingTransformer.Summary,
UserName = ImageResizingTransformer.UserName, ShortName = ImageResizingTransformer.LoaderSignature)]
public static CommonOutputs.TransformOutput ImageResizer(IHostEnvironment env, ImageResizingTransformer.Arguments input)
{
var h = EntryPointUtils.CheckArgsAndCreateHost(env, "ImageResizerTransform", input);
var xf = ImageResizerTransformer.Create(h, input, input.Data);
var xf = ImageResizingTransformer.Create(h, input, input.Data);
return new CommonOutputs.TransformOutput()
{
Model = new TransformModelImpl(h, xf, input.Data),
OutputData = xf
};
}

[TlcModule.EntryPoint(Name = "Transforms.ImagePixelExtractor", Desc = ImagePixelExtractorTransformer.Summary,
UserName = ImagePixelExtractorTransformer.UserName, ShortName = ImagePixelExtractorTransformer.LoaderSignature)]
public static CommonOutputs.TransformOutput ImagePixelExtractor(IHostEnvironment env, ImagePixelExtractorTransformer.Arguments input)
[TlcModule.EntryPoint(Name = "Transforms.ImagePixelExtractor", Desc = ImagePixelExtractingTransformer.Summary,
UserName = ImagePixelExtractingTransformer.UserName, ShortName = ImagePixelExtractingTransformer.LoaderSignature)]
public static CommonOutputs.TransformOutput ImagePixelExtractor(IHostEnvironment env, ImagePixelExtractingTransformer.Options input)
{
var h = EntryPointUtils.CheckArgsAndCreateHost(env, "ImagePixelExtractorTransform", input);
var xf = ImagePixelExtractorTransformer.Create(h, input, input.Data);
var xf = ImagePixelExtractingTransformer.Create(h, input, input.Data);
return new CommonOutputs.TransformOutput()
{
Model = new TransformModelImpl(h, xf, input.Data),
Expand All @@ -52,7 +52,7 @@ public static CommonOutputs.TransformOutput ImagePixelExtractor(IHostEnvironment

[TlcModule.EntryPoint(Name = "Transforms.ImageGrayscale", Desc = ImageGrayscaleTransformer.Summary,
UserName = ImageGrayscaleTransformer.UserName, ShortName = ImageGrayscaleTransformer.LoaderSignature)]
public static CommonOutputs.TransformOutput ImageGrayscale(IHostEnvironment env, ImageGrayscaleTransformer.Arguments input)
public static CommonOutputs.TransformOutput ImageGrayscale(IHostEnvironment env, ImageGrayscaleTransformer.Options input)
{
var h = EntryPointUtils.CheckArgsAndCreateHost(env, "ImageGrayscaleTransform", input);
var xf = ImageGrayscaleTransformer.Create(h, input, input.Data);
Expand Down
Loading