Skip to content

Cleaning and Fixing public API for set of learners. #2765

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Mar 2, 2019
Merged

Cleaning and Fixing public API for set of learners. #2765

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
19d25e2
Scrubbing FieldAwareFactorizationMachine learner.
zeahmed Feb 26, 2019
b89c63b
Addressed reviewers' comments.
zeahmed Feb 26, 2019
0bc0c92
Fixed entrypoint json.
zeahmed Feb 26, 2019
4051d6b
Fixed entrypoint json.
zeahmed Feb 26, 2019
17f83a9
Merge branch 'learners_check' of https://github.com/zeahmed/machinele…
zeahmed Feb 26, 2019
7d4e7ed
Resolved merged conflicts.
zeahmed Feb 26, 2019
3f7cf53
Resolved a compilation bug.
zeahmed Feb 26, 2019
30faca7
Addressed reviewers' comments.
zeahmed Feb 26, 2019
c2746e2
Resolved merged conflicts.
zeahmed Feb 27, 2019
003b6a0
Addressed reviewers' comments.
zeahmed Feb 27, 2019
b16b1e3
Fixing and cleaning public API surface.
zeahmed Feb 27, 2019
f43e8d3
Resolved merged conflicts.
zeahmed Feb 27, 2019
d1a24f8
Resolved merged conflicts.
zeahmed Feb 27, 2019
0039fc6
Resolved merged conflicts.
zeahmed Feb 27, 2019
69d654e
Resolved merged conflicts.
zeahmed Feb 28, 2019
8235b0a
Resolved merged conflicts.
zeahmed Feb 28, 2019
03957a0
Addressed reviewers' comments.
zeahmed Mar 1, 2019
5c139a1
Updated entrypoint list file.
zeahmed Mar 1, 2019
06977ed
Merge remote-tracking branch 'upstream/master' into learners_check_2
zeahmed Mar 1, 2019
88a14c4
Addressed reviewers' comments.
zeahmed Mar 1, 2019
14f6b76
Addressed reviewers' comments.
zeahmed Mar 1, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 29 additions & 24 deletions ...es/Dynamic/Trainers/PriorTrainerSample.cs → ...inaryClassification/PriorTrainerSample.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,59 +7,64 @@ public class PriorTrainer
{
public static void Example()
{
// Downloading the dataset from github.com/dotnet/machinelearning.
// This will create a sentiment.tsv file in the filesystem.
// You can open this file, if you want to see the data.
string dataFile = SamplesUtils.DatasetUtils.DownloadSentimentDataset()[0];
// Create a new context for ML.NET operations. It can be used for exception tracking and logging,
// as a catalog of available operations and as the source of randomness.
var mlContext = new MLContext();

// Download and featurize the dataset.
var dataFiles = SamplesUtils.DatasetUtils.DownloadSentimentDataset();
var trainFile = dataFiles[0];
var testFile = dataFiles[1];

// A preview of the data.
// Sentiment SentimentText
// 0 " :Erm, thank you. "
// 1 ==You're cool==

// Create a new context for ML.NET operations. It can be used for exception tracking and logging,
// as a catalog of available operations and as the source of randomness.
var mlContext = new MLContext();

// Step 1: Load the data as an IDataView.
// First, we define the loader: specify the data columns and where to find them in the text file.
var loader = mlContext.Data.CreateTextLoader(
// Step 1: Read the data as an IDataView.
// First, we define the reader: specify the data columns and where to find them in the text file.
var reader = mlContext.Data.CreateTextLoader(
Copy link
Member

@sfilipi sfilipi Mar 1, 2019 •
edited by zeahmed
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reader [](start = 16, length = 6)

load. Artidoro had a whole PR to replace Read with Load.
#Resolved

columns: new[]
{
new TextLoader.Column("Sentiment", DataKind.Single, 0),
new TextLoader.Column("SentimentText", DataKind.String, 1)
},
hasHeader: true
);

// Load the data
var data = loader.Load(dataFile);

// Split it between training and test data
var trainTestData = mlContext.BinaryClassification.TrainTestSplit(data);
// Read the data
var trainData = reader.Load(trainFile);

// Step 2: Pipeline
// Featurize the text column through the FeaturizeText API.
// Then append a binary classifier, setting the "Label" column as the label of the dataset, and
// the "Features" column produced by FeaturizeText as the features column.
var pipeline = mlContext.Transforms.Text.FeaturizeText("Features", "SentimentText")
.AppendCacheCheckpoint(mlContext) // Add a data-cache step within a pipeline.
.AppendCacheCheckpoint(mlContext)
.Append(mlContext.BinaryClassification.Trainers.Prior(labelColumnName: "Sentiment"));

// Step 3: Train the pipeline
var trainedPipeline = pipeline.Fit(trainTestData.TrainSet);
var trainedPipeline = pipeline.Fit(trainData);

// Step 4: Evaluate on the test set
var transformedData = trainedPipeline.Transform(trainTestData.TestSet);
var transformedData = trainedPipeline.Transform(reader.Load(testFile));
var evalMetrics = mlContext.BinaryClassification.Evaluate(transformedData, label: "Sentiment");

// Step 5: Inspect the output
Console.WriteLine("Accuracy: " + evalMetrics.Accuracy);
SamplesUtils.ConsoleUtils.PrintMetrics(evalMetrics);

// The Prior trainer outputs the proportion of a label in the dataset as the probability of that label.
// In this case it means that there is a split of around 64%-36% of positive and negative labels in the dataset.
// In this case 'Accuracy: 0.50' means that there is a split of around 50%-50% of positive and negative labels in the test dataset.
// Expected output:
// Accuracy: 0.647058823529412

// Accuracy: 0.50
// AUC: 0.50
// F1 Score: 0.67
// Negative Precision: 0.00
// Negative Recall: 0.00
// Positive Precision: 0.50
// Positive Recall: 1.00
// LogLoss: 1.05
// LogLossReduction: -4.89
// Entropy: 1.00
}
}
}
53 changes: 29 additions & 24 deletions ...s/Dynamic/Trainers/RandomTrainerSample.cs → ...naryClassification/RandomTrainerSample.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,59 +7,64 @@ public static class RandomTrainer
{
public static void Example()
{
// Downloading the dataset from github.com/dotnet/machinelearning.
// This will create a sentiment.tsv file in the filesystem.
// You can open this file, if you want to see the data.
string dataFile = SamplesUtils.DatasetUtils.DownloadSentimentDataset()[0];
// Create a new context for ML.NET operations. It can be used for exception tracking and logging,
// as a catalog of available operations and as the source of randomness.
var mlContext = new MLContext(seed: 1);

// Download and featurize the dataset.
var dataFiles = SamplesUtils.DatasetUtils.DownloadSentimentDataset();
var trainFile = dataFiles[0];
var testFile = dataFiles[1];

// A preview of the data.
// Sentiment SentimentText
// 0 " :Erm, thank you. "
// 1 ==You're cool==

// Create a new context for ML.NET operations. It can be used for exception tracking and logging,
// as a catalog of available operations and as the source of randomness.
var mlContext = new MLContext(seed: 1);

// Step 1: Load the data as an IDataView.
// First, we define the loader: specify the data columns and where to find them in the text file.
var loader = mlContext.Data.CreateTextLoader(
// Step 1: Read the data as an IDataView.
// First, we define the reader: specify the data columns and where to find them in the text file.
var reader = mlContext.Data.CreateTextLoader(
columns: new[]
{
new TextLoader.Column("Sentiment", DataKind.Single, 0),
new TextLoader.Column("SentimentText", DataKind.String, 1)
},
hasHeader: true
);

// Load the data
var data = loader.Load(dataFile);

// Split it between training and test data
var trainTestData = mlContext.BinaryClassification.TrainTestSplit(data);
// Read the data
var trainData = reader.Load(trainFile);
Copy link
Member

@sfilipi sfilipi Mar 1, 2019 •
edited by zeahmed
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reader.Load [](start = 28, length = 11)

nit.: revert #Resolved


// Step 2: Pipeline
// Featurize the text column through the FeaturizeText API.
// Then append a binary classifier, setting the "Label" column as the label of the dataset, and
// the "Features" column produced by FeaturizeText as the features column.
var pipeline = mlContext.Transforms.Text.FeaturizeText("Features", "SentimentText")
.AppendCacheCheckpoint(mlContext) // Add a data-cache step within a pipeline.
.AppendCacheCheckpoint(mlContext)
.Append(mlContext.BinaryClassification.Trainers.Random());

// Step 3: Train the pipeline
var trainedPipeline = pipeline.Fit(trainTestData.TrainSet);
var trainedPipeline = pipeline.Fit(trainData);

// Step 4: Evaluate on the test set
var transformedData = trainedPipeline.Transform(trainTestData.TestSet);
var transformedData = trainedPipeline.Transform(reader.Load(testFile));
var evalMetrics = mlContext.BinaryClassification.Evaluate(transformedData, label: "Sentiment");

// Step 5: Inspect the output
Console.WriteLine("Accuracy: " + evalMetrics.Accuracy);
SamplesUtils.ConsoleUtils.PrintMetrics(evalMetrics);

// We expect an output probability closet to 0.5 as the Random trainer outputs a random prediction.
// Regardless of the input features, the trainer will predict either positive or negative label with equal probability.
// Expected output (close to 0.5):
// Accuracy: 0.588235294117647
// Expected output: (close to 0.5):

// Accuracy: 0.56
// AUC: 0.57
// F1 Score: 0.60
// Negative Precision: 0.57
// Negative Recall: 0.44
// Positive Precision: 0.55
// Positive Recall: 0.67
// LogLoss: 1.53
// LogLossReduction: -53.37
// Entropy: 1.00
}
}
}
2 changes: 1 addition & 1 deletion ...les/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescentWithOptions.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ public static void Example()
var split = mlContext.BinaryClassification.TrainTestSplit(data, testFraction: 0.1);
// Create data training pipeline
var pipeline = mlContext.BinaryClassification.Trainers.SymbolicStochasticGradientDescent(
new ML.Trainers.HalLearners.SymSgdClassificationTrainer.Options()
new ML.Trainers.HalLearners.SymbolicStochasticGradientDescentClassificationTrainer.Options()
{
LearningRate = 0.2f,
NumberOfIterations = 10,
Expand Down
22 changes: 11 additions & 11 deletions src/Microsoft.ML.HalLearners/HalLearnersCatalog.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
namespace Microsoft.ML
{
/// <summary>
/// The trainer catalog extensions for the <see cref="OlsLinearRegressionTrainer"/> and <see cref="SymSgdClassificationTrainer"/>.
/// The trainer catalog extensions for the <see cref="OlsLinearRegressionTrainer"/> and <see cref="SymbolicStochasticGradientDescentClassificationTrainer"/>.
/// </summary>
public static class HalLearnersCatalog
{
Expand Down Expand Up @@ -68,7 +68,7 @@ public static OlsLinearRegressionTrainer OrdinaryLeastSquares(
}

/// <summary>
/// Predict a target using a linear binary classification model trained with the <see cref="SymSgdClassificationTrainer"/>.
/// Predict a target using a linear binary classification model trained with the <see cref="SymbolicStochasticGradientDescentClassificationTrainer"/>.
/// </summary>
/// <param name="catalog">The <see cref="BinaryClassificationCatalog"/>.</param>
/// <param name="labelColumnName">The name of the label column.</param>
Expand All @@ -81,43 +81,43 @@ public static OlsLinearRegressionTrainer OrdinaryLeastSquares(
/// ]]>
/// </format>
/// </example>
public static SymSgdClassificationTrainer SymbolicStochasticGradientDescent(this BinaryClassificationCatalog.BinaryClassificationTrainers catalog,
public static SymbolicStochasticGradientDescentClassificationTrainer SymbolicStochasticGradientDescent(this BinaryClassificationCatalog.BinaryClassificationTrainers catalog,
string labelColumnName = DefaultColumnNames.Label,
string featureColumnName = DefaultColumnNames.Features,
int numberOfIterations = SymSgdClassificationTrainer.Defaults.NumberOfIterations)
int numberOfIterations = SymbolicStochasticGradientDescentClassificationTrainer.Defaults.NumberOfIterations)
{
Contracts.CheckValue(catalog, nameof(catalog));
var env = CatalogUtils.GetEnvironment(catalog);

var options = new SymSgdClassificationTrainer.Options
var options = new SymbolicStochasticGradientDescentClassificationTrainer.Options
{
LabelColumn = labelColumnName,
FeatureColumn = featureColumnName,
};

return new SymSgdClassificationTrainer(env, options);
return new SymbolicStochasticGradientDescentClassificationTrainer(env, options);
}

/// <summary>
/// Predict a target using a linear binary classification model trained with the <see cref="SymSgdClassificationTrainer"/>.
/// Predict a target using a linear binary classification model trained with the <see cref="SymbolicStochasticGradientDescentClassificationTrainer"/>.
/// </summary>
/// <param name="catalog">The <see cref="BinaryClassificationCatalog"/>.</param>
/// <param name="options">Algorithm advanced options. See <see cref="SymSgdClassificationTrainer.Options"/>.</param>
/// <param name="options">Algorithm advanced options. See <see cref="SymbolicStochasticGradientDescentClassificationTrainer.Options"/>.</param>
/// <example>
/// <format type="text/markdown">
/// <![CDATA[
/// [!code-csharp[SymbolicStochasticGradientDescent](~/../docs/samples/docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescentWithOptions.cs)]
/// ]]>
/// </format>
/// </example>
public static SymSgdClassificationTrainer SymbolicStochasticGradientDescent(
public static SymbolicStochasticGradientDescentClassificationTrainer SymbolicStochasticGradientDescent(
this BinaryClassificationCatalog.BinaryClassificationTrainers catalog,
SymSgdClassificationTrainer.Options options)
SymbolicStochasticGradientDescentClassificationTrainer.Options options)
{
Contracts.CheckValue(catalog, nameof(catalog));
Contracts.CheckValue(options, nameof(options));
var env = CatalogUtils.GetEnvironment(catalog);
return new SymSgdClassificationTrainer(env, options);
return new SymbolicStochasticGradientDescentClassificationTrainer(env, options);
}

/// <summary>
Expand Down
Loading