-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Move Learner* input base and Transform* input base out of Entrypoints… #2748
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 13 commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
42ca927
Move Learner* input base and Transform* input base out of Entrypoints…
ganik e3fe31e
Merge pull request #1 from dotnet/master
ganik 34ec4ea
Merge
ganik c957969
fix build
ganik 199f02b
Learner to Trainer
ganik be52686
Rename file
ganik c2eebe1
fix build
ganik 1fefaa0
Merge pull request #2 from dotnet/master
ganik e43a4a6
Merge
ganik 37bca1b
fix build
ganik ed30b45
fix commens
ganik 9cd32bb
merge
ganik d457e90
fix #2760
ganik fcf85a2
Merge pull request #3 from dotnet/master
ganik 90c6846
Merge
ganik c5f4204
fix tests
ganik cca1d13
fix tests
ganik 0ff67fe
fix test
ganik File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,113 @@ | ||
// Licensed to the .NET Foundation under one or more agreements. | ||
// The .NET Foundation licenses this file to you under the MIT license. | ||
// See the LICENSE file in the project root for more information. | ||
|
||
using System; | ||
using System.Collections.Generic; | ||
using Microsoft.Data.DataView; | ||
using Microsoft.ML.Calibrators; | ||
using Microsoft.ML.CommandLine; | ||
using Microsoft.ML.Data; | ||
using Microsoft.ML.Data.IO; | ||
using Microsoft.ML.EntryPoints; | ||
|
||
namespace Microsoft.ML.Trainers | ||
{ | ||
/// <summary> | ||
/// The base class for all trainer inputs. | ||
/// </summary> | ||
[TlcModule.EntryPointKind(typeof(CommonInputs.ITrainerInput))] | ||
public abstract class TrainerInputBase | ||
{ | ||
private protected TrainerInputBase() { } | ||
|
||
/// <summary> | ||
/// The data to be used for training. Used only in entry-points, since in the API the expected mechanism is | ||
/// that the user will use the <see cref="IEstimator{TTransformer}.Fit(IDataView)"/> or some other train | ||
/// method. | ||
/// </summary> | ||
[BestFriend] | ||
[Argument(ArgumentType.Required, ShortName = "data", HelpText = "The data to be used for training", SortOrder = 1, Visibility = ArgumentAttribute.VisibilityType.EntryPointsOnly)] | ||
internal IDataView TrainingData; | ||
|
||
/// <summary> | ||
/// Column to use for features. | ||
/// </summary> | ||
[Argument(ArgumentType.AtMostOnce, HelpText = "Column to use for features", ShortName = "feat", SortOrder = 2, Visibility = ArgumentAttribute.VisibilityType.EntryPointsOnly)] | ||
public string FeatureColumnName = DefaultColumnNames.Features; | ||
|
||
/// <summary> | ||
/// Normalize option for the feature column. Used only in entry-points, since in the API the user is expected to do this themselves. | ||
/// </summary> | ||
[BestFriend] | ||
[Argument(ArgumentType.AtMostOnce, HelpText = "Normalize option for the feature column", ShortName = "norm", SortOrder = 5, Visibility = ArgumentAttribute.VisibilityType.EntryPointsOnly)] | ||
internal NormalizeOption NormalizeFeatures = NormalizeOption.Auto; | ||
|
||
/// <summary> | ||
/// Whether learner should cache input training data. Used only in entry-points, since the intended API mechanism | ||
/// is that the user will use the <see cref="DataOperationsCatalog.Cache(IDataView, string[])"/> or other method | ||
/// like <see cref="EstimatorChain{TLastTransformer}.AppendCacheCheckpoint(IHostEnvironment)"/>. | ||
/// </summary> | ||
[BestFriend] | ||
[Argument(ArgumentType.LastOccurenceWins, HelpText = "Whether learner should cache input training data", ShortName = "cache", SortOrder = 6, Visibility = ArgumentAttribute.VisibilityType.EntryPointsOnly)] | ||
internal CachingOptions Caching = CachingOptions.Auto; | ||
} | ||
|
||
/// <summary> | ||
/// The base class for all learner inputs that support a Label column. | ||
/// </summary> | ||
[TlcModule.EntryPointKind(typeof(CommonInputs.ITrainerInputWithLabel))] | ||
public abstract class TrainerInputBaseWithLabel : TrainerInputBase | ||
{ | ||
private protected TrainerInputBaseWithLabel() { } | ||
|
||
/// <summary> | ||
/// Column to use for labels. | ||
/// </summary> | ||
[Argument(ArgumentType.AtMostOnce, HelpText = "Column to use for labels", ShortName = "lab", SortOrder = 3, Visibility = ArgumentAttribute.VisibilityType.EntryPointsOnly)] | ||
public string LabelColumnName = DefaultColumnNames.Label; | ||
} | ||
|
||
// REVIEW: This is a known antipattern, but the solution involves the decorator pattern which can't be used in this case. | ||
/// <summary> | ||
/// The base class for all learner inputs that support a weight column. | ||
/// </summary> | ||
[TlcModule.EntryPointKind(typeof(CommonInputs.ITrainerInputWithWeight))] | ||
public abstract class TrainerInputBaseWithWeight : TrainerInputBaseWithLabel | ||
{ | ||
private protected TrainerInputBaseWithWeight() { } | ||
|
||
/// <summary> | ||
/// Column to use for example weight. | ||
/// </summary> | ||
[Argument(ArgumentType.AtMostOnce, HelpText = "Column to use for example weight", ShortName = "weight", SortOrder = 4, Visibility = ArgumentAttribute.VisibilityType.EntryPointsOnly)] | ||
public string ExampleWeightColumnName = null; | ||
} | ||
|
||
/// <summary> | ||
/// The base class for all unsupervised learner inputs that support a weight column. | ||
/// </summary> | ||
[TlcModule.EntryPointKind(typeof(CommonInputs.IUnsupervisedTrainerWithWeight))] | ||
public abstract class UnsupervisedTrainerInputBaseWithWeight : TrainerInputBase | ||
{ | ||
private protected UnsupervisedTrainerInputBaseWithWeight() { } | ||
|
||
/// <summary> | ||
/// Column to use for example weight. | ||
/// </summary> | ||
[Argument(ArgumentType.AtMostOnce, HelpText = "Column to use for example weight", ShortName = "weight", SortOrder = 4, Visibility = ArgumentAttribute.VisibilityType.EntryPointsOnly)] | ||
public string ExampleWeightColumnName = null; | ||
} | ||
|
||
[TlcModule.EntryPointKind(typeof(CommonInputs.ITrainerInputWithGroupId))] | ||
public abstract class TrainerInputBaseWithGroupId : TrainerInputBaseWithWeight | ||
{ | ||
private protected TrainerInputBaseWithGroupId() { } | ||
|
||
/// <summary> | ||
/// Column to use for example groupId. | ||
/// </summary> | ||
[Argument(ArgumentType.AtMostOnce, HelpText = "Column to use for example groupId", ShortName = "groupId", SortOrder = 5, Visibility = ArgumentAttribute.VisibilityType.EntryPointsOnly)] | ||
public string RowGroupColumnName = null; | ||
} | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
// Licensed to the .NET Foundation under one or more agreements. | ||
// The .NET Foundation licenses this file to you under the MIT license. | ||
// See the LICENSE file in the project root for more information. | ||
|
||
using Microsoft.Data.DataView; | ||
using Microsoft.ML.CommandLine; | ||
using Microsoft.ML.EntryPoints; | ||
|
||
namespace Microsoft.ML.Transforms | ||
{ | ||
/// <summary> | ||
/// The base class for all transform inputs. | ||
/// </summary> | ||
[TlcModule.EntryPointKind(typeof(CommonInputs.ITransformInput))] | ||
public abstract class TransformInputBase | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This class should have a |
||
{ | ||
private protected TransformInputBase() { } | ||
|
||
/// <summary> | ||
/// The input dataset. Used only in entry-point methods, since the normal API mechanism for feeding in a dataset to | ||
/// create an <see cref="ITransformer"/> is to use the <see cref="IEstimator{TTransformer}.Fit(IDataView)"/> method. | ||
/// </summary> | ||
[BestFriend] | ||
[Argument(ArgumentType.Required, HelpText = "Input dataset", Visibility = ArgumentAttribute.VisibilityType.EntryPointsOnly, SortOrder = 1)] | ||
internal IDataView Data; | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This class should have a
[BestFriend] private protected
constructor. We do not want people forming base classes from it willy nilly. Similar with all the other abstract classes defined here. #ResolvedThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could potentially be part of another PR. But, it's relatively easy to do if you have to change the PR for some other reason anyway.
In reply to: 260837670 [](ancestors = 260837670)