Skip to content

Conversion of Whitening Transform to estimator with pigstensions #1452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 45 commits into from
Nov 8, 2018
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
371c802
opening new pr
artidoro Oct 30, 2018
72a2d69
moving file
artidoro Oct 30, 2018
d6cb65f
return back
artidoro Oct 30, 2018
61774cb
move it
artidoro Oct 30, 2018
49ca333
moving it back
artidoro Oct 30, 2018
0d8220a
fixing some issues
artidoro Oct 30, 2018
1eda6d2
added more tests
artidoro Oct 30, 2018
ce7553f
adding summary comments
artidoro Oct 30, 2018
201225c
unused usings
artidoro Oct 30, 2018
4dce5ba
Merge branch 'master' of https://github.com/dotnet/machinelearning in…
artidoro Oct 30, 2018
cfa256f
merging with master required a small change
artidoro Oct 30, 2018
a677d0c
correcting comments
artidoro Oct 30, 2018
dc6cb96
change precision for x86
artidoro Oct 30, 2018
7b86ad5
merge
artidoro Oct 30, 2018
7d7ae63
fixing review comments
artidoro Oct 31, 2018
652ab2d
fixing review comments and making training into static method
artidoro Nov 1, 2018
8434722
renaming to vectorwhiteningestimator and adding it to mlcontext
artidoro Nov 1, 2018
3f622df
merge
artidoro Nov 1, 2018
0e82095
prevent training on empty data
artidoro Nov 6, 2018
adaa7cd
turn on hosted mac
artidoro Nov 6, 2018
fb3be27
remove wine quality dataset from build download
artidoro Nov 6, 2018
80481ff
remove the hosted mac change as that will come in another pr
artidoro Nov 6, 2018
c4c004f
remove winequality change as this will come in another pr
artidoro Nov 6, 2018
f0157ac
Update src/Microsoft.ML.Transforms/WhiteningTransform.cs
wschin Nov 6, 2018
3b9e385
Update src/Microsoft.ML.Transforms/WhiteningTransform.cs
wschin Nov 6, 2018
6eed55a
Update src/Microsoft.ML.Transforms/WhiteningTransform.cs
wschin Nov 6, 2018
f9441b4
Update src/Microsoft.ML.Transforms/WhiteningTransform.cs
wschin Nov 6, 2018
4e38239
review comments
artidoro Nov 6, 2018
a51f016
Merge branch 'whitening' of https://github.com/artidoro/machinelearni…
artidoro Nov 6, 2018
609d4c2
review comments
artidoro Nov 6, 2018
143ebfb
Merge branch 'master' of https://github.com/dotnet/machinelearning in…
artidoro Nov 6, 2018
23eb3b1
completed merge
artidoro Nov 6, 2018
51c2fde
Update src/Microsoft.ML.Transforms/WhiteningTransform.cs
wschin Nov 6, 2018
4bb83fe
Merge branch 'master' of https://github.com/dotnet/machinelearning in…
artidoro Nov 7, 2018
44f8891
updating code
artidoro Nov 7, 2018
180f715
review comments
artidoro Nov 7, 2018
9931da8
Merge branch 'master' of https://github.com/dotnet/machinelearning in…
artidoro Nov 7, 2018
d8c2d30
Merge branch 'master' of https://github.com/dotnet/machinelearning in…
artidoro Nov 7, 2018
4ea9c7a
review comment
artidoro Nov 7, 2018
c8488a7
reduce precision
artidoro Nov 7, 2018
cd1ea46
adjusting precision
artidoro Nov 8, 2018
39dbcb7
adjusting precision
artidoro Nov 8, 2018
e98ca69
merge
artidoro Nov 8, 2018
528aa74
adjust precision to initial value 4
artidoro Nov 8, 2018
f8c39d4
review comments
artidoro Nov 8, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions src/Microsoft.ML.Transforms/ProjectionCatalog.cs
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,30 @@ public static RandomFourierFeaturizingEstimator CreateRandomFourierFeatures(this
/// <param name="columns">The input columns to use for the transformation.</param>
public static RandomFourierFeaturizingEstimator CreateRandomFourierFeatures(this TransformsCatalog.ProjectionTransforms catalog, params RffTransform.ColumnInfo[] columns)
=> new RandomFourierFeaturizingEstimator(CatalogUtils.GetEnvironment(catalog), columns);

/// <include file='doc.xml' path='doc/members/member[@name="Whitening"]/*'/>
/// <param name="catalog">The transform's catalog.</param>
/// <param name="env">The environment.</param>
/// <param name="inputColumn">Name of the input column.</param>
/// <param name="outputColumn">Name of the column resulting from the transformation of <paramref name="inputColumn"/>. Null means <paramref name="inputColumn"/> is replaced. </param>
/// <param name="kind">Whitening kind (PCA/ZCA).</param>
/// <param name="eps">Whitening constant, prevents division by zero.</param>
/// <param name="maxRows">Maximum number of rows used to train the transform.</param>
/// <param name="saveInverse">Whether to save inverse (recovery) matrix.</param>
/// <param name="pcaNum">In case of PCA whitening, indicates the number of components to retain.</param>
public static VectorWhiteningEstimator VectorWhiten(this TransformsCatalog.ProjectionTransforms catalog, IHostEnvironment env, string inputColumn, string outputColumn,
Copy link
Contributor

@Ivanidzo4ka Ivanidzo4ka Nov 7, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IHostEnvironment env, [](start = 113, length = 21)

you don't need this. you use catalog to get it. #Closed

WhiteningKind kind = VectorWhiteningTransform.Defaults.Kind,
float eps = VectorWhiteningTransform.Defaults.Eps,
int maxRows = VectorWhiteningTransform.Defaults.MaxRows,
bool saveInverse = VectorWhiteningTransform.Defaults.SaveInverse,
int pcaNum = VectorWhiteningTransform.Defaults.PcaNum)
=> new VectorWhiteningEstimator(CatalogUtils.GetEnvironment(catalog), inputColumn, outputColumn, kind, eps, maxRows, saveInverse, pcaNum);

/// <include file='doc.xml' path='doc/members/member[@name="Whitening"]/*'/>
/// <param name="catalog">The transform's catalog.</param>
/// <param name="env">The environment.</param>
/// <param name="columns"> Describes the parameters of the whitening process for each column pair.</param>
public static VectorWhiteningEstimator VectorWhiten(this TransformsCatalog.ProjectionTransforms catalog, IHostEnvironment env, params VectorWhiteningTransform.ColInfo[] columns)
Copy link
Contributor

@Ivanidzo4ka Ivanidzo4ka Nov 7, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IHostEnvironment env [](start = 113, length = 20)

drop it. Use CatalogUtils.GetEnvironment(catalog) #Closed

=> new VectorWhiteningEstimator(env, columns);
}
}
837 changes: 550 additions & 287 deletions src/Microsoft.ML.Transforms/WhiteningTransform.cs

Large diffs are not rendered by default.

167 changes: 0 additions & 167 deletions src/Microsoft.ML.Transforms/WrappedWhiteningTransformer.cs

This file was deleted.

9 changes: 9 additions & 0 deletions test/BaselineOutput/Common/NormalizerEstimator/whitened.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#@ TextLoader{
#@ sep=tab
#@ col=whitened1:R4:0-10
#@ col=whitened2:R4:11-15
#@ }
-2.604605 0.829638362 -0.5992434 0.19860521 1.33247662 0.369197041 -0.5760094 -0.5490271 -1.94509208 -0.393351972 0.507488966 1.75005662 -0.546613038 2.462052 1.32538271 -0.57087183
-0.5923902 -0.324390084 -0.114805378 0.6855182 -1.055579 0.8767955 -0.0392023772 1.21807373 -0.160801888 -0.47570774 -2.22817 0.7284955 -0.8200103 0.4638015 -1.0152092 0.3444226
-0.9132714 -0.911281645 1.814283 0.07471426 -0.8969923 -1.44387519 -1.19571114 -0.6542767 0.887983143 -0.4604767 -0.17543222 0.0112341344 0.913079262 -0.134250313 -0.118262529 -1.16476536
0.236966148 1.004758 -0.233154371 0.3862052 -1.02724624 0.240614042 0.299898773 -1.03102541 -1.13852251 -0.6675951 0.766793966 0.490669161 -0.489173561 -0.5981086 1.18466234 1.05758965
Loading