Skip to content

Hide much infrastructure in data #2300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jan 29, 2019
Merged

Hide much infrastructure in data #2300

merged 9 commits into from
Jan 29, 2019

Conversation

TomFinley
Copy link
Contributor

Another of many steps towards #1602. Commits logically structured. No overall theme, just lots of hiding of individual components, most notably command line parsing, entry-point declarations, and other such things.

@TomFinley TomFinley added the API Issues pertaining the friendly API label Jan 29, 2019
@codecov
Copy link

codecov bot commented Jan 29, 2019

Codecov Report

Merging #2300 into master will increase coverage by <.01%.
The diff coverage is 94.44%.

@@            Coverage Diff             @@
##           master    #2300      +/-   ##
==========================================
+ Coverage   71.14%   71.14%   +<.01%     
==========================================
  Files         779      779              
  Lines      140243   140251       +8     
  Branches    16024    16024              
==========================================
+ Hits        99773    99783      +10     
+ Misses      36038    36035       -3     
- Partials     4432     4433       +1
Flag Coverage Δ
#Debug 71.14% <94.44%> (ø) ⬆️
#production 67.57% <94.44%> (ø) ⬆️
#test 85.05% <ø> (ø) ⬆️

@@ -31,7 +31,8 @@ namespace Microsoft.ML.Transforms
/// It will be used in conjunction with a filter transform to create random
/// partitions of the data, used in cross validation.
/// </summary>
public sealed class GenerateNumberTransform : RowToRowTransformBase
[BestFriend]
internal sealed class GenerateNumberTransform : RowToRowTransformBase
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

internal [](start = 4, length = 8)

why not public?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pursuant to #1995, all IDataTransforms (as opposed to IEstimator/ITransformers) are to be hidden. So when hiding the entry-point exposing this class, I happened to opportunistically hide this as well, while I was at it.

@sfilipi
Copy link
Member

sfilipi commented Jan 29, 2019

    public const string TakeFilterShortName = "Take";

hurts my eyes :)


Refers to: src/Microsoft.ML.Data/Transforms/SkipTakeFilter.cs:48 in 3ac69ce. [](commit_id = 3ac69ce, deletion_comment = False)

@codemzs
Copy link
Member

codemzs commented Jan 29, 2019

Can you please sync to the latest in master because I’m really curious what the new code coverage is?

@dotnet dotnet deleted a comment from codemzs Jan 29, 2019
Copy link
Member

@sfilipi sfilipi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@TomFinley
Copy link
Contributor Author

TomFinley commented Jan 29, 2019

Can you please sync to the latest in master because I’m really curious what the new code coverage is?

If it naturally comes up due to an actual required change I'd happily rebase again to master, as I always do. In the absence of any requested changes, I see no particular urgency. This might not be the best PR if you're curious about changes in code coverage, since I did not change any tests, or any infrastructure really. I just made things internal.

If you are curious about this, you can always of course check out my branch yourself and run the coverage numbers locally. In principle it shouldn't change much, since I did not change much infrastructure, just made it internal. (But still according to the report in its current form, I have a +3.29% change in coverage. Who can account for such things.)

I've seen lots of strange stuff like that. Are we certain those code coverage numbers are reliable?

@TomFinley
Copy link
Contributor Author

hurts my eyes :)

The goggles do nothing!

@shauheen shauheen added this to the 0119 milestone Jan 29, 2019
@@ -0,0 +1,137 @@
Data.CustomTextLoader Import a dataset from a text file Microsoft.ML.EntryPoints.ImportTextData ImportText Microsoft.ML.EntryPoints.ImportTextData+Input Microsoft.ML.EntryPoints.ImportTextData+Output
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CustomTextLoader [](start = 5, length = 16)

Why we need this file?

Copy link
Contributor Author

@TomFinley TomFinley Jan 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't, sorry. 😄 I'll remove it. WinMerge keeps creating these stupid files...

Copy link
Contributor

@shauheen shauheen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Copy link
Contributor

@Ivanidzo4ka Ivanidzo4ka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@eerhardt
Copy link
Member

[BestFriend]

what does BestFriend on a public class mean?


Refers to: src/Microsoft.ML.HalLearners/OlsLinearRegression.cs:38 in 90096c7. [](commit_id = 90096c7, deletion_comment = False)

@eerhardt
Copy link
Member

Did you mean to check this core_ep-list.tsv.bak file in?

@@ -37,10 +44,13 @@ public enum CachingOptions
public abstract class LearnerInputBase
{
/// <summary>
/// The data to be used for training.
/// The data to be used for training. Used only in entry-points, since in the API the expected mechanism is
/// that the user iwll use the <see cref="IEstimator{TTransformer}.Fit(IDataView)"/> or some other train
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typeo- iwll

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to check this core_ep-list.tsv.bak file in?

No. @Ivanidzo4ka had the same comment, I think if you refresh the view you're using you should note that it is removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typeo- iwll

Whoops. Since internal only I will fix in next phase, if that's OK.

@TomFinley TomFinley merged commit 3dd3a1e into dotnet:master Jan 29, 2019
@TomFinley
Copy link
Contributor Author

[BestFriend]

what does BestFriend on a public class mean?

Refers to: src/Microsoft.ML.HalLearners/OlsLinearRegression.cs:38 in 90096c7. [](commit_id = 90096c7, deletion_comment = False)

It means I'm silly. I'll fix that in next pass.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
API Issues pertaining the friendly API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants