Skip to content

WIP: adding a nuget to package the datasets. #1166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

sfilipi
Copy link
Member

@sfilipi sfilipi commented Oct 5, 2018

The samples that appear in our documentation should have code that the user can copy/pate and execute.
For that purpose packaging the datasets in a NuGet whose namespace we can reference from the samples.

Addresses #1137

@sfilipi sfilipi self-assigned this Oct 5, 2018
@sfilipi sfilipi requested review from eerhardt, Ivanidzo4ka and Zruty0 and removed request for Zruty0 October 5, 2018 21:03
@sfilipi
Copy link
Member Author

sfilipi commented Oct 5, 2018

@eerhardt @Ivanidzo4ka where can i find more info about the nupkgproj syntax?
I need to add a post install step, to copy the files to some known project directory, so we can reference them from the samples.

@sfilipi sfilipi added documentation Related to documentation of ML.NET test related to tests labels Oct 5, 2018

<ItemGroup>
<Content Include="..\common\CommonPackage.props" Pack="true" PackagePath="build\netstandard2.0\$(MSBuildProjectName).props" />
<Content Include="..\..\test\data\syntheticRegressionTrain.txt" Pack="true" BuildAction="Content" target=".\content\regression\" />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[](start = 0, length = 1)

SHOCK

<ItemGroup>
<Content Include="..\common\CommonPackage.props" Pack="true" PackagePath="build\netstandard2.0\$(MSBuildProjectName).props" />
<Content Include="..\..\test\data\syntheticRegressionTrain.txt" Pack="true" BuildAction="Content" target=".\content\regression\" />
<Content Include="..\..\test\data\syntheticRegressionTest.txt" Pack="true" BuildAction="Content" target=".\content\regression\" />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[](start = 0, length = 1)

AND HORROR

Copy link
Member Author

@sfilipi sfilipi Oct 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go on vacation :) #Closed

@Ivanidzo4ka
Copy link
Contributor

Ivanidzo4ka commented Oct 5, 2018

in nuspec world I use targets file (to force certain actions) and I put files into specific folder https://docs.microsoft.com/en-us/nuget/reference/nuspec#including-content-files
Don't know anything about nupkgproj


In reply to: 427498975 [](ancestors = 427498975)


<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
<PackageDescription>ML.NET additional learners making use of hardware acceleration. They use Intel Mkl.</PackageDescription>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is out of date.

@@ -0,0 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk" DefaultTargets="Pack">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'd probably want to use a different nuget package name than Microsoft.ML.Datasets. That seems like it could be a useful package name in the future that we may not want to take here. Maybe adding Test or Sample or some other such word would help users realize that this NuGet package isn't for typical consumption.

@eerhardt
Copy link
Member

eerhardt commented Oct 8, 2018

in nuspec world I use targets file (to force certain actions) and I put files into specific folder https://docs.microsoft.com/en-us/nuget/reference/nuspec#including-content-files

I would use contentFiles, since they are respected when using PackageReferences:

https://docs.microsoft.com/en-us/nuget/reference/nuspec#using-the-contentfiles-element-for-content-files

@eerhardt
Copy link
Member

eerhardt commented Oct 8, 2018

@mairaw @JRAlexander - do we have other instances of where MSDN/docs.microsoft.com examples require you to install a NuGet package? I'm not sure I've seen this before.

I'm wondering if we can't just have a simple loop that creates the data into a CollectionDataSource instead?

@sfilipi
Copy link
Member Author

sfilipi commented Oct 8, 2018

The File operations in the .NEt docs indeed create the data in memory than write it/read it.
In the ML packages (R, Python) usually the datasets are packaged and shipped separately.

@GalOshri, to weigh in wrt what to snap to.


In reply to: 427859939 [](ancestors = 427859939)

@sfilipi
Copy link
Member Author

sfilipi commented Oct 9, 2018

We won't package the datasets in a nuget, we will point users to downloading them from github, in the samples.

@sfilipi sfilipi closed this Oct 9, 2018
@sfilipi sfilipi deleted the samplesDatasetPackage branch October 20, 2018 03:09
@ghost ghost locked as resolved and limited conversation to collaborators Mar 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Related to documentation of ML.NET test related to tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants