merge with master

Author: Ivan Matantsev

Microsoft.ML.sln

@@ -260,6 +260,12 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Microsoft.Data.DataView", "
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "RemoteExecutorConsoleApp", "test\RemoteExecutorConsoleApp\RemoteExecutorConsoleApp.csproj", "{5E920CAC-5A28-42FB-936E-49C472130953}"
 EndProject
+Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Microsoft.ML.Ensemble", "Microsoft.ML.Ensemble", "{AD7058C9-5608-49A8-BE23-58C33A74EE91}"
+	ProjectSection(SolutionItems) = preProject
+		pkg\Microsoft.ML.Ensemble\Microsoft.ML.Ensemble.nupkgproj = pkg\Microsoft.ML.Ensemble\Microsoft.ML.Ensemble.nupkgproj
+		pkg\Microsoft.ML.Ensemble\Microsoft.ML.Ensemble.symbols.nupkgproj = pkg\Microsoft.ML.Ensemble\Microsoft.ML.Ensemble.symbols.nupkgproj
+	EndProjectSection
+EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Any CPU = Debug|Any CPU
@@ -1026,6 +1032,7 @@ Global
 		{85D0CAFD-2FE8-496A-88C7-585D35B94243} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
 		{31D38B21-102B-41C0-9E0A-2FE0BF68D123} = {D3D38B03-B557-484D-8348-8BADEE4DF592}
 		{5E920CAC-5A28-42FB-936E-49C472130953} = {AED9C836-31E3-4F3F-8ABC-929555D3F3C4}
+		{AD7058C9-5608-49A8-BE23-58C33A74EE91} = {D3D38B03-B557-484D-8348-8BADEE4DF592}
 	EndGlobalSection
 	GlobalSection(ExtensibilityGlobals) = postSolution
 		SolutionGuid = {41165AF1-35BB-4832-A189-73060F82B01D}

build/BranchInfo.props

@@ -1,7 +1,7 @@
 <Project>
   <PropertyGroup>
     <MajorVersion>0</MajorVersion>
-    <MinorVersion>11</MinorVersion>
+    <MinorVersion>12</MinorVersion>
     <PatchVersion>0</PatchVersion>
     <PreReleaseLabel>preview</PreReleaseLabel>
   </PropertyGroup>

docs/code/IDataViewDesignPrinciples.md

@@ -47,7 +47,7 @@ only when needed to satisfy a local request for information.
 The IDataView design fulfills the following design requirements:
 
 * **General schema**: Each view carries schema information, which specifies
-  the names and types of the view's columns, together with metadata associated
+  the names and types of the view's columns, together with annotations associated
   with the columns. The system is optimized for a reasonably small number of
   columns (hundreds). See [here](#basics).
 
@@ -112,14 +112,14 @@ The IDataView system design does *not* include the following:
 * **Multi-view schema information**: There is no direct support for specifying
   cross-view schema information, for example, that certain columns are primary
   keys, and that there are foreign key relationships among tables. However,
-  the column metadata support, together with conventions, may be used to
+  the column annotation support, together with conventions, may be used to
   represent such information.
 
 * **Standard ML schema**: The IDataView system does not define, nor prescribe,
   standard ML schema representation. For example, it does not dictate
   representation of nor distinction between different semantic interpretations
   of columns, such as label, feature, score, weight, etc. However, the column
-  metadata support, together with conventions, may be used to represent such
+  annotation support, together with conventions, may be used to represent such
   interpretations.
 
 * **Row count**: A view is not required to provide its row count. The
@@ -149,7 +149,7 @@ The IDataView system design does *not* include the following:
 
 IDataView has general schema support, in that a view can have an arbitrary
 number of columns, each having an associated name, index, data type, and
-optional metadata.
+optional annotation.
 
 Column names are case sensitive. Multiple columns can share the same name, in
 which case, one of the columns hides the others, in the sense that the name
@@ -177,7 +177,7 @@ The set of standard types will likely be expanded over time.
 The IDataView type system is specified in a separate document, *IDataView Type
 System Specification*.
 
-IDataView provides a general mechanism for associating semantic metadata with
+IDataView provides a general mechanism for associating semantic annotations with
 columns, such as designating sets of score columns, names associated with the
 individual slots of a vector-valued column, values associated with a key type
 column, whether a column's data is normalized, etc.

docs/code/IDataViewImplementation.md

@@ -313,10 +313,10 @@ are initialized using the pseudo-random number generator in an `IHost` that
 changes from one to another. But, that's a bit nit-picky.
 
 Note also: when we say functionally identical we include everything about it:
-not just the data, but the schema, its metadata, the implementation of
+not just the data, but the schema, its annotations, the implementation of
 shuffling, etc. For this reason, while serializing the data *model* has
 guarantees of consistency, serializing the *data* has no such guarantee: if
-you serialize data using the text saver, practically all metadata (except slot
+you serialize data using the text saver, practically all annotations (except slot
 names) will be completely lost, which can have implications on how some
 transforms and downstream processes work. Or: if you serialize data using the
 binary saver, suddenly it may become shufflable whereas it may not have been
@@ -475,7 +475,7 @@ helpful).
 
 The schema contains information about the columns. As we see in [the design
 principles](IDataViewDesignPrinciples.md), it has index, data type, and
-optional metadata.
+optional annotations.
 
 While *programmatically* accesses to an `IDataView` are by index, from a
 user's perspective the indices are by name; most training algorithms
@@ -498,20 +498,20 @@ things like key-types and vector-types, when returned, should not be created
 in the function itself (thereby creating a new object every time), but rather
 stored somewhere and returned.
 
-## Metadata
+## Annotations
 
-Since metadata is *optional*, one is not obligated to necessarily produce it,
+Since annotations are *optional*, one is not obligated to necessarily produce it,
 or conform to any particular schemas for any particular kinds (beyond, say,
 the obvious things like making sure that the types and values are consistent).
 However, the flip side of that freedom given to *producers*, is that
 *consumers* are obligated, when processing a data view input, to react
-gracefully when metadata of a certain kind is absent, or not in a form that
-one expects. One should *never* fail when input metadata is in a form one does
+gracefully when an annotation of a certain kind is absent, or not in a form that
+one expects. One should *never* fail when input annotations are in a form one does
 not expect.
 
 To give a practical example of this: many transforms, learners, or other
 components that process `IDataView`s will do something with the slot names,
-but when the `SlotNames` metadata kind for a given column is either absent,
+but when the `SlotNames` annotation kind for a given column is either absent,
 *or* not of the right type (vectors of strings), *or* not of the right size
 (same length vectors as the input), the behavior is not to throw or yield
 errors or do anything of the kind, but to simply say, "oh, I don't really have

docs/code/IDataViewTypeSystem.md

@@ -63,7 +63,7 @@ components. At a high level, it is analogous to the .Net interface
 While `IEnumerable<T>` is a sequence of objects of type `T`, `IDataView` is a
 sequence of rows. An `IDataView` object has an associated `ISchema` object
 that defines the `IDataView`'s columns, including their names, types, indices,
-and associated metadata. Each row of the `IDataView` has a value for each
+and associated annotations. Each row of the `IDataView` has a value for each
 column defined by the schema.
 
 Just as `IEnumerable<T>` has an associated enumerator interface, namely
@@ -224,29 +224,29 @@ to a dense representation having the suppressed entries filled in with the
 entries are emphatically *not* the missing/`NA` value of the item type, unless
 the missing and default values are identical, as they are for key types.
 
-### Metadata
+### Annotations
 
-A column in an `ISchema` can have additional column-wide information, known as
-metadata. For each string value, known as a metadata kind, a column may have a
-value associated with that metadata kind. The value also has an associated
+A column in an `DataViewSchema` can have additional column-wide information, known as
+annotations. For each string value, known as an annotation kind, a column may have a
+value associated with that annotation kind. The value also has an associated
 type, which is a compatible column type.
 
 For example:
 
 * A column may indicate that it is normalized, by providing a `BL` valued
-  piece of metadata named `IsNormalized`.
+  annotation named `IsNormalized`.
 
 * A column whose type is `V<R4,17>`, meaning a vector of length 17 whose items
-  are single-precision floating-point values, might have `SlotNames` metadata
+  are single-precision floating-point values, might have `SlotNames` annotation
   of type `V<TX,17>`, meaning a vector of length 17 whose items are text.
 
 * A column produced by a scorer may have several pieces of associated
-  metadata, indicating the "scoring column group id" that it belongs to, what
+  annotations, indicating the "scoring column group id" that it belongs to, what
   kind of scorer produced the column (for example, binary classification), and the
   precise semantics of the column (for example, predicted label, raw score,
   probability).
 
-The `ISchema` interface, including the metadata API, is fully specified in
+The `DataViewSchema` class, including the annotations API, is fully specified in
 another document.
 
 ## Text Type