OneVsAllClassifier fits two trainers for binary classification task

[rtanase]

System information

• OS version/distro:
• .NET Version (eg., dotnet --info):

Issue

• What did you do?
• What happened?
• What did you expect?
  I expected OneVsAll to identify there are only two classes and train a single learner.

Source code / logs

Internally the OneVersusAllTrainer will always instantiate as many binary classifiers as the number of classes. This is inefficient for binary classification, as just a single trainer is needed. I understand that I can just use an out of the box binary classification for this, but it will simplify the usage.

[wschin]

So you're saying that if the number of classes is 2, OneVersusAllTrainer should only produce one binary classifier?

[rtanase]

yeah, I don't think it should be necessary to train 2 binary classifiers, one for each of the classes, when a single one would do the trick. Right?

[wschin]

From the perspective you pointed out, yes. On the other hand, the model trained by OVA is a list of model, if we follow your suggestion ML.NET will end up with

• OVA with 2 classes --> output model is OutputModel = new Model().
• OVA with > 2 classes --> output model is OutputModel = new Model[number of classes].

It looks a bit wired because the index i in OutputModel[i] means differently in the two cases above. In other words, it's not a desired strongly typed way to ML.NET's public APIs.

Inspired by your comment, if there are only two classes, ML.NET can internally train a binary classifier and copy it to OutputModel[0] and OutputModel[1].

[rtanase]

As long as there shouldn't be 2 trainings involved, then it should be fine. The public API should remain same. I wanna avoid having to go through the dataset to figure out how many classes are there to choose between a straight BinaryClassifier or a OneVsRest (which will again go through the dataset to determine number of classes).

If going with training a single model and storing it within OutputModel[0] and OutputModel[1], can those 2 models be exactly the same or does one need to be inverse of the first one?

[wschin]

The later one should have negative weights of the first one, I assume. For example, if OutputModel[0]'s weight vector is [1, 2, 3], OutputModel[1]'s weight vector should be [-1, -2, -3]. This PR will expose the trained models to users so that we can check if that assumption is true. If that assumption is true, I'd expect a small change to support your idea.

[wschin]

Now I feel doing one binary classification is better than copying the weights.