Lockdown of Microsoft.ML.LightGBM public surface. #2476

zeahmed · 2019-02-08T02:56:10Z

This PR fixes #2271. Fixes #2534 fixes #2459

The changes included in this PR are as follows.

Changed unnecessary public items to internal.
Created samples in the samples project.
Referenced the samples as an example in public API exposed through MLContext.
Resolved a bug where one of the constructor of LightGbmMulticlassTrainer was using MakeBoolScalarLabel instead of MakeU4ScalarColumn.

zeahmed · 2019-02-08T02:56:50Z

@shmoradims, do you know which parameters you were talking about documenting in LightGBM? #Resolved

zeahmed · 2019-02-08T02:58:12Z

@sfilipi and @shmoradims, do we have standard explanation for the public Train methods in trainers? #Resolved

zeahmed · 2019-02-08T03:02:04Z

@TomFinley, these are the items I see exposed now after my changes.

#Resolved

codecov · 2019-02-08T03:29:32Z

Codecov Report

Merging #2476 into master will decrease coverage by <.01%.
The diff coverage is 50%.

@@            Coverage Diff             @@
##           master    #2476      +/-   ##
==========================================
- Coverage   71.43%   71.42%   -0.01%     
==========================================
  Files         801      801              
  Lines      141784   141793       +9     
  Branches    16135    16135              
==========================================
+ Hits       101278   101281       +3     
- Misses      36039    36045       +6     
  Partials     4467     4467

Flag	Coverage Δ
#Debug	`71.42% <50%> (-0.01%)`	⬇️
#production	`67.72% <50%> (-0.01%)`	⬇️
#test	`85.51% <ø> (ø)`	⬆️

Ivanidzo4ka · 2019-02-08T20:22:12Z

docs/samples/Microsoft.ML.Samples/Dynamic/LightGBM/LightGBMBinaryClassification.cs

@@ -0,0 +1,103 @@
+using System;
+using Microsoft.ML.Data;


Microsoft [](start = 6, length = 9)

We already have Transforms folder, I'm creating Trainers folder with Recommendation subfolder in #2451
maybe it's worth to put them in Trainers/BinaryClassification, Trainers/MultiClassification, Trainers/Regression?

Overwise it start look like trash pile :) #Resolved

+1 for the Trainers folder.

In reply to: 255223584 [](ancestors = 255223584)

Ivanidzo4ka · 2019-02-08T20:23:28Z

src/Microsoft.ML.LightGBM/LightGbmArguments.cs


    [TlcModule.ComponentKind("BoosterParameterFunction")]
    public interface ISupportBoosterParameterFactory : IComponentFactory<IBoosterParameter>
    {
    }
+
    public interface IBoosterParameter


public [](start = 4, length = 6)

hide this one as well. #Resolved

I can do things like this right now:
new Options.DartBooster.Arguments { }.CreateComponent(ML).UpdateParameters(new Dictionary<string,object>);
does they make any sense for you?

In reply to: 255223941 [](ancestors = 255223941)

The problem is ISupportBoosterParameterFactory is public. It is also used as a parameter of Options class which is public too. This does not allow us to make it internal.

In reply to: 255258378 [](ancestors = 255258378,255223941)

You can mark public ISupportBoosterParameterFactory Booster = new TreeBooster.Arguments(); as internal and put in arguments attribute CmdLine only.
You can also create something like public BoosterParameter BoosterParameters = TreeBooster.Arguments() and mark it EntryPoint only
At least we can try to hide it.

In reply to: 255667107 [](ancestors = 255667107,255258378,255223941)

CreateComponent(ML) and UpdateParameters are hidden now.

I am now using

public ISupportBoosterParameterFactory Booster = new TreeBooster.Arguments();

in the samples. So I think ISupportBoosterParameterFactory cannot be hidden.

In reply to: 256153739 [](ancestors = 256153739,255667107,255258378,255223941)

I agree with Ivan I think you should be able to do as he suggested: make public ISupportBoosterParameterFactory Booster internal, and add a new public field that is of type BoosterParameter, or even TreeBooster.Arguments.

In reply to: 256250752 [](ancestors = 256250752,256153739,255667107,255258378,255223941)

Ivanidzo4ka · 2019-02-08T20:24:06Z

        internal const int NumBoostRound = 100;

can be public, class is internal anyway. #Closed

Refers to: src/Microsoft.ML.LightGBM/LightGbmArguments.cs:97 in 59d1a08. [](commit_id = 59d1a08, deletion_comment = False)

Ivanidzo4ka · 2019-02-08T20:29:51Z

        public virtual void UpdateParameters(Dictionary<string, object> res)

internal #Closed

Refers to: src/Microsoft.ML.LightGBM/LightGbmArguments.cs:58 in 59d1a08. [](commit_id = 59d1a08, deletion_comment = False)

Ivanidzo4ka · 2019-02-08T20:30:07Z

        public const string FriendlyName = "Tree Booster";

internal #Closed

Refers to: src/Microsoft.ML.LightGBM/LightGbmArguments.cs:103 in 59d1a08. [](commit_id = 59d1a08, deletion_comment = False)

Ivanidzo4ka · 2019-02-08T20:30:43Z

            public virtual IBoosterParameter CreateComponent(IHostEnvironment env) => new TreeBooster(this);

hide it #Closed

Refers to: src/Microsoft.ML.LightGBM/LightGbmArguments.cs:168 in 59d1a08. [](commit_id = 59d1a08, deletion_comment = False)

Ivanidzo4ka · 2019-02-08T20:31:28Z

        public TreeBooster(Arguments args)

not sure it should be public #Closed

Refers to: src/Microsoft.ML.LightGBM/LightGbmArguments.cs:171 in 59d1a08. [](commit_id = 59d1a08, deletion_comment = False)

Ivanidzo4ka · 2019-02-08T20:31:37Z

        public const string FriendlyName = "Tree Dropout Tree Booster";

internal #Closed

Refers to: src/Microsoft.ML.LightGBM/LightGbmArguments.cs:191 in 59d1a08. [](commit_id = 59d1a08, deletion_comment = False)

Ivanidzo4ka · 2019-02-08T20:34:08Z

public delegate void ReduceScatterFunction([MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 1)]byte[] input, int inputSize, int typeSize,

any reason why it should be public? #Closed

Refers to: src/Microsoft.ML.LightGBM/Parallel/IParallel.cs:25 in 59d1a08. [](commit_id = 59d1a08, deletion_comment = False)

Ivanidzo4ka · 2019-02-08T20:34:17Z

public delegate void AllgatherFunction([MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 1)]byte[] input, int inputSize,

any reason why it should be public? #Closed

Refers to: src/Microsoft.ML.LightGBM/Parallel/IParallel.cs:33 in 59d1a08. [](commit_id = 59d1a08, deletion_comment = False)

Ivanidzo4ka · 2019-02-08T20:34:41Z

public interface IParallel

Definitely should be internal. #Closed

Refers to: src/Microsoft.ML.LightGBM/Parallel/IParallel.cs:37 in 59d1a08. [](commit_id = 59d1a08, deletion_comment = False)

Ivanidzo4ka · 2019-02-08T20:34:59Z

public interface ISupportParallel : IComponentFactory<IParallel>

internal #Closed

Refers to: src/Microsoft.ML.LightGBM/Parallel/IParallel.cs:71 in 59d1a08. [](commit_id = 59d1a08, deletion_comment = False)

Ivanidzo4ka · 2019-02-08T22:48:17Z

docs/samples/Microsoft.ML.Samples/Dynamic/LightGBM/LightGBMMulticlassClassification.cs

+            var pipeline = mlContext.Transforms.Conversion.MapValueToKey("LabelIndex", "Label")
+                        .Append(mlContext.MulticlassClassification.Trainers.LightGbm(labelColumn: "LabelIndex"))
+                        .Append(mlContext.Transforms.Conversion.MapValueToKey("PredictedLabelIndex", "PredictedLabel"))
+                        .Append(mlContext.Transforms.CopyColumns("Scores", "Score"));


.Append(mlContext.Transforms.CopyColumns("Scores", "Score") [](start = 24, length = 59)

why this one is needed? #Closed

I took the example from

machinelearning/docs/samples/Microsoft.ML.Samples/Static/LightGBMMulticlassWithInMemoryData.cs

Line 11 in b863ac2

class LightGBMMulticlassWithInMemoryData

I see there are two scores in dataview after transformation 1) Scores 2) Score. If there is no copy from Score to Scores, all the probabilities are zero. Any reason why?

In reply to: 255259910 [](ancestors = 255259910)

That sound like a bug.

In reply to: 255679272 [](ancestors = 255679272,255259910)

#2542 to track it

In reply to: 256220736 [](ancestors = 256220736,255679272,255259910)

Ivanidzo4ka · 2019-02-08T22:52:00Z

docs/samples/Microsoft.ML.Samples/Dynamic/LightGBM/LightGBMBinaryClassification.cs

+                                                        "hours-per-week",
+                                                        "native-country"))
+            .Append(mlContext.Transforms.Normalize("Features"))
+            .Append(mlContext.BinaryClassification.Trainers.LightGbm());


LightGbm [](start = 60, length = 8)

it's line 83 and i can finally see example of how to use LightGBM! #Closed

Ivanidzo4ka · 2019-02-08T22:58:14Z

        Options options)

I think this method deserves sample more than simple method.
Especially how to set specific booster.
Right now I'm creating Options class, type Booster inside it, and I have no idea what to do next.
Booster = new Options.GossBooster.Arguments { }
it should be something like this but how I have no idea how user would find out what he need to type Options first. And there is zero help from Intellisense. #Closed

Refers to: src/Microsoft.ML.LightGBM/LightGbmCatalog.cs:99 in 59d1a08. [](commit_id = 59d1a08, deletion_comment = False)

sfilipi · 2019-02-09T00:08:44Z

docs/samples/Microsoft.ML.Samples/Dynamic/LightGBM/LightGBMBinaryClassification.cs

+{
+    public class LightGbmBinaryClassification
+    {
+        public static void LightGbmBinaryClassificationExample()


LightGbmBinaryClassificationExample [](start = 27, length = 35)

think Shahab started calling the methods "Example" #Resolved

sfilipi · 2019-02-09T00:09:36Z

docs/samples/Microsoft.ML.Samples/Dynamic/LightGBM/LightGBMBinaryClassification.cs

+            // Creating the ML.Net IHostEnvironment object, needed for the pipeline
+            var mlContext = new MLContext();
+
+            var reader = mlContext.Data.ReadFromTextFile(dataFilePath, new TextLoader.Arguments


reader = mlContext.Data.ReadFromTextFile(dataFi [](start = 16, length = 47)

look at Shahab's PR on AP. He's creating a SamplesUtils for this one. #Resolved

sfilipi · 2019-02-09T00:12:03Z

docs/samples/Microsoft.ML.Samples/Dynamic/LightGBM/LightGBMBinaryClassification.cs

+            Console.WriteLine($"Negative Precision: {metrics.NegativePrecision}"); // 0.88
+            Console.WriteLine($"Negative Recall: {metrics.NegativeRecall}"); // 0.91
+            Console.WriteLine($"Positive Precision: {metrics.PositivePrecision}"); // 0.67
+            Console.WriteLine($"Positive Recall: {metrics.PositiveRecall}"); // 0.58       


see : #2483 #Resolved

To be consistent, I am going to change it the way #2483 is using it.

However, I think that this sort of code reusability does not make these sample standalone which I think is more important for documentation samples. What do you think?

In reply to: 255272930 [](ancestors = 255272930)

sfilipi · 2019-02-09T00:26:34Z

docs/samples/Microsoft.ML.Samples/Dynamic/LightGBM/LightGBMMulticlassClassification.cs

+            var examples = DatasetUtils.GenerateRandomMulticlassClassificationExamples(1000);
+
+            // Convert native C# class to IDataView, a consumble format to ML.NET functions.
+            var dataView = mlContext.Data.ReadFromEnumerable(examples);


dataView [](start = 16, length = 8)

preview #Resolved

sfilipi · 2019-02-09T00:26:53Z

docs/samples/Microsoft.ML.Samples/Dynamic/LightGBM/LightGBMMulticlassClassification.cs

+            var model = pipeline.Fit(trainingData);
+
+            // Do prediction on the test set.
+            var dataWithPredictions = model.Transform(testingData);


dataWithPredictions [](start = 16, length = 19)

preview #Resolved

sfilipi · 2019-02-09T00:27:29Z

docs/samples/Microsoft.ML.Samples/Dynamic/LightGBM/LightGBMMulticlassClassification.cs

+            var metrics = mlContext.MulticlassClassification.Evaluate(dataWithPredictions, label: "LabelIndex");
+
+            // Check if metrics are resonable.
+            Console.WriteLine("Macro accuracy: {0}, Micro accuracy: {1}.", 0.863482146891263, 0.86309523809523814);


863482146891263 [](start = 77, length = 15)

metrics[0] ? #Resolved

artidoro · 2019-02-13T09:04:37Z

        string labelColumn = DefaultColumnNames.Label,

Are we getting a sample for ranking too? #Resolved

Refers to: src/Microsoft.ML.LightGBM/LightGbmCatalog.cs:133 in 49509c1. [](commit_id = 49509c1, deletion_comment = False)

artidoro · 2019-02-13T09:14:00Z

...s/Microsoft.ML.Samples/Dynamic/Trainers/BinaryClassification/LightGBMBinaryClassification.cs

+    public class LightGbmBinaryClassification
+    {
+        /// <summary>
+        /// This example require installation of addition nuget package <a href="https://www.nuget.org/packages/Microsoft.ML.LightGBM/">Microsoft.ML.LightGBM</a>


require [](start = 25, length = 7)

Could you replace "requires" instead of "require". Here and in the other samples. #Resolved

zeahmed · 2019-02-13T18:29:22Z

        string labelColumn = DefaultColumnNames.Label,

As this PR is already getting very long, I have created an issue for this to address it in my next PR.
#2530

In reply to: 463116505 [](ancestors = 463116505)

Refers to: src/Microsoft.ML.LightGBM/LightGbmCatalog.cs:133 in 49509c1. [](commit_id = 49509c1, deletion_comment = False)

Ivanidzo4ka · 2019-02-13T19:14:18Z

...s/Microsoft.ML.Samples/Dynamic/Trainers/BinaryClassification/LightGBMBinaryClassification.cs

+{
+    public class LightGbmBinaryClassification
+    {
+        /// <summary>


I think @sfilipi ask me to use regular comments instead of summary.

no need for xml style comments #Resolved

Ivanidzo4ka · 2019-02-13T19:15:45Z

src/Microsoft.ML.SamplesUtils/SamplesDatasetUtils.cs

@@ -86,6 +86,38 @@ public static string DownloadSentimentDataset()
        public static string DownloadAdultDataset()
            => Download("https://raw.githubusercontent.com/dotnet/machinelearning/244a8c2ac832657af282aa312d568211698790aa/test/data/adult.train", "adult.txt");

+        public static IDataView LoadAdultDataset(MLContext mlContext)


LoadAdultDataset [](start = 32, length = 16)

I prefere Shahab approach https://github.com/dotnet/machinelearning/pull/2517/files#diff-eb95ea0c54ebcf8d695d8d73d5849b0c #Resolved

I actually took it from his PR...:)
He then changed. I am updating.

In reply to: 256547096 [](ancestors = 256547096)

Ivanidzo4ka

artidoro · 2019-02-14T01:02:20Z

src/Microsoft.ML.LightGBM/LightGbmArguments.cs

        }

        public sealed class TreeBooster : BoosterParameter<TreeBooster.Arguments>
        {
            public const string Name = "gbdt";
-            public const string FriendlyName = "Tree Booster";
+            internal const string FriendlyName = "Tree Booster";

            [TlcModule.Component(Name = Name, FriendlyName = FriendlyName, Desc = "Traditional Gradient Boosting Decision Tree.")]
            public class Arguments : ISupportBoosterParameterFactory


Arguments [](start = 25, length = 9)

Can you rename this to Options? you would also need to rename the local variable of type Arguments, they are usually called args and we usually rename to options. #Resolved

There is already a class named Options at line 43 above. I am not sure how its going to impact the cmdline.

In reply to: 256654714 [](ancestors = 256654714)

Could you name it with something that's related to Options? Before the general renaming of Arguments to Options both this and the above at line 43 were named Arguments, and it wasn't an issue. But I understand this is a more specific Options class, that's why it could make sense to add a prefix to the name.

In reply to: 256671757 [](ancestors = 256671757,256654714)

artidoro · 2019-02-14T01:07:54Z

src/Microsoft.ML.LightGBM/LightGbmArguments.cs

@@ -187,7 +191,7 @@ public override void UpdateParameters(Dictionary<string, object> res)
        public class DartBooster : BoosterParameter<DartBooster.Arguments>
        {
            public const string Name = "dart";
-            public const string FriendlyName = "Tree Dropout Tree Booster";
+            internal const string FriendlyName = "Tree Dropout Tree Booster";


Why not making name internal as well? #Resolved

artidoro · 2019-02-14T01:08:33Z

src/Microsoft.ML.LightGBM/LightGbmArguments.cs

@@ -231,7 +235,7 @@ public override void UpdateParameters(Dictionary<string, object> res)
        public class GossBooster : BoosterParameter<GossBooster.Arguments>
        {
            public const string Name = "goss";
-            public const string FriendlyName = "Gradient-based One-Size Sampling";
+            internal const string FriendlyName = "Gradient-based One-Size Sampling";


Here too? #Resolved

artidoro · 2019-02-14T01:17:36Z

src/Microsoft.ML.LightGBM/LightGbmArguments.cs

@@ -24,12 +24,13 @@

 namespace Microsoft.ML.LightGBM
 {
-    public delegate void SignatureLightGBMBooster();
+    internal delegate void SignatureLightGBMBooster();

    [TlcModule.ComponentKind("BoosterParameterFunction")]
    public interface ISupportBoosterParameterFactory : IComponentFactory<IBoosterParameter>


public [](start = 4, length = 6)

Can you make this interface internal? #WontFix

I tried making it internal but it requires more changes than I thought. So, I am not doing it in this PR. I opened an issue #2559

In reply to: 256657661 [](ancestors = 256657661)

artidoro · 2019-02-14T01:21:08Z

src/Microsoft.ML.LightGBM/LightGbmArguments.cs

@@ -92,14 +95,13 @@ private static string GetArgName(string name)
        [BestFriend]
        internal static class Defaults
        {
-            [BestFriend]
-            internal const int NumBoostRound = 100;
+            public const int NumBoostRound = 100;
        }

        public sealed class TreeBooster : BoosterParameter<TreeBooster.Arguments>
        {
            public const string Name = "gbdt";


public [](start = 12, length = 6)

Why does the name need to be public? #Resolved

Ivanidzo4ka · 2019-02-14T18:52:53Z

src/Microsoft.ML.LightGBM/LightGbmArguments.cs

@@ -187,7 +191,7 @@ public override void UpdateParameters(Dictionary<string, object> res)
        public class DartBooster : BoosterParameter<DartBooster.Arguments>


DartBooster [](start = 21, length = 11)

would be nice to make all this boosters sealed classes. #Resolved

…dline.

…earning into LightGBM_refact

…internal.

artidoro

Ivanidzo4ka

Lockdown of Microsoft.ML.LightGBM public surface.

59d1a08

zeahmed requested review from TomFinley, artidoro, shmoradims and sfilipi February 8, 2019 02:56

Ivanidzo4ka reviewed Feb 8, 2019

View reviewed changes

sfilipi reviewed Feb 9, 2019

View reviewed changes

Merged with the base and resolved merge conflicts.

49509c1

artidoro reviewed Feb 13, 2019

View reviewed changes

Addressed reviewers' comments.

3e27a88

Ivanidzo4ka reviewed Feb 13, 2019

View reviewed changes

Ivanidzo4ka mentioned this pull request Feb 13, 2019

LightGbmMulti System.ArgumentOutOfRangeException: Schema mismatch for label column '': expected Bool, got Key<U4> #2534

Closed

Ivanidzo4ka approved these changes Feb 13, 2019

View reviewed changes

artidoro reviewed Feb 14, 2019

View reviewed changes

zeahmed added 3 commits February 13, 2019 18:25

Merge remote-tracking branch 'upstream/master' into LightGBM_refact

6c8ac35

Addressed reviewers' comments.

571575f

Merge remote-tracking branch 'upstream/master' into LightGBM_refact

112636b

Ivanidzo4ka reviewed Feb 14, 2019

View reviewed changes

zeahmed added 5 commits February 14, 2019 14:18

Created two parameters for Booster. One for EntryPoint and one for cm…

fc60edd

…dline.

Merge branch 'LightGBM_refact' of https://github.com/zeahmed/machinel…

69e51bf

…earning into LightGBM_refact

Making all the Booster classes sealed.

31ceabb

Resolved merged conflicts.

0a56dc8

Resolved conflicts in SamplesDatasetUtils.cs

1b54da4

zeahmed mentioned this pull request Feb 14, 2019

Make ISupportBoosterParameterFactory internal in LightGBM project #2559

Closed

zeahmed added 2 commits February 14, 2019 15:46

Reverted changes related to making ISupportBoosterParameterFactory …

ed2ea83

…internal.

Changed Arguments classes to Options.

d1d4109

artidoro approved these changes Feb 15, 2019

View reviewed changes

Ivanidzo4ka approved these changes Feb 15, 2019

View reviewed changes

zeahmed merged commit 856c7e8 into dotnet:master Feb 15, 2019

ghost locked as resolved and limited conversation to collaborators Mar 24, 2022

		@@ -187,7 +191,7 @@ public override void UpdateParameters(Dictionary<string, object> res)
		public class DartBooster : BoosterParameter<DartBooster.Arguments>

Lockdown of Microsoft.ML.LightGBM public surface. #2476

Lockdown of Microsoft.ML.LightGBM public surface. #2476

Conversation

zeahmed commented Feb 8, 2019 • edited by Ivanidzo4ka Loading

zeahmed commented Feb 8, 2019 • edited Loading

zeahmed commented Feb 8, 2019 • edited Loading

zeahmed commented Feb 8, 2019 • edited Loading

codecov bot commented Feb 8, 2019 • edited Loading

Codecov Report

Ivanidzo4ka Feb 8, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivanidzo4ka Feb 8, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zeahmed Feb 13, 2019 • edited Loading

Choose a reason for hiding this comment

artidoro Feb 14, 2019 • edited Loading

Choose a reason for hiding this comment

Ivanidzo4ka commented Feb 8, 2019 • edited Loading

Ivanidzo4ka commented Feb 8, 2019 • edited Loading

Ivanidzo4ka commented Feb 8, 2019 • edited Loading

Ivanidzo4ka commented Feb 8, 2019 • edited Loading

Ivanidzo4ka commented Feb 8, 2019 • edited Loading

Ivanidzo4ka commented Feb 8, 2019 • edited Loading

Ivanidzo4ka commented Feb 8, 2019 • edited Loading

Ivanidzo4ka commented Feb 8, 2019 • edited Loading

Ivanidzo4ka commented Feb 8, 2019 • edited Loading

Ivanidzo4ka commented Feb 8, 2019 • edited Loading

Ivanidzo4ka Feb 8, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivanidzo4ka Feb 8, 2019 • edited Loading

Choose a reason for hiding this comment

Ivanidzo4ka commented Feb 8, 2019 • edited Loading

sfilipi Feb 9, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

sfilipi Feb 9, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

sfilipi Feb 9, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfilipi Feb 9, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

sfilipi Feb 9, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

sfilipi Feb 9, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

artidoro commented Feb 13, 2019 • edited by zeahmed Loading

artidoro Feb 13, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

zeahmed commented Feb 13, 2019

Ivanidzo4ka Feb 13, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

Ivanidzo4ka Feb 13, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivanidzo4ka left a comment

Choose a reason for hiding this comment

artidoro Feb 14, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

zeahmed Feb 14, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

artidoro Feb 14, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

artidoro Feb 14, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

artidoro Feb 14, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

artidoro Feb 14, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

Ivanidzo4ka Feb 14, 2019 • edited by zeahmed Loading

Choose a reason for hiding this comment

artidoro left a comment

zeahmed commented Feb 8, 2019 •

edited by Ivanidzo4ka

Loading

zeahmed commented Feb 8, 2019 •

edited

Loading

zeahmed commented Feb 8, 2019 •

edited

Loading

zeahmed commented Feb 8, 2019 •

edited

Loading

codecov bot commented Feb 8, 2019 •

edited

Loading

Ivanidzo4ka Feb 8, 2019 •

edited by zeahmed

Loading

Ivanidzo4ka Feb 8, 2019 •

edited by zeahmed

Loading

zeahmed Feb 13, 2019 •

edited

Loading

artidoro Feb 14, 2019 •

edited

Loading

Ivanidzo4ka commented Feb 8, 2019 •

edited

Loading

Ivanidzo4ka commented Feb 8, 2019 •

edited

Loading

Ivanidzo4ka commented Feb 8, 2019 •

edited

Loading

Ivanidzo4ka commented Feb 8, 2019 •

edited

Loading

Ivanidzo4ka commented Feb 8, 2019 •

edited

Loading

Ivanidzo4ka commented Feb 8, 2019 •

edited

Loading

Ivanidzo4ka commented Feb 8, 2019 •

edited

Loading

Ivanidzo4ka commented Feb 8, 2019 •

edited

Loading

Ivanidzo4ka commented Feb 8, 2019 •

edited

Loading

Ivanidzo4ka commented Feb 8, 2019 •

edited

Loading

Ivanidzo4ka Feb 8, 2019 •

edited

Loading

Ivanidzo4ka Feb 8, 2019 •

edited

Loading

Ivanidzo4ka commented Feb 8, 2019 •

edited

Loading

sfilipi Feb 9, 2019 •

edited by zeahmed

Loading

sfilipi Feb 9, 2019 •

edited by zeahmed

Loading

sfilipi Feb 9, 2019 •

edited by zeahmed

Loading

sfilipi Feb 9, 2019 •

edited by zeahmed

Loading

sfilipi Feb 9, 2019 •

edited by zeahmed

Loading

sfilipi Feb 9, 2019 •

edited by zeahmed

Loading

artidoro commented Feb 13, 2019 •

edited by zeahmed

Loading

artidoro Feb 13, 2019 •

edited by zeahmed

Loading

Ivanidzo4ka Feb 13, 2019 •

edited by zeahmed

Loading

Ivanidzo4ka Feb 13, 2019 •

edited by zeahmed

Loading

artidoro Feb 14, 2019 •

edited by zeahmed

Loading

zeahmed Feb 14, 2019 •

edited

Loading

artidoro Feb 14, 2019 •

edited by zeahmed

Loading

artidoro Feb 14, 2019 •

edited by zeahmed

Loading

artidoro Feb 14, 2019 •

edited by zeahmed

Loading

artidoro Feb 14, 2019 •

edited by zeahmed

Loading

Ivanidzo4ka Feb 14, 2019 •

edited by zeahmed

Loading