Documentation for BinaryClassification.AveragedPerceptron (V2) #2517

shmoradims · 2019-02-12T23:49:17Z

#2483 got messed up with a bad rebase. Recreating the PR here.

Docs & sample for BinaryClassification.AveragedPerceptron. Related to #1209.

…elearning into doc_binary_clf

For sample code, changed numIterations to 10 as per Justin's request

Ivanidzo4ka · 2019-02-13T00:21:26Z

src/Microsoft.ML.StandardLearners/Standard/Online/AveragedPerceptron.cs

@@ -41,12 +41,21 @@ public sealed class AveragedPerceptronTrainer : AveragedLinearTrainer<BinaryPred

        public sealed class Options : AveragedLinearArguments
        {
+            /// <summary>
+            /// The custom <a href="tmpurl_loss">loss</a>.


tmpurl_loss [](start = 36, length = 11)

What's the story with this tmpurl? #Resolved

Shahab is working on a concepts doc. Those tmpUrl are placeholders for referencing those concepts.

In reply to: 256202773 [](ancestors = 256202773)

please see #2356

In reply to: 256251776 [](ancestors = 256251776,256202773)

Ivanidzo4ka

codecov · 2019-02-13T01:00:57Z

Codecov Report

Merging #2517 into master will decrease coverage by 0.02%.
The diff coverage is 21.42%.

@@            Coverage Diff             @@
##           master    #2517      +/-   ##
==========================================
- Coverage   71.45%   71.42%   -0.03%     
==========================================
  Files         800      801       +1     
  Lines      141700   141749      +49     
  Branches    16135    16135              
==========================================
- Hits       101247   101240       -7     
- Misses      35987    36041      +54     
- Partials     4466     4468       +2

Flag	Coverage Δ
#Debug	`71.42% <21.42%> (-0.03%)`	⬇️
#production	`67.72% <15.38%> (-0.04%)`	⬇️
#test	`85.49% <100%> (-0.01%)`	⬇️

rogancarr · 2019-02-13T02:00:00Z

docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/BinaryClassification/AveragedPerceptron.cs

+            // Setting the seed to a fixed number in this example to make outputs deterministic.
+            var mlContext = new MLContext(seed: 0);
+
+            // Download and featurize the dataset


// Downl [](start = 12, length = 8)

Period at the end of all comments. #Resolved

done

In reply to: 256221898 [](ancestors = 256221898)

rogancarr · 2019-02-13T02:00:30Z

docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/BinaryClassification/AveragedPerceptron.cs

+            var metrics = mlContext.BinaryClassification.EvaluateNonCalibrated(dataWithPredictions, "IsOver50K");
+            SamplesUtils.ConsoleUtils.PrintMetrics(metrics);
+
+            // Output:


Output [](start = 15, length = 6)

Expected output, then an extra space on the output below. #Resolved

done

In reply to: 256221990 [](ancestors = 256221990)

Add space between Expected output and line below it.

In reply to: 256538945 [](ancestors = 256538945,256221990)

not sure what you mean. This is how other samples look like.

In reply to: 256622797 [](ancestors = 256622797,256538945,256221990)

rogancarr · 2019-02-13T02:02:55Z

src/Microsoft.ML.SamplesUtils/SamplesDatasetUtils.cs

+                    "occupation", "relationship", "ethnicity", "native-country", "age", "education-num",
+                    "capital-gain", "capital-loss", "hours-per-week"))
+                // Min-max normalized all the features
+                .Append(mlContext.Transforms.Normalize("Features"));


.Append [](start = 16, length = 7)

Can you add a CopyColumns("Label", "IsOver50K")? That way you don't have to specify the label or remember what it is. #Resolved

done

In reply to: 256222438 [](ancestors = 256222438)

rogancarr · 2019-02-13T02:03:33Z

src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs

@@ -190,17 +190,46 @@ public static class StandardLearnersCatalog
        }

        /// <summary>
-        /// Predict a target using a linear binary classification model trained with the AveragedPerceptron trainer.
+        /// Predict a target using a linear binary classification model trained with averaged perceptron trainer.


The #Resolved

done

In reply to: 256222543 [](ancestors = 256222543)

rogancarr · 2019-02-13T02:03:48Z

src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs

        /// </summary>
+        /// <remarks>
+        /// Perceptron is a classification algorithm that makes its predictions by finding a separating hyperplane.


Perceptron [](start = 12, length = 11)

The Perceptron #Resolved

done

In reply to: 256222592 [](ancestors = 256222592)

rogancarr · 2019-02-13T02:04:35Z

src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs

+        ///
+        /// The perceptron is an online algorithm, which means it processes the instances in the training set one at a time.
+        /// It starts with a set of initial weights (zero, random, or initialized from a previous learner). Then, for each example in the training set, the weighted sum of the features (sigma[0, D-1] (w_i * f_i)) is computed.
+        /// If this value has the same sign as the label of the current example, the weights remain the same.If they have opposite signs,


.If [](start = 108, length = 4)

Space on "same. If" #Resolved

good eyes

In reply to: 256222755 [](ancestors = 256222755)

rogancarr · 2019-02-13T02:05:14Z

src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs

+        /// The perceptron is an online algorithm, which means it processes the instances in the training set one at a time.
+        /// It starts with a set of initial weights (zero, random, or initialized from a previous learner). Then, for each example in the training set, the weighted sum of the features (sigma[0, D-1] (w_i * f_i)) is computed.
+        /// If this value has the same sign as the label of the current example, the weights remain the same.If they have opposite signs,
+        /// the weights vector is updated by either subtracting or adding (if the label is negative or positive, respectively) the feature vector of the current example,


subtracting or adding [](start = 52, length = 21)

"adding or subtracting" (common form of speech) #Resolved

done

In reply to: 256222892 [](ancestors = 256222892)

rogancarr · 2019-02-13T02:05:45Z

src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs

+        /// It starts with a set of initial weights (zero, random, or initialized from a previous learner). Then, for each example in the training set, the weighted sum of the features (sigma[0, D-1] (w_i * f_i)) is computed.
+        /// If this value has the same sign as the label of the current example, the weights remain the same.If they have opposite signs,
+        /// the weights vector is updated by either subtracting or adding (if the label is negative or positive, respectively) the feature vector of the current example,
+        /// multiplied by a factor 0 &lt; a &lt;= 1, called the learning rate.In a generalization of this algorithm, the weights are updated by adding the feature vector multiplied by the learning rate,


e.In [](start = 76, length = 5)

Space #Resolved

done

In reply to: 256222984 [](ancestors = 256222984)

rogancarr · 2019-02-13T02:07:45Z

src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs

        /// <param name="catalog">The binary classification catalog trainer object.</param>
        /// <param name="labelColumn">The name of the label column, or dependent variable.</param>
        /// <param name="featureColumn">The features, or independent variables.</param>
-        /// <param name="lossFunction">The custom loss.</param>
+        /// <param name="lossFunction">The custom <a href="tmpurl_loss">loss</a>. If <see langword="null"/>, hinge loss will be used resulting in max-margin averaged perceptron.</param>


The [](start = 38, length = 4)

A custom #Resolved

done

In reply to: 256223351 [](ancestors = 256223351)

rogancarr · 2019-02-13T02:08:17Z

        IClassificationLoss lossFunction = null,

How do we make one of these? Do we want to have these public? #Resolved

Refers to: src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs:238 in 59673b8. [](commit_id = 59673b8, deletion_comment = False)

rogancarr · 2019-02-13T02:09:11Z

src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs

@@ -219,10 +248,18 @@ public static class StandardLearnersCatalog
        }

        /// <summary>
-        /// Predict a target using a linear binary classification model trained with the AveragedPerceptron trainer.
+        /// Predict a target using a linear binary classification model trained with averaged perceptron trainer using advanced options.
+        /// For usage details, please see <see cref="AveragedPerceptron(BinaryClassificationCatalog.BinaryClassificationTrainers, string, string, string, IClassificationLoss, float, bool, float, int)"/>


For usage details [](start = 12, length = 17)

This goes in remarks, so it doesn't show up in the table of contents. #Resolved

+1

In reply to: 256223581 [](ancestors = 256223581)

moved

In reply to: 256250971 [](ancestors = 256250971,256223581)

rogancarr · 2019-02-13T02:09:31Z

src/Microsoft.ML.StandardLearners/Standard/Online/AveragedLinear.cs

@@ -17,38 +17,89 @@ namespace Microsoft.ML.Trainers.Online
 {
    public abstract class AveragedLinearArguments : OnlineLinearArguments
    {
+        /// <summary>
+        /// <a href="tmpurl_lr">Learning rate</a>.


Learning rate [](start = 32, length = 13)

Remarks? #Resolved

IMO, if it is short, it can stay in summary. This does need more text, if that's what you mean by remarks.

In reply to: 256223629 [](ancestors = 256223629)

i was thinking the same as Senja. The remark will be simply the link, so we might as well leave it in the summary.

In reply to: 256251344 [](ancestors = 256251344,256223629)

rogancarr · 2019-02-13T02:09:47Z

src/Microsoft.ML.StandardLearners/Standard/Online/AveragedLinear.cs

+        /// Determine whether to decrease the <see cref="LearningRate"/> or not.
+        /// </summary>
+        /// <value>
+        /// <see langword="true" /> to decrease the <see cref="LearningRate"/> as iterations progress; otherwise, <see langword="false" />.


<see lan [](start = 12, length = 8)

Remarks on exactly how it decreases? #Resolved

added

In reply to: 256223703 [](ancestors = 256223703)

rogancarr · 2019-02-13T02:10:30Z

src/Microsoft.ML.StandardLearners/Standard/Online/AveragedPerceptron.cs

-    //     - Prediction calibration to produce probabilities. Off by default, if on, uses exponential (aka Platt) calibration.
-    /// <include file='doc.xml' path='doc/members/member[@name="AP"]/*' />
+    /// <summary>
+    /// This is averaged perceptron trainer.


This is averaged perceptron trainer. [](start = 8, length = 36)

See the email on standard summary text. #Resolved

+1

In reply to: 256223833 [](ancestors = 256223833)

updated

In reply to: 256251639 [](ancestors = 256251639,256223833)

rogancarr · 2019-02-13T02:11:24Z

src/Microsoft.ML.StandardLearners/Standard/Online/OnlineLinear.cs

@@ -19,27 +19,41 @@ namespace Microsoft.ML.Trainers.Online

    public abstract class OnlineLinearArguments : LearnerInputBaseWithLabel
    {
+        /// <summary>
+        /// Number of training iterations through the data.


Number of training iterations [](start = 12, length = 29)

Number of passes through the dataset. #Resolved

done

In reply to: 256223998 [](ancestors = 256223998)

rogancarr · 2019-02-13T02:12:41Z

src/Microsoft.ML.StandardLearners/Standard/Online/OnlineLinear.cs


+        /// <summary>
+        /// <see langword="true" /> to shuffle data for each training iteration; otherwise, <see langword="false" />.
+        /// Default is <see langword="true" />.


Default is <see lan [](start = 11, length = 20)

in value #Resolved

moved

In reply to: 256224234 [](ancestors = 256224234)

sfilipi · 2019-02-13T05:18:58Z

docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/BinaryClassification/AveragedPerceptron.cs

+            // In this examples we will use the adult income dataset. The goal is to predict
+            // if a person's income is above $50K or not, based on different pieces of information about that person.
+            // For more details about this dataset, please see https://archive.ics.uci.edu/ml/datasets/adult
+


would this be better atop the Example() #Resolved

moved up

In reply to: 256250307 [](ancestors = 256250307)

sfilipi · 2019-02-13T05:21:57Z

src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs

        /// <param name="catalog">The binary classification catalog trainer object.</param>
        /// <param name="labelColumn">The name of the label column, or dependent variable.</param>
        /// <param name="featureColumn">The features, or independent variables.</param>
-        /// <param name="lossFunction">The custom loss.</param>
+        /// <param name="lossFunction">The custom <a href="tmpurl_loss">loss</a>. If <see langword="null"/>, hinge loss will be used resulting in max-margin averaged perceptron.</param>


tmpurl_loss [](start = 59, length = 11)

replace before checking in #WontFix

we don't have the final links yet. find-replace needs to be done later.

In reply to: 256250676 [](ancestors = 256250676)

sfilipi · 2019-02-13T05:22:12Z

src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs

-        /// <param name="learningRate">The learning Rate.</param>
-        /// <param name="decreaseLearningRate">Decrease learning rate as iterations progress.</param>
-        /// <param name="l2RegularizerWeight">L2 regularization weight.</param>
+        /// <param name="learningRate"><a href="tmpurl_lr">Learning rate</a>.</param>


tmpurl_lr [](start = 48, length = 9)

replace #WontFix

same as other one

In reply to: 256250732 [](ancestors = 256250732)

sfilipi · 2019-02-13T05:23:00Z

src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs

+        /// <example>
+        /// <format type="text/markdown">
+        /// <![CDATA[
+        /// [!code-csharp[AveragedPerceptron](~/../docs/samples/docs/samples/Microsoft.ML.Samples/Dynamic/BinaryClassification/AveragedPerceptron.cs)]


A [](start = 127, length = 1)

Trainers/AveragedPerceptron.cs #Resolved

aaah, good catch :)

In reply to: 256250880 [](ancestors = 256250880)

sfilipi · 2019-02-13T05:28:35Z

src/Microsoft.ML.StandardLearners/Standard/Online/OnlineLinear.cs

@@ -19,27 +19,41 @@ namespace Microsoft.ML.Trainers.Online

    public abstract class OnlineLinearArguments : LearnerInputBaseWithLabel


OnlineLinearArguments [](start = 26, length = 21)

xml #Resolved

added

In reply to: 256251874 [](ancestors = 256251874)

sfilipi · 2019-02-13T05:28:58Z

src/Microsoft.ML.StandardLearners/Standard/Online/AveragedLinear.cs

@@ -17,38 +17,89 @@ namespace Microsoft.ML.Trainers.Online
 {
    public abstract class AveragedLinearArguments : OnlineLinearArguments


AveragedLinearArguments [](start = 26, length = 23)

xml #Resolved

added

In reply to: 256251939 [](ancestors = 256251939)

sfilipi · 2019-02-13T05:30:08Z

src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs

@@ -219,10 +248,18 @@ public static class StandardLearnersCatalog
        }

        /// <summary>
-        /// Predict a target using a linear binary classification model trained with the AveragedPerceptron trainer.
+        /// Predict a target using a linear binary classification model trained with averaged perceptron trainer using advanced options.
+        /// For usage details, please see <see cref="AveragedPerceptron(BinaryClassificationCatalog.BinaryClassificationTrainers, string, string, string, IClassificationLoss, float, bool, float, int)"/>
        /// </summary>


i would prefer we have full text on all extension, and not redirect ppl to the other one just to read a paragraph.. This is where the doc.xml includes can come handy. #Resolved

as discussed offline, moving this text to the estimator class

In reply to: 256252156 [](ancestors = 256252156)

sfilipi · 2019-02-13T05:32:21Z

src/Microsoft.ML.StandardLearners/Standard/Online/OnlineLinear.cs

        [Argument(ArgumentType.AtMostOnce, HelpText = "Number of iterations", ShortName = "iter", SortOrder = 50)]
        [TGUI(Label = "Number of Iterations", Description = "Number of training iterations through data", SuggestedSweeps = "1,10,100")]
        [TlcModule.SweepableLongParamAttribute("NumIterations", 1, 100, stepSize: 10, isLogScale: true)]
-        public int NumIterations = OnlineDefaultArgs.NumIterations;


NumIterations [](start = 19, length = 13)

Set the Name attribute to be the old name everytime you change one of them; for maml backwards compat. #Resolved

i added it to shortname. did i do it right?

In reply to: 256252619 [](ancestors = 256252619)

shmoradims · 2019-02-13T19:07:17Z

        IClassificationLoss lossFunction = null,

I've covered it in AveragedPerceptronWithOptions.cs sample. We have public losses but they don't look good. I'll bring it up in the scrum.

In reply to: 463025252 [](ancestors = 463025252)

Refers to: src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs:238 in 59673b8. [](commit_id = 59673b8, deletion_comment = False)

sfilipi · 2019-02-13T22:47:10Z

.../Microsoft.ML.Samples/Dynamic/Trainers/BinaryClassification/AveragedPerceptronWithOptions.cs

+            var metrics = mlContext.BinaryClassification.EvaluateNonCalibrated(dataWithPredictions);
+            SamplesUtils.ConsoleUtils.PrintMetrics(metrics);
+
+            // Expected output:


Expected output: [](start = 15, length = 16)

space #Pending

sfilipi · 2019-02-13T22:47:41Z

src/Microsoft.ML.SamplesUtils/SamplesDatasetUtils.cs

@@ -86,6 +87,57 @@ public static string DownloadSentimentDataset()
        public static string DownloadAdultDataset()
            => Download("https://raw.githubusercontent.com/dotnet/machinelearning/244a8c2ac832657af282aa312d568211698790aa/test/data/adult.train", "adult.txt");

+        public static IDataView LoadFeaturizedAdultDataset(MLContext mlContext)


LoadFeaturizedAdultDataset [](start = 32, length = 26)

xml doc since this is public #Resolved

done.

In reply to: 256623075 [](ancestors = 256623075)

sfilipi · 2019-02-13T22:49:38Z

src/Microsoft.ML.StandardLearners/Standard/Online/OnlineLinear.cs

-
+    /// <summary>
+    /// Arguments class for online linear trainers.
+    /// </summary>
    public abstract class OnlineLinearArguments : LearnerInputBaseWithLabel


OnlineLinearArguments [](start = 26, length = 21)

think those are supposed to be called Options. cc @artidoro, @abgoswam

Not really. This serves as the base class (for the online linear learners) from which the public facing Options class is derived.

It should be internal though. I believe Issue #2264 should take care of making it internal

I would like to disagree regarding internal. Why? If you make it internal all this property will disappear for user, and they are good properties.
What is wrong with having hierarchy of classes?

In reply to: 256639493 [](ancestors = 256639493)

I was going by smell test -- I do not see any samples (yet) of how this class would be helpful for a public facing API. I may be wrong though.

Would like to hear @yaeldekel / @codemzs thoughts on this since they are working on "lockdown" issues.

If we do want to keep this public perhaps we should rename to OnlineLinearOptions ?

We have following structure:

OnlineLinear Options -> LinearSVM Options | |____> AverageLinear Options--> AveragePerceptron Options | |____________> OnlineGradientDescent Options

base options contains properties which belong to all of them, derived classes contains specific options.

Only way how you can get rid of this tree structure is to create new flatten object which will be flat and have all required properties by himself. But that would also mean you need to repeat some properties in this flatten structures multiple times.

In reply to: 256646030 [](ancestors = 256646030)

sfilipi · 2019-02-13T22:55:28Z

src/Microsoft.ML.StandardLearners/Standard/Online/OnlineLinear.cs

+        /// <summary>
+        /// Number of passes through the training dataset.
+        /// </summary>
+        [Argument(ArgumentType.AtMostOnce, HelpText = "Number of iterations", ShortName = "iter, numIterations", SortOrder = 50)]


numIterations", [](start = 96, length = 16)

Add a Name attribute property, for backwards compatibility, here and below in InitialWeightsDiameter.

See: https://github.com/dotnet/machinelearning/blob/master/src/Microsoft.ML.Data/DataLoadSave/Text/TextLoader.cs#L359

#Resolved

ShortName and Name work same way, so you can have it in either place.
On other hand ShortName assumes what it's a short name, which numIteration is not so short.
Also I'm not sure if you presence of space would make system happy iter, numIterations ->iter,numIterations
#Resolved

i'll remove the space. numIterations is now "short", given the new name NumberOfIterations :)

In reply to: 257009222 [](ancestors = 257009222)

rogancarr

Shahab Moradi added 12 commits February 7, 2019 16:10

Updated docs for AveragedPerceptron

bf9d946

Added a sample

4222e85

Addressed PR comments

f4f03ba

Added sample for the second overload with trainer options.

ceb3aa2

Fixed the failing tests

2b96f6d

Updated docs for AveragedPerceptron

87be8dc

Added a sample

ba0abff

Addressed PR comments

a5538ed

Added sample for the second overload with trainer options.

6b56065

Fixed the failing tests

c487733

Merge branch 'doc_binary_clf' of https://github.com/shmoradims/machin…

5219d0c

…elearning into doc_binary_clf

Fixed breaking changes from master.

59673b8

For sample code, changed numIterations to 10 as per Justin's request

shmoradims mentioned this pull request Feb 12, 2019

Documentation for BinaryClassification.AveragedPerceptron #2483

Closed

shmoradims requested review from sfilipi, rogancarr and Ivanidzo4ka February 12, 2019 23:50

shmoradims added the documentation Related to documentation of ML.NET label Feb 12, 2019

shmoradims closed this Feb 13, 2019

shmoradims reopened this Feb 13, 2019

Ivanidzo4ka reviewed Feb 13, 2019

View reviewed changes

Ivanidzo4ka approved these changes Feb 13, 2019

View reviewed changes

rogancarr reviewed Feb 13, 2019

View reviewed changes

sfilipi reviewed Feb 13, 2019

View reviewed changes

Shahab Moradi added 2 commits February 13, 2019 14:06

Addressed the PR comments

3daf7bd

Moved algorithm remarks to the estimator class.

b10d7e6

sfilipi reviewed Feb 13, 2019

View reviewed changes

Shahab Moradi added 2 commits February 14, 2019 12:13

Merged with master

5900a41

Added doc xml docs

056a887

rogancarr approved these changes Feb 14, 2019

View reviewed changes

shmoradims merged commit 17c037f into dotnet:master Feb 14, 2019

shmoradims deleted the avg_perceptron branch February 14, 2019 22:14

shmoradims mentioned this pull request Feb 21, 2019

Docs and samples for the API reference site (P0 & P1 Trainers) #2522

Closed

ghost locked as resolved and limited conversation to collaborators Mar 24, 2022

		@@ -19,27 +19,41 @@ namespace Microsoft.ML.Trainers.Online

		public abstract class OnlineLinearArguments : LearnerInputBaseWithLabel

		@@ -17,38 +17,89 @@ namespace Microsoft.ML.Trainers.Online
		{
		public abstract class AveragedLinearArguments : OnlineLinearArguments

Documentation for BinaryClassification.AveragedPerceptron (V2) #2517

Documentation for BinaryClassification.AveragedPerceptron (V2) #2517

Conversation

shmoradims commented Feb 12, 2019

Ivanidzo4ka Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivanidzo4ka left a comment

Choose a reason for hiding this comment

codecov bot commented Feb 13, 2019 • edited Loading

Codecov Report

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr commented Feb 13, 2019 • edited by shmoradims Loading

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfilipi Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfilipi Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfilipi Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfilipi Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfilipi Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfilipi Feb 13, 2019 • edited by shmoradims Loading

Choose a reason for hiding this comment

Ivanidzo4ka Feb 13, 2019 •

edited by shmoradims

Loading

codecov bot commented Feb 13, 2019 •

edited

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr commented Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

rogancarr Feb 13, 2019 •

edited by shmoradims

Loading

sfilipi Feb 13, 2019 •

edited by shmoradims

Loading

sfilipi Feb 13, 2019 •

edited by shmoradims

Loading

sfilipi Feb 13, 2019 •

edited by shmoradims

Loading

sfilipi Feb 13, 2019 •

edited by shmoradims

Loading

sfilipi Feb 13, 2019 •

edited by shmoradims

Loading

sfilipi Feb 13, 2019 •

edited by shmoradims

Loading

sfilipi Feb 13, 2019 •

edited by shmoradims

Loading

sfilipi Feb 13, 2019 •

edited by shmoradims

Loading

sfilipi Feb 13, 2019 •

edited by shmoradims

Loading

sfilipi Feb 13, 2019 •

edited by shmoradims

Loading

Ivanidzo4ka Feb 14, 2019 •

edited

Loading

sfilipi Feb 13, 2019 •

edited by shmoradims

Loading

Ivanidzo4ka Feb 14, 2019 •

edited by shmoradims

Loading