Term transformer implementation #759

Ivanidzo4ka · 2018-08-28T20:11:08Z

implementation for #754

Ivanidzo4ka · 2018-08-28T20:14:40Z

src/Microsoft.ML.Data/Transforms/TermTransform.cs

-        /// </summary>
-        private TermTransform(IHostEnvironment env, TermTransform transform, IDataView newSource)
-            : base(env, RegistrationName, transform, newSource, TestIsKnownDataKind)
+        public void CreateInfos(SourceNameColumnBase[] column, ISchema input)


CreateInfos [](start = 20, length = 11)

need be shared with Mapper #Closed

Ivanidzo4ka · 2018-08-28T20:15:40Z

src/Microsoft.ML.Data/Transforms/TermTransform.cs

+#pragma warning restore MSML_NoMessagesForLoadContext
+            boundMap = termMap;
+        }
+
        private static string TestIsKnownDataKind(ColumnType type)


TestIsKnownDataKind [](start = 30, length = 19)

remove or incorporate into current logic #Closed

Ivanidzo4ka · 2018-08-28T20:18:12Z

src/Microsoft.ML.Data/Transforms/TermTransformImpl.cs

+                        var info = new MetadataInfo<VBuffer<DvText>>(columnType, getter);
+                        colMetaInfo.Add(MetadataUtils.Kinds.KeyValues, info);
+                        /*bldr.AddGetter<VBuffer<DvText>>(MetadataUtils.Kinds.KeyValues,
+                            new VectorType(TextType.Instance, TypedMap.OutputType.KeyCount), getter);*/


remove #Closed

Ivanidzo4ka · 2018-08-28T20:18:18Z

src/Microsoft.ML.Data/Transforms/TermTransformImpl.cs

+                        var info = new MetadataInfo<VBuffer<T>>(columnType, getter);
+                        colMetaInfo.Add(MetadataUtils.Kinds.KeyValues, info);
+                        /*bldr.AddGetter<VBuffer<T>>(MetadataUtils.Kinds.KeyValues,
+                            new VectorType(TypedMap.ItemType, TypedMap.OutputType.KeyCount), getter);*/


remove #Closed

Ivanidzo4ka · 2018-08-28T20:18:33Z

src/Microsoft.ML.Data/Transforms/TermTransformImpl.cs

+                        var info = new MetadataInfo<VBuffer<DvText>>(columnType, mgetter);
+                        colMetaInfo.Add(MetadataUtils.Kinds.KeyValues, info);
+                        /*bldr.AddGetter<VBuffer<DvText>>(MetadataUtils.Kinds.KeyValues,
+                            new VectorType(TextType.Instance, TypedMap.OutputType.KeyCount), mgetter);*/


remove #Closed

Ivanidzo4ka · 2018-08-28T20:18:39Z

src/Microsoft.ML.Data/Transforms/TermTransformImpl.cs

+                        var info = new MetadataInfo<VBuffer<TMeta>>(columnType, mgetter);
+                        colMetaInfo.Add(MetadataUtils.Kinds.KeyValues, info);
+                        /*bldr.AddGetter<VBuffer<TMeta>>(MetadataUtils.Kinds.KeyValues,
+                            new VectorType(srcMetaType.ItemType.AsPrimitive, TypedMap.OutputType.KeyCount), mgetter);*/


remove #Closed

fix getters

Ivanidzo4ka · 2018-08-29T21:09:57Z

@Zruty0 can you take a glance of changes in term transform?
@codemzs I've change some interfaces in Onnx and PFA, are you ok with this?

Ivanidzo4ka · 2018-08-30T16:46:25Z

src/Microsoft.ML.Data/DataView/RowToRowMapperTransform.cs

+            }
+        }
+
+        private sealed class Row : IRow


Row [](start = 29, length = 3)

Can I use SimpleRow? #Closed

I suppose you can

In reply to: 214102719 [](ancestors = 214102719)

A better question is: can you reuse RowCursor somehow? Maybe derive RowCursor from Row or something. It's bad to have two implementations of GetGetter

In reply to: 214105932 [](ancestors = 214105932,214102719)

It's rude to intervene my dialogue with myself!

In reply to: 214106201 [](ancestors = 214106201,214105932,214102719)

Zruty0 · 2018-08-30T16:55:15Z

src/Microsoft.ML.Data/DataView/RowToRowMapperTransform.cs

@@ -318,6 +324,96 @@ public override IRowCursor[] GetRowCursorSet(out IRowCursorConsolidator consolid
            return cursors;
        }

+        public void SaveAsOnnx(OnnxContext ctx)
+        {
+            if (_mapper is ISaveAsOnnx onnx)


Host.CheckValue(ctx) #Closed

Zruty0 · 2018-08-30T16:55:47Z

src/Microsoft.ML.Data/DataView/RowToRowMapperTransform.cs

+        public Func<int, bool> GetDependencies(Func<int, bool> predicate)
+        {
+            Func<int, bool> predicateInput;
+            var active = _bindings.GetActive(predicate, out predicateInput);


var active = [](start = 12, length = 12)

no need to assign anything, just call GetActive #Closed

Zruty0 · 2018-08-30T16:56:25Z

src/Microsoft.ML.Data/DataView/RowToRowMapperTransform.cs

+                return fn;
+            }
+
+            public ValueGetter<UInt128> GetIdGetter()


GetIdGetter [](start = 40, length = 11)

=> #Closed

Zruty0 · 2018-08-30T17:00:03Z

src/Microsoft.ML.Data/Model/Onnx/ICanSaveOnnx.cs

+    /// <summary>
+    /// This data model component is savable as ONNX.
+    /// </summary>
+    public interface ITransformCanSaveOnnx : ICanSaveOnnx, ISaveAsOnnx, IDataTransform


ICanSaveOnnx, ISaveAsOnnx [](start = 45, length = 25)

this is confusing: why are there two interfaces? #Closed

I see a reason, but it doesn't make it less confusing to be honest

In reply to: 214106801 [](ancestors = 214106801)

Zruty0 · 2018-08-30T18:09:34Z

src/Microsoft.ML.Data/Transforms/TermEstimatorcs.cs

+                var originalColumn = inputSchema.FindColumn(column.Source);
+                if (originalColumn != null)
+                {
+                    var col = new SchemaShape.Column(column.Name, originalColumn.Kind, DataKind.U4, true, originalColumn.MetadataKinds);


MetadataKinds [](start = 121, length = 13)

is this correct? #Closed

Zruty0 · 2018-08-30T18:13:05Z

src/Microsoft.ML.Data/Transforms/TermEstimatorcs.cs

+                    throw _host.ExceptParam(nameof(inputSchema), $"{column.Source} not found in {nameof(inputSchema)}");
+                }
+            }
+            return new SchemaShape(resultDic.Values.ToArray());


.ToArray() [](start = 51, length = 10)

No need #Closed

Zruty0 · 2018-08-30T18:13:28Z

src/Microsoft.ML.Data/Transforms/TermTransform.cs

+using System.IO;
+using System.Linq;
+using System.Text;
+using static Microsoft.ML.Runtime.Data.TermTransform;


using static [](start = 0, length = 12)

can we avoid this? #Closed

Zruty0 · 2018-08-30T18:16:02Z

src/Microsoft.ML.Data/Transforms/TermTransform.cs

@@ -226,70 +227,154 @@ private CodecFactory CodecFactory
        /// Public constructor corresponding to SignatureDataTransform.
        /// </summary>
        public TermTransform(IHostEnvironment env, Arguments args, IDataView input)


public TermTransform(IHostEnvironment env, Arguments args, IDataView input) [](start = 8, length = 75)

it doesn't correspond to the signature any more, right? #Closed

Actually, I'd suggest having the 'training' happen in public static TermTransform Train(env, args, trainData), and ctors should be private.

In reply to: 214131316 [](ancestors = 214131316)

Zruty0 · 2018-08-30T18:20:53Z

src/Microsoft.ML.Data/Transforms/TermTransform.cs

+        internal readonly ISchema Schema;
+        private readonly Dictionary<int, int> _colNewToOldMapping;
+        private readonly ColumnType[] _types;
+        private readonly (string Source, string Name)[] _columns;


if you look at the image transforms, I adopted the paradigm that row mapper has a transformer _parent. When you load a row mapper, you actually load the parent, and call MakeRowMapper on it.
This way you don't need a static method to 'save stuff' / 'load stuff': you make only the transformer truly save-able, and the mapper will just save the parent. #Closed

TomFinley · 2018-08-31T17:47:22Z

src/Microsoft.ML.Data/DataView/RowToRowMapperTransform.cs

@@ -209,6 +211,10 @@ private static VersionInfo GetVersionInfo()

        public override ISchema Schema { get { return _bindings; } }

+        public bool CanSaveOnnx => _mapper is ISaveAsOnnx onnxMapper ? true : false;
+
+        public bool CanSavePfa => _mapper is ISaveAsOnnx pfaMapper ? true : false;


ISaveAsOnnx [](start = 45, length = 11)

You meant PFA I'm sure. #Closed

TomFinley · 2018-08-31T17:51:29Z

src/Microsoft.ML.Data/Model/Pfa/ICanSavePfa.cs

    /// </summary>
-    public interface ITransformCanSavePfa : ICanSavePfa, IDataTransform
+    public interface ISaveAsPfa


ISaveAsPfa [](start = 21, length = 10)

Same comment... can't do this. #Closed

TomFinley · 2018-08-31T18:00:15Z

src/Microsoft.ML.Data/Model/Onnx/ICanSaveOnnx.cs

    /// </summary>
-    public interface ITransformCanSaveOnnx: ICanSaveOnnx, IDataTransform
+    public interface ISaveAsOnnx


ISaveAsOnnx [](start = 21, length = 11)

This thing needs to descend from ICanSaveOnnx, because it is often the case, as you'll see if you inspect the actual implementations of CanSaveOnnx, that the behavior of whether or not we can in fact save as ONNX is far more complex than whether we implement the interface or not. #Closed

For example, if there were an implementation of IRowMapper that wrapped any existing mapper, obviously that meta-wrapper would have to report it can save as ONNX, but internally any of the things it is wrapping do not it is hosed, which is bad.

By having the core interface always have this runtime check, we avoid the problem that we have elsewhere with things like exporting as text, exporting as INI, and so on, where we have things that report they can save as something, but really cannot, and fail in strange and interesting ways.

In reply to: 214432408 [](ancestors = 214432408)

TomFinley · 2018-08-31T18:02:14Z

src/Microsoft.ML.Data/DataView/RowToRowMapperTransform.cs

@@ -209,6 +211,10 @@ private static VersionInfo GetVersionInfo()

        public override ISchema Schema { get { return _bindings; } }

+        public bool CanSaveOnnx => _mapper is ISaveAsOnnx onnxMapper ? true : false;


_mapper is ISaveAsOnnx onnxMapper ? true : false; [](start = 35, length = 49)

Once you correct the interface this could be:

(_mapper as ISaveAsOnnx)?.CanSaveOnnx ?? false

See other instances of CanSaveOnnx to see this pattern.

Moot point, note that if you have an expression of the form <boolExpr> ? true : false this can be simplified to <boolExpr>. #Closed

TomFinley · 2018-08-31T18:03:00Z

src/Microsoft.ML.Data/DataView/RowToRowMapperTransform.cs

+        {
+            Host.CheckValue(ctx, nameof(ctx));
+            if (_mapper is ISaveAsOnnx onnx)
+                onnx.SaveAsOnnx(ctx);


I think elsewhere we fail if CanSaveOnnx is false, but double check me. #Closed

This is fine I guess. Alternate would just be to have at the head of this method Host.Check(CanSaveOnnx) but that's oK.

In reply to: 214433162 [](ancestors = 214433162)

TomFinley · 2018-08-31T18:25:11Z

src/Microsoft.ML.Data/Transforms/TermTransform.cs

@@ -97,7 +100,7 @@ public enum SortOrder : byte
            // other things, like case insensitive (where appropriate), culturally aware, etc.?
        }

-        private static class Defaults
+        public static class Defaults


public static class Defaults [](start = 8, length = 28)

Must this be public? Can it not be `internal? #Closed

TomFinley · 2018-08-31T20:09:26Z

src/Microsoft.ML.Data/Transforms/TermTransform.cs

+
+        public class ColumnInfo
+        {
+            public ColumnInfo(string input, string output, int maxNumTerms = Defaults.MaxNumTerms, SortOrder sort = Defaults.Sort, string[] term = null, string terms = null, bool textKeyValues = false)


string terms = null [](start = 153, length = 19)

Certainly don't need both of these. #Pending

Considering how TermMap class and Train function implemented...
I can spend probably few hours and refactor it, but this PR already covering so much stuff.

In reply to: 214463561 [](ancestors = 214463561)

One easy thing we can do is to pass terms = null everywhere, without allowing user to override it

In reply to: 214469361 [](ancestors = 214469361,214463561)

Right. In the public facing method, we definitely do not want both exposed. If an internal method requires it (why would you, if terms only possible use is the command line?), that's fine? I guess?

In reply to: 214479433 [](ancestors = 214479433,214469361,214463561)

Is this better?

In reply to: 214481635 [](ancestors = 214481635,214479433,214469361,214463561)

TomFinley · 2018-08-31T20:09:55Z

src/Microsoft.ML.Data/Transforms/TermTransform.cs

+            }
+        }
+
+        public class ColumnInfo


ColumnInfo [](start = 21, length = 10)

This is a little unfortunate. It may be necessary in the short term, but it seems like we have two objects to control the behavior, is it not so?

One class(ColumnInfo) used to pass parameters and basically a copy of Arguments.Column and used to pass parameters of how to train transformer, second is just convenient class to store information about schema we work with.

In reply to: 214463662 [](ancestors = 214463662)

Right, I was talking about this being a copy of Arguments.Column.

In reply to: 214468160 [](ancestors = 214468160,214463662)

It's a weak excuse, but I'm just following what @Zruty0 doing right now. I also don't like duplication, but in same time, Arguments.Column and ColumnInfo is a slightly different, ColumnInfo doesn't have any nullable fields, which is the case for Arguments.Column.

In reply to: 214482328 [](ancestors = 214482328,214468160,214463662)

Yes, the pattern I have been following up until now is, I keep Arguments.Column intact, and create a separate set of ColumnInfo when needed, to pass to my estimators/transformers.
The idea is, ColumnInfo doesn't have to be mutable and only have fields that can be populated via cmdline.

Ultimately, I guess, we will sweep up all the Arguments into the Microsoft.ML.CommandLine assembly, together with the corresponding loadable classes and factory methods

In reply to: 214484933 [](ancestors = 214484933,214482328,214468160,214463662)

TomFinley · 2018-08-31T20:14:47Z

src/Microsoft.ML.Data/Transforms/TermTransformImpl.cs

+            private static volatile CodecFactory _codecFactory;
+            private static object _factoryLock = new object();
+
+            private static CodecFactory CodecFactory(IHostEnvironment env)


private static [](start = 12, length = 14)

By making this object static, we are effectively making the environment and host composed out of that environment static, which we have indicated many, many times we do not want to do. If this CodecFactory must be defined with an environment, then it is a mistake to make instances of it static, by that token. #Closed

moved back to TermTransform

In reply to: 214464720 [](ancestors = 214464720)

TomFinley · 2018-08-31T20:15:34Z

src/Microsoft.ML.Data/Transforms/TermTransformImpl.cs

+                        Interlocked.CompareExchange(ref _codecFactory, new CodecFactory(env, _codecFactoryPool), null);
+                    }
+                }
+                env.Assert(_codecFactory != null);


Having a lock, or using InterlockedCompare.Exchange, are both possible choices. (We tend to favor InterlockedCompare.Exchange throughout the rest of the codebase, but whatever.) Using both however seems somewhat pointless. #Closed

Zruty0 · 2018-08-31T21:32:32Z

src/Microsoft.ML.Data/Transforms/TermTransform.cs

-            Contracts.Assert(CanSavePfa);
+            private readonly ColumnType[] _types;
+            private readonly TermTransform _parent;
+            private readonly ColInfo[] _infos;


_infos [](start = 39, length = 6)

can you zip _types into _infos? #Closed

I mostly use ColInfo to pass information about schema to TermMap builders. So I can go scan data and build dictionaries.
_types is output type of the column, and depends on TermMap (it's gonna be Key of U4, but you don't know cardinality of it until you build map.

I can put them together, but they would be somewhat weird.

In reply to: 214480038 [](ancestors = 214480038)

I see

In reply to: 214482092 [](ancestors = 214482092,214480038)

Zruty0 · 2018-08-31T21:37:40Z

test/Microsoft.ML.Tests/TermEstimatorTests.cs

+                    new TermTransform.ColumnInfo("C", "TermC")
+                });
+                var invalidData = ComponentCreation.CreateDataView(env, xydata);
+                TestEstimatorCore(pipe, dataView, null, invalidData);


TestEstimatorCore [](start = 16, length = 17)

do another one with stringData. Or make TestEstimatorCore accept an array of invalidData's #Closed

TestEstimator except to fail in both cases, one for schema, one for transformation, but for schema string fields are perfectly fine, so this lead to test failure. and I'm not sure I want to change TestEstimatorCore

In reply to: 214480934 [](ancestors = 214480934)

Zruty0 · 2018-08-31T21:38:48Z

test/Microsoft.ML.Tests/TermEstimatorTests.cs

+        }
+
+        [Fact]
+        void TestBadTransformSchema()


TestBadTransformSchema [](start = 13, length = 22)

I suspect that all the tests except TestWorking are now not needed, as all is covered by TestEstimatorCore. #Closed

Keep only TestWorking, and also baseline the output.

In reply to: 214481129 [](ancestors = 214481129)

I still want to have other tests

In reply to: 214481205 [](ancestors = 214481205,214481129)

Zruty0

🕐

Zruty0

Zruty0 · 2018-09-01T00:34:09Z

test/Microsoft.ML.TestFramework/DataPipe/TestDataPipeBase.cs

+                mustFail(() => loadedTransformer.GetOutputSchema(validForFitNotValidForTransformInput.Schema));
+                mustFail(() => loadedTransformer.Transform(validForFitNotValidForTransformInput));
+            }
+

            // Schema verification between estimator and transformer.


remove empty line

Zruty0

TomFinley

All right thanks @Ivanidzo4ka !

first iteration

23170de

Ivanidzo4ka changed the title ~~Term transformer implementation~~ WIP Term transformer implementation Aug 28, 2018

Ivanidzo4ka commented Aug 28, 2018

View reviewed changes

ctrl+k+d

b7cc054

fix getters

Zruty0 mentioned this pull request Aug 28, 2018

New API for ML.NET #754

Closed

Ivan Matantsev added 6 commits August 28, 2018 16:03

some improvments

99b2e2f

some more metadata magic

1e848c5

fix onnx story

577d62f

RowToRowToRowToRowToRow

53260d0

prepare for review

058b04c

ColumnBase first

8c9823d

Ivanidzo4ka changed the title ~~WIP Term transformer implementation~~ Term transformer implementation Aug 29, 2018

Ivanidzo4ka requested a review from Zruty0 August 29, 2018 21:10

Ivanidzo4ka commented Aug 30, 2018

View reviewed changes

Zruty0 added the API Issues pertaining the friendly API label Aug 30, 2018

Zruty0 reviewed Aug 30, 2018

View reviewed changes

more changes

72be9d3

TomFinley reviewed Aug 31, 2018

View reviewed changes

Onnx comments

76bcd54

TomFinley reviewed Aug 31, 2018

View reviewed changes

shuffle codecs, onnx, private

b9e1473

Zruty0 reviewed Aug 31, 2018

View reviewed changes

Zruty0 suggested changes Aug 31, 2018

View reviewed changes

silly solution

f7e3a12

dotnet deleted a comment from pete Aug 31, 2018

Zruty0 approved these changes Aug 31, 2018

View reviewed changes

improvments in tests

e95cd43

Zruty0 reviewed Sep 1, 2018

View reviewed changes

Zruty0 approved these changes Sep 1, 2018

View reviewed changes

Ivan Matantsev added 2 commits August 31, 2018 17:37

Merge branch 'master' into ivanidze/Term_Transformer

e3fce58

extra line

ec3e5d3

TomFinley approved these changes Sep 3, 2018

View reviewed changes

Merge branch 'master' into ivanidze/Term_Transformer

fa14729

TomFinley merged commit 622e028 into dotnet:master Sep 4, 2018

sfilipi mentioned this pull request Oct 10, 2018

API reference - Samples for Transforms #1209

Closed

ghost locked as resolved and limited conversation to collaborators Mar 29, 2022

		@@ -209,6 +211,10 @@ private static VersionInfo GetVersionInfo()

		public override ISchema Schema { get { return _bindings; } }

		public bool CanSaveOnnx => _mapper is ISaveAsOnnx onnxMapper ? true : false;

Term transformer implementation #759

Term transformer implementation #759

Conversation

Ivanidzo4ka commented Aug 28, 2018 • edited Loading

Ivanidzo4ka Aug 28, 2018 • edited Loading

Choose a reason for hiding this comment

Ivanidzo4ka Aug 28, 2018 • edited by Zruty0 Loading

Choose a reason for hiding this comment

Ivanidzo4ka Aug 28, 2018 • edited Loading

Choose a reason for hiding this comment

Ivanidzo4ka Aug 28, 2018 • edited Loading

Choose a reason for hiding this comment

Ivanidzo4ka Aug 28, 2018 • edited Loading

Choose a reason for hiding this comment

Ivanidzo4ka Aug 28, 2018 • edited Loading

Choose a reason for hiding this comment

Ivanidzo4ka commented Aug 29, 2018

Ivanidzo4ka Aug 30, 2018 • edited by Zruty0 Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zruty0 Aug 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zruty0 Aug 30, 2018 • edited Loading

Choose a reason for hiding this comment

Zruty0 Aug 30, 2018 • edited Loading

Choose a reason for hiding this comment

Zruty0 Aug 30, 2018 • edited Loading

Choose a reason for hiding this comment

Zruty0 Aug 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zruty0 Aug 30, 2018 • edited Loading

Choose a reason for hiding this comment

Zruty0 Aug 30, 2018 • edited Loading

Choose a reason for hiding this comment

Zruty0 Aug 30, 2018 • edited Loading

Choose a reason for hiding this comment

Zruty0 Aug 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zruty0 Aug 30, 2018 • edited Loading

Choose a reason for hiding this comment

TomFinley Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

TomFinley Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

TomFinley Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomFinley Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

TomFinley Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomFinley Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

TomFinley Aug 31, 2018 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

Ivanidzo4ka Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivanidzo4ka Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivanidzo4ka Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomFinley Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomFinley Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

Zruty0 Aug 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivanidzo4ka commented Aug 28, 2018 •

edited

Loading

Ivanidzo4ka Aug 28, 2018 •

edited

Loading

Ivanidzo4ka Aug 28, 2018 •

edited by Zruty0

Loading

Ivanidzo4ka Aug 28, 2018 •

edited

Loading

Ivanidzo4ka Aug 28, 2018 •

edited

Loading

Ivanidzo4ka Aug 28, 2018 •

edited

Loading

Ivanidzo4ka Aug 28, 2018 •

edited

Loading

Ivanidzo4ka Aug 30, 2018 •

edited by Zruty0

Loading

Zruty0 Aug 30, 2018 •

edited

Loading

Zruty0 Aug 30, 2018 •

edited

Loading

Zruty0 Aug 30, 2018 •

edited

Loading

Zruty0 Aug 30, 2018 •

edited

Loading

Zruty0 Aug 30, 2018 •

edited

Loading

Zruty0 Aug 30, 2018 •

edited

Loading

Zruty0 Aug 30, 2018 •

edited

Loading

Zruty0 Aug 30, 2018 •

edited

Loading

Zruty0 Aug 30, 2018 •

edited

Loading

Zruty0 Aug 30, 2018 •

edited

Loading

TomFinley Aug 31, 2018 •

edited

Loading

TomFinley Aug 31, 2018 •

edited

Loading

TomFinley Aug 31, 2018 •

edited

Loading

TomFinley Aug 31, 2018 •

edited

Loading

TomFinley Aug 31, 2018 •

edited

Loading

TomFinley Aug 31, 2018 •

edited

Loading

TomFinley Aug 31, 2018 •

edited by Ivanidzo4ka

Loading

Ivanidzo4ka Aug 31, 2018 •

edited

Loading

Ivanidzo4ka Aug 31, 2018 •

edited

Loading

Ivanidzo4ka Aug 31, 2018 •

edited

Loading

TomFinley Aug 31, 2018 •

edited

Loading

TomFinley Aug 31, 2018 •

edited

Loading

Zruty0 Aug 31, 2018 •

edited

Loading

Zruty0 Aug 31, 2018 •

edited

Loading

Zruty0 Aug 31, 2018 •

edited

Loading

Zruty0 Sep 1, 2018 •

edited

Loading